Fall Research Expo 2020

UPENN Aviary: Latent Space Analysis of Cowbird Song

Computational ethology is a rapidly expanding field with the advancements in computational power and machine learning. However, within the field, auditory research has been lagging behind due to the complexity of the data. For example, large amounts of labeled data are commonly needed to train neural networks when analyzing bird songs, but it might take a researcher many years to fully understand and recognize the subtle elements and cues the birds use to create this data set. Thus, the purpose of my research with the UPenn cowbird aviary team was creating latent space representations to investigate cowbird song features and characteristics. Additionally, I examined what information cowbird song can tell us, such as which individual the vocalization came from. I used UMAP to reduce the dimensionality of the data, or in other words visualize the song in a 2D space, requiring few a priori assumptions about the vocalizations and overcoming the challenge of expertly labeled data. The results demonstrated that the complex features of cowbird vocalizations, including song elements and the identity of the bird singing. Additionally, little to none labeled data was required to construct the latent space, significantly decreasing the amount of time and expertise needed to perform analyses. Future explorations include combining this data with visual data, such as pose and movement to further explore cowbird behavior.

PRESENTED BY
Team Grants for Interdisciplinary Activities
Engineering & Applied Sciences
Advised By
Marc Schmidt
Marc Badger
PRESENTED BY
Team Grants for Interdisciplinary Activities
Engineering & Applied Sciences
Advised By
Marc Schmidt
Marc Badger

Comments

Hi, I thought your research was incredibly compelling! I was wondering to what extent this ability to map cowbird song into 2D space can be applied to other birds, or even birds in nature rather than a controlled setting, like an aviary?