Learning Embodied Semantics via Music and Dance Semiotic Correlations

03/25/2019
by   Francisco Afonso Raposo, et al.
0

Music semantics is embodied, in the sense that meaning is biologically mediated by and grounded in the human body and brain. This embodied cognition perspective also explains why music structures modulate kinetic and somatosensory perception. We leverage this aspect of cognition, by considering dance as a proxy for music perception, in a statistical computational model that learns semiotic correlations between music audio and dance video. We evaluate the ability of this model to effectively capture underlying semantics in a cross-modal retrieval task. Quantitative results, validated with statistical significance testing, strengthen the body of evidence for embodied cognition in music and show the model can recommend music audio for dance video queries and vice-versa.

READ FULL TEXT
research
08/24/2023

Emotion-Aligned Contrastive Learning Between Images and Music

Traditional music search engines rely on retrieval methods that match na...
research
12/14/2017

Towards Deep Modeling of Music Semantics using EEG Regularizers

Modeling of music audio semantics has been previously tackled through le...
research
03/14/2023

Improving Music Genre Classification from multi-modal properties of music and genre correlations Perspective

Music genre classification has been widely studied in past few years for...
research
01/06/2023

Multimodal Lyrics-Rhythm Matching

Despite the recent increase in research on artificial intelligence for m...
research
09/16/2020

A Human-Computer Duet System for Music Performance

Virtual musicians have become a remarkable phenomenon in the contemporar...
research
06/23/2020

Audeo: Audio Generation for a Silent Performance Video

We present a novel system that gets as an input video frames of a musici...
research
11/01/2015

Using Raspberry Pi for scientific video observation of pedestrians during a music festival

The document serves as a reference for researchers trying to capture a l...

Please sign up or login with your details

Forgot password? Click here to reset