Mugeetion: Musical Interface Using Facial Gesture and Emotion

09/14/2018
by   Eunjeong Stella Koh, et al.
0

People feel emotions when listening to music. However, emotions are not tangible objects that can be exploited in the music composition process as they are difficult to capture and quantify in algorithms. We present a novel musical interface, Mugeetion, designed to capture occurring instances of emotional states from users' facial gestures and relay that data to associated musical features. Mugeetion can translate qualitative data of emotional states into quantitative data, which can be utilized in the sound generation process. We also presented and tested this work in the exhibition of sound installation, Hearing Seascape, using the audiences' facial expressions. Audiences heard changes in the background sound based on their emotional state. The process contributes multiple research areas, such as gesture tracking systems, emotion-sound modeling, and the connection between sound and facial gesture.

READ FULL TEXT

page 1

page 4

research
10/07/2020

Sonification of Facial Actions for Musical Expression

The central role of the face in social interaction and non-verbal commun...
research
01/11/2020

Establishing Human-Robot Trust through Music-Driven Robotic Emotion Prosody and Gesture

As human-robot collaboration opportunities continue to expand, trust bec...
research
06/04/2021

Musical Prosody-Driven Emotion Classification: Interpreting Vocalists Portrayal of Emotions Through Machine Learning

The task of classifying emotions within a musical track has received wid...
research
03/02/2023

AI as mediator between composers, sound designers, and creative media producers

Musical professionals who produce material for non-musical stakeholders ...
research
10/07/2020

Designing, Playing, and Performing with a Vision-based Mouth Interface

The role of the face and mouth in speech production as well asnon-verbal...
research
07/01/2021

iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis

We introduce a new dataset for the emotional artificial intelligence res...
research
09/07/2022

ESSYS* Sharing #UC: An Emotion-driven Audiovisual Installation

We present ESSYS* Sharing #UC, an audiovisual installation artwork that ...

Please sign up or login with your details

Forgot password? Click here to reset