Sonification of Facial Actions for Musical Expression

10/07/2020
by   Mathias Funk, et al.
0

The central role of the face in social interaction and non-verbal communication suggests we explore facial action as a means of musical expression. This paper presents the design, implementation, and preliminary studies of a novel system utilizing face detection and optic flow algorithms to associate facial movements with sound synthesis in a topographically specific fashion. We report on our experience with various gesture-to-sound mappings and applications, and describe our preliminary experiments at musical performance using the system.

READ FULL TEXT

page 2

page 3

page 4

10/07/2020

Designing, Playing, and Performing with a Vision-based Mouth Interface

The role of the face and mouth in speech production as well asnon-verbal...
09/14/2018

Mugeetion: Musical Interface Using Facial Gesture and Emotion

People feel emotions when listening to music. However, emotions are not ...
11/26/2020

Interactive Machine Learning of Musical Gesture

This chapter presents an overview of Interactive Machine Learning (IML) ...
10/04/2020

Facial gesture interfaces for expression and communication

Considerable effort has been devoted to the automatic extraction of info...
03/09/2010

Facial Gesture Recognition Using Correlation And Mahalanobis Distance

Augmenting human computer interaction with automated analysis and synthe...
12/11/2020

Exploring Facial Expressions and Affective Domains for Parkinson Detection

Parkinson's Disease (PD) is a neurological disorder that affects facial ...
09/23/2016

The face-space duality hypothesis: a computational model

Valentine's face-space suggests that faces are represented in a psycholo...