It's A Match! Gesture Generation Using Expressive Parameter Matching

03/04/2021
by   Ylva Ferstl, et al.
0

Automatic gesture generation from speech generally relies on implicit modelling of the nondeterministic speech-gesture relationship and can result in averaged motion lacking defined form. Here, we propose a database-driven approach of selecting gestures based on specific motion characteristics that have been shown to be associated with the speech audio. We extend previous work that identified expressive parameters of gesture motion that can both be predicted from speech and are perceptually important for a good speech-gesture match, such as gesture velocity and finger extension. A perceptual study was performed to evaluate the appropriateness of the gestures selected with our method. We compare our method with two baseline selection methods. The first respects timing, the desired onset and duration of a gesture, but does not match gesture form in other ways. The second baseline additionally disregards the original gesture timing for selecting gestures. The gesture sequences from our method were rated as a significantly better match to the speech than gestures selected by either baseline method.

READ FULL TEXT

Authors

page 1

page 2

page 3

06/28/2021

Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech

We propose a new framework for gesture generation, aiming to allow data-...
10/02/2020

Understanding the Predictability of Gesture Parameters from Speech and their Perceptual Importance

Gesture behavior is a natural part of human conversation. Much work has ...
02/23/2021

A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020

Co-speech gestures, gestures that accompany speech, play an important ro...
08/17/2020

Sequence-to-Sequence Predictive Model: From Prosody To Communicative Gestures

Communicative gestures and speech prosody are tightly linked. Our object...
10/13/2020

Labeling the Phrase Set of the Conversation Agent, Rinna

Mapping spoken text to gestures is an important research area for robots...
07/16/2020

Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation

This paper presents a novel framework for speech-driven gesture producti...
11/05/2002

Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures

Although speech and gesture recognition has been studied extensively, al...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.