It's A Match! Gesture Generation Using Expressive Parameter Matching

by   Ylva Ferstl, et al.

Automatic gesture generation from speech generally relies on implicit modelling of the nondeterministic speech-gesture relationship and can result in averaged motion lacking defined form. Here, we propose a database-driven approach of selecting gestures based on specific motion characteristics that have been shown to be associated with the speech audio. We extend previous work that identified expressive parameters of gesture motion that can both be predicted from speech and are perceptually important for a good speech-gesture match, such as gesture velocity and finger extension. A perceptual study was performed to evaluate the appropriateness of the gestures selected with our method. We compare our method with two baseline selection methods. The first respects timing, the desired onset and duration of a gesture, but does not match gesture form in other ways. The second baseline additionally disregards the original gesture timing for selecting gestures. The gesture sequences from our method were rated as a significantly better match to the speech than gestures selected by either baseline method.



page 1

page 2

page 3


Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech

We propose a new framework for gesture generation, aiming to allow data-...

Understanding the Predictability of Gesture Parameters from Speech and their Perceptual Importance

Gesture behavior is a natural part of human conversation. Much work has ...

A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020

Co-speech gestures, gestures that accompany speech, play an important ro...

Sequence-to-Sequence Predictive Model: From Prosody To Communicative Gestures

Communicative gestures and speech prosody are tightly linked. Our object...

Labeling the Phrase Set of the Conversation Agent, Rinna

Mapping spoken text to gestures is an important research area for robots...

Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation

This paper presents a novel framework for speech-driven gesture producti...

Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures

Although speech and gesture recognition has been studied extensively, al...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.