Naturalistic Head Motion Generation from Speech

10/26/2022
by   Trisha Mittal, et al.
0

Synthesizing natural head motion to accompany speech for an embodied conversational agent is necessary for providing a rich interactive experience. Most prior works assess the quality of generated head motion by comparing them against a single ground-truth using an objective metric. Yet there are many plausible head motion sequences to accompany a speech utterance. In this work, we study the variation in the perceptual quality of head motions sampled from a generative model. We show that, despite providing more diverse head motions, the generative model produces motions with varying degrees of perceptual quality. We finally show that objective metrics commonly used in previous research do not accurately reflect the perceptual quality of generated head motions. These results open an interesting avenue for future work to investigate better objective metrics that correlate with human perception of quality.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/18/2022

Towards a Perceptual Model for Estimating the Quality of Visual Speech

Generating realistic lip motions to simulate speech production is key fo...
research
07/24/2019

A neural network based post-filter for speech-driven head motion synthesis

Despite the fact that neural networks are widely used for speech-driven ...
research
09/09/2023

Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video

Synthesizing realistic videos according to a given speech is still an op...
research
11/17/2022

SPACEx: Speech-driven Portrait Animation with Controllable Expression

Animating portraits using speech has received growing attention in recen...
research
11/02/2022

Autoregressive GAN for Semantic Unconditional Head Motion Generation

We address the task of unconditional head motion generation to animate s...
research
03/21/2022

Generative Adversarial Network for Future Hand Segmentation from Egocentric Video

We introduce the novel problem of anticipating a time series of future h...
research
02/05/2020

Prediction of head motion from speech waveforms with a canonical-correlation-constrained autoencoder

This study investigates the direct use of speech waveforms to predict he...

Please sign up or login with your details

Forgot password? Click here to reset