Toward Automated Generation of Affective Gestures from Text:A Theory-Driven Approach

by   Micol Spitale, et al.

Communication in both human-human and human-robot interac-tion (HRI) contexts consists of verbal (speech-based) and non-verbal(facial expressions, eye gaze, gesture, body pose, etc.) components.The verbal component contains semantic and affective information;accordingly, HRI work on the gesture component so far has focusedon rule-based (mapping words to gestures) and data-driven (deep-learning) approaches to generating speech-paired gestures basedon either semantics or the affective state. Consequently, most ges-ture systems are confined to producing either semantically-linkedor affect-based gesticures. This paper introduces an approach forenabling human-robot communication based on a theory-drivenapproach to generate speech-paired robot gestures using both se-mantic and affective information. Our model takes as input textand sentiment analysis, and generates robot gestures in terms oftheir shape, intensity, and speed.



page 1

page 2

page 3


Gesticulator: A framework for semantically-aware speech-driven gesture generation

During speech, people spontaneously gesticulate, which plays a key role ...

Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots

Co-speech gestures enhance interaction experiences between humans as wel...

Speech-Gesture Mapping and Engagement Evaluation in Human Robot Interaction

A robot needs contextual awareness, effective speech production and comp...

Learning to gesticulate by observation using a deep generative approach

The goal of the system presented in this paper is to develop a natural t...

Speech-driven Animation with Meaningful Behaviors

Conversational agents (CAs) play an important role in human computer int...

Learning,Generating and Adapting Wave Gestures for Expressive Human-Robot Interaction

This study proposes a novel imitation learning approach for the stochast...

Labeling the Phrase Set of the Conversation Agent, Rinna

Mapping spoken text to gestures is an important research area for robots...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.