Interaction Transformer for Human Reaction Generation

07/04/2022
by   Baptiste Chopin, et al.
0

We address the challenging task of human reaction generation which aims to generate a corresponding reaction based on an input action. Most of the existing works do not focus on generating and predicting the reaction and cannot generate the motion when only the action is given as input. To address this limitation, we propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attentions. Specifically, the temporal attention captures the temporal dependencies of the motion of both characters and of their interaction, while the spatial attention learns the dependencies between the different body parts of each character and those which are part of the interaction. Moreover, we propose using graphs to increase the performance of the spatial attention via an interaction distance module that helps focus on nearby joints from both characters. Extensive experiments on the SBU interaction, K3HI, and DuetDance datasets demonstrate the effectiveness of InterFormer. Our method is general and can be used to generate more complex and long-term interactions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/25/2022

IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition

Human interaction recognition is very important in many applications. On...
research
10/06/2022

Focal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition

Despite great progress achieved by transformer in various vision tasks, ...
research
10/01/2021

GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interaction

Creating realistic characters that can react to the users' or another ch...
research
07/05/2023

MRecGen: Multimodal Appropriate Reaction Generator

Verbal and non-verbal human reaction generation is a challenging task, a...
research
06/20/2020

Technical Note: Generating Realistic Fighting Scenes by Game Tree

Recently, there have been a lot of researches to synthesize / edit the m...
research
03/16/2023

Narrator: Towards Natural Control of Human-Scene Interaction Generation via Relationship Reasoning

Naturally controllable human-scene interaction (HSI) generation has an i...
research
02/13/2023

Multiple Appropriate Facial Reaction Generation in Dyadic Interaction Settings: What, Why and How?

According to the Stimulus Organism Response (SOR) theory, all human beha...

Please sign up or login with your details

Forgot password? Click here to reset