3d human motion generation from the text via gesture action classification and the autoregressive model

11/18/2022
by   Gwantae Kim, et al.
0

In this paper, a deep learning-based model for 3D human motion generation from the text is proposed via gesture action classification and an autoregressive model. The model focuses on generating special gestures that express human thinking, such as waving and nodding. To achieve the goal, the proposed method predicts expression from the sentences using a text classification model based on a pretrained language model and generates gestures using the gate recurrent unit-based autoregressive model. Especially, we proposed the loss for the embedding space for restoring raw motions and generating intermediate motions well. Moreover, the novel data augmentation method and stop token are proposed to generate variable length motions. To evaluate the text classification model and 3D human motion generation model, a gesture action classification dataset and action-based gesture dataset are collected. With several experiments, the proposed method successfully generates perceptually natural and realistic 3D human motion from the text. Moreover, we verified the effectiveness of the proposed method using a public-available action recognition dataset to evaluate cross-dataset generalization performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/16/2023

AMD: Autoregressive Motion Diffusion

Human motion generation aims to produce plausible human motion sequences...
research
10/13/2022

Deep Gesture Generation for Social Robots Using Type-Specific Libraries

Body language such as conversational gesture is a powerful way to ease c...
research
02/22/2023

Soft Pneumatic Actuator Capable of Generating Various Bending and Extension Motions Inspired by an Elephant Trunk

Inspired by the dexterous handling ability of an elephant's trunk, we pr...
research
10/30/2018

DeepGRU: Deep Gesture Recognition Utility

We introduce DeepGRU, a deep learning based gesture and action recognize...
research
09/04/2023

DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion

We present DiverseMotion, a new approach for synthesizing high-quality h...
research
12/08/2022

Generating Holistic 3D Human Motion from Speech

This work addresses the problem of generating 3D holistic body motions f...
research
07/20/2023

Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification

In this paper, we briefly introduce the solution of our team HFUT-VUT fo...

Please sign up or login with your details

Forgot password? Click here to reset