SDMTL: Semi-Decoupled Multi-grained Trajectory Learning for 3D human motion prediction

by   Xiaoli Liu, et al.

Predicting future human motion is critical for intelligent robots to interact with humans in the real world, and human motion has the nature of multi-granularity. However, most of the existing work either implicitly modeled multi-granularity information via fixed modes or focused on modeling a single granularity, making it hard to well capture this nature for accurate predictions. In contrast, we propose a novel end-to-end network, Semi-Decoupled Multi-grained Trajectory Learning network (SDMTL), to predict future poses, which not only flexibly captures rich multi-grained trajectory information but also aggregates multi-granularity information for predictions. Specifically, we first introduce a Brain-inspired Semi-decoupled Motion-sensitive Encoding module (BSME), effectively capturing spatiotemporal features in a semi-decoupled manner. Then, we capture the temporal dynamics of motion trajectory at multi-granularity, including fine granularity and coarse granularity. We learn multi-grained trajectory information using BSMEs hierarchically and further capture the information of temporal evolutional directions at each granularity by gathering the outputs of BSMEs at each granularity and applying temporal convolutions along the motion trajectory. Next, the captured motion dynamics can be further enhanced by aggregating the information of multi-granularity with a weighted summation scheme. Finally, experimental results on two benchmarks, including Human3.6M and CMU-Mocap, show that our method achieves state-of-the-art performance, demonstrating the effectiveness of our proposed method. The code will be available if the paper is accepted.



There are no comments yet.


page 1

page 2

page 3

page 4


AGVNet: Attention Guided Velocity Learning for 3D Human Motion Prediction

Human motion prediction plays a vital role in human-robot interaction wi...

Temporal Pyramid Network for Pedestrian Trajectory Prediction with Multi-Supervision

Predicting human motion behavior in a crowd is important for many applic...

Multi-grained Trajectory Graph Convolutional Networks for Habit-unrelated Human Motion Prediction

Human motion prediction is an essential part for human-robot collaborati...

Motion Prediction via Joint Dependency Modeling in Phase Space

Motion prediction is a classic problem in computer vision, which aims at...

Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion

Human behavior has the nature of indeterminacy, which requires the pedes...

Potential Field: Interpretable and Unified Representation for Trajectory Prediction

Predicting an agent's future trajectory is a challenging task given the ...

Impact of Change Granularity in Refactoring Detection

Detecting refactorings in commit history is essential to improve the com...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.