RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition

by   Yi Zhang, et al.

Despite the rapid growth in datasets for video activity, stable robust activity recognition with neural networks remains challenging. This is in large part due to the explosion of possible variation in video – including lighting changes, object variation, movement variation, and changes in surrounding context. An alternative is to make use of simulation data, where all of these factors can be artificially controlled. In this paper, we propose the Randomized Simulation as Augmentation (RSA) framework which augments real-world training data with synthetic data to improve the robustness of action recognition networks. We generate large-scale synthetic datasets with randomized nuisance factors. We show that training with such extra data, when appropriately constrained, can significantly improve the performance of the state-of-the-art I3D networks or, conversely, reduce the number of labeled real videos needed to achieve good performance. Experiments on two real-world datasets NTU RGB+D and VIRAT demonstrate the effectiveness of our method.


page 1

page 3

page 4

page 6

page 7


Synthetic Humans for Action Recognition from Unseen Viewpoints

Our goal in this work is to improve the performance of human action reco...

Improving Human Activity Recognition Through Ranking and Re-ranking

We propose two well-motivated ranking-based methods to enhance the perfo...

Nuisance-Label Supervision: Robustness Improvement by Free Labels

In this paper, we present a Nuisance-label Supervision (NLS) module, whi...

ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

To train deep learning models for vision-based action recognition of eld...

Phone2Proc: Bringing Robust Robots Into Our Chaotic World

Training embodied agents in simulation has become mainstream for the emb...

A Hybrid Framework for Action Recognition in Low-Quality Video Sequences

Vision-based activity recognition is essential for security, monitoring ...

Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances

Human action recognition is a challenging problem, particularly when the...

Please sign up or login with your details

Forgot password? Click here to reset