RSA: Randomized Simulation as Augmentation for Robust Human Action Recognition

12/03/2019
by   Yi Zhang, et al.
8

Despite the rapid growth in datasets for video activity, stable robust activity recognition with neural networks remains challenging. This is in large part due to the explosion of possible variation in video – including lighting changes, object variation, movement variation, and changes in surrounding context. An alternative is to make use of simulation data, where all of these factors can be artificially controlled. In this paper, we propose the Randomized Simulation as Augmentation (RSA) framework which augments real-world training data with synthetic data to improve the robustness of action recognition networks. We generate large-scale synthetic datasets with randomized nuisance factors. We show that training with such extra data, when appropriately constrained, can significantly improve the performance of the state-of-the-art I3D networks or, conversely, reduce the number of labeled real videos needed to achieve good performance. Experiments on two real-world datasets NTU RGB+D and VIRAT demonstrate the effectiveness of our method.

READ FULL TEXT

page 1

page 3

page 4

page 6

page 7

research
12/09/2019

Synthetic Humans for Action Recognition from Unseen Viewpoints

Our goal in this work is to improve the performance of human action reco...
research
12/11/2015

Improving Human Activity Recognition Through Ranking and Re-ranking

We propose two well-motivated ranking-based methods to enhance the perfo...
research
10/14/2021

Nuisance-Label Supervision: Robustness Improvement by Free Labels

In this paper, we present a Nuisance-label Supervision (NLS) module, whi...
research
10/28/2020

ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

To train deep learning models for vision-based action recognition of eld...
research
12/08/2022

Phone2Proc: Bringing Robust Robots Into Our Chaotic World

Training embodied agents in simulation has become mainstream for the emb...
research
03/11/2019

A Hybrid Framework for Action Recognition in Low-Quality Video Sequences

Vision-based activity recognition is essential for security, monitoring ...
research
03/17/2023

Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances

Human action recognition is a challenging problem, particularly when the...

Please sign up or login with your details

Forgot password? Click here to reset