Fine-grained Human Activity Recognition Using Virtual On-body Acceleration Data

11/02/2022
by   Zikang Leng, et al.
0

Previous work has demonstrated that virtual accelerometry data, extracted from videos using cross-modality transfer approaches like IMUTube, is beneficial for training complex and effective human activity recognition (HAR) models. Systems like IMUTube were originally designed to cover activities that are based on substantial body (part) movements. Yet, life is complex, and a range of activities of daily living is based on only rather subtle movements, which bears the question to what extent systems like IMUTube are of value also for fine-grained HAR, i.e., When does IMUTube break? In this work we first introduce a measure to quantitatively assess the subtlety of human movements that are underlying activities of interest–the motion subtlety index (MSI)–which captures local pixel movements and pose changes in the vicinity of target virtual sensor locations, and correlate it to the eventual activity recognition accuracy. We then perform a "stress-test" on IMUTube and explore for which activities with underlying subtle movements a cross-modality transfer approach works, and for which not. As such, the work presented in this paper allows us to map out the landscape for IMUTube applications in practical scenarios.

READ FULL TEXT

page 2

page 3

page 5

research
07/08/2020

An Efficient Data Imputation Technique for Human Activity Recognition

The tremendous applications of human activity recognition are surging it...
research
05/04/2023

Generating Virtual On-body Accelerometer Data from Virtual Textual Descriptions for Human Activity Recognition

The development of robust, generalized models in human activity recognit...
research
02/23/2015

Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data

Activity recognition has shown impressive progress in recent years. Howe...
research
05/11/2019

Follow the Attention: Combining Partial Pose and Object Motion for Fine-Grained Action Detection

Activity recognition in shopping environments is an important and challe...
research
12/28/2020

Computational Analysis of Subscapularis Tears and Pectoralis Major Transfers on Muscular Activity

Muscle transfers are commonly performed to restore joint function after ...
research
06/07/2014

Fine-grained Activity Recognition with Holistic and Pose based Features

Holistic methods based on dense trajectories are currently the de facto ...
research
08/09/2017

What Actions are Needed for Understanding Human Actions in Videos?

What is the right way to reason about human activities? What directions ...

Please sign up or login with your details

Forgot password? Click here to reset