MARS: Mixed Virtual and Real Wearable Sensors for Human Activity Recognition with Multi-Domain Deep Learning Model

by   Ling Pei, et al.

Human activity recognition (HAR) using wearable Inertial Measurement Unit (IMU) sensors is a promising technology for many research areas. Recently, deep learning-based methods pave a new way of understanding and performing analysis of the complex data in the HAR system. However, the performance of these methods is mostly based on the quality and quantity of the collected data. In this paper, we innovatively propose to build a large database based on virtual IMUs and then address technical issues by introducing a multiple-domain deep learning framework consisting of three technical parts. In the first part, we propose to learn the single-frame human activity from the noisy IMU data with hybrid convolutional neural networks (CNNs) in the semi-supervised form. For the second part, the extracted data features are fused according to the principle of uncertainty-aware consistency, which reduces the uncertainty by weighting the importance of the features. The transfer learning is performed in the last part based on the newly released Archive of Motion Capture as Surface Shapes (AMASS) dataset, containing abundant synthetic human poses, which enhances the variety and diversity of the training dataset and is beneficial for the process of training and feature transfer in the proposed method. The efficiency and effectiveness of the proposed method have been demonstrated in the real deep inertial poser (DIP) dataset. The experimental results show that the proposed methods can surprisingly converge within a few iterations and outperform all competing methods.


page 1

page 5

page 9

page 11


A Deep Learning Method for Complex Human Activity Recognition Using Virtual Wearable Sensors

Sensor-based human activity recognition (HAR) is now a research hotspot ...

TASKED: Transformer-based Adversarial learning for human activity recognition using wearable sensors via Self-KnowledgE Distillation

Wearable sensor-based human activity recognition (HAR) has emerged as a ...

Human Activity Recognition with Convolutional Neural Netowrks

The problem of automatic identification of physical activities performed...

An Interpretable Machine Vision Approach to Human Activity Recognition using Photoplethysmograph Sensor Data

The current gold standard for human activity recognition (HAR) is based ...

CROMOSim: A Deep Learning-based Cross-modality Inertial Measurement Simulator

With the prevalence of wearable devices, inertial measurement unit (IMU)...

Comparison Study of Inertial Sensor Signal Combination for Human Activity Recognition based on Convolutional Neural Networks

Human Activity Recognition (HAR) is one of the essential building blocks...

Optimised Convolutional Neural Networks for Heart Rate Estimation and Human Activity Recognition in Wrist Worn Sensing Applications

Wrist-worn smart devices are providing increased insights into human hea...