Machine-Generated Hierarchical Structure of Human Activities to Reveal How Machines Think

by   Mahsun Altın, et al.

Deep-learning based computer vision models have proved themselves to be ground-breaking approaches to human activity recognition (HAR). However, most existing works are dedicated to improve the prediction accuracy through either creating new model architectures, increasing model complexity, or refining model parameters by training on larger datasets. Here, we propose an alternative idea, differing from existing work, to increase model accuracy and also to shape model predictions to align with human understandings through automatically creating higher-level summarizing labels for similar groups of human activities. First, we argue the importance and feasibility of constructing a hierarchical labeling system for human activity recognition. Then, we utilize the predictions of a black box HAR model to identify similarities between different activities. Finally, we tailor hierarchical clustering methods to automatically generate hierarchical trees of activities and conduct experiments. In this system, the activity labels on the same level will have a designed magnitude of accuracy and reflect a specific amount of activity details. This strategy enables a trade-off between the extent of the details in the recognized activity and the user privacy by masking some sensitive predictions; and also provides possibilities for the use of formerly prohibited invasive models in privacy-concerned scenarios. Since the hierarchy is generated from the machine's perspective, the predictions at the upper levels provide better accuracy, which is especially useful when there are too detailed labels in the training set that are rather trivial to the final prediction goal. Moreover, the analysis of the structure of these trees can reveal the biases in the prediction model and guide future data collection strategies.



There are no comments yet.


page 2

page 4

page 5

page 6

page 7

page 8

page 9

page 10


ReHAR: Robust and Efficient Human Activity Recognition

Designing a scheme that can achieve a good performance in predicting sin...

AudioAR: Audio-Based Activity Recognition with Large-Scale Acoustic Embeddings from YouTube Videos

Activity sensing and recognition have been demonstrated to be critical i...

Dual Residual Network for Accurate Human Activity Recognition

Human Activity Recognition (HAR) using deep neural network has become a ...

On Multi-resident Activity Recognition in Ambient Smart-Homes

Increasing attention to the research on activity monitoring in smart hom...

Learning to Abstract and Predict Human Actions

Human activities are naturally structured as hierarchies unrolled over t...

Description of Structural Biases and Associated Data in Sensor-Rich Environments

In this article, we study activity recognition in the context of sensor-...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.