TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

03/02/2022
by   Kunyu Peng, et al.
39

Traditional video-based human activity recognition has experienced remarkable progress linked to the rise of deep learning, but this effect was slower as it comes to the downstream task of driver behavior understanding. Understanding the situation inside the vehicle cabin is essential for Advanced Driving Assistant System (ADAS) as it enables identifying distraction, predicting driver's intent and leads to more convenient human-vehicle interaction. At the same time, driver observation systems face substantial obstacles as they need to capture different granularities of driver states, while the complexity of such secondary activities grows with the rising automation and increased driver freedom. Furthermore, a model is rarely deployed under conditions identical to the ones in the training set, as sensor placements and types vary from vehicle to vehicle, constituting a substantial obstacle for real-life deployment of data-driven models. In this work, we present a novel vision-based framework for recognizing secondary driver behaviours based on visual transformers and an additional augmented feature distribution calibration module. This module operates in the latent feature-space enriching and diversifying the training set at feature-level in order to improve generalization to novel data appearances, (e.g., sensor changes) and general feature quality. Our framework consistently leads to better recognition rates, surpassing previous state-of-the-art results of the public Drive Act benchmark on all granularity levels. Our code will be made publicly available at https://github.com/KPeng9510/TransDARC.

READ FULL TEXT

page 1

page 3

page 5

research
04/10/2022

Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates

Driver observation models are rarely deployed under perfect conditions. ...
research
03/02/2023

MuscleMap: Towards Video-based Activated Muscle Group Estimation

In this paper, we tackle the new task of video-based Activated Muscle Gr...
research
08/03/2022

Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living

Domain shifts, such as appearance changes, are a key challenge in real-w...
research
04/10/2022

A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding

Visual recognition inside the vehicle cabin leads to safer driving and m...
research
06/16/2023

Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos

Recognizing the activities, causing distraction, in real-world driving s...
research
01/24/2018

When Vehicles See Pedestrians with Phones:A Multi-Cue Framework for Recognizing Phone-based Activities of Pedestrians

The intelligent vehicle community has devoted considerable efforts to mo...
research
06/26/2023

A Preference-aware Meta-optimization Framework for Personalized Vehicle Energy Consumption Estimation

Vehicle Energy Consumption (VEC) estimation aims to predict the total en...

Please sign up or login with your details

Forgot password? Click here to reset