Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving

06/07/2018
by   Mahdyar Ravanbakhsh, et al.
6

This paper presents a novel approach for learning self-awareness models for autonomous vehicles. The proposed technique is based on the availability of synchronized multi-sensor dynamic data related to different maneuvering tasks performed by a human operator. It is shown that different machine learning approaches can be used to first learn single modality models using coupled Dynamic Bayesian Networks; such models are then correlated at event level to discover contextual multi-modal concepts. In the presented case, visual perception and localization are used as modalities. Cross-correlations among modalities in time is discovered from data and are described as probabilistic links connecting shared and private multi-modal DBNs at the event (discrete) level. Results are presented on experiments performed on an autonomous vehicle, highlighting potentiality of the proposed approach to allow anomaly detection and autonomous decision making based on learned self-awareness models.

READ FULL TEXT

page 5

page 6

page 7

page 8

research
10/28/2020

Collective Awareness for Abnormality Detection in Connected Autonomous Vehicles

The advancements in connected and autonomous vehicles in these times dem...
research
08/21/2020

Towards Autonomous Driving: a Multi-Modal 360^∘ Perception Proposal

In this paper, a multi-modal 360^∘ framework for 3D object detection and...
research
06/08/2018

Hierarchy of GANs for learning embodied self-awareness model

In recent years several architectures have been proposed to learn embodi...
research
10/16/2012

Factorized Multi-Modal Topic Model

Multi-modal data collections, such as corpora of paired images and text ...
research
09/30/2019

On Assessing Driver Awareness of Situational Criticalities: Multi-modal Bio-sensing and Vision-based Analysis, Evaluations and Insights

Automobiles for our roadways are increasingly utilizing advanced driver ...
research
02/08/2021

The Autonomous Siemens Tram

This paper presents the Autonomous Siemens Tram that was publicly demons...
research
04/23/2023

Learning-enabled multi-modal motion prediction in urban environments

Motion prediction is a key factor towards the full deployment of autonom...

Please sign up or login with your details

Forgot password? Click here to reset