Conversational Intent Understanding for Passengers in Autonomous Vehicles

12/14/2018
by   Eda Okur, et al.
0

Understanding passenger intents and extracting relevant slots are important building blocks towards developing a contextual dialogue system responsible for handling certain vehicle-passenger interactions in autonomous vehicles (AV). When the passengers give instructions to AMIE (Automated-vehicle Multimodal In-cabin Experience), the agent should parse such commands properly and trigger the appropriate functionality of the AV system. In our AMIE scenarios, we describe usages and support various natural commands for interacting with the vehicle. We collected a multimodal in-cabin data-set with multi-turn dialogues between the passengers and AMIE using a Wizard-of-Oz scheme. We explored various recent Recurrent Neural Networks (RNN) based techniques and built our own hierarchical models to recognize passenger intents along with relevant slots associated with the action to be performed in AV scenarios. Our experimental results achieved F1-score of 0.91 on utterance-level intent recognition and 0.96 on slot extraction models.

READ FULL TEXT

page 1

page 2

page 3

research
04/23/2019

Natural Language Interactions in Autonomous Vehicles: Intent Detection and Slot Filling from Passenger Utterances

Understanding passenger intents and extracting relevant slots are import...
research
09/20/2019

Towards Multimodal Understanding of Passenger-Vehicle Interactions in Autonomous Vehicles: Intent/Slot Recognition Utilizing Audio-Visual Data

Understanding passenger intents from spoken interactions and car's visio...
research
07/08/2020

Audio-Visual Understanding of Passenger Intents for In-Cabin Conversational Agents

Building multimodal dialogue understanding capabilities situated in the ...
research
03/06/2018

MIRIAM: A Multimodal Chat-Based Interface for Autonomous Systems

We present MIRIAM (Multimodal Intelligent inteRactIon for Autonomous sys...
research
07/26/2018

Naturalistic Driver Intention and Path Prediction using Recurrent Neural Networks

Understanding the intentions of drivers at intersections is a critical c...
research
09/28/2022

RADACS: Towards Higher-Order Reasoning using Action Recognition in Autonomous Vehicles

When applied to autonomous vehicle settings, action recognition can help...
research
09/18/2020

Commands 4 Autonomous Vehicles (C4AV) Workshop Summary

The task of visual grounding requires locating the most relevant region ...

Please sign up or login with your details

Forgot password? Click here to reset