Dialog Acts for Task-Driven Embodied Agents

09/26/2022
by   Spandana Gella, et al.
0

Embodied agents need to be able to interact in natural language understanding task descriptions and asking appropriate follow up questions to obtain necessary information to be effective at successfully accomplishing tasks for a wide range of users. In this work, we propose a set of dialog acts for modelling such dialogs and annotate the TEACh dataset that includes over 3,000 situated, task oriented conversations (consisting of 39.5k utterances in total) with dialog acts. TEACh-DA is one of the first large scale dataset of dialog act annotations for embodied task completion. Furthermore, we demonstrate the use of this annotated dataset in training models for tagging the dialog acts of a given utterance, predicting the dialog act of the next response given a dialog history, and use the dialog acts to guide agent's non-dialog behaviour. In particular, our experiments on the TEACh Execution from Dialog History task where the model predicts the sequence of low level actions to be executed in the environment for embodied task completion, demonstrate that dialog acts can improve end task success rate by up to 2 points compared to the system without dialog acts.

READ FULL TEXT

page 3

page 5

page 8

research
06/26/2020

Dialog as a Vehicle for Lifelong Learning

Dialog systems research has primarily been focused around two main types...
research
10/30/2018

Prosodic entrainment in dialog acts

We examined prosodic entrainment in spoken dialogs separately for severa...
research
05/26/2023

Multimodal Recommendation Dialog with Subjective Preference: A New Challenge and Benchmark

Existing multimodal task-oriented dialog data fails to demonstrate the d...
research
07/05/2021

What Helps Transformers Recognize Conversational Structure? Importance of Context, Punctuation, and Labels in Dialog Act Recognition

Dialog acts can be interpreted as the atomic units of a conversation, mo...
research
02/02/2022

Knowledge Engineering in the Long Game of Artificial Intelligence: The Case of Speech Acts

This paper describes principles and practices of knowledge engineering t...
research
05/07/2016

Adobe-MIT submission to the DSTC 4 Spoken Language Understanding pilot task

The Dialog State Tracking Challenge 4 (DSTC 4) proposes several pilot ta...

Please sign up or login with your details

Forgot password? Click here to reset