DeepAI AI Chat
Log In Sign Up

SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

04/18/2021
by   Satwik Kottur, et al.
0

We present a new corpus for the Situated and Interactive Multimodal Conversations, SIMMC 2.0, aimed at building a successful multimodal assistant agent. Specifically, the dataset features 11K task-oriented dialogs (117K utterances) between a user and a virtual assistant on the shopping domain (fashion and furniture), grounded in situated and photo-realistic VR scenes. The dialogs are collected using a two-phase pipeline, which first generates simulated dialog flows via a novel multimodal dialog simulator we propose, followed by manual paraphrasing of the generated utterances. In this paper, we provide an in-depth analysis of the collected dataset, and describe in detail the four main benchmark tasks we propose for SIMMC 2.0. The preliminary analysis with a baseline model highlights the new challenges that the SIMMC 2.0 dataset brings, suggesting new directions for future research. Our dataset and code will be made publicly available.

READ FULL TEXT

page 1

page 2

page 11

06/02/2020

Situated and Interactive Multimodal Conversations

Next generation virtual assistants are envisioned to handle multimodal i...
11/15/2022

Navigating Connected Memories with a Task-oriented Dialog System

Recent years have seen an increasing trend in the volume of personal med...
10/29/2017

Finding Dominant User Utterances And System Responses in Conversations

There are several dialog frameworks which allow manual specification of ...
04/01/2017

Multimodal Dialogs (MMD): A large-scale dataset for studying multimodal domain-aware conversations

While multimodal conversation agents are gaining importance in several d...
11/08/2022

Tell Your Story: Task-Oriented Dialogs for Interactive Content Creation

People capture photos and videos to relive and share memories of persona...
09/15/2021

End-to-End Learning of Flowchart Grounded Task-Oriented Dialogs

We propose a novel problem within end-to-end learning of task-oriented d...
10/22/2020

Migratable AI: Personalizing Dialog Conversations with migration context

The migration of conversational AI agents across different embodiments i...