EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion in Task-Oriented Dialogue Systems

09/10/2021
by   Shutong Feng, et al.
0

The ability to recognise emotions lends a conversational artificial intelligence a human touch. While emotions in chit-chat dialogues have received substantial attention, emotions in task-oriented dialogues have been largely overlooked despite having an equally important role, such as to signal failure or success. Existing emotion-annotated task-oriented corpora are limited in size, label richness, and public availability, creating a bottleneck for downstream tasks. To lay a foundation for studies on emotions in task-oriented dialogues, we introduce EmoWOZ, a large-scale manually emotion-annotated corpus of task-oriented dialogues. EmoWOZ is based on MultiWOZ, a multi-domain task-oriented dialogue dataset. It contains more than 11K dialogues with more than 83K emotion annotations of user utterances. In addition to Wizzard-of-Oz dialogues from MultiWOZ, we collect human-machine dialogues within the same set of domains to sufficiently cover the space of various emotions that can happen during the lifetime of a data-driven dialogue system. To the best of our knowledge, this is the first large-scale open-source corpus of its kind. We propose a novel emotion labelling scheme, which is tailored to task-oriented dialogues. We report a set of experimental results to show the usability of this corpus for emotion recognition and state tracking in task-oriented dialogues.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/24/2022

Building a Dialogue Corpus Annotated with Expressed and Experienced Emotions

In communication, a human would recognize the emotion of an interlocutor...
research
08/24/2023

From Chatter to Matter: Addressing Critical Steps of Emotion Recognition Learning in Task-oriented Dialogue

Emotion recognition in conversations (ERC) is a crucial task for buildin...
research
05/21/2023

JNV Corpus: A Corpus of Japanese Nonverbal Vocalizations with Diverse Phrases and Emotions

We present JNV (Japanese Nonverbal Vocalizations) corpus, a corpus of Ja...
research
05/31/2022

APPReddit: a Corpus of Reddit Posts Annotated for Appraisal

Despite the large number of computational resources for emotion recognit...
research
05/15/2018

A Manually Annotated Chinese Corpus for Non-task-oriented Dialogue Systems

This paper presents a large-scale corpus for non-task-oriented dialogue ...
research
12/12/2021

ValueNet: A New Dataset for Human Value Driven Dialogue System

Building a socially intelligent agent involves many challenges, one of w...
research
06/02/2023

EmoUS: Simulating User Emotions in Task-Oriented Dialogues

Existing user simulators (USs) for task-oriented dialogue systems only m...

Please sign up or login with your details

Forgot password? Click here to reset