The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems

04/06/2022
by   Caleb Ziems, et al.
3

Conversational agents have come increasingly closer to human competence in open-domain dialogue settings; however, such models can reflect insensitive, hurtful, or entirely incoherent viewpoints that erode a user's trust in the moral integrity of the system. Moral deviations are difficult to mitigate because moral judgments are not universal, and there may be multiple competing judgments that apply to a situation simultaneously. In this work, we introduce a new resource, not to authoritatively resolve moral ambiguities, but instead to facilitate systematic understanding of the intuitions, values and moral judgments reflected in the utterances of dialogue systems. The Moral Integrity Corpus, MIC, is such a resource, which captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs). Each RoT reflects a particular moral conviction that can explain why a chatbot's reply may appear acceptable or problematic. We further organize RoTs with a set of 9 moral and social attributes and benchmark performance for attribute classification. Most importantly, we show that current neural language models can automatically generate new RoTs that reasonably describe previously unseen interactions, but they still struggle with certain scenarios. Our findings suggest that MIC will be a useful resource for understanding and language models' implicit moral assumptions and flexibly benchmarking the integrity of conversational agents. To download the data, see https://github.com/GT-SALT/mic

READ FULL TEXT

page 1

page 5

page 6

research
07/19/2023

DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI

Despite advancements in conversational AI, language models encounter cha...
research
04/30/2022

Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models

Recent open-domain dialogue models have brought numerous breakthroughs. ...
research
05/24/2023

Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark

Large language models (LLMs) have been shown to perform well at a variet...
research
10/09/2022

Controllable Dialogue Simulation with In-Context Learning

Building dialogue systems requires a large corpus of annotated dialogues...
research
08/13/2019

Getting To Know You: User Attribute Extraction from Dialogues

User attributes provide rich and useful information for user understandi...
research
06/15/2023

DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoning

Pragmatic reasoning plays a pivotal role in deciphering implicit meaning...

Please sign up or login with your details

Forgot password? Click here to reset