DeepAI
Log In Sign Up

Assessing Political Prudence of Open-domain Chatbots

06/11/2021
by   Yejin Bang, et al.
2

Politically sensitive topics are still a challenge for open-domain chatbots. However, dealing with politically sensitive content in a responsible, non-partisan, and safe behavior way is integral for these chatbots. Currently, the main approach to handling political sensitivity is by simply changing such a topic when it is detected. This is safe but evasive and results in a chatbot that is less engaging. In this work, as a first step towards a politically safe chatbot, we propose a group of metrics for assessing their political prudence. We then conduct political prudence analysis of various chatbots and discuss their behavior from multiple angles through our automatic metric and human evaluation metrics. The testsets and codebase are released to promote research in this area.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/04/2018

When Politicians Talk About Politics: Identifying Political Tweets of Brazilian Congressmen

Since June 2013, when Brazil faced the largest and most significant mass...
09/18/2021

Popping the bubble may not be enough: news media role in online political polarization

Politics in different countries show diverse degrees of polarization, wh...
04/30/2021

Mitigating Political Bias in Language Models Through Reinforced Calibration

Current large-scale language models can be politically biased as a resul...
05/02/2019

Characterizing Attention Cascades in WhatsApp Groups

An important political and social phenomena discussed in several countri...
08/23/2022

Parameter Sensitivity of Deep-Feature based Evaluation Metrics for Audio Textures

Standard evaluation metrics such as the Inception score and Fréchet Audi...
04/17/2018

Ontologies for Representing Relations among Political Agents

The Internet and the Web are now an integral part of the way most modern...
08/18/2022

Untangling the Dueling Expert Witnesses: Comparing Ensemble Methods in Pennsylvania's Redistricting Plans

Ensembles of random legislative districts are a valuable tool for assess...