Log In Sign Up

Assessing Political Prudence of Open-domain Chatbots

by   Yejin Bang, et al.

Politically sensitive topics are still a challenge for open-domain chatbots. However, dealing with politically sensitive content in a responsible, non-partisan, and safe behavior way is integral for these chatbots. Currently, the main approach to handling political sensitivity is by simply changing such a topic when it is detected. This is safe but evasive and results in a chatbot that is less engaging. In this work, as a first step towards a politically safe chatbot, we propose a group of metrics for assessing their political prudence. We then conduct political prudence analysis of various chatbots and discuss their behavior from multiple angles through our automatic metric and human evaluation metrics. The testsets and codebase are released to promote research in this area.


page 1

page 2

page 3

page 4


When Politicians Talk About Politics: Identifying Political Tweets of Brazilian Congressmen

Since June 2013, when Brazil faced the largest and most significant mass...

Popping the bubble may not be enough: news media role in online political polarization

Politics in different countries show diverse degrees of polarization, wh...

Mitigating Political Bias in Language Models Through Reinforced Calibration

Current large-scale language models can be politically biased as a resul...

Characterizing Attention Cascades in WhatsApp Groups

An important political and social phenomena discussed in several countri...

Parameter Sensitivity of Deep-Feature based Evaluation Metrics for Audio Textures

Standard evaluation metrics such as the Inception score and Fréchet Audi...

Ontologies for Representing Relations among Political Agents

The Internet and the Web are now an integral part of the way most modern...

Untangling the Dueling Expert Witnesses: Comparing Ensemble Methods in Pennsylvania's Redistricting Plans

Ensembles of random legislative districts are a valuable tool for assess...