DeepAI AI Chat
Log In Sign Up

Critical Perspectives: A Benchmark Revealing Pitfalls in PerspectiveAPI

by   Lorena Piedras, et al.

Detecting "toxic" language in internet content is a pressing social and technical challenge. In this work, we focus on PERSPECTIVE from Jigsaw, a state-of-the-art tool that promises to score the "toxicity" of text, with a recent model update that claims impressive results (Lees et al., 2022). We seek to challenge certain normative claims about toxic language by proposing a new benchmark, Selected Adversarial SemanticS, or SASS. We evaluate PERSPECTIVE on SASS, and compare to low-effort alternatives, like zero-shot and few-shot GPT-3 prompt models, in binary classification settings. We find that PERSPECTIVE exhibits troubling shortcomings across a number of our toxicity categories. SASS provides a new tool for evaluating performance on previously undetected toxic language that avoids common normative pitfalls. Our work leads us to emphasize the importance of questioning assumptions made by tools already in deployment for toxicity detection in order to anticipate and prevent disparate harms.


page 1

page 2

page 3

page 4


Evaluating Prompts Across Multiple Choice Tasks In a Zero-Shot Setting

Large language models have shown that impressive zero-shot performance c...

Anchor-Based Adversarially Robust Zero-Shot Learning Driven by Language

Deep neural networks are vulnerable to adversarial attacks. We consider ...

A Systematic Investigation of Commonsense Understanding in Large Language Models

Large language models have shown impressive performance on many natural ...

Adversarial Learning for Zero-Shot Stance Detection on Social Media

Stance detection on social media can help to identify and understand sla...

Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

The tremendous success of CLIP (Radford et al., 2021) has promoted the r...

Zero-Shot Learning of Text Adventure Games with Sentence-Level Semantics

Reinforcement learning algorithms such as Q-learning have shown great pr...