Competency Problems: On Finding and Removing Artifacts in Language Data

04/17/2021
by   Matt Gardner, et al.
0

Much recent work in NLP has documented dataset artifacts, bias, and spurious correlations between input features and output labels. However, how to tell which features have "spurious" instead of legitimate correlations is typically left unspecified. In this work we argue that for complex language understanding tasks, all simple feature correlations are spurious, and we formalize this notion into a class of problems which we call competency problems. For example, the word "amazing" on its own should not give information about a sentiment label independent of the context in which it appears, which could include negation, metaphor, sarcasm, etc. We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account, showing that realistic datasets will increasingly deviate from competency problems as dataset size increases. This analysis gives us a simple statistical test for dataset artifacts, which we use to show more subtle biases than were described in prior work, including demonstrating that models are inappropriately affected by these less extreme biases. Our theoretical treatment of this problem also allows us to analyze proposed solutions, such as making local edits to dataset instances, and to give recommendations for future data collection and model design efforts that target competency problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/03/2023

Stubborn Lexical Bias in Data and Models

In NLP, recent work has seen increased focus on spurious correlations be...
research
05/30/2023

Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biases

NLP models often rely on superficial cues known as dataset biases to ach...
research
04/09/2022

Uninformative Input Features and Counterfactual Invariance: Two Perspectives on Spurious Correlations in Natural Language

Spurious correlations are a threat to the trustworthiness of natural lan...
research
04/27/2022

On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations

Recent work has shown that deep learning models in NLP are highly sensit...
research
05/31/2023

How to Construct Perfect and Worse-than-Coin-Flip Spoofing Countermeasures: A Word of Warning on Shortcut Learning

Shortcut learning, or `Clever Hans effect` refers to situations where a ...
research
02/15/2021

Expected Exploitability: Predicting the Development of Functional Vulnerability Exploits

Assessing the exploitability of software vulnerabilities at the time of ...
research
07/01/2021

Combining Feature and Instance Attribution to Detect Artifacts

Training the large deep neural networks that dominate NLP requires large...

Please sign up or login with your details

Forgot password? Click here to reset