Your fairness may vary: Group fairness of pretrained language models in toxic text classification

08/03/2021
by   Ioana Baldini, et al.
ibm
0

We study the performance-fairness trade-off in more than a dozen fine-tuned LMs for toxic text classification. We empirically show that no blanket statement can be made with respect to the bias of large versus regular versus compressed models. Moreover, we find that focusing on fairness-agnostic performance metrics can lead to models with varied fairness characteristics.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

01/18/2018

Fine-tuned Language Models for Text Classification

Transfer learning has revolutionized computer vision, but existing appro...
06/21/2021

Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification

Existing bias mitigation methods to reduce disparities in model outcomes...
12/14/2021

Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Models

An increasing awareness of biased patterns in natural language processin...
03/11/2019

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Unintended bias in Machine Learning can manifest as systemic differences...
04/18/2021

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

When primed with only a handful of training samples, very large pretrain...
11/04/2020

Debiasing classifiers: is reality at variance with expectation?

Many methods for debiasing classifiers have been proposed, but their eff...
11/17/2021

Fairness Testing of Deep Image Classification with Adequacy Metrics

As deep image classification applications, e.g., face recognition, becom...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

References