Your fairness may vary: Group fairness of pretrained language models in toxic text classification

08/03/2021

∙

We study the performance-fairness trade-off in more than a dozen fine-tuned LMs for toxic text classification. We empirically show that no blanket statement can be made with respect to the bias of large versus regular versus compressed models. Moreover, we find that focusing on fairness-agnostic performance metrics can lead to models with varied fairness characteristics.

READ FULL TEXT

Your fairness may vary: Group fairness of pretrained language models in toxic text classification

On Bias and Fairness in NLP: How to have a fairer text classification?

Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

Model and Evaluation: Towards Fairness in Multilingual Text Classification

Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification

Building for Tomorrow: Assessing the Temporal Persistence of Text Classifiers

Your fairness may vary: Group fairness of pretrained language models in toxic text classification

Related Research

On Bias and Fairness in NLP: How to have a fairer text classification?

Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

Model and Evaluation: Towards Fairness in Multilingual Text Classification

Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification

Building for Tomorrow: Assessing the Temporal Persistence of Text Classifiers