Log In Sign Up

Mitigating Bias in Facial Analysis Systems by Incorporating Label Diversity

by   Camila Kolling, et al.

Facial analysis models are increasingly applied in real-world applications that have significant impact on peoples' lives. However, as previously shown, models that automatically classify facial attributes might exhibit algorithmic discrimination behavior with respect to protected groups, potentially posing negative impacts on individuals and society. It is therefore critical to develop techniques that can mitigate unintended biases in facial classifiers. Hence, in this work, we introduce a novel learning method that combines both subjective human-based labels and objective annotations based on mathematical definitions of facial traits. Specifically, we generate new objective annotations from a large-scale human-annotated dataset, each capturing a different perspective of the analyzed facial trait. We then propose an ensemble learning method, which combines individual models trained on different types of annotations. We provide an in-depth analysis of the annotation procedure as well as the dataset distribution. Moreover, we empirically demonstrate that, by incorporating label diversity, and without additional synthetic images, our method successfully mitigates unintended biases, while maintaining significant accuracy on the downstream task.


page 1

page 2

page 3

page 4


Understanding and Mitigating Annotation Bias in Facial Expression Recognition

The performance of a computer vision model depends on the size and quali...

Fair Attribute Classification through Latent Space De-biasing

Fairness in visual recognition is becoming a prominent and critical topi...

Fairness in Deep Learning: A Computational Perspective

Deep learning is increasingly being used in high-stake decision making a...

Mitigating Algorithmic Bias with Limited Annotations

Existing work on fairness modeling commonly assumes that sensitive attri...

Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition

As Facial Expression Recognition (FER) systems become integrated into ou...

The Price of Diversity

Systemic bias with respect to gender, race and ethnicity, often unconsci...