Person Perception Biases Exposed: Revisiting the First Impressions Dataset

This work revisits the ChaLearn First Impressions database, annotated for personality perception using pairwise comparisons via crowdsourcing. We analyse for the first time the original pairwise annotations, and reveal existing person perception biases associated to perceived attributes like gender, ethnicity, age and face attractiveness. We show how person perception bias can influence data labelling of a subjective task, which has received little attention from the computer vision and machine learning communities by now. We further show that the mechanism used to convert pairwise annotations to continuous values may magnify the biases if no special treatment is considered. The findings of this study are relevant for the computer vision community that is still creating new datasets on subjective tasks, and using them for practical applications, ignoring these perceptual biases.

READ FULL TEXT

page 3

page 4

page 6

research
08/05/2016

Deep Learning the City : Quantifying Urban Perception At A Global Scale

Computer vision methods that quantify the perception of urban environmen...
research
05/03/2019

Auditing ImageNet: Towards a Model-driven Framework for Annotating Demographic Attributes of Large-Scale Image Datasets

The ImageNet dataset ushered in a flood of academic and industry interes...
research
10/28/2021

IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisons

Today, comprehensive evaluation of large-scale machine learning models i...
research
05/31/2023

Crowdsourcing subjective annotations using pairwise comparisons reduces bias and error compared to the majority-vote method

How to better reduce measurement variability and bias introduced by subj...
research
09/12/2019

On the Effect of Observed Subject Biases in Apparent Personality Analysis from Audio-visual Signals

Personality perception is implicitly biased due to many subjective facto...
research
09/28/2022

Racial Bias in the Beautyverse

This short paper proposes a preliminary and yet insightful investigation...
research
08/27/2020

Listener-Position and Orientation Dependency of Auditory Perception in an Enclosed Space: Elicitation of Salient Attributes

This paper presents a subjective study conducted on the perception of sa...

Please sign up or login with your details

Forgot password? Click here to reset