Uncovering the Source of Machine Bias

by   Xiyang Hu, et al.

We develop a structural econometric model to capture the decision dynamics of human evaluators on an online micro-lending platform, and estimate the model parameters using a real-world dataset. We find two types of biases in gender, preference-based bias and belief-based bias, are present in human evaluators' decisions. Both types of biases are in favor of female applicants. Through counterfactual simulations, we quantify the effect of gender bias on loan granting outcomes and the welfare of the company and the borrowers. Our results imply that both the existence of the preference-based bias and that of the belief-based bias reduce the company's profits. When the preference-based bias is removed, the company earns more profits. When the belief-based bias is removed, the company's profits also increase. Both increases result from raising the approval probability for borrowers, especially male borrowers, who eventually pay back loans. For borrowers, the elimination of either bias decreases the gender gap of the true positive rates in the credit risk evaluation. We also train machine learning algorithms on both the real-world data and the data from the counterfactual simulations. We compare the decisions made by those algorithms to see how evaluators' biases are inherited by the algorithms and reflected in machine-based decisions. We find that machine learning algorithms can mitigate both the preference-based bias and the belief-based bias.


page 5

page 9


On the Basis of Sex: A Review of Gender Bias in Machine Learning Applications

Machine Learning models have been deployed across almost every aspect of...

Raw Audio for Depression Detection Can Be More Robust Against Gender Imbalance than Mel-Spectrogram Features

Depression is a large-scale mental health problem and a challenging area...

Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting

We present a large-scale study of gender bias in occupation classificati...

Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation

Models often easily learn biases present in the training data, and their...

Pay No Attention to the Model Behind the Curtain

Many widely used models amount to an elaborate means of making up number...

What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring

Although systematic biases in decision-making are widely documented, the...

Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution

Spurious correlations were found to be an important factor explaining mo...

Please sign up or login with your details

Forgot password? Click here to reset