DeepAI
Log In Sign Up

Transitioning from Real to Synthetic data: Quantifying the bias in model

05/10/2021
by   Aman Gupta, et al.
13

With the advent of generative modeling techniques, synthetic data and its use has penetrated across various domains from unstructured data such as image, text to structured dataset modeling healthcare outcome, risk decisioning in financial domain, and many more. It overcomes various challenges such as limited training data, class imbalance, restricted access to dataset owing to privacy issues. To ensure the trained model used for automated decisioning purposes makes a fair decision there exist prior work to quantify and mitigate those issues. This study aims to establish a trade-off between bias and fairness in the models trained using synthetic data. Variants of synthetic data generation techniques were studied to understand bias amplification including differentially private generation schemes. Through experiments on a tabular dataset, we demonstrate there exist a varying levels of bias impact on models trained using synthetic data. Techniques generating less correlated feature performs well as evident through fairness metrics with 94%, 82%, and 88% relative drop in DPD (demographic parity difference), EoD (equality of odds) and EoP (equality of opportunity) respectively, and 24% relative improvement in DRP (demographic parity ratio) with respect to the real dataset. We believe the outcome of our research study will help data science practitioners understand the bias in the use of synthetic data.

READ FULL TEXT

page 4

page 7

06/15/2021

An Analysis of the Deployment of Models Trained on Private Tabular Synthetic Data: Unexpected Surprises

Diferentially private (DP) synthetic datasets are a powerful approach fo...
10/24/2022

FairGen: Fair Synthetic Data Generation

With the rising adoption of Machine Learning across the domains like ban...
08/04/2022

Customs Import Declaration Datasets

Given the huge volume of cross-border flows, effective and efficient con...
08/24/2021

Bias Mitigated Learning from Differentially Private Synthetic Data: A Cautionary Tale

Increasing interest in privacy-preserving machine learning has led to ne...
05/24/2018

Fairness GAN

In this paper, we introduce the Fairness GAN, an approach for generating...
04/07/2021

Representative Fair Synthetic Data

Algorithms learn rules and associations based on the training data that ...
06/30/2018

Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction

Recidivism prediction scores are used across the USA to determine senten...