Model Patching: Closing the Subgroup Performance Gap with Data Augmentation

08/15/2020
by   Karan Goel, et al.
18

Classifiers in machine learning are often brittle when deployed. Particularly concerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting disparities in skin cancer classification in the presence or absence of a spurious bandage. To mitigate these performance differences, we introduce model patching, a two-stage framework for improving robustness that encourages the model to be invariant to subgroup differences, and focus on class information shared by subgroups. Model patching first models subgroup features within a class and learns semantic transformations between them, and then trains a classifier with data augmentations that deliberately manipulate subgroup features. We instantiate model patching with CAMEL, which (1) uses a CycleGAN to learn the intra-class, inter-subgroup augmentations, and (2) balances subgroup performance using a theoretically-motivated subgroup consistency regularizer, accompanied by a new robust objective. We demonstrate CAMEL's effectiveness on 3 benchmark datasets, with reductions in robust error of up to 33 a model that fails due to spurious features on a real-world skin cancer dataset.

READ FULL TEXT
10/29/2019

Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasets

Recent advances in computer vision and deep learning have led to breakth...
08/21/2020

Method to Classify Skin Lesions using Dermoscopic images

Skin cancer is the most common cancer in the existing world constituting...
12/12/2012

Enhanced skin colour classifier using RGB Ratio model

Skin colour detection is frequently been used for searching people, face...
08/03/2022

RealPatch: A Statistical Matching Framework for Model Patching with Real Samples

Machine learning classifiers are typically trained to minimise the avera...
11/20/2019

DermGAN: Synthetic Generation of Clinical Skin Images with Pathology

Despite the recent success in applying supervised deep learning to medic...
06/06/2019

Image Synthesis with a Single (Robust) Classifier

We show that the basic classification framework alone can be used to tac...

Code Repositories

model-patching

Model Patching: Closing the Subgroup Performance Gap with Data Augmentation


view repo