ExCon: Explanation-driven Supervised Contrastive Learning for Image Classification

11/28/2021
by   Zhibo Zhang, et al.
0

Contrastive learning has led to substantial improvements in the quality of learned embedding representations for tasks such as image classification. However, a key drawback of existing contrastive augmentation methods is that they may lead to the modification of the image content which can yield undesired alterations of its semantics. This can affect the performance of the model on downstream tasks. Hence, in this paper, we ask whether we can augment image data in contrastive learning such that the task-relevant semantic content of an image is preserved. For this purpose, we propose to leverage saliency-based explanation methods to create content-preserving masked augmentations for contrastive learning. Our novel explanation-driven supervised contrastive learning (ExCon) methodology critically serves the dual goals of encouraging nearby image embeddings to have similar content and explanation. To quantify the impact of ExCon, we conduct experiments on the CIFAR-100 and the Tiny ImageNet datasets. We demonstrate that ExCon outperforms vanilla supervised contrastive learning in terms of classification, explanation quality, adversarial robustness as well as calibration of probabilistic predictions of the model in the context of distributional shift.

READ FULL TEXT
research
05/23/2022

Conditional Supervised Contrastive Learning for Fair Text Classification

Contrastive representation learning has gained much attention due to its...
research
11/01/2021

When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?

Contrastive learning (CL) can learn generalizable feature representation...
research
06/09/2021

CLCC: Contrastive Learning for Color Constancy

In this paper, we present CLCC, a novel contrastive learning framework f...
research
06/06/2023

Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage

We investigate the role of projection heads, also known as projectors, w...
research
03/05/2023

CoRTX: Contrastive Framework for Real-time Explanation

Recent advancements in explainable machine learning provide effective an...
research
06/10/2023

Two-Stage Holistic and Contrastive Explanation of Image Classification

The need to explain the output of a deep neural network classifier is no...
research
11/25/2020

ImCLR: Implicit Contrastive Learning for Image Classification

Contrastive learning is an effective method for learning visual represen...

Please sign up or login with your details

Forgot password? Click here to reset