Learning Robust Variational Information Bottleneck with Reference

04/29/2021
by   Weizhu Qian, et al.
0

We propose a new approach to train a variational information bottleneck (VIB) that improves its robustness to adversarial perturbations. Unlike the traditional methods where the hard labels are usually used for the classification task, we refine the categorical class information in the training phase with soft labels which are obtained from a pre-trained reference neural network and can reflect the likelihood of the original class labels. We also relax the Gaussian posterior assumption in the VIB implementation by using the mutual information neural estimation. Extensive experiments have been performed with the MNIST and CIFAR-10 datasets, and the results show that our proposed approach significantly outperforms the benchmarked models.

READ FULL TEXT
research
05/17/2021

Disentangled Variational Information Bottleneck for Multiview Representation Learning

Multiview data contain information from multiple modalities and have pot...
research
10/11/2022

Towards Consistency and Complementarity: A Multiview Graph Information Bottleneck Approach

The empirical studies of Graph Neural Networks (GNNs) broadly take the o...
research
08/29/2023

Advancing Adversarial Robustness Through Adversarial Logit Update

Deep Neural Networks are susceptible to adversarial perturbations. Adver...
research
12/24/2022

Visualizing Information Bottleneck through Variational Inference

The Information Bottleneck theory provides a theoretical and computation...
research
12/18/2021

Improving Subgraph Recognition with Variational Graph Information Bottleneck

Subgraph recognition aims at discovering a compressed substructure of a ...
research
07/02/2022

Eliciting and Learning with Soft Labels from Every Annotator

The labels used to train machine learning (ML) models are of paramount i...
research
03/04/2022

Sparsity-Inducing Categorical Prior Improves Robustness of the Information Bottleneck

The information bottleneck framework provides a systematic approach to l...

Please sign up or login with your details

Forgot password? Click here to reset