Task-Agnostic Robust Representation Learning

by   A. Tuan Nguyen, et al.

It has been reported that deep learning models are extremely vulnerable to small but intentionally chosen perturbations of its input. In particular, a deep network, despite its near-optimal accuracy on the clean images, often mis-classifies an image with a worst-case but humanly imperceptible perturbation (so-called adversarial examples). To tackle this problem, a great amount of research has been done to study the training procedure of a network to improve its robustness. However, most of the research so far has focused on the case of supervised learning. With the increasing popularity of self-supervised learning methods, it is also important to study and improve the robustness of their resulting representation on the downstream tasks. In this paper, we study the problem of robust representation learning with unlabeled data in a task-agnostic manner. Specifically, we first derive an upper bound on the adversarial loss of a prediction model (which is based on the learned representation) on any downstream task, using its loss on the clean data and a robustness regularizer. Moreover, the regularizer is task-independent, thus we propose to minimize it directly during the representation learning phase to make the downstream prediction model more robust. Extensive experiments show that our method achieves preferable adversarial performance compared to relevant baselines.


page 1

page 2

page 3

page 4


Robustness of Unsupervised Representation Learning without Labels

Unsupervised representation learning leverages large unlabeled datasets ...

Online Adversarial Purification based on Self-Supervision

Deep neural networks are known to be vulnerable to adversarial examples,...

Adversarial Attack on Network Embeddings via Supervised Network Poisoning

Learning low-level node embeddings using techniques from network represe...

Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations

Perceptual understanding of the scene and the relationship between its d...

Learning task-agnostic representation via toddler-inspired learning

One of the inherent limitations of current AI systems, stemming from the...

DyG2Vec: Representation Learning for Dynamic Graphs with Self-Supervision

The challenge in learning from dynamic graphs for predictive tasks lies ...

REaaS: Enabling Adversarially Robust Downstream Classifiers via Robust Encoder as a Service

Encoder as a service is an emerging cloud service. Specifically, a servi...

Please sign up or login with your details

Forgot password? Click here to reset