Residual Relaxation for Multi-view Representation Learning

10/28/2021
by   Yifei Wang, et al.
4

Multi-view methods learn representations by aligning multiple views of the same image and their performance largely depends on the choice of data augmentation. In this paper, we notice that some other useful augmentations, such as image rotation, are harmful for multi-view methods because they cause a semantic shift that is too large to be aligned well. This observation motivates us to relax the exact alignment objective to better cultivate stronger augmentations. Taking image rotation as a case study, we develop a generic approach, Pretext-aware Residual Relaxation (Prelax), that relaxes the exact alignment by allowing an adaptive residual vector between different views and encoding the semantic shift through pretext-aware learning. Extensive experiments on different backbones show that our method can not only improve multi-view methods with existing augmentations, but also benefit from stronger image augmentations like rotation.

READ FULL TEXT

page 10

page 16

research
10/19/2022

Two-level Data Augmentation for Calibrated Multi-view Detection

Data augmentation has proven its usefulness to improve model generalizat...
research
01/15/2022

Uncertainty-Aware Multi-View Representation Learning

Learning from different data views by exploring the underlying complemen...
research
05/24/2020

Multi-view Alignment and Generation in CCA via Consistent Latent Encoding

Multi-view alignment, achieving one-to-one correspondence of multi-view ...
research
06/26/2023

Multi-View Attention Learning for Residual Disease Prediction of Ovarian Cancer

In the treatment of ovarian cancer, precise residual disease prediction ...
research
05/04/2023

Semantic-aware Generation of Multi-view Portrait Drawings

Neural radiance fields (NeRF) based methods have shown amazing performan...
research
12/12/2019

Towards Disentangled Representations for Human Retargeting by Multi-view Learning

We study the problem of learning disentangled representations for data a...
research
05/18/2018

Multi-view Sentence Representation Learning

Multi-view learning can provide self-supervision when different views ar...

Please sign up or login with your details

Forgot password? Click here to reset