Multi-view Alignment and Generation in CCA via Consistent Latent Encoding

05/24/2020
by   Yaxin Shi, et al.
10

Multi-view alignment, achieving one-to-one correspondence of multi-view inputs, is critical in many real-world multi-view applications, especially for cross-view data analysis problems. Recently, an increasing number of works study this alignment problem with Canonical Correlation Analysis (CCA). However, existing CCA models are prone to misalign the multiple views due to either the neglect of uncertainty or the inconsistent encoding of the multiple views. To tackle these two issues, this paper studies multi-view alignment from the Bayesian perspective. Delving into the impairments of inconsistent encodings, we propose to recover correspondence of the multi-view inputs by matching the marginalization of the joint distribution of multi-view random variables under different forms of factorization. To realize our design, we present Adversarial CCA (ACCA) which achieves consistent latent encodings by matching the marginalized latent encodings through the adversarial training paradigm. Our analysis based on conditional mutual information reveals that ACCA is flexible for handling implicit distributions. Extensive experiments on correlation analysis and cross-view generation under noisy input settings demonstrate the superiority of our model.

READ FULL TEXT

page 22

page 23

page 24

page 25

page 26

page 27

page 29

page 31

research
07/04/2019

Probabilistic CCA with Implicit Distributions

Canonical Correlation Analysis (CCA) is a classic technique for multi-vi...
research
12/21/2018

Canonical Correlation Analysis for Misaligned Satellite Image Change Detection

Canonical correlation analysis (CCA) is a statistical learning method th...
research
07/07/2023

Unpaired Multi-View Graph Clustering with Cross-View Structure Matching

Multi-view clustering (MVC), which effectively fuses information from mu...
research
07/30/2017

Deep Multi-View Learning with Stochastic Decorrelation Loss

Multi-view learning aims to learn an embedding space where multiple view...
research
10/28/2021

Residual Relaxation for Multi-view Representation Learning

Multi-view methods learn representations by aligning multiple views of t...
research
09/16/2017

A LBP Based Correspondence Identification Scheme for Multi-view Sensing Network

In this paper, we describes a correspondence identification method betwe...
research
12/12/2019

Towards Disentangled Representations for Human Retargeting by Multi-view Learning

We study the problem of learning disentangled representations for data a...

Please sign up or login with your details

Forgot password? Click here to reset