Multi-view Alignment and Generation in CCA via Consistent Latent Encoding

by   Yaxin Shi, et al.

Multi-view alignment, achieving one-to-one correspondence of multi-view inputs, is critical in many real-world multi-view applications, especially for cross-view data analysis problems. Recently, an increasing number of works study this alignment problem with Canonical Correlation Analysis (CCA). However, existing CCA models are prone to misalign the multiple views due to either the neglect of uncertainty or the inconsistent encoding of the multiple views. To tackle these two issues, this paper studies multi-view alignment from the Bayesian perspective. Delving into the impairments of inconsistent encodings, we propose to recover correspondence of the multi-view inputs by matching the marginalization of the joint distribution of multi-view random variables under different forms of factorization. To realize our design, we present Adversarial CCA (ACCA) which achieves consistent latent encodings by matching the marginalized latent encodings through the adversarial training paradigm. Our analysis based on conditional mutual information reveals that ACCA is flexible for handling implicit distributions. Extensive experiments on correlation analysis and cross-view generation under noisy input settings demonstrate the superiority of our model.


page 22

page 23

page 24

page 25

page 26

page 27

page 29

page 31


Probabilistic CCA with Implicit Distributions

Canonical Correlation Analysis (CCA) is a classic technique for multi-vi...

Canonical Correlation Analysis for Misaligned Satellite Image Change Detection

Canonical correlation analysis (CCA) is a statistical learning method th...

SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence

In this paper, we study the problem of multi-view sketch correspondence,...

Deep Multi-View Learning with Stochastic Decorrelation Loss

Multi-view learning aims to learn an embedding space where multiple view...

Residual Relaxation for Multi-view Representation Learning

Multi-view methods learn representations by aligning multiple views of t...

A LBP Based Correspondence Identification Scheme for Multi-view Sensing Network

In this paper, we describes a correspondence identification method betwe...

MCGNet: Partial Multi-view Few-shot Learning via Meta-alignment and Context Gated-aggregation

In this paper, we propose a new challenging task named as partial multi-...