Disentangling Multi-view Representations Beyond Inductive Bias

by   Guanzhou Ke, et al.

Multi-view (or -modality) representation learning aims to understand the relationships between different view representations. Existing methods disentangle multi-view representations into consistent and view-specific representations by introducing strong inductive biases, which can limit their generalization ability. In this paper, we propose a novel multi-view representation disentangling method that aims to go beyond inductive biases, ensuring both interpretability and generalizability of the resulting representations. Our method is based on the observation that discovering multi-view consistency in advance can determine the disentangling information boundary, leading to a decoupled learning objective. We also found that the consistency can be easily extracted by maximizing the transformation invariance and clustering consistency between views. These observations drive us to propose a two-stage framework. In the first stage, we obtain multi-view consistency by training a consistent encoder to produce semantically-consistent representations across views as well as their corresponding pseudo-labels. In the second stage, we disentangle specificity from comprehensive representations by minimizing the upper bound of mutual information between consistent and comprehensive representations. Finally, we reconstruct the original data by concatenating pseudo-labels and view-specific representations. Our experiments on four multi-view datasets demonstrate that our proposed method outperforms 12 comparison methods in terms of clustering and classification performance. The visualization results also show that the extracted consistency and specificity are compact and interpretable. Our code can be found at <https://github.com/Guanzhou-Ke/DMRIB>.


Multi-View Clustering from the Perspective of Mutual Information

Exploring the complementary information of multi-view data to improve cl...

Multi-view Fuzzy Representation Learning with Rules based Model

Unsupervised multi-view representation learning has been extensively stu...

MORI-RAN: Multi-view Robust Representation Learning via Hybrid Contrastive Fusion

Multi-view representation learning is essential for many multi-view task...

A Clustering-guided Contrastive Fusion for Multi-view Representation Learning

The past two decades have seen increasingly rapid advances in the field ...

One-step Multi-view Clustering with Diverse Representation

Multi-view clustering has attracted broad attention due to its capacity ...

Multi-View MOOC Quality Evaluation via Information-Aware Graph Representation Learning

In this paper, we study the problem of MOOC quality evaluation which is ...

View-labels Are Indispensable: A Multifacet Complementarity Study of Multi-view Clustering

Consistency and complementarity are two key ingredients for boosting mul...

Please sign up or login with your details

Forgot password? Click here to reset