DeepAI AI Chat
Log In Sign Up

Contrastive Multiview Coding

by   Yonglong Tian, et al.

Humans view the world through many sensory channels, e.g., the long-wavelength light channel, viewed by the left eye, or the high-frequency vibrations channel, viewed by the right ear. Each view is noisy and incomplete, but important factors, such as physics, geometry, and semantics, tend to be shared between all views (e.g., a "dog" can be seen, heard, and felt). We hypothesize that a powerful representation is one that models view-invariant factors. Based on this hypothesis, we investigate a contrastive coding scheme, in which a representation is learned that aims to maximize mutual information between different views but is otherwise compact. Our approach scales to any number of views, and is view-agnostic. The resulting learned representations perform above the state of the art for downstream tasks such as object classification, compared to formulations based on predictive learning or single view reconstruction, and improve as more views are added. Code and reference implementations are released on our project page:


Rethinking Minimal Sufficient Representation in Contrastive Learning

Contrastive learning between different views of the data achieves outsta...

What makes for good views for contrastive learning

Contrastive learning between multiple views of the data has recently ach...

Region Embedding with Intra and Inter-View Contrastive Learning

Unsupervised region representation learning aims to extract dense and ef...

Contrastive Multiview Coding for Enzyme-Substrate Interaction Prediction

Characterizing Enzyme function is an important requirement for predictin...

Multi-view Contrastive Coding of Remote Sensing Images at Pixel-level

Our planet is viewed by satellites through multiple sensors (e.g., multi...

Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations

Self-supervision is recently surging at its new frontier of graph learni...

Learning Oculomotor Behaviors from Scanpath

Identifying oculomotor behaviors relevant for eye-tracking applications ...

Code Repositories