Log In Sign Up

Semi-Supervised Learning for Multi-Task Scene Understanding by Neural Graph Consensus

by   Marius Leordeanu, et al.

We address the challenging problem of semi-supervised learning in the context of multiple visual interpretations of the world by finding consensus in a graph of neural networks. Each graph node is a scene interpretation layer, while each edge is a deep net that transforms one layer at one node into another from a different node. During the supervised phase edge networks are trained independently. During the next unsupervised stage edge nets are trained on the pseudo-ground truth provided by consensus among multiple paths that reach the nets' start and end nodes. These paths act as ensemble teachers for any given edge and strong consensus is used for high-confidence supervisory signal. The unsupervised learning process is repeated over several generations, in which each edge becomes a "student" and also part of different ensemble "teachers" for training other students. By optimizing such consensus between different paths, the graph reaches consistency and robustness over multiple interpretations and generations, in the face of unknown labels. We give theoretical justifications of the proposed idea and validate it on a large dataset. We show how prediction of different representations such as depth, semantic segmentation, surface normals and pose from RGB input could be effectively learned through self-supervised consensus in our graph. We also compare to state-of-the-art methods for multi-task and semi-supervised learning and show superior performance.


page 2

page 5

page 7


Unsupervised Domain Adaptation through Iterative Consensus Shift in a Multi-Task Graph

Babies learn with very little supervision by observing the surrounding w...

Semi-supervised classification by reaching consensus among modalities

This paper introduces transductive consensus network (TCNs), as an exten...

Graph Representation Learning by Ensemble Aggregating Subgraphs via Mutual Information Maximization

Graph Neural Networks have shown tremendous potential on dealing with ga...

Ensemble Manifold Segmentation for Model Distillation and Semi-supervised Learning

Manifold theory has been the central concept of many learning methods. H...

Comment on "Ensemble Projection for Semi-supervised Image Classification"

In a series of papers by Dai and colleagues [1,2], a feature map (or ker...

AssemblyNet: A large ensemble of CNNs for 3D Whole Brain MRI Segmentation

Whole brain segmentation using deep learning (DL) is a very challenging ...

To Supervise or Not: How to Effectively Learn Wireless Interference Management Models?

Machine learning has become successful in solving wireless interference ...