Ensemble Manifold Segmentation for Model Distillation and Semi-supervised Learning

04/06/2018
by   Dengxin Dai, et al.
0

Manifold theory has been the central concept of many learning methods. However, learning modern CNNs with manifold structures has not raised due attention, mainly because of the inconvenience of imposing manifold structures onto the architecture of the CNNs. In this paper we present ManifoldNet, a novel method to encourage learning of manifold-aware representations. Our approach segments the input manifold into a set of fragments. By assigning the corresponding segmentation id as a pseudo label to every sample, we convert the problem of preserving the local manifold structure into a point-wise classification task. Due to its unsupervised nature, the segmentation tends to be noisy. We mitigate this by introducing ensemble manifold segmentation (EMS). EMS accounts for the manifold structure by dividing the training data into an ensemble of classification training sets that contain samples of local proximity. CNNs are trained on these ensembles under a multi-task learning framework to conform to the manifold. ManifoldNet can be trained with only the pseudo labels or together with task-specific labels. We evaluate ManifoldNet on two different tasks: network imitation (distillation) and semi-supervised learning. Our experiments show that the manifold structures are effectively utilized for both unsupervised and semi-supervised learning.

READ FULL TEXT
research
08/13/2023

Manifold DivideMix: A Semi-Supervised Contrastive Learning Framework for Severe Label Noise

Deep neural networks have proven to be highly effective when large amoun...
research
08/18/2018

Tangent-Normal Adversarial Regularization for Semi-supervised Learning

The ever-increasing size of modern datasets combined with the difficulty...
research
09/12/2023

Multi-dimensional Fusion and Consistency for Semi-supervised Medical Image Segmentation

In this paper, we introduce a novel semi-supervised learning framework t...
research
12/20/2022

C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation

Temporal action segmentation tags action labels for every frame in an in...
research
03/13/2021

Ensemble Learning with Manifold-Based Data Splitting for Noisy Label Correction

Label noise in training data can significantly degrade a model's general...
research
01/07/2020

A semi-supervised learning framework for quantitative structure-activity regression modelling

Supervised learning models, also known as quantitative structure-activit...

Please sign up or login with your details

Forgot password? Click here to reset