MUSCLE: Strengthening Semi-Supervised Learning Via Concurrent Unsupervised Learning Using Mutual Information Maximization

11/30/2020
by   Hanchen Xie, et al.
5

Deep neural networks are powerful, massively parameterized machine learning models that have been shown to perform well in supervised learning tasks. However, very large amounts of labeled data are usually needed to train deep neural networks. Several semi-supervised learning approaches have been proposed to train neural networks using smaller amounts of labeled data with a large amount of unlabeled data. The performance of these semi-supervised methods significantly degrades as the size of labeled data decreases. We introduce Mutual-information-based Unsupervised Semi-supervised Concurrent LEarning (MUSCLE), a hybrid learning approach that uses mutual information to combine both unsupervised and semi-supervised learning. MUSCLE can be used as a stand-alone training scheme for neural networks, and can also be incorporated into other learning approaches. We show that the proposed hybrid model outperforms state of the art on several standard benchmarks, including CIFAR-10, CIFAR-100, and Mini-Imagenet. Furthermore, the performance gain consistently increases with the reduction in the amount of labeled data, as well as in the presence of bias. We also show that MUSCLE has the potential to boost the classification performance when used in the fine-tuning phase for a model pre-trained only on unlabeled data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/21/2019

EnAET: Self-Trained Ensemble AutoEncoding Transformations for Semi-Supervised Learning

Deep neural networks have been successfully applied to many real-world a...
research
06/22/2021

Recent Deep Semi-supervised Learning Approaches and Related Works

The author of this work proposes an overview of the recent semi-supervis...
research
10/12/2020

Unsupervised Semantic Aggregation and Deformable Template Matching for Semi-Supervised Learning

Unlabeled data learning has attracted considerable attention recently. H...
research
06/28/2015

Neural Simpletrons - Minimalistic Directed Generative Networks for Learning with Few Labels

Classifiers for the semi-supervised setting often combine strong supervi...
research
10/02/2016

Deep unsupervised learning through spatial contrasting

Convolutional networks have marked their place over the last few years a...
research
12/19/2019

Semi-Supervised Deep Learning Using Improved Unsupervised Discriminant Projection

Deep learning demands a huge amount of well-labeled data to train the ne...
research
10/06/2021

Data-Centric Semi-Supervised Learning

We study unsupervised data selection for semi-supervised learning (SSL),...

Please sign up or login with your details

Forgot password? Click here to reset