Semi-Supervised Learning with Self-Supervised Networks

06/25/2019
by   Phi Vu Tran, et al.
9

Recent advances in semi-supervised learning have shown tremendous potential in overcoming a major barrier to the success of modern machine learning algorithms: access to vast amounts of human-labeled training data. Algorithms based on self-ensemble learning and virtual adversarial training can harness the abundance of unlabeled data to produce impressive state-of-the-art results on a number of semi-supervised benchmarks, approaching the performance of strong supervised baselines using only a fraction of the available labeled data. However, these methods often require careful tuning of many hyper-parameters and are usually not easy to implement in practice. In this work, we present a conceptually simple yet effective semi-supervised algorithm based on self-supervised learning to combine semantic feature representations from unlabeled data. Our models are efficiently trained end-to-end for the joint, multi-task learning of labeled and unlabeled data in a single stage. Striving for simplicity and practicality, our approach requires no additional hyper-parameters to tune for optimal performance beyond the standard set for training convolutional neural networks. We conduct a comprehensive empirical evaluation of our models for semi-supervised image classification on SVHN, CIFAR-10 and CIFAR-100, and demonstrate results competitive with, and in some cases exceeding, prior state of the art. Reference code and data are available at https://github.com/vuptran/sesemi

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/18/2020

FROST: Faster and more Robust One-shot Semi-supervised Training

Recent advances in one-shot semi-supervised learning have lowered the ba...
research
11/21/2019

EnAET: Self-Trained Ensemble AutoEncoding Transformations for Semi-Supervised Learning

Deep neural networks have been successfully applied to many real-world a...
research
02/07/2021

Self-supervised driven consistency training for annotation efficient histopathology image analysis

Training a neural network with a large labeled dataset is still a domina...
research
08/08/2023

Improving Performance of Semi-Supervised Learning by Adversarial Attacks

Semi-supervised learning (SSL) algorithm is a setup built upon a realist...
research
02/17/2021

Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training

Self-training is a standard approach to semi-supervised learning where t...
research
03/21/2021

ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised Learning

In this paper, we address the problem of training deep neural networks i...

Please sign up or login with your details

Forgot password? Click here to reset