Boosting the Performance of Semi-Supervised Learning with Unsupervised Clustering

by   Boaz Lerner, et al.

Recently, Semi-Supervised Learning (SSL) has shown much promise in leveraging unlabeled data while being provided with very few labels. In this paper, we show that ignoring the labels altogether for whole epochs intermittently during training can significantly improve performance in the small sample regime. More specifically, we propose to train a network on two tasks jointly. The primary classification task is exposed to both the unlabeled and the scarcely annotated data, whereas the secondary task seeks to cluster the data without any labels. As opposed to hand-crafted pretext tasks frequently used in self-supervision, our clustering phase utilizes the same classification network and head in an attempt to relax the primary task and propagate the information from the labels without overfitting them. On top of that, the self-supervised technique of classifying image rotations is incorporated during the unsupervised learning phase to stabilize training. We demonstrate our method's efficacy in boosting several state-of-the-art SSL algorithms, significantly improving their results and reducing running time in various standard semi-supervised benchmarks, including 92.6 class in each task. We also notably improve the results in the extreme cases of 1,2 and 3 labels per class, and show that features learned by our model are more meaningful for separating the data.



There are no comments yet.


page 7

page 8


A Semi-Supervised Self-Organizing Map for Clustering and Classification

There has been an increasing interest in semi-supervised learning in the...

Improving colonoscopy lesion classification using semi-supervised deep learning

While data-driven approaches excel at many image analysis tasks, the per...

A Semi-Supervised Self-Organizing Map with Adaptive Local Thresholds

In the recent years, there is a growing interest in semi-supervised lear...

Unsupervised Semantic Aggregation and Deformable Template Matching for Semi-Supervised Learning

Unlabeled data learning has attracted considerable attention recently. H...

Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation

Supervised learning in large discriminative models is a mainstay for mod...

Self-Supervised Learning by Estimating Twin Class Distributions

We present TWIST, a novel self-supervised representation learning method...

Multitask Hopfield Networks

Multitask algorithms typically use task similarity information as a bias...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.