Meta-Semi: A Meta-learning Approach for Semi-supervised Learning

07/05/2020
by   Yulin Wang, et al.
0

Deep learning based semi-supervised learning (SSL) algorithms have led to promising results in recent years. However, they tend to introduce multiple tunable hyper-parameters, making them less practical in real SSL scenarios where the labeled data is scarce for extensive hyper-parameter search. In this paper, we propose a novel meta-learning based SSL algorithm (Meta-Semi) that requires tuning only one additional hyper-parameter, compared with a standard supervised deep learning algorithm, to achieve competitive performance under various conditions of SSL. We start by defining a meta optimization problem that minimizes the loss on labeled data through dynamically reweighting the loss on unlabeled samples, which are associated with soft pseudo labels during training. As the meta problem is computationally intensive to solve directly, we propose an efficient algorithm to dynamically obtain the approximate solutions. We show theoretically that Meta-Semi converges to the stationary point of the loss function on labeled data under mild conditions. Empirically, Meta-Semi outperforms state-of-the-art SSL algorithms significantly on the challenging semi-supervised CIFAR-100 and STL-10 tasks, and achieves competitive performance on CIFAR-10 and SVHN.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/03/2019

Learning to Self-Train for Semi-Supervised Few-Shot Classification

Few-shot classification (FSC) is challenging due to the scarcity of labe...
research
09/26/2020

Domain Generalization via Semi-supervised Meta Learning

The goal of domain generalization is to learn from multiple source domai...
research
06/17/2019

Particle Swarm Optimization for Great Enhancement in Semi-Supervised Retinal Vessel Segmentation with Generative Adversarial Networks

Retinal vessel segmentation based on deep learning requires a lot of man...
research
07/22/2020

MetAL: Active Semi-Supervised Learning on Graphs via Meta Learning

The objective of active learning (AL) is to train classification models ...
research
08/27/2019

MetaMixUp: Learning Adaptive Interpolation Policy of MixUp with Meta-Learning

MixUp is an effective data augmentation method to regularize deep neural...
research
09/07/2022

A Novel Semi-supervised Meta Learning Method for Subject-transfer Brain-computer Interface

Brain-computer interface (BCI) provides a direct communication pathway b...
research
03/23/2020

Meta Pseudo Labels

Many training algorithms of a deep neural network can be interpreted as ...

Please sign up or login with your details

Forgot password? Click here to reset