Enhancing Self-Training Methods

Semi-supervised learning approaches train on small sets of labeled data along with large sets of unlabeled data. Self-training is a semi-supervised teacher-student approach that often suffers from the problem of "confirmation bias" that occurs when the student model repeatedly overfits to incorrect pseudo-labels given by the teacher model for the unlabeled data. This bias impedes improvements in pseudo-label accuracy across self-training iterations, leading to unwanted saturation in model performance after just a few iterations. In this work, we describe multiple enhancements to improve the self-training pipeline to mitigate the effect of confirmation bias. We evaluate our enhancements over multiple datasets showing performance gains over existing self-training design choices. Finally, we also study the extendability of our enhanced approach to Open Set unlabeled data (containing classes not seen in labeled data).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/18/2022

Self-Transriber: Few-shot Lyrics Transcription with Self-training

The current lyrics transcription approaches heavily rely on supervised l...
research
11/20/2020

SLADE: A Self-Training Framework For Distance Metric Learning

Most existing distance metric learning approaches use fully labeled data...
research
07/17/2021

Self Training with Ensemble of Teacher Models

In order to train robust deep learning models, large amounts of labelled...
research
02/23/2023

Uncertainty Guided Ensemble Self-Training for Semi-Supervised Global Field Reconstruction

Recovering a globally accurate complex physics field from limited sensor...
research
04/12/2021

Self-Training with Weak Supervision

State-of-the-art deep neural networks require large-scale labeled traini...
research
09/30/2019

Revisiting Self-Training for Neural Sequence Generation

Self-training is one of the earliest and simplest semi-supervised method...
research
04/20/2020

Local Clustering with Mean Teacher for Semi-supervised Learning

The Mean Teacher (MT) model of Tarvainen and Valpola has shown favorable...

Please sign up or login with your details

Forgot password? Click here to reset