FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning

05/15/2022
by   Yidong Wang, et al.
0

Pseudo labeling and consistency regularization approaches with confidence-based thresholding have made great progress in semi-supervised learning (SSL). In this paper, we theoretically and empirically analyze the relationship between the unlabeled data distribution and the desirable confidence threshold. Our analysis shows that previous methods might fail to define favorable threshold since they either require a pre-defined / fixed threshold or an ad-hoc threshold adjusting scheme that does not reflect the learning effect well, resulting in inferior performance and slow convergence, especially for complicated unlabeled data distributions. We hence propose FreeMatch to define and adjust the confidence threshold in a self-adaptive manner according to the model's learning status. To handle complicated unlabeled data distributions more effectively, we further propose a self-adaptive class fairness regularization method that encourages the model to produce diverse predictions during training. Extensive experimental results indicate the superiority of FreeMatch especially when the labeled data are extremely rare. FreeMatch achieves 5.78%, 13.59%, and 1.28% error rate reduction over the latest state-of-the-art method FlexMatch on CIFAR-10 with 1 label per class, STL-10 with 4 labels per class, and ImageNet with 100k labels respectively.

READ FULL TEXT
research
05/21/2022

ADT-SSL: Adaptive Dual-Threshold for Semi-Supervised Learning

Semi-Supervised Learning (SSL) has advanced classification tasks by inpu...
research
11/11/2015

Universum Prescription: Regularization using Unlabeled Data

This paper shows that simply prescribing "none of the above" labels to u...
research
03/20/2023

Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

Semi-supervised learning (SSL) has attracted enormous attention due to i...
research
08/15/2023

Boosting Semi-Supervised Learning by bridging high and low-confidence predictions

Pseudo-labeling is a crucial technique in semi-supervised learning (SSL)...
research
10/15/2021

FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

The recently proposed FixMatch achieved state-of-the-art results on most...
research
05/23/2020

Power Pooling Operators and Confidence Learning for Semi-Supervised Sound Event Detection

In recent years, the involvement of synthetic strongly labeled data,weak...
research
08/15/2023

Semi-Supervised Learning with Multiple Imputations on Non-Random Missing Labels

Semi-Supervised Learning (SSL) is implemented when algorithms are traine...

Please sign up or login with your details

Forgot password? Click here to reset