The information-theoretic value of unlabeled data in semi-supervised learning

01/16/2019
by   Alexander Golovnev, et al.
0

We quantify the separation between the numbers of labeled examples required to learn in two settings: Settings with and without the knowledge of the distribution of the unlabeled data. More specifically, we prove a separation by Θ( n) multiplicative factor for the class of projections over the Boolean hypercube of dimension n. We prove that there is no separation for the class of all functions on domain of any size. Learning with the knowledge of the distribution (a.k.a. fixed-distribution learning) can be viewed as an idealized scenario of semi-supervised learning where the number of unlabeled data points is so great that the unlabeled distribution is known exactly. For this reason, we call the separation the value of unlabeled data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/22/2020

Semi-Supervised Learning: the Case When Unlabeled Data is Equally Useful

Semi-supervised learning algorithms attempt to take advantage of relativ...
research
04/06/2019

Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift

Recent work has shown that using unlabeled data in semi-supervised learn...
research
05/31/2023

A rule-general abductive learning by rough sets

In real-world tasks, there is usually a large amount of unlabeled data a...
research
10/19/2012

On Information Regularization

We formulate a principle for classification with the knowledge of the ma...
research
12/12/2012

Continuation Methods for Mixing Heterogenous Sources

A number of modern learning tasks involve estimation from heterogeneous ...
research
05/02/2022

Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding

Semantic understanding of 3D point cloud relies on learning models with ...
research
05/22/2022

Deep Low-Density Separation for Semi-Supervised Classification

Given a small set of labeled data and a large set of unlabeled data, sem...

Please sign up or login with your details

Forgot password? Click here to reset