Coresets for Classification – Simplified and Strengthened

06/08/2021
by   Tung Mai, et al.
0

We give relative error coresets for training linear classifiers with a broad class of loss functions, including the logistic loss and hinge loss. Our construction achieves (1±ϵ) relative error with Õ(d ·μ_y(X)^2/ϵ^2) points, where μ_y(X) is a natural complexity measure of the data matrix X ∈ℝ^n × d and label vector y ∈{-1,1}^n, introduced in by Munteanu et al. 2018. Our result is based on subsampling data points with probabilities proportional to their ℓ_1 Lewis weights. It significantly improves on existing theoretical bounds and performs well in practice, outperforming uniform subsampling along with other importance sampling methods. Our sampling distribution does not depend on the labels, so can be used for active learning. It also does not depend on the specific loss function, so a single coreset can be used in multiple training scenarios.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/10/2019

Active Learning with Importance Sampling

We consider an active learning setting where the algorithm has access to...
research
04/26/2021

Efficient training of physics-informed neural networks via importance sampling

Physics-Informed Neural Networks (PINNs) are a class of deep neural netw...
research
12/24/2018

Improving MMD-GAN Training with Repulsive Loss Function

Generative adversarial nets (GANs) are widely used to learn the data sam...
research
02/19/2018

Understanding the Loss Surface of Neural Networks for Binary Classification

It is widely conjectured that the reason that training algorithms for ne...
research
07/06/2020

Surprise sampling: improving and extending the local case-control sampling

Fithian and Hastie (2014) proposed a new sampling scheme called local ca...
research
06/20/2022

Actively Learning Deep Neural Networks with Uncertainty Sampling Based on Sum-Product Networks

Active learning is popular approach for reducing the amount of data in t...
research
02/10/2020

Stability for the Training of Deep Neural Networks and Other Classifiers

We examine the stability of loss-minimizing training processes that are ...

Please sign up or login with your details

Forgot password? Click here to reset