Coresets for Near-Convex Functions

06/09/2020
by   Murad Tukan, et al.
0

Coreset is usually a small weighted subset of n input points in R^d, that provably approximates their loss function for a given set of queries (models, classifiers, etc.). Coresets become increasingly common in machine learning since existing heuristics or inefficient algorithms may be improved by running them possibly many times on the small coreset that can be maintained for streaming distributed data. Coresets can be obtained by sensitivity (importance) sampling, where its size is proportional to the total sum of sensitivities. Unfortunately, computing the sensitivity of each point is problem dependent and may be harder to compute than the original optimization problem at hand. We suggest a generic framework for computing sensitivities (and thus coresets) for wide family of loss functions which we call near-convex functions. This is by suggesting the f-SVD factorization that generalizes the SVD factorization of matrices to functions. Example applications include coresets that are either new or significantly improves previous results, such as SVM, Logistic regression, M-estimators, and ℓ_z-regression. Experimental results and open source are also provided.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/02/2019

Tight Sensitivity Bounds For Smaller Coresets

An ε-coreset for Least-Mean-Squares (LMS) of a matrix A∈R^n× d is a smal...
02/21/2018

Coresets For Monotonic Functions with Applications to Deep Learning

Coreset (or core-set) in this paper is a small weighted subset Q of the ...
06/12/2019

Coresets for Gaussian Mixture Models of Any Shape

An ε-coreset for a given set D of n points, is usually a small weighted ...
06/09/2020

Faster PAC Learning and Smaller Coresets via Smoothed Analysis

PAC-learning usually aims to compute a small subset (ε-sample/net) from ...
11/04/2021

A Unified Approach to Coreset Learning

Coreset of a given dataset and loss function is usually a small weighed ...
11/04/2019

Importance Sampling via Local Sensitivity

Given a loss function F:X→R^+ that can be written as the sum of losses o...
10/19/2019

Introduction to Coresets: Accurate Coresets

A coreset (or core-set) of an input set is its small summation, such tha...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.