Optimally Compressed Nonparametric Online Learning

09/25/2019
by   Alec Koppel, et al.
0

Batch training of machine learning models based on neural networks is now well established, whereas to date streaming methods are largely based on linear models. To go beyond linear in the online setting, nonparametric methods are of interest due to their universality and ability to stably incorporate new information via convexity or Bayes' Rule. Unfortunately, when used online, nonparametric methods suffer a classic "curse of dimensionality" which precludes their use: their complexity scales at least with the time index. We survey online compression tools which bring their memory under control while attaining approximate convergence. The asymptotic bias depends on a compression parameter which trades off memory and accuracy, echoing Shannon-Nyquist sampling for nonparametric statistics. Applications to autonomous robotics, communications, economics, and power are scoped, as well as extensions to multi-agent systems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/20/2020

Real-Time Optimisation for Online Learning in Auctions

In display advertising, a small group of sellers and bidders face each o...
research
12/05/2017

An Online Algorithm for Nonparametric Correlations

Nonparametric correlations such as Spearman's rank correlation and Kenda...
research
12/31/2016

Lazily Adapted Constant Kinky Inference for Nonparametric Regression and Model-Reference Adaptive Control

Techniques known as Nonlinear Set Membership prediction, Lipschitz Inter...
research
02/22/2021

You Only Compress Once: Optimal Data Compression for Estimating Linear Models

Linear models are used in online decision making, such as in machine lea...
research
08/20/2020

Unified Rules of Renewable Weighted Sums for Various Online Updating Estimations

This paper establishes unified frameworks of renewable weighted sums (RW...
research
08/09/2023

Bayes Risk Consistency of Nonparametric Classification Rules for Spike Trains Data

Spike trains data find a growing list of applications in computational n...
research
04/23/2020

Consistent Online Gaussian Process Regression Without the Sample Complexity Bottleneck

Gaussian processes provide a framework for nonlinear nonparametric Bayes...

Please sign up or login with your details

Forgot password? Click here to reset