Dynamic Least-Squares Regression

by   Shunhua Jiang, et al.

A common challenge in large-scale supervised learning, is how to exploit new incremental data to a pre-trained model, without re-training the model from scratch. Motivated by this problem, we revisit the canonical problem of dynamic least-squares regression (LSR), where the goal is to learn a linear model over incremental training data. In this setup, data and labels (𝐀^(t), 𝐛^(t)) ∈ℝ^t × d×ℝ^t evolve in an online fashion (t≫ d), and the goal is to efficiently maintain an (approximate) solution to min_𝐱^(t)𝐀^(t)𝐱^(t) - 𝐛^(t)_2 for all t∈ [T]. Our main result is a dynamic data structure which maintains an arbitrarily small constant approximate solution to dynamic LSR with amortized update time O(d^1+o(1)), almost matching the running time of the static (sketching-based) solution. By contrast, for exact (or even 1/poly(n)-accuracy) solutions, we show a separation between the static and dynamic settings, namely, that dynamic LSR requires Ω(d^2-o(1)) amortized update time under the OMv Conjecture (Henzinger et al., STOC'15). Our data structure is conceptually simple, easy to implement, and fast both in theory and practice, as corroborated by experiments over both synthetic and real-world datasets.



page 1

page 2

page 3

page 4

∙ 11/01/2021

Dynamic Geometric Set Cover, Revisited

Geometric set cover is a classical problem in computational geometry, wh...
∙ 03/14/2021

More Dynamic Data Structures for Geometric Set Cover with Sublinear Update Time

We study geometric set cover problems in dynamic settings, allowing inse...
∙ 06/03/2020

Dynamic Longest Common Substring in Polylogarithmic Time

The longest common substring problem consists in finding a longest strin...
∙ 09/27/2018

Point Location in Incremental Planar Subdivisions

We study the point location problem in incremental (possibly disconnecte...
∙ 07/16/2020

Dynamic Products of Ranks

We describe a data structure that can maintain a dynamic set of points g...
∙ 05/31/2020

Efficient fully dynamic elimination forests with applications to detecting long paths and cycles

We present a data structure that in a dynamic graph of treedepth at most...
∙ 02/11/2020

Incremental Fast Subclass Discriminant Analysis

This paper proposes an incremental solution to Fast Subclass Discriminan...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.