Statistical inference using SGD

05/21/2017
by   Tianyang Li, et al.
0

We present a novel method for frequentist statistical inference in M-estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the Ornstein-Uhlenbeck process suggests that such averages are asymptotically normal. From a practical perspective, our SGD-based inference procedure is a first order method, and is well-suited for large scale problems. To show its merits, we apply it to both synthetic and real datasets, and demonstrate that its accuracy is comparable to classical statistical methods, while requiring potentially far less computation.

READ FULL TEXT

page 10

page 11

page 30

research
11/04/2019

Statistical Inference for Model Parameters in Stochastic Gradient Descent via Batch Means

Statistical inference of true model parameters based on stochastic gradi...
research
10/27/2016

Statistical Inference for Model Parameters in Stochastic Gradient Descent

The stochastic gradient descent (SGD) algorithm has been widely used in ...
research
02/24/2023

Statistical Inference with Stochastic Gradient Methods under φ-mixing Data

Stochastic gradient descent (SGD) is a scalable and memory-efficient opt...
research
02/13/2018

Statistical Inference for Online Learning and Stochastic Approximation via Hierarchical Incremental Gradient Descent

Stochastic gradient descent (SGD) is an immensely popular approach for o...
research
06/06/2021

Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scaling

We develop a new method of online inference for a vector of parameters e...
research
02/10/2021

Once is Never Enough: Foundations for Sound Statistical Inference in Tor Network Experimentation

Tor is a popular low-latency anonymous communication system that focuses...
research
10/19/2016

An Efficient Minibatch Acceptance Test for Metropolis-Hastings

We present a novel Metropolis-Hastings method for large datasets that us...

Please sign up or login with your details

Forgot password? Click here to reset