DeepAI AI Chat
Log In Sign Up

An efficient Averaged Stochastic Gauss-Newtwon algorithm for estimating parameters of non linear regressions models

06/23/2020
by   Peggy Cénac, et al.
0

Non linear regression models are a standard tool for modeling real phenomena, with several applications in machine learning, ecology, econometry... Estimating the parameters of the model has garnered a lot of attention during many years. We focus here on a recursive method for estimating parameters of non linear regressions. Indeed, these kinds of methods, whose most famous are probably the stochastic gradient algorithm and its averaged version, enable to deal efficiently with massive data arriving sequentially. Nevertheless, they can be, in practice, very sensitive to the case where the eigen-values of the Hessian of the functional we would like to minimize are at different scales. To avoid this problem, we first introduce an online Stochastic Gauss-Newton algorithm. In order to improve the estimates behavior in case of bad initialization, we also introduce a new Averaged Stochastic Gauss-Newton algorithm and prove its asymptotic efficiency.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/23/2020

An efficient Averaged Stochastic Gauss-Newton algorithm for estimating parameters of non linear regressions models

Non linear regression models are a standard tool for modeling real pheno...
03/01/2023

Non asymptotic analysis of Adaptive stochastic gradient algorithms and applications

In stochastic optimization, a common tool to deal sequentially with larg...
04/16/2019

An efficient stochastic Newton algorithm for parameter estimation in logistic regressions

Logistic regression is a well-known statistical model which is commonly ...
04/03/2023

Online stochastic Newton methods for estimating the geometric median and applications

In the context of large samples, a small number of individuals might spo...
10/21/2020

Progressive Batching for Efficient Non-linear Least Squares

Non-linear least squares solvers are used across a broad range of offlin...
11/19/2020

On the asymptotic rate of convergence of Stochastic Newton algorithms and their Weighted Averaged versions

The majority of machine learning methods can be regarded as the minimiza...
12/02/2020

Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression

The Shapley value solution concept from cooperative game theory has beco...