Perturbed Bayesian Inference for Online Parameter Estimation
share
We introduce a new Bayesian based approach for online parameter inference that we term perturbed Bayesian inference. Given a sequence of stationary observations (Y_t)_t≥ 1, a parametric model {f_θ,θ∈R^d} for a single observation and θ_:=argmax_θ∈R^dE[ f_θ(Y_1)], the sequence (π̃_t^N)_t≥ 1 of perturbed posterior distributions has the following properties: (i) π̃_t^N does not depend on (Y_s)_s>t, (ii) the time and space complexity of computing π̃_t^N from π̃^N_t-1 and Y_t is cN, and c<+∞ is independent of t, and (iii) for N large enough π̃^N_t converges almost surely as t→+∞ to θ_ at rate (t)^(1+ε)/2t^-1/2, with ε>0 arbitrary, under classical conditions that can be found in the literature on maximum likelihood estimation and on Bayesian asymptotics. Properties (i) and (ii) make perturbed Bayesian inference applicable in the context of streaming data, which requires to update the estimated value of θ_ in real time when a new observation is available. The proposed method also allows for performing statistical inference with datasets containing a large number of observations, which is known to be computationally challenging for current statistical methods.
READ FULL TEXT