DeepAI

# General proof of a limit related to AR(k) model of Statistics

Computing moments of various parameter estimators related to an autoregressive model of Statistics, one needs to evaluate several non-trivial limits. This was done by arXiv:1506.03131 for the case of two, three and four dimensions; in this article, we present a proof of a fully general formula, based on an ingenious solution of https://mathoverflow.net/users/4312/fedor-petrov.

12/14/2012

### Machine Learning in Proof General: Interfacing Interfaces

We present ML4PG - a machine learning extension for Proof General. It al...
09/19/2021

### Moments of permutation statistics and central limit theorems

We show that if a permutation statistic can be written as a linear combi...
05/24/2021

### Asymptotic Moments Matching to Uniformly Minimum Variance Unbiased Estimation under Ewens Sampling Formula

The Ewens sampling formula is a distribution related to the random parti...
02/25/2022

### Robust Estimation of Loss Models for Truncated and Censored Severity Data

In this paper, we consider robust estimation of claim severity models in...
01/05/2023

### Least absolute deviation estimation for AR(1) processes with roots close to unity

We establish the asymptotic theory of least absolute deviation estimator...
09/05/2019

### Number of Sign Changes: Segment of AR(1)

Let X_t denote a stationary first-order autoregressive process. Consider...
05/17/2020

### A Tutorial on Multivariate k-Statistics and their Computation

This document aims to provide an accessible tutorial on the unbiased est...

## 1 Introduction

The autoregressive model of Statistics generates a random sequence of observations by

 Xi=α1Xi−1+α2Xi−2+...+αkXi−k+εi (1)

where

are independent, Normally distributed random variables with the mean of

and the same standard deviation, and

is a fixed integer, usually quite small (e.g.

defines the so called Markov model). The sufficient and necessary condition for the resulting sequence to be asymptotically stationary is that all

solutions of the characteristic polynomial

 λk=α1λk−1+α2λk−2+...+αk (2)

are, in absolute value, smaller than (this is then assumed from now on).

The -order serial correlation coefficient (between and ) is then computed by

 ρj=A1λ|j|1+A2λ|j|2+...+Akλ|j|k (3)

where the ’s are the roots of (2), and the coefficients are themselves simple functions of these roots. Note that the absolute value of each root must be smaller than if the resulting stochastic process is be stationary.

Computing the first few moments of various estimators (of the parameters) boils down to computing moments of expressions of the

 n∑i=1Xi (4)

and

 n−j∑i=1XiXi+j (5)

type, where is a collection of consecutive observations (assuming that the process has already reached its stationary phase).

This in turn requires evaluating various summations (see [4]), of which the most difficult has the form of

 ~n∑i1,i2,...ik=1λ|i1−i2+s1|1λ|i2−i3+s2|2...λ|ik−i1+sk|k (6)

where are the roots (some may be multiple), are (small) integers, and indicates that the upper limit equals to , adjusted in the manner of (5).

For small , it is possible (but rather messy - see [2]) to exactly evaluate (6) and realize that the answer will always consist of three parts:

• terms proportional to which all tend to zero (as increases) ‘exponentially’,

• terms which stay constant as increases,

• terms proportional to

Luckily, to build an approximation which is usually deemed sufficient (see [4]), we need to find only the proportional terms. These can be extracted by dividing (6) by and taking the limit. Incidentally, this results in the following (and most welcomed) simplification: the corresponding answer will be the same regardless of the adjustments (thus, we may as well use instead), and will similarly not depend on the individual ’s, but only on the absolute value of their sum, as the following statement indicates.

## 2 The main theorem

where

Proof. Define

 BSdef=∑m1+m2+...+mk=Sλ|m1|1λ|m2|2...λ|mk|k (8)

where is the non-negative integer of the theorem.

When , a term of and a term of the summation are considered identical (we also say that they match each other) only when (implying since the ’s and ’s must add up to the same and the ’s cancel); note that this also implies (but not the reverse) that such matching terms have the same value. On the other hand, when , we declare them identical when etc. instead. From now on, we assume that to avoid a trivial duplication of all subsequent arguments.

Clearly, each term of the summation matches a term of : just take where , with the understanding that .

At the same time, no term of is matched by more than terms of the summation, since once you select (from any of its possible values), all the remaining ’s are uniquely determined by etc., resulting in a term of only when all of these turn out to be between and (inclusive).

This proves that

Since

 BS<∞∑m2,m3,...mk=−∞|λ2||m2||λ3||m3|...|λk||mk|

implying that the sum is (absolutely) convergent; let denote its actual value. This means that any number smaller that (say ) can be exceeded by a sum of finitely many terms of (this is true for any convergent series).

Now, let us go back to counting how many terms of the summation match a single, specific term of ; we have already seen that, starting with any one of the possible values of the subsequent ’s would be computed by

 ip=i1+p−1∑j=1(mj−sj)         where \ \ \ p=2...k

matching a term of the summation only when they are all in the to range, i.e. when

 1≤i1+minp=2...kp−1∑j=1(mj−sj)

and

 i1+maxp=2...kp−1∑j=1(mj−sj)≤n

This implies that, for each choice of which meet

 1−minp=2..kp−1∑j=1(mj−sj)≤i1≤n−maxp=2..kp−1∑j=1(mj−sj)

we get a legitimate term of the summation (matching and having the same value as the specific term of ); we thus have

 n−maxp=2...kp−1∑j=1(mj−sj)+minp=2..kp−1∑j=1(mj−sj)

such terms in total. Dividing their sum by and taking the limit thus yields the value of the specific term.

This can be repeated for any term of the finite sum of the previous paragraph; thus we get And, since we can make as close to as we wish, this implies that

We have thus shown that (2) and (8) have the same value.

We now define the following Laurent series of the sequence (allowing to have any integer value, and assuming that ), namely

 F(t)def=∞∑S=−∞tS∑m1+m2+...+mk=Sλ|m1|1λ|m2|2...λ|mk|k= ∞∑m1,m2,...,mk=−∞tm1λ|m1|1tm2λ|m2|2...tmkλ|mk|k= ∞∑m1,m2,...,mk=−∞k∏ℓ=1tmℓλℓ|mℓ|=k∏ℓ=1∞∑m=−∞tmλℓ|m|= k∏ℓ=1(∞∑m=0(tλℓ)m+−1∑m=−∞tmλℓ−m)=k∏ℓ=1(∞∑m=0(tλℓ)m+∞∑m=1(λℓt)m)= =k∏ℓ=1⎛⎜⎝11−t λℓ+λℓt1−λℓt⎞⎟⎠=k∑j=1(Cj1−t λj+Djt−λj)

where the last expression is the partial-fraction expansion of the previous rational function of (the roots of the common denominator are the ’s and their inverses). We can now get a formula for (and thus for our limit) as a coefficient of of the last expression. Since only the part contributes to non-negative powers of and

 Cj=F(t)(1−t λj)∣∣t=λ−1j=k∏ℓ=1ℓ≠j⎛⎜ ⎜⎝11−λℓλj+λjλℓ1−λjλℓ⎞⎟ ⎟⎠= =λk−1jk∏ℓ=1ℓ≠j(1λj−λℓ+λℓ1−λjλℓ)=λk−1jk∏ℓ=1ℓ≠j1−λ2ℓ(λj−λℓ)(1−λjλℓ)

the final formula is therefore given by

 k∑j=1λSjCj

(note that the coefficient of in the expansion of is ).

This proves the original statement.

## 3 Conclusion

The formula of (2) then enables us to evaluate all the expected values needed to deal with any autoregressive model of type (1). Note that in some cases the set of values may consist of only a subset of of roots of (2); this only reduces the value of and makes the result that much easier.

A modification of the formula is needed when some of the ’s are identical; in that case all we have to do is to evaluate the formula’s corresponding limit, such as when the two ’s have the same value (in the case of triple roots, we would need to take two consecutive limits, etc.). This yields a multitude of new (and rather messy) formulas not worth quoting - suffices to say that they all result (as they must) in a finite expression.

A further challenge would be to find the constant part of (6).