The autoregressive model of Statistics generates a random sequence of observations by
and the same standard deviation, andis a fixed integer, usually quite small (e.g.
defines the so called Markov model). The sufficient and necessary condition for the resulting sequence to be asymptotically stationary is that allsolutions of the characteristic polynomial
are, in absolute value, smaller than (this is then assumed from now on).
The -order serial correlation coefficient (between and ) is then computed by
where the ’s are the roots of (2), and the coefficients are themselves simple functions of these roots. Note that the absolute value of each root must be smaller than if the resulting stochastic process is be stationary.
Computing the first few moments of various estimators (of the parameters) boils down to computing moments of expressions of the
type, where is a collection of consecutive observations (assuming that the process has already reached its stationary phase).
This in turn requires evaluating various summations (see ), of which the most difficult has the form of
where are the roots (some may be multiple), are (small) integers, and indicates that the upper limit equals to , adjusted in the manner of (5).
terms proportional to which all tend to zero (as increases) ‘exponentially’,
terms which stay constant as increases,
terms proportional to
Luckily, to build an approximation which is usually deemed sufficient (see ), we need to find only the proportional terms. These can be extracted by dividing (6) by and taking the limit. Incidentally, this results in the following (and most welcomed) simplification: the corresponding answer will be the same regardless of the adjustments (thus, we may as well use instead), and will similarly not depend on the individual ’s, but only on the absolute value of their sum, as the following statement indicates.
2 The main theorem
where is the non-negative integer of the theorem.
When , a term of and a term of the summation are considered identical (we also say that they match each other) only when … (implying since the ’s and ’s must add up to the same and the ’s cancel); note that this also implies (but not the reverse) that such matching terms have the same value. On the other hand, when , we declare them identical when etc. instead. From now on, we assume that to avoid a trivial duplication of all subsequent arguments.
Clearly, each term of the summation matches a term of : just take where , with the understanding that .
At the same time, no term of is matched by more than terms of the summation, since once you select (from any of its possible values), all the remaining ’s are uniquely determined by etc., resulting in a term of only when all of these turn out to be between and (inclusive).
This proves that
implying that the sum is (absolutely) convergent; let denote its actual value. This means that any number smaller that (say ) can be exceeded by a sum of finitely many terms of (this is true for any convergent series).
Now, let us go back to counting how many terms of the summation match a single, specific term of ; we have already seen that, starting with any one of the possible values of the subsequent ’s would be computed by
matching a term of the summation only when they are all in the to range, i.e. when
This implies that, for each choice of which meet
we get a legitimate term of the summation (matching and having the same value as the specific term of ); we thus have
such terms in total. Dividing their sum by and taking the limit thus yields the value of the specific term.
This can be repeated for any term of the finite sum of the previous paragraph; thus we get And, since we can make as close to as we wish, this implies that
We now define the following Laurent series of the sequence (allowing to have any integer value, and assuming that ), namely
where the last expression is the partial-fraction expansion of the previous rational function of (the roots of the common denominator are the ’s and their inverses). We can now get a formula for (and thus for our limit) as a coefficient of of the last expression. Since only the part contributes to non-negative powers of and
the final formula is therefore given by
(note that the coefficient of in the expansion of is ).
This proves the original statement.
The formula of (2) then enables us to evaluate all the expected values needed to deal with any autoregressive model of type (1). Note that in some cases the set of values may consist of only a subset of of roots of (2); this only reduces the value of and makes the result that much easier.
A modification of the formula is needed when some of the ’s are identical; in that case all we have to do is to evaluate the formula’s corresponding limit, such as when the two ’s have the same value (in the case of triple roots, we would need to take two consecutive limits, etc.). This yields a multitude of new (and rather messy) formulas not worth quoting - suffices to say that they all result (as they must) in a finite expression.
A further challenge would be to find the constant part of (6).
-  Fedor Petrov (https://mathoverflow.net/users/4312/fedor-petrov), Prove an existing formula for a limit of a specific sum, URL (version: 2019-07-09): https://mathoverflow.net/q/335816
-  Yuhao Liu: ”Finding moments of AR(k)-model parameter estimators” Brock Reports in Mathematics and Statistics No. 150504 (May 4, 2015)
-  Yuhao Liu and Jan Vrbik: https://arxiv.org/abs/1506.03131
-  Jan Vrbik: ”Moments of AR(k) parameter estimators” Communications in Statistics - Simulation and Computation 44 (2015) 1239-1252