Joint Probability Distribution of Prediction Errors of ARIMA

11/12/2018 ∙ by Xin Qin, et al. ∙ 0

Producing probabilistic guarantee for several steps of a predicted signal follow a temporal logic defined behavior has its rising importance in monitoring. In this paper, we derive a method to compute the joint probability distribution of prediction errors of multiple steps based on Autoregressive Integrated Moving Average(ARIMA) model. We cover scenarios in stationary process and intrinsically stationary process for univariate and multivariate.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Signal Temporal Logic(STL) specify the simultaneous behavior of a signal across different time point, while time series prediction based on Autoregressive Integrated Moving Average (ARIMA) model only give guarantee on each single time point. In this paper, we give the proof of calculating joint probability distribution of prediction errors of multiple steps.

2 Error of Prediction in Univariate Process

The goal is to get the joint distribution of

2.1 Error of Stationary Process

Previous work [1] [2] concluded that for stationary process

Assuming that

is a normal distribution, we have

.

Definition 1

The best linear prediction of is[2]:

(1)

The optimized choice of are

(2)

and

(3)

where

Definition 2
(4)
Definition 3

For best linear predictor [2]

Corollary 1

The value of best linear predictor at already observed datapoint follows

Definition 4

If the distribution of

is a multivariate normal distribution, then the random variable vector

, where is a matrix, is also a multivariate normal distribution.And if , then

Definition 5

If

are independent white noise, the joint distribution of

is a multivariate normal distribution with

(5)
Lemma 1

Given , let denote , then

(6)

where is a transform matrix from to .

(7)
Proof

From the definition of ARMA(p,q) process,

(8)

where .

Lemma 2
(9)

where is a matrix, is a matrix.

(10)
(11)
Proof

is a stationary process starting from with zero mean, the prediction of follows

(12)

where . C3 is the coefficient matrix of when calculating , the best linear predictor of , a stationary MA(q) process with zero mean.

Theorem 2.1

The covariance matrix of prediction error of a moving average process is

(13)
Proof

Let denote , denote , denote , denote , denote

Error is defined as

(14)

From lemma 1 and 2

(15)

After augment the coefficient matrix and to and by

(16)

equation 15 is transformed to

(17)

Let denote . Follow Lemma 4, , where

(18)

where the size of here is .

Theorem 2.2

is a multivariate normal distribution in ARMA(p,q) process , using the best linear predictor . The covariance matrix of prediction error is

(19)

where

(20)

in which is the coefficient of when calculating .

can be recursively given by: For any (),

(21)

For any (),

(22)
Proof

From ARIMA we know

(23)

Then from definition 23

(24)

From definition 3

(25)

From corollary 1

(26)

Then we have

(27)

which can be represent in matrix form

(28)

Let denote

(29)

Using Lemma 4, we have where

(30)

Q.E.D.

From definition 2,

(31)

which lead to the following conclusion:

Theorem 2.3

Now we have a guarantee for any prediction interval of a time step , denote the event as event , then the joint probability of can be calculated by using theorem 2.2 when is a stationary process.

2.2 Error of Intrinsically Stationary Process

Definition 6

When the d-ordered differencing of a time series is a stationary process while its -ordered process is still a non-stationary process, we call this process a d-ordered intrinsically stationary process.

A d-ordered differencing is defined as:

(32)
Lemma 3

To a d-ordered intrinsically stationary process, the best linear prediction of based on observation is:

(33)
Proof

From the definition of differencing function, equation 32,

(34)

which means

(35)

using Property 3,

(36)

Introducing Property 1, when , is observed, so:

(37)

When ,

(38)

Q.E.D.

Lemma 4

The error of the -th time step can be represent recursively by .

(39)
Proof

When, , using the definition of differencing (equation 32):

(40)

and Lemma 3, the error is:

(41)

Then for , similarly, using the definition (equation 32):

(42)

and Theorem 3, the error is:

(43)

Q.E.D.

Theorem 2.4

The joint distribution among errors of different time steps is a multivariate normal distribution

(44)

where

(45)

in which is the covariance matrix of given by Theorem 2.2 and

(46)

in which is the coefficient of when calculating .

can be recursively given by: For any (),

(47)

For any (),

(48)
Proof

From Lemma 4

(49)

in which is the coefficient of when calculating .

can be recursively given by: For any (),

(50)

For any (),

(51)

Denoting

Following Definition 4, where

(52)

Q.E.D.

Now we have the joint guarantee of any prediction interval of a series of time steps .We can use the conclusion of Theorem 2.4 when dealing with an intrinsically stationary process.

3 Error of Prediction of Multivariate Process

3.1 Error of Stationary Multivariate Process

All of the time series we have discussed are univariate time series, now we want to generalize our conclusion to multivariate cases. Similarly, we will deal with the stationary case firstly.

Definition 7

The best linear prediction of is[2]:

(53)

in which is a time series with m variate, is a vector and are matrices.

The optimized choice of are

(54)
Definition 8

Denoting , where is the white noise of time step i.

(55)

where

(56)
Lemma 5

Defining a multivariate MA process

(57)

where is a matrix.

Denoting , then

(58)

where is a transform matrix from to .

(59)
Proof

From the definition of multivariate ARMA(p,q) process,

(60)

Q.E.D.

Lemma 6

The best linear predictor of is a linear combination of .

(61)

where

(62)

is a transform matrix from to .

And

(63)

is the coefficient matrix of when calculating , the best linear predictor of , where a multivariate stationary MA(q) process with zero mean.

will be given by equation 54.

Theorem 3.1

The joint distribution among errors of different time steps of is a multivariate distribution