Consider stochastic sequences defined over . For convenience, let the index be time. A sequence is Markov if and only if (iff) conditioned on the state at any time , the segment before is independent of the segment after . A sequence is reciprocal iff conditioned on the states at any two times and , the segment inside the interval is independent of the two segments outside . As defined in , a sequence is () over iff conditioned on the state at time (), the sequence is Markov over (). The Markov sequence is a special case of the reciprocal sequence (i.e., each Markov sequence is a reciprocal sequence, but not vice versa) and the reciprocal sequence is a special case of the CM sequence (i.e., each reciprocal sequence is a CM sequence, but not vice versa).
Markov processes have been widely studied and used. However, they are not general enough for some problems –, and more general processes are needed. Reciprocal processes are one generalization of Markov processes. The CM process (including the reciprocal process as a special case) provides a systematic and convenient generalization of the Markov process (based on conditioning) leading to various classes of processes .
Being a motivation for defining reciprocal processes , the problem posed by E. Schrodinger  about some behavior of particles can be studied in the reciprocal process setting. In  reciprocal processes were discussed in the context of stochastic mechanics. In a quantized state space, finite-state reciprocal sequences were used in – for detection of anomalous trajectory patterns, intent inference, and tracking. The approach presented in – for intent inference in an intelligent interactive vehicle’s display is implicitly based on the idea of reciprocal processes. In , the relation between acausal systems and reciprocal processes was discussed. Applications of reciprocal processes in image processing can be found in –. Some CM sequences were used in – for trajectory modeling and prediction.
Gaussian CM processes were introduced in  based on mean and covariance functions, where the processes were assumed nonsingular on the interior of the index (time) interval.  considered conditioning at the first time of the CM interval.  extended the definition of Gaussian CM processes (presented in ) to the general (Gaussian/non-Gaussian) case. In  we presented definitions of different (Gaussian/non-Gaussian) CM processes based on conditioning at the first or the last time of the CM interval, studied (stationary/non-stationary) NG CM sequences, and presented their dynamic models and characterizations. Two of these models for two important classes of NG CM sequences (i.e., sequences being or over ) are called and models. Applications of CM sequences for trajectory modeling in different scenarios were also discussed. In addition,  provided a foundation and preliminaries for studying the reciprocal sequence from the viewpoint of the CM sequence in .
Reciprocal processes were introduced in  and studied in – and others. – studied reciprocal processes in a general setting.  made an inspiring comment that reciprocal and CM processes are related, and discussed the relationship between the Gaussian reciprocal process and the Gaussian CM process.  elaborated on the comment of  and obtained a relationship between (Gaussian/non-Gaussian) CM and reciprocal processes. It was shown in  that a NG continuous-time CM (including reciprocal) process can be represented in terms of a Wiener process and an uncorrelated NG vector. Following , – obtained some results about continuous-time Gaussian reciprocal processes. – presented state evolution models of Gaussian reciprocal processes. In , a dynamic model and a characterization of the NG reciprocal sequence were presented. It was shown that the evolution of a reciprocal sequence can be described by a second-order nearest-neighbor model driven by locally correlated dynamic noise 
. That model is a natural generalization of the Markov model. Due to the dynamic noise correlation and the nearest-neighbor structure, the corresponding state estimation is not straightforward. Recursive estimation of the sequence based on the model presented in was discussed in –. A covariance extension problem for reciprocal sequences was addressed in . Modeling and estimation of finite-state reciprocal sequences were discussed in –. Based on the results of , in  reciprocal sequences were studied from the CM viewpoint leading to simple and revealing results.
A typical application of reciprocal and CM sequences is in trajectory modeling and prediction with an intent or destination. One group of papers focuses on trajectory prediction without explicitly modeling trajectories. They use estimation approaches developed for the case of no intent/destination. Then, they utilize intent/destination information to improve the trajectory prediction performance (e.g., –). The underlying trajectory model is not clear in such approaches. However, to study and generate trajectories, and analyze problems, it is desired to have a clear model. A rigorous mathematical model of the trajectories provides a solid basis for systematically handling relevant problems. Another group of papers tries to explicitly model trajectories. Due to many sources of uncertainty, trajectories are mathematically modeled as some stochastic processes (e.g., –). After quantizing the state space, – used finite-state reciprocal sequences for intent inference and trajectory modeling with destination/waypoint information. Reciprocal sequences provide an interesting mathematical tool for motion problems with destination information. However, it is not always feasible or efficient to quantize the state space. So, it is desirable to use continuous-state reciprocal sequences to model such trajectories. Gaussian sequences have a continuous-state space. A dynamic model of NG reciprocal sequences was presented in , which is the most significant paper on Gaussian reciprocal sequences. However, as mentioned above, due to the nearest neighbor structure and the colored dynamic noise, the model of  is not easy to apply for trajectory modeling and prediction. For example, in the model of , the current state depends on the previous state and the next state. As a result, for estimation of the current state, prior information (density) of the next state is required. However, such information is not available.
We presented a different dynamic model of NG reciprocal sequence (called reciprocal model) in  from the CM viewpoint. That model has a good structure for trajectory modeling with a destination. More specifically, its structure can naturally model a destination. Also, recursive estimation based on the model of  is straightforward. Like any model-based approach, to use it in application, we need to design its model parameters.
obtained a transition probability function of a finite-state reciprocal sequence from a transition probability function of a finite-state Markov sequence in a quantized state space for a problem of intent inference and trajectory modeling. However, did not discuss if all reciprocal transition probability functions can be obtained from a Markov transition probability function, which is critical for the application considered in . In this paper, we make this issue clear based on our reciprocal model. – obtained a transition density of a Gaussian bridging distribution from a Markov transition density. However, – did not show what type of stochastic process was obtained for modeling their problem of intent inference. In other words, – did not discuss what type of transition density was obtained. In this paper, we address this issue and make it clear. Including reciprocal sequences as a special case (, ), CM sequences are more general for trajectory modeling with waypoints and/or a destination , for example, sequences for trajectory modeling with destination information. However, guidelines for parameter design of a model are lacking. In this paper, application of sequences to trajectory modeling is discussed and guidelines for parameter design of models are presented. Some classes of CM sequences provide models for more complicated trajectories. For example, a - sequence (i.e., a sequence which is over both and ) can be applied to modeling trajectories with a waypoint and a destination. However, a dynamic model of - sequences is not available in the literature. We discuss such application and present a dynamic model for these CM sequences. Systematic modeling of trajectories in the above scenarios is desired but challenging. Different classes of CM sequences make it possible to achieve this goal. Then, for application of these CM sequences, we need to have their dynamic models and design their parameters. This is a main topic of this paper.
The main goal of this paper is three-fold: 1) to present approaches/guidelines for parameter design of , , and reciprocal models in general and their application in trajectory modeling with destination in particular, 2) to obtain a representation of NG , , and reciprocal sequences, revealing a key fact behind these sequences, and to demonstrate the significance of studying reciprocal sequences from the CM viewpoint, and 3) to present a full spectrum of dynamic models from a model to a reciprocal model and show how models of various intersections of CM classes can be obtained.
The main contributions of this paper are as follows. From the CM viewpoint, we not only show how a Markov model induces a reciprocal model, but also prove that every reciprocal model can be induced by a Markov model. Then, we give formulas to obtain parameters of the reciprocal model from those of the Markov model. This approach is more intuitive than a direct parameter design of a reciprocal model, because one usually has a much better intuitive understanding of Markov models. This is particularly useful for parameter design of a reciprocal model for trajectory modeling with destination. In addition, our results make it clear that the transition density obtained in – is actually a reciprocal transition density. A full spectrum of dynamic models from a model to a reciprocal model is presented. This spectrum helps to understand the gradual change from a model to a reciprocal model. Also, it is demonstrated how dynamic models for intersections of NG CM sequences can be obtained. In addition to their usefulness for application (e.g., application of - sequences in trajectory modeling with a waypoint and a destination), these models are particularly useful to describe the evolution of a sequence (e.g., a reciprocal sequence) in more than one CM class. Based on a valuable observation,  discussed representations of NG continuous-time CM (including reciprocal) processes in terms of a Wiener process and an uncorrelated NG vector. First, we show that the representation presented in  is not sufficient for a Gaussian process to be reciprocal (although  stated that it was sufficient, which has not been challenged or corrected so far). Then, we present a simple (necessary and sufficient) representation for NG reciprocal sequences from the CM viewpoint. This demonstrates the significance of studying reciprocal sequences from the CM viewpoint. Second, inspired by , we show that a NG () sequence can be represented by a NG Markov sequence plus an uncorrelated NG vector. This (necessary and sufficient) representation makes a key fact of CM sequences clear and is very helpful for parameter design of and models from a Markov model plus an uncorrelated NG vector. Third, we study the obtained representations of NG , , and reciprocal sequences and, as a by-product, obtain new representations of some matrices, which characterize NG , , and reciprocal sequences.
A preliminary conference version of this paper is , where results were presented without proof. In this paper, we present all proofs and detailed discussion. Other significant results beyond  include the following. The notion of a model induced by a Markov model is defined and such a model is studied in Subsection 3.1 (Definition 3.3, Corollary 3.4, and Lemma 3.1). Dynamic models are obtained for intersections of CM classes (Proposition 3.5 and Proposition 3.6). Uniqueness of the representation of a () sequence (as a sum of a NG Markov sequence and an uncorrelated NG vector) is proved (Corollary 4.3). Such a representation is also presented for reciprocal sequences (Proposition 4.4 and Proposition 4.6). Due to its usefulness for application, the notion of a model constructed from a Markov model is introduced and is compared with that of a model induced by a Markov model (Section 4). As a by-product, representations of some matrices (that characterize CM sequences) are obtained in Corollary 4.2 and 4.5.
The paper is organized as follows. Section 2 reviews some definitions and results required for later sections. In Section 3, a reciprocal model and its parameter design are discussed. Also, it is shown how dynamic models for intersections of CM classes can be obtained. In Section 4, a representation of NG () sequences are presented and parameter design of and models is discussed. Illustrative examples and discussion are presented in Section 5. Section 6 contains a summary and conclusions.
2 Definitions and Preliminaries
We consider stochastic sequences defined over the interval , which is a general discrete index interval. For convenience this discrete index is called time. Also, we consider:
where in (or ) is a dummy variable. denotes the conditional cumulative distribution function (CDF). denotes the Gaussian distribution with mean
) is a dummy variable.is a stochastic sequence. The symbols “” and “ ” are used for set subtraction and matrix transposition, respectively. is a covariance function and . is the covariance matrix of the whole sequence (i.e., ). For a matrix , denotes its submatrix consisting of (block) rows to and (block) columns to of . Also, may denote a zero scalar, vector, or matrix, as is clear from the context.
denotes the conditional cumulative distribution function (CDF).
denotes the Gaussian distribution with meanand covariance . Also, denotes the corresponding Gaussian density with (dummy) variable . The abbreviations ZMNG and NG are used for “zero-mean nonsingular Gaussian” and “nonsingular Gaussian”.
2.1 Definitions and Notations
A sequence is - (i.e., CM over ) iff conditioned on the state at time (or ), the sequence is Markov over (). The above definition is equivalent to the following lemma .
is -, iff for every , , where is the dimension of .
The interval of the - sequence is called the CM interval of the sequence.
We consider the following notation ()
where the subscript “” or “” is used because the conditioning is at the first or the last time of the CM interval.
When the CM interval of a sequence is the whole time interval, it is dropped: the - sequence is called .
A sequence is and a sequence is . For different values of , , and , there are different classes of CM sequences. For example, and - are two classes. By a - sequence we mean a sequence being both and -. We define that every sequence with a length smaller than 3 (i.e., , , and ) is Markov. Similarly, every sequence is -, . So, and -, are equivalent.
A sequence is reciprocal iff conditioned on the states at any two times and , the segment inside the interval is independent of the two segments outside . In other words, inside and outside are independent given the boundaries.
is reciprocal iff for every (), , where is the dimension of .
is reciprocal iff it is -, , and .
A symmetric positive definite matrix is called if it has form or if it has form :
Here , , and are matrices in general. We call both and matrices . A matrix is for and for .
A NG sequence with covariance matrix is: (i) iff is , (ii) reciprocal iff is cyclic (block) tri-diagonal (i.e. both and ), (iii) Markov iff is (block) tri-diagonal.
A NG sequence with its covariance inverse is
(i) - () iff has the form, where
, , and .
(ii) - () iff has the form, where
, , and .
A positive definite matrix is called a - ( -) matrix if () in () has the form.
A ZMNG is , , iff it obeys
where () is a zero-mean white NG sequence, and boundary condition111Note that means that for we have and ; for we have . Likewise for .
or equivalently222 and in are not necessarily the same as and in . Just for simplicity we use the same notation.
A ZMNG is reciprocal iff it satisfies along with or , and
for , or for . Moreover, for , is Markov iff in addition to , we have for , or equivalently for . Also, for , is Markov iff in addition to , we have .
A reciprocal sequence is a special sequence. Theorem 2.10 gives a necessary and sufficient condition for a model to be a model of the ZMNG reciprocal sequence. A model of this sequence was also presented in . In other words, the ZMNG reciprocal sequence can be modeled by either what we call the “reciprocal model” of Theorem 2.10 or what we call the “reciprocal model” of .
Similarly, a Markov sequence is a special sequence. Theorem 2.10 gives a necessary and sufficient condition for a model to be a model of the ZMNG Markov sequence. A model of a Markov sequence is called a “Markov model”. A different model of the ZMNG Markov sequence is as follows.
A ZMNG is Markov iff it obeys
where and () is a zero-mean white NG sequence.
3 Dynamic Models of Reciprocal and Intersections of CM Classes
3.1 Reciprocal Sequences
By Theorem 2.10, one can determine whether a model describes a reciprocal sequence or not. In other words, it gives the required conditions on the parameters of a model to be a reciprocal model. However, it does not provide an approach for designing the parameters. Theorem 3.2 below provides such an approach. First, we need a lemma.
The set of reciprocal sequences modeled by a reciprocal model with parameters , includes Markov sequences.
Now, consider a reciprocal sequence modeled by satisfying with parameters , and boundary condition with parameters , , and , where
meaning that . This reciprocal sequence is Markov (Theorem 2.7). Note that since for every possible value of the parameters of the boundary condition the sequence is nonsingular reciprocal modeled by the same reciprocal model, choice is valid. Thus, there exist Markov sequences belonging to the set of reciprocal sequences modeled by a reciprocal model with the parameters , . ∎
(Markov-induced model) A ZMNG is reciprocal iff it can be modeled by a model – (for ) induced by a Markov model , that is, iff the parameters , , of the model – of can be determined by the parameters , , of a Markov model as
where , , , , where , , are square matrices, and , , are positive definite having the dimension of .
First, we show how – are obtained and prepare the setting for our proof.
Given the square matrices , and the positive definite matrices , there exists a ZMNG Markov sequence (Lemma 2.11):
where is a zero-mean white NG sequence with covariances .
Since every Markov sequence is , we can obtain a model of as
where is a zero-mean white NG sequence with covariances , and boundary condition
Parameters of can be obtained as follows. By , we have . Since is Markov, we have, for ,
and it turns out that , , and are given by – , where we have .
Now, we construct a sequence modeled by the same model as
where is a zero-mean white NG sequence with covariances , and boundary condition
but with different parameters of the boundary condition (i.e., ). By Theorem 2.9, is a ZMNG sequence. Note that parameters of and are the same ( ), but parameters of () and () are different.
Sufficiency: we prove sufficiency; that is, a model with the parameters – is a reciprocal model. It suffices to show that the parameters – satisfy and consequently is reciprocal. Substituting – in , for the right hand side of , we have
and for the left hand side of , we have , where from the matrix inversion lemma it follows that holds. Therefore, is reciprocal. So, equations – with – model a ZMNG reciprocal sequence.
Necessity: Let be ZMNG reciprocal. By Theorem 2.10 obeys – with . By Lemma 3.1, the set of reciprocal sequences modeled by a reciprocal model contains Markov and non-Markov sequences (depending on the parameters of the boundary condition). So, a sequence modeled by a reciprocal model and a boundary condition determined as in the proof of Lemma 3.1 (i.e., satisfying ) is actually a Markov sequence whose is (block) tri-diagonal (i.e., with ). Given this , we can obtain parameters of Markov model (, ) of a Markov sequence with the given as follows. of a Markov sequence can be calculated in terms of parameters of a Markov model or those of a Markov model. Equating these two formulations of , parameters of the Markov model are obtained in terms of those of the Markov model. Thus, for ,
Following to get a reciprocal model from this Markov model, we have –.
What remains to be proven is that the parameters of the model obtained by – are the same as those of the model calculated directly based on the covariance function of . By Theorem 2.9, the model constructed from – is a valid model. In addition, given a matrix (a positive definite cyclic (block) tri-diagonal matrix is a special matrix) as the of a sequence, the set of parameters of the model and boundary condition of the sequence is unique (it can be seen by the almost sure uniqueness of a conditional expectation ). Thus, the parameters – must be the same as those obtained directly from the covariance function of . Thus, a ZMNG reciprocal sequence obeys – with –. ∎
Note that by matrix inversion lemma, is equivalent to .
Note that Theorem 3.2 holds true for every combination of the parameters (i.e., square matrices and positive definite matrices ). By –, parameters of every reciprocal model are obtained from , which are parameters of a Markov model . This is particularly useful for parameter design of a reciprocal model. We explain it for the problem of motion trajectory modeling with destination information as follows. Such trajectories can be modeled by combining two key assumptions: (i) the object motion follows a Markov model (e.g., a nearly constant velocity model) without considering the destination information, and (ii) the joint origin and destination density is known (which can be different from that of the Markov model in (i)). In reality, if the joint density is not known, an approximate density can be used. Now, (by (i)) let be Markov modeled by (e.g., a nearly constant velocity model without considering the destination information) with parameters . can be also modeled by a model –. By the Markov property, parameters of are obtained as – based on . Next, we construct modeled by –. By Theorem 2.9, is a sequence. Since parameters of are arbitrary, can have any joint density of and . So, and have the same model ( and ) (i.e., the same transition ), but can have any joint distribution of the states at the endpoints. In other words,
can have any joint distribution of the states at the endpoints. In other words,can model any origin and destination. Therefore, combining the two assumptions (i) and (ii) above naturally leads to a sequence whose model is the same as that of while the former can model any origin and destination. Thus, model with – is the desired model for destination-directed trajectory modeling based on (i) and (ii) above.
Markov sequences modeled by the same reciprocal model of  were studied in . This is an important topic in the theory of reciprocal processes . In the following, Markov sequences modeled by the same model are studied and determined. Following the notion of a reciprocal transition density derived from a Markov transition density , a model induced by a Markov model is defined as follows. A Markov sequence can be modeled by either a Markov model or a model . Such a model is called the model induced by the Markov model since parameters of the former can be obtained from those of the latter (see or –). Definition 3.3 is for the Gaussian case.
Consider a Markov model with parameters . The model with parameters , , given by – is called the Markov-induced model.
A model is for a reciprocal sequence iff it can be so induced by a Markov model .
See our proof of Theorem 3.2. ∎
By the proof of Theorem 3.2, given a reciprocal model (satisfying ), we can choose a boundary condition satisfying and then obtain a Markov model for a Markov sequence that obeys the given reciprocal model (see –). Since parameters of the boundary condition (i.e., ,