## I Introduction

Millimeter-wave (mmWave) mobile communication is currently a hot topic due to its much wider bandwidth compared with sub-6GHz spectrum. In mmWave channels, the much shorter wavelength leads to much higher propagation attenuation. Fortunately, with the shorter wavelength, the narrower beam and larger array gain can be formed with a given array aperture providing compensation for the propagation loss [1, 2, 3, 4, 5]. However, the narrower beam also implies the requirement of more accurate beam alignment, which can be achieved by beam training at the expense of significant exploration overhead in static or quasi-static scenarios [6, 7, 8, 9, 10, 11]. While in mobile communication, the fast change of wireless channel either in the direction or the channel gain makes beam training inefficient and inaccurate. Therefore, accurate beam tracking with low exploration overhead is crucial for serving fast-moving users in mmWave communication system.

Moreover, the large array gain requirement of mmWave mobile communication implies a large number of antenna elements in the array. However, due to the high cost of AD/DA devices and the high energy consumption of mmWave RF chains, the number of RF chains should be kept much smaller than that of antenna elements, leading to hybrid beamforming or even analog beamforming for cost-effective and energy-efficient systems [4, 12, 3, 13, 5]. In analog beamforming, since only one RF chain is available with a certain set of programmable phase shifters at a certain time (forming a so-called

exploring beamforming vector (EBV)

in this paper), only one direction (or dimension) of the channel can be observed. A randomly selected EBV may most probably lead to missing the real propagation ray and result in an observation of very low signal-to-noise ratio (SNR). What is interesting is that an EBV resulting highest SNR is also inappropriate, since the observation could not provide any information of direction variation. Hence it is crucial to select informative EBVs dynamically to achieve as accurate direction information as possible. Since the EBVs should be designed according to historical observations with noise, this process is really tracking rather than estimation.

Some beam tracking methods have been proposed in [14, 15, 9, 16], which utilize historical exploring directions and observations to obtain current estimates. However, the EBVs are not optimized in those tracking algorithms, which may lead to poor observations. A beam tracking algorithm is proposed in [17], trying to optimize the EBVs, assuming that the channel gain is known. In [18], the authors start to jointly track the channel gain and the beam direction with optimal EBVs. Despite the progress, the proposed algorithm is based on uniform linear array (ULA) antennas, which can only support one-dimensional (1D) beam tracking. While in several mobile scenarios, e.g., dense urban area [19] or unmanned aerial vehicle (UAV) scenarios [20], both horizontal and vertical directions are variable and need to be tracked. The algorithm designed for 1D beam tracking in [18] could not be efficiently applied to the two-dimensional (2D) problems directly.

In this paper, we focus on accurate 2D beam and channel tracking problem. The mmWave wireless system in this paper periodically works in exploration and communication mode. In the exploration stage of one exploration and communication cycle (ECC), the transmitter sends pre-defined pilot sequences. Then the receiver points in one exploring direction in the duration of each pilot sequence and makes an estimate of the channel gain and the direction of the incoming beam at the end of the -th exploration. In the communication stage of one ECC, the beam is aligned in the estimated direction. Based on this structure, we care about the following questions:

1) What is the minimum exploration overhead in each ECC?

2) How to determine the exploring directions?

3) How to track the beam direction and the channel gain for different time-varying channels, e.g., slowly changing channels and fast fading channels?

4) How is the accuracy, convergence and stability of the tracking algorithm?

To answer these questions, we perform theoretical analysis on 2D beam tracking and propose corresponding algorithms to efficiently track different time-varying channels. The main contributions of this paper are summarized as follows:

1) It is proved that for the unique estimate of the channel gain and the 2D beam direction within one ECC, the minimum exploring overhead in each ECC is .

2) Dynamic beam and channel tracking strategies for three different time-varying channels are proposed and optimized. The salient advantages of these tracking algorithms are given below:

i) For channels changing slowly, i.e., with quasi-static channel gain and beam direction (called Quasi-static Case in this paper), optimal exploring directions are derived which are proved to be determined only by the array size, and approach constants as the array size goes large enough. The tracking error of our proposed algorithm is proved to converge to the minimum Cramér-Rao lower bound (CRLB).

ii) For channels with quasi-static beam direction and fast-changing channel gain (called Dynamic Case I in this paper), the scenario where the channel gain satisfies Rayleigh distribution is studied as a special case in this paper. An algorithm for beam only tracking is proposed, and it is proved to converge and achieve the minimum CRLB on beam direction.

iii) For channels with fast-changing beam direction and channel gain (called Dynamic Case II in this paper), simulation result shows that our algorithm can achieve faster and more accurate tracking performance compared with existing methods.

The rest of this paper is organized as follows: The system model is described in Section II. In Section III, the tracking problem with some constraints is formulated. Then the minimum exploration overhead of joint 2D beam and channel tracking is given in theory in Section IV. In Section V and Section VI, tracking problem for Quasi-static Case (Section V) and Dynamic Case I (Section VI) are studied separately. The tracking performance bounds are derived and corresponding tracking algorithms are developed with convergence and optimality analysis. In Section VII, a tracking algorithm is developed for Dynamic Case II. Numerical results are obtained to verify the performance of our proposed tracking algorithms In Section VIII.

*Notations*: We use lower case letters such as and a to denote scalars and column vectors. Respectively, and represent the module and 2-norm of the vector a. Upper case boldface letters, e.g., A, are used to denote matrices. The superscript , , are utilized to denote conjugate, transpose and conjugate-transpose. For a matrix A, its inverse and pseudo-inverse are written as and

. The identity matrix of order

is denoted by . Letrepresent the symmetric complex Gaussian distribution with mean

and variance

, and stand for the real Gaussian distribution with mean and variance . Statistical expectation is denoted by . The real (imaginary) part is represented as . The natural logarithm of a scalar is obtained by and the phase angle of a complex number is written as .## Ii System Model

We consider a mmWave receiver equipped with a planar phased antenna array^{1}^{1}1Note that tracking is needed at both the transmitter and receiver. However, considering the transmitter-receiver reciprocity, the beam and channel tracking of both sides have similar designs. Hence, we focus on beam and channel tracking on the receiver side., as shown in Fig. 1. The planar array consists of antenna elements that are placed in a rectangular area, where () antenna elements are equally distributed along -axis (-axis) with a distance () between neighboring antenna elements^{2}^{2}2To obtain different resolutions in the horizontal direction and vertical direction, the antenna numbers along different directions may not be the same, i.e., [21]. To suppress sidelobe, the antennas may be unequally spaced, i.e., [22].. These antenna elements are connected to the same RF chain via different phase shifters. The system periodically works in exploration and communication mode. The angle of arrival (AoA) and channel gain in each ECC are assumed to be constant and
can be different in different ECCs. In the exploration stage of one ECC, the transmitter sends same pre-defined pilot sequence s, where s contains symbols, i.e., , and is the transmit energy of each pilot sequence. Then the receiver points in one exploring direction in the duration of each pilot sequence and makes an estimate of the channel gain and the direction of the incoming beam at the end of the -th exploration. In the communication stage of one ECC, the beam is aligned in the estimated direction.

### Ii-a Channel Model

In mmWave channels, only a few paths exist due to the weak scattering effect [4]. Because the beam formed by a large array in the mmWave system is quite narrow, the interaction between multi-paths is relatively weak. In other words, the incoming beam paths are usually sparse in space, making it possible to track each path independently [23]. Hence, we focus on the method for tracking one path. Different paths can be tracked separately by using the same method.

In -th ECC, the direction of the incoming beam path is denoted by (), where is the elevation AoA and is the azimuth AoA. Then the channel vector of this path during -th ECC is

(1) |

where is the complex channel gain, is the direction parameter vector (DPV) determined by (),

(2) |

is the steering vector with , and is the wavelength.

### Ii-B RF and Base Band Preprocessing

Let be the EBV for receiving the -th pilot sequence in -th ECC. The entries of are of the same amplitude with , where denotes the -th element of . After phase shifting and combining, the -th received sequence in -th ECC at the baseband output of the RF chain is given by

(3) |

where is a circularly symmetric complex Gaussian random vector with denoting the identity matrix of order. By match filtering on the received sequence , the -th observation in -th ECC is given below:

(4) |

where Step (a) is due to the definition . It is clear that

is a complex Gaussian random variable with

. Let , and denote the exploring beamforming matrix (EBM), the noise vector and the observation vector respectively. Then we can rewrite (4) as follows:(5) |

### Ii-C Tracking Loop

The frame structure of our system is given in Fig. 2.

It is assumed that the beam estimator can output an initial estimate that falls in the main lobe of the DPV:

(6) |

Then our tracking starts from this initial estimate of the channel gain and the DPV.

In the exploration stage of -th ECC, the receiver needs to choose an EBM based on previously used EBMs and historical observation vectors . By applying , we can get the observation vector . Then the estimate of the channel parameter vector is obtained by using all EBMs and observation vectors available. From a control system perspective, is the system state, is the estimate of the system state, the EBM is the control action and is a non-linear noisy observation determined by the system state and control action. Hence, the task of a tracking design is to find the following strategy:

(7) | ||||

(8) |

where denotes the control function in -th ECC while denotes the estimation function in -th ECC.

## Iii Problem Formulation

Let denote a *causal* beam and channel tracking scheme in -th ECC. Then the beam and channel tracking problem is formulated as:

(9) | ||||

s.t. | (10) | |||

where the constraint (10) ensures that

is an unbiased estimate of the channel vector

.Problem (9) is challenging due to the following reasons:

1) It is a constrained partially observed Markov decision process (C-POMDP) that is quite difficult to solve optimally

[24][25].2) There are phase shifts to adjust in each EBV . This makes the optimization of EBV too complicated due to the high degree of design freedom.

3) To obtain in -th ECC, EBMs, i.e., , need to be designed, making it difficult to optimize so many beamforming matrices simultaneously when is not quite small.

4) The time-varying features of the channel in (1) restrict the tracking algorithm and system performance. Hence, it is hard to design a tracking method for a general channel model in (1).

These challenges above make it extremely difficult to solve this problem optimally. Hence, we add some reasonable constraints in this paper to take the first step of optimal tracking policy:

1) EBV constraint. Instead of general phase shifts, we use steering vectors to design the EBVs,

(11) |

where denotes the -th exploring DPV in -th ECC. This ensures that only two variables need to be designed for each EBV.

2) Exploring direction constraint. Although the exploring DPV in (11) can be any form, however, considering the tracking accuracy, it is better to make sure that falls within the main lobe of the DPV in (3). Thus, it is a reasonable choice to use the currently estimated direction to perform a certain offset for exploring. For this purpose, we use such an architecture in this paper. That is, the -th exploring DPV in -th ECC, i.e., in (11), is determined by the previous estimated DPV plus an exploring offset . Considering the design of the offsets that change in different ECCs is also very complicated, we adopt fixed exploring offsets in this paper:

(12) |

Therefore, the EBV in (11) can be rewritten as

(13) |

3) Time-varying channel constraint.
The channel vector in (1) is determined by two parts: the channel gain and the DPV , both of which may change slowly or fast. Therefore, four possible cases exist:

(i) Both the channel gain and the DPV change slowly;

(ii) The channel gain changes fast while the DPV changes slowly;

(iii) The channel gain changes slowly while the DPV changes fast;

(iv) Both the channel gain and the DPV change fast.

These four cases correspond to different practical scenarios, which can be modeled as follows:

### Iii-a Quasi-static Case:

When both and change slowly, e.g., the user keeps static or quasi-static, the channel can be seen as approximately fixed. For the sake of convenience, we assume that , in this case.

### Iii-B Dynamic Case:

For channels that changes fast while changes slowly, e.g., the user moves fast without rotating, the beam direction can be seen as approximately fixed, i.e., , when the distance between transmitter and receiver is very large compared with the wavelength. In order to distinguish with other dynamic scenarios, this case is called Dynamic Case I.

### Iii-C Dynamic Case:

This case requires that the channel gain keeps static or quasi-static while the beam direction changes fast. However, in mmWave channels, the fast change of beam direction usually leads to the fast change of channel gain since the propagation paths change. This case exists only when the antenna array rotates around the first antenna element which keeps static. This is not the usual case and is not studied in this paper.

### Iii-D Dynamic Case:

This case happens in most fast moving scenarios except Dynamic Case I, e.g., the user moves while the receiver array rotates. In order to distinguish with Dynamic Case I, we call it Dynamic Case II.

With the EBV constraint, the exploring direction constraint and the time-varying channel constraint, the beam and channel tracking problem in -th ECC can be reformulated as:

(14) | ||||

s.t. |

## Iv How Many Explorations Are Needed In each ECC?

Before further studying the tracking problem in (14), we will first study the number of explorations needed in this section.

Under the constraint in (11), two explorations in each ECC are sufficient to jointly track the channel gain and 1D beam direction according to [18]. When tracking the horizontal and vertical beam direction simultaneously, it is straight forward that four explorations are feasible by separately using two explorations to track each dimension of the 2D beam direction. However, with four explorations, the channel gain is updated twice in each ECC, possibly leading to redundancy. Since it will cost time resource for each exploration, we may try to answer the question that can we reduce the times of exploration, or what is the minimum number of the explorations required?

Then the following lemma is proposed to help determine the minimum exploration overhead in each ECC:

###### Lemma 1.

*If the EBVs are of steering vector forms, i.e.,* *, and the observation vector in (5) is noiseless, then*

*1) to obtain a unique estimate of the channel parameter vector within one ECC, at least 3 explorations are needed in each ECC;*

*2) to obtain a unique estimate of the DPV within one ECC, at least 3 explorations are needed in each ECC.*

###### Proof.

proof See Appendix A. ∎

Lemma 1 tells us that at least three explorations are required in each ECC no matter we want to jointly estimate and or just estimate . Hence, the minimum exploration overhead in each ECC is , i.e., the EBM .

In the next three sections, we will separately study the tracking problem for Quasi-static Case, Dynamic Case I and Dynamic Case II. In these three cases, a set of same symbols are used for the sake of writing convenience: the EBM , the observation vector and the estimate of channel parameter vector . The values of these symbols may vary for different cases. However, it will cause little confusion as long as noticing the case where these symbols appear.

## V Quasi-static Tracking: Performance Bound, Convergence and Optimality

The beam and channel tracking problem for Quasi-static Case is studied here. In Quasi-static Case where and

, the conditional probability density function of the observation vector

is given by(15) |

In this section, we will first provide the lower bound of tracking error in Quasi-static Case. Then we develop a tracking algorithm and prove this algorithm can converge to the minimum CRLB asymptotically.

### V-a Cramér-Rao Lower Bound of Tracking Error

The Cramér-Rao lower bound theory gives the lower bound of the unbiased estimation error [26]. Based on this, we introduce the following lemma to obtain the lower bound of tracking error in Quasi-static Case:

###### Lemma 2.

*In Quasi-static Case, given *, *the MSE of the channel vector in (14) is lower bounded as follows:*

(16) |

*where* V *is the Jacobian matrix given by*

(17) |

*and the Fisher information matrix* *is given by*

(18) |

###### Proof.

See Appendix B. ∎

The CRLB in (16) is a function of the EBMs . It is hard to optimize so many EBMs simultaneously. Consider any tracking algorithm that can converge to the DPV x. Then given any error , there exists some so that when , . In other words, when , , where is defined below:

(19) |

with denoting the fixed exploring offsets in Quasi-static Case. Hence, as , the asymptotic CRLB in (16) is given by

(20) | ||||

where is the normalized CRLB since the CRLB in (16) decreases with .

### V-B Asymptotically Optimal EBM

Let us consider the optimal EBM . In (21), three EBVs need to be optimized, i.e., three 2D exploring offsets need to be determined. It is hard to get analytical results for such a six-dimensional non-convex problem. Numerical search is a feasible way to obtain the three optimal exploring offsets. However, these optimal exploring offsets may be related to some system parameters, e.g., the channel gain , the DPV x and the antenna array size . Once these system parameters change, numerical search has to be re-conducted, resulting in high complexity. This leads to the question: how do these parameters affect optimal offsets and cause the difference of CRLB. Then the following lemma is provided to answer this question:

###### Lemma 3.

*In Quasi-static Case, the optimal exploring offsets have the following three properties*:

*1)* *are invariant to the channel gain* ;

*2)* *are invariant to the DPV* x;

*3)* *converge to constant values as* :

###### Proof.

See Appendix C. ∎

Lemma 3 reveals that are only related to array size . Hence, the numerical search times can be reduced to one for a particular array size . Numerically, we find later that even if may change for different array sizes, can be used to take the place of as long as and are sufficiently large. Therefore, the numerical search times is reduced to one in the end.

By numerical search in the main lobe of the DPV in (3), we can obtain the asymptotically optimal exploring offsets in TABLE I and Fig. 3.

With these offsets, a general way to generate the asymptotically optimal EBM is obtained to achieve the minimum CRLB as below:

(23) |

By adopting to smaller size antenna arrays, we compare the minimum CRLB and the CRLB achieved by in TABLE I.

As illustrated in Fig. 4, when the antenna number , we can approach the minimum CRLB with a relative error less than by using . Therefore, with , the minimum CRLB is obtained for different beam directions, different channel gains and different antenna numbers when .

### V-C Joint Beam and Channel Tracking

Before we have provided a low-complexity numerical method to design the optimal EBM and obtain the minimum CRLB, given that the DPV x is known. However, in a real tracking problem, the DPV x is unknown and the EBMs need to be adjusted dynamically. In addition, a sequence of optimal beamforming matrices can only tell us what the minimum CRLB is, but it can not tell us which tracking algorithm can achieve the minimum CRLB. In this subsection, we propose a specific tracking algorithm to approach the minimum CRLB.

The proposed tracker is motivated by the following minimization problem:

(24) | ||||

s.t. | (25) | |||

(26) | ||||

(27) |

where the exact value and the gradient of the objective function are not available and can only be observed via the noisy vectors . Hence, (24) is a stochastic optimization problem [28].

A two-layer nested optimization algorithm is proposed to find the solution of (24). In the *inner layer* of (24), we use stochastic Newton’s method [29] to update the estimate, given by

(28) | ||||

where is the step-size for Stochastic Newton’s method, Step (a) is obtained by substituting (15) into (28) and is the Hessian matrix defined below:

(29) |

In the end, the estimate is updated as follow:

(30) |

where Step (b) results from (V-C) and the definition that . Here, is the tracking step-size that will be specified after.

In the outer layer of (24), as to be shown in Section V-D, it is equivalent to minimize the CRLB, i.e., , which leads to that the exploring offsets equal to .

Finally, the proposed tracking algorithm is summarized in Algorithm 1.

(31) |

(32) |

### V-D Asymptotic Optimality Analysis

We care about the following three questions of the proposed algorithm:

*1) Is the tracking algorithm convergent?*

*2) If the tracking algorithm is convergent, can it converge to the channel parameter vector *?

*3) If the tracking algorithm converges to the channel parameter vector , what is the gap between the tracking error and the minimum CRLB in (21)?*

Theorem 10.2.2 in [30, Section 10.2] proves that the tracking procedure in the form of (V-C) can converge to and achieve the minimum CRLB under some constraints. Unfortunately, our tracking algorithm cannot satisfy the necessary requirements for this theorem. Since it is hard to answer the three questions at once, we try to deal with them one by one.

To answer the first question, i.e., the convergence of the proposed algorithm, we can apply Theorem 5.2.1 in [31, Section 5.2.1] to the tracking problem, which gives the conditions that ensure converges. In Theorem 5.2.1 of [31], the stable point is a crucial concept. To study the stable points of our problem, we first rewrite (V-C) as (33):

(33) |

where is the updating direction defined as:

(34) |

This updating direction is a random vector, which can be divided into a deterministic part and a zero mean stochastic part , i.e.,

(35) |

where is defined as follows:

(36) |

and is given by

(37) |

Thus, the tracking procedure in (V-C) can be rewritten as

(38) |

According to [31, Section 4.3], a stable point of satisfies two conditions: 1) ; 2) is negative definite. Hence, we define the stable points set as below: