# Robust Adaptive Beam Tracking for Mobile Millimetre Wave Communications

Millimetre wave (mmWave) beam tracking is a challenging task because tracking algorithms are required to provide consistent high accuracy with low probability of loss of track and minimal overhead. To meet these requirements, we propose in this paper a new analog beam tracking framework namely Adaptive Tracking with Stochastic Control (ATSC). Under this framework, beam direction updates are made using a novel mechanism based on measurements taken from only two beam directions perturbed from the current data beam. To achieve high tracking accuracy and reliability, we provide a systematic approach to jointly optimise the algorithm parameters. The complete framework includes a method for adapting the tracking rate together with a criterion for realignment (perceived loss of track). ATSC adapts the amount of tracking overhead that matches well to the mobility level, without incurring frequent loss of track, as verified by an extensive set of experiments under both representative statistical channel models as well as realistic urban scenarios simulated by ray-tracing software. In particular, numerical results show that ATSC can track dominant channel directions with high accuracy for vehicles moving at 72 km/hour in complicated urban scenarios, with an overhead of less than 1%.

## Authors

• 4 publications
• 26 publications
• 8 publications
• 7 publications
• 5 publications
• 5 publications
• 6 publications
02/06/2018

### How to Mobilize mmWave: A Joint Beam and Channel Tracking Approach

Maintaining reliable millimeter wave (mmWave) connections to many fast-m...
06/29/2019

### Fast Accurate Beam and Channel Tracking for Two-dimensional Phased Antenna Array

The sparsity and the severe attenuation of millimeter-wave (mmWave) chan...
12/29/2019

### A Machine Learning Solution for Beam Tracking in mmWave Systems

Utilizing millimeter-wave (mmWave) frequencies for wireless communicatio...
02/16/2021

This paper discusses the opportunity of bringing the concept of zero-sho...
06/29/2019

### Fast Accurate Beam and Channel Tracking for Two-dimensional Phased Antenna Arrays

The sparsity and the severe attenuation of millimeter-wave (mmWave) chan...
12/02/2020

### Millimeter Wave Communications on Overhead Messenger Wire: Deep Reinforcement Learning-Based Predictive Beam Tracking

This paper discusses the feasibility of beam tracking against dynamics i...
09/29/2021

### Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning

We investigate the problem of wireless beam tracking on mmWave bands wit...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## I Introduction

Millimetre wave (mmWave) frequencies have been recognized as one of the most promising means to increase wireless capacity for evolving 5G and beyond cellular networks and have attracted considerable attention [19, 5, 25, 11, 33, 16, 8, 35, 24, 29, 27]. However, employment of mmWave frequencies in mobile communications comes with significant challenges. In particular, the severe path loss in the mmWave band needs to be compensated via beamforming techniques at Base Station (BS) and/or User Equipment (UE). To ensure high beamforming gain, the directions of the beams used in transmission must align well with the dominant path of mmWave channels at all times.

Establishing and maintaining such accurate alignment is usually obtained via an initial beam search followed by beam tracking/management [6]. Initial beam search on the one hand is primarily concerned with finding good beamforming directions with minimal access delay. This is generally achieved by conducting a full search over the entire angular space and hence long training times are required [15, 3]. A wide range of solutions have been proposed using techniques such as spatial scanning [26, 32, 15, 13] and compressive sensing [3, 2, 21].

Beam tracking on the other hand is concerned with maintaining high alignment accuracy between the BS/UE beams and the directions of the dominant channel paths. This is a challenging task because the directions of dominant channel paths often change rapidly due to mobility and radio environmental variations. Moreover, good mmWave beam tracking algorithms are required to (i) provide consistent high tracking accuracy and therefore high beamforming gain in various fading scenarios and (ii) use a minimum amount of tracking overhead to achieve (i). To attain (i), tracking errors must be kept small throughout the process of communications, not just on the average. This is because short-term large excursions of error can lead to disruptive losses of beamforming gain, with consequent packet losses or even loss of the entire link. As far as (ii) is concerned, it is crucial that the rate of taking tracking measurements is matched to the mobility level. Using a high tracking rate on the one hand limits the time available for data transmission and tracking multiple users simultaneously, not to mention the need for UEs to track multiple BSs. Using a low tracking rate, on the other hand, will lead to frequent loss of track for high-mobility UEs, incurring costly re-alignments.

There has already been considerable effort devoted to the problem of mmWave beam tracking [9, 12, 18, 28, 31, 22, 10, 34, 36]. In [9, 12], the search scheme of the IEEE 802.11ad standard is described. This scheme tracks mmWave channels by scanning using three beams in each tracking update: the data beam currently adopted and its two adjacent beams from a pre-determined codebook. The beam yielding the strongest measurement is selected as the next data beam which finishes the tracking udpate. This method avoids a full search of the codebook and thus reduces the tracking overhead as compared to initial alignment methods [26, 32, 15, 13]. However, its accuracy is limited by the resolution of the codebooks [36].

In [18], it is proposed to use pilots sent in data slots to track angular changes in a hybrid mmWave beamforming architecture. Here, assumptions on the prior distributions of the angular changes are needed in order for the algorithm to operate. Reference [28] proposed a tracking algorithm that requires only one measurement per tracking update. This is taken at a direction perturbed with respect to the current beam direction. However, this method requires knowledge of the instantaneous strength of the path being tracked, which cannot be obtained accurately because of the unknown path angle and channel fading. References[31, 22, 10]

developed Kalman Filter (KF) based algorithms to track the time-varying directions. As has been shown in

[22, 10], such KF-based methods can suffer from error propagation and eventual loss of track, i.e., the angle of the dominant path is no longer covered by the beam used for data transmission.

Reference [34] proposed a method that requires two measurements taken at different directions in each tracking update. This method is not suitable for urban scenarios as it was developed for a mmWave UAV BS communicating with a moving ship on the sea surface, which is a line-of-sight (LOS) scenario. Furthermore, it relies on knowledge of the height of the UAV BS and the velocity of the ship to determine the measurement directions. The algorithm in [36] takes two measurements using beams perturbed in angle with respect to the current beamforming direction. Beam updates are calculated based on the amplitude-comparison monopulse [17, Chapt. 9]. While its main contribution lies in the consideration of hardware impairments, important algorithmic parameters such as the perturbing distance were not optimised. As we will show later, the algorithm in [36] can suffer frequent and significant losses of beamforming gain. Moreover, none of these works provides a systematic treatment as to how much tracking overhead is needed for a given mobility level nor do they consider how to adapt the tracking overhead to different mobility levels in real-time.

In this paper, we propose a new mmWave beam tracking method which we call Adaptive Tracking with Stochastic Control (ATSC) where both BS and UE employ analog beamforming. As we will show later, ATSC is able to adapt its tracking overhead to different mobility levels, that is to use a suitable amount of overhead for a certain mobility level. Meanwhile, it can provide high tracking accuracy, as measured not only by long-term Signal-to-Noise Ratio (SNR) average but also by metrics that account for worst-case variations.

The core part of ATSC is a novel stochastic control (SC) based algorithm we developed to control the beamforming direction to achieve high alignment accuracy. In SC, two measurements taken with two sampling beams that are perturbed with respect to the direction of the current beam (data beam) are needed to get beam updates, similar to [36, 34]. By analysing the statistical behaviour of the measurement signals, we design a simple yet effective beam updating mechanism. This mechanism enables us to analytically derive upper bounds on the probability of losing tracking (PLT) as the channel changes, i.e., the probability that the pointing error goes outside a given fraction of the beam width between two updates. In order to achieve consistently high tracking accuracy in both high-mobility and low-mobility scenarios, we further optimise the parameters for the SC algorithm, including the perturbing distance, pilot sequence length and the stepsize of SC update, based on the derived PLT upper bound and the mean absolute error (MAE) between the angle of the dominant path and the beam angle updated by SC.

Finally, we introduce a method to adaptively set the tracking rate according to the underlying speed of angular variations. This angular speed can be estimated directly from the angle updates produced by SC, owing to its high tracking accuracy. With the SC algorithm and the adaptive tracking rate, ATSC is able to maintain high beam alignment accuracy without suffering error propagation. It adaptively reduces the tracking rate (hence the overhead) when the dominant path angle varies slowly to save overhead, and increases the tracking rate when the angle varies quickly (due to faster mobility). The performance of the proposed ATSC protocol is evaluated via an extensive set of numerical results, including performance evaluations in complicated urban scenarios simulated via high-accuracy ray-tracing software

[20].

The remainder of the paper is organized as follows. Section II presents the system model and describes the beam tracking protocol at high-level. Section III elaborates the SC algorithm, the core of ASTC, while Section IV details the full ASTC framework. Numerical simulations are presented in Section V and conclusions are finally drawn in Section VI.

## Ii System Model and Beam Tracking Protocol

### Ii-a Beam Tracking Protocol

We consider a mmWave communication system where a BS and a UE are both equipped with one-dimensional Uniform Linear Arrays (ULAs) with isotropic antennas, a single RF chain and both use analog beamforming. The BS and UE attempt to track the beams used for data transmission such that the data beams are always aligned well with the time-varying dominant path of the channel. Fig. 1 (a) is a time frame structure of our proposed tracking protocol. The time frame consists of times slots with beam tracking measurements, i.e., tracking slots, and time slots without tracking measurements, i.e., data slots. As illustrated in Fig. 1 (a), the tracking operations start after a full search over the entire angular space, where the full-search corresponds to an initial beam alignment [13] when a UE wakes up from idle mode. Immediately after the full-search, tracking measurements are collected frequently, i.e., every time slot becomes a tracking slot. In each of these slots, the beam used for data transmission, i.e., the data beam, is updated. After a number of tracking updates, the angular speed is estimated, based on which, the tracking rate (how frequently tracking updates are performed) is determined so as to match the tracking overhead to the mobility level. To deal with the possibilities of sudden blockage of the existing path being tracked and failure to track the path of interest, realignment is also considered in the proposed protocol, see Fig. 1(a).

Fig. 1(b) gives a more detailed view of a tracking slot, where the tracking measurements take a fraction of the slot, leaving the remaining parts for other uses such as data transmission. To track the beam at either the BS or the UE, two measurements are collected, as indicated in Fig. 1 (b). If both the beams at the BS and the UE are to be tracked, four measurements are needed in total for ULA beams: two consecutive measurements at the BS and two consecutive measurements at the UE. For each side, the two measurements are taken by measuring pilot sequences, sent by the other side using the current data beam. The two measurements are collected using two sampling beams which point in two directions that are slightly perturbed with respect to the direction of the data beam used prior to the tracking measurements, as illustrated in Fig. 1 (c).

###### Remark 1.

The above mentioned protocol can be generalised to track multiple users/beams. Suppose the users are served in a time-division manner. Then the tracking measurements of different users can be scheduled to be collected at different times. For instance, a tracking slot now can contain tracking measurements from different users such that the BS can update its beam directions for the users. Similar treatments can be made to track multiple paths. As we will show later, the overhead due to tracking a single path for one user is small. This makes it possible to support more complicated scenarios including multi-user, multi-BS and multi-path.

Before proceeding to Section III to elaborate the control algorithm, we present the channel model and the signal model in the remaining part of this section.

### Ii-B Channel Model

We consider a block fading model where the channel remains fixed during a slot. Denote as the channel matrix between the BS and the UE at slot , where and are the number of antennas at the BS and the UE, respectively. Following a geometric model, can be represented as:

 H(t)=L∑l=1gl(t)u(~ϕl(t))v†(~ψl(t)), (1)

where is the number of multipath components, is the complex coefficient of the -th path in slot , and and are the angles of the

-th path with respect to the UE and the BS, respectively. Vectors

and are the array response vectors at the UE and the BS, respectively. For ULA, it is often convenient to consider and . Using these notations, the array response vectors can be represented as:

 u(ϕ)=[1,e−j2πdλϕ,⋯,e−j2πdλ(NR−1)ϕ]T and v(ψ)=[1,e−j2πdλψ,⋯,e−j2πdλ(NT−1)ψ]T,

where denotes vector transpose, is the antenna spacing and the carrier wavelength.

### Ii-C Signal Model and Tracking Measurements

We go on to present the signal model and explain how measurements are obtained under our tracking algorithm. Denote the pilot sequence used in the beam tracking process as , where with being the transmit power. As explained in Section II-A, for the BS to track the beam, the UE sends the pilot sequence twice in the uplink using the UE data beam used in the previous time slot, while the BS takes the corresponding measurements using two beamformers that are perturbed slightly with respect to the BS data beam. The same process applies in the downlink such that the UE updates its data beam. We consider ULA beamformers at both the UE and the BS for both the data beams and the sampling beams. UE beam pointing in direction and BS beam pointing in direction can be represented as:

 w(Φ)=1√NR[1,e−j2πdλΦ,⋯,e−j2πdλ(NR−1)Φ]T and f(Ψ)=1√NT[1,e−j2πdλΨ,⋯,e−j2πdλ(NT−1)Ψ]T.

We note that the ULA beams achieve the highest possible gain if the beamformer direction is aligned perfectly with the path direction, assuming there is only one path.

Let and be the directions of the data beams for the BS and UE, respectively, and and be the angular perturbation applied at the BS and the UE, respectively. In the -th slot, for BS beam tracking, two measurements are taken with beamformers pointing to . Similarly, we can define for UE beam tracking. Denote further

 Track BS beam:  h±T=w†(Φ(t−1))H(t)f(ψ±(t)) (2)

and

 Track UE beam:  h±R=w†(ϕ±(t))H(t)f(Ψ(t−1)) (3)

as the effective channels of the BS and the UE measurements, respectively. The measurements collected by the BS and the UE can be represented as:

 y±T(t)=h±T(t)s+z±T(t),    and    y±R(t)=h±R(t)s+z±R(t), (4)

respectively, where and

are vectors of circularly symmetric Gaussian random variables with zero mean and variance

. By matched filtering the measurements with pilot squence , we obtain

 Q±T(t)=2σ2||s||2∣∣s†y±T(t)∣∣2,    and    Q±R(t)=2σ2||s||2∣∣s†y±R(t)∣∣2. (5)

It follows directly from (4) that and are independent non-central chi-square (

) random variables with Degrees of Freedom (DOF) equal to 2 and non-centrality parameters

 η±T(t)=2nPT∣∣h±T(t)∣∣2σ2,    and    η±R(t)=2nPT∣∣h±R(t)∣∣2σ2. (6)

The above results are used in the derivation of the update formula for our SC algorithm.

## Iii Stochastic-Control Algorithm for Beam Tracking

In this section, we present the core part of the proposed beam tracking framework, the stochastic-control (SC) algorithm. For ease of exposition, we use beam tracking at the BS side to describe the algorithm, assuming that the UE beam is fixed and does not require tracking. We also assume that the channel has a single path when developing the SC algorithm, i.e., , and drop the subscript for convenience. We note that the single-path channel model assumption is reasonable because narrow beams are used for tracking and mmWave channels tend to be sparse in the angular domain [1]. In this case, the chance that two or more channel paths falling into the same BS and UE beam is small [36]. Therefore, the single-path model provides sufficient accuracy even in the presence of multiple paths. We finally assume in the section that the path coefficient  does not change over time111In the numerical experiments, we will examine the performance of the proposed algorithm using multi-path fading channels. and the error between the UE data beam and the angle at the UE is constant: .

Under the assumptions made above, the effective channel gains of the two BS measurements can be represented as:

 |h±T(t)|2 =|w†(Φ(t−1))H(t)f(ψ±(t))|2=|gw†(Φ(t−1))u(ϕ(t))v†(ψ(t))f(ψ±(t))|2 (7) =|g|2GR(ϵR)GT(Ψ±(t)−ψ(t)) (8)

where are the gains of the two sampling beams at the channel direction. is defined similarly. Clearly, . Moreover, only depends the relative error .

Fig. 2 plots with respect to the relative error . It can be seen that and is about dB lower than . A larger relative angular error will lead to even more significant loss of beamforming gain, which is to be avoided in beam tracking.

Let be the initial angular error prior to the tracking measurements at slot . Since , . With the notations defined above, the non-centrality parameters of in (6) can be further represented as:

 η±(t)=2nPT|g|2GR(ϵ)GT(ϵψ(t)±ΔT)σ2=2nγGR(ϵR)GT(ϵψ(t)±ΔT), (9)

where is the SNR of the dominant path without beamforming.

### Iii-a Stochastic-Control algorithm outline

Consider the outcome of an exhaustive initial search based on a set of equally spaced ULA beams [15, 13], with the space between two beam centres equal to in the domain. A successful initial search finds the best beam from the codebook, whose angular difference from the channel is smaller than that of any other beams in the codebook. In this case, the angular error (after a successful initial search) is within and the beamforming gain will be close to the maximal gain that can be achieved.

In the process of beam tracking, it is also desirable to maintain the angular error within . Suppose this goal is achievable, then it is reasonable to consider that prior to a tracking update, . The control algorithm will then update the beam angle from to at slot , ideally to reduce the post-tracking error

 ϵ′ψ(t)=Ψ(t)−ψ(t) (10)

such that .

The two measurements provide useful information to fulfil this goal. To see this, consider without loss of generality, then . If are not too far from , i.e., being still in , then it can be seen that .222If , it can be seen that is close to zero as the beamforming gain is very small. With a sufficiently long pilot sequence, i.e., is large enough, there is a high chance that . In this case, to achieve the goal that , the angular update can take the form: , where is a function that if and if . With this update, the post-tracking error becomes and will be reduced in magnitude if is properly chosen.

In this paper, we take a simple yet effective choice of : , where is a stepsize parameter controlling the size of the update/correction. Following this choice, itself is a random variable, whose expectation is the difference between the two corresponding non-centrality parameters

 η+T(t)−η−T(t)=2nγGR(ϵR)[GT(ϵψ(t)+ΔT)−GT(ϵψ(t)−ΔT)],

as can be seen from (9). As the channel is being tracked and data transmitted, it is reasonable to consider that an accurate estimate of the post-beamforming SNR prior to the tracking measurements at slot is available, i.e., . Note that the path strength, or equivalently the pre-beamforming SNR is generally not available due to the unknown angular error and .

Using to normalise , we obtained a modified as follows:

 h(Q+T(t)−Q−T(t))=δTQ+T(t)−Q−T(t)Γ(t−1), (11)

where we call function the drift function and is a stepsize.

Before continuing, we note that for any choice of the stepsize , it is possible that , because are random. When , an over correction occurs because the initial error . We therefore apply a truncation to and obtain the following beam updating formula for the control algorithm

 Ψ(t)=Ψ(t−1)+~h(Q+T(t)−Q−T(t)), (12)

where

 ~h(Q+T(t)−Q−T(t))=min(∣∣h(Q+T(t)−Q−T(t))∣∣,BT)⋅z(h(Q+T(t)−Q−T(t))), (13)

where is the sign function. With tracking update , the post-tracking error defined in (10) becomes:

 ϵ′ψ(t)=ϵψ(t)+~h(Q+T(t)−Q−T(t)), (14)

Eq. (13) specifies the control algorithm except that the sampling angle and the step size have not yet been specified.

###### Remark 2.

The linear drift given (11) is motivated by considering in the noiseless case (), where and become deterministic and so does the drift function: . With proper choices of parameter and , the noiseless drift after truncation, i.e., , is close to . In this case, following (14), the error after update will be small: . Fig. 3 plots the drift given by (13) when , with respect to the initial error . Four parameter choices are presented: , , and . The drift and the initial error plotted are normalised by such that they are in . It can be seen that the two parameter choices and provide a drift close to the curve . However, with or , the gap between the drift and is fairly large, leading to higher expected error after tracking updates. This result shows that it is possible to achieve low tracking error by taking a linear drift. It also shows that parameter and are crucial to the accuracy of the tracking algorithm.

### Iii-B Choosing the Control Parameters

In this subsection, we investigate the impact of the parameter pair , together with the pilot sequence length , on the performance of the tracking algorithm. We consider two important performance metrics: the mean absolute error (MAE) and the probability of losing track (PLT) between two tracking corrections. The MAE is an expectation of the alignment error immediately after the tracking updates, i.e., , assuming that the initial error is random. To define PLT, we will say that the algorithm has lost track if the beam pointing error is outside the interval , i.e., the channel direction is no longer covered by the mainlobe of the data beam. In what follows, we will use these metrics to choose the parameters.

#### Iii-B1 Mean Absolute Error E{|ϵ′ψ(t)|}

In computing the MAE, we consider uniform random in

which corresponds to the error distribution immediately after the initial alignment or a realignment where an exhaustive search based on a fixed codebook is used and the path angle is uniformly distributed in

(see [13] as an example). The MAE captures the tracking accuracy immediately after tracking updates and also reflects the tracking accuracy in static scenarios where angles do not change after beam updates. We note that the tracking accuracy in static scenarios is an important performance metric for the beam tracking algorithm, because a good tracking algorithm should not only be able to track angles when there are changes, but also be able to keep the angle close to the true one when there are no changes.

Fig. 4 plots the MAE normalised by , i.e., , as a function of , for a number of values of the pilot sequence length , when (Fig. 4 (a)) and (Fig. 4 (b)). The results are obtained for the one sided case, where only the BS has multiple antennas and UE has an omni antenna. In this case, .

As can be seen from Fig. 4, the best that minimises the MAE, i.e., , generally depends on the pilot sequence length and also the perturbing distance . For instance, for the sampling distance of as shown in Fig. 4(a), appears to be around to for the various values of . With , varies more significantly with respect to , where for , is around while for , is around .

Fig. 5 presents the optimised MAE obtained with for a range of values on and . The curve with shows that the smaller the perturbing distance, the smaller the MAE, which appears to suggest that a smaller is better than a larger one. However, with finite up to , as shown by the curves in Fig. 5, the MAE first decreases and then increases when increases. The high MAE for small (e.g., ) when is small results from the difficulty in differentiating two beams that are pointing in similar directions and thus have comparable and high gains. The high MAE for large , e.g., , comes with a similar difficulty, which is caused by two beams that are both far apart from the dominant path and hence have low gains.

Fig. 5 also shows that the best perturbing distance varies with : for a larger , the best perturbing distance is smaller and vice versa. For instance, for , the best perturbing distance is around . When , the best perturbing distance becomes close to . However, for up to , it appears that is a robust choice as the corresponding MAE is close to the minimum MAE that can be achieved by other values of . Recall that in Fig. 4(a), the best step size is when . Therefore, is a good parameter choice.

The fact that is a good parameter choice does not rule out other choices also being good choices. For instance, has similar MAEs to . With this observation, we also consider the PLT that captures the performance in dynamic scenarios, in order to further optimise the parameter choices.

#### Iii-B2 Probability of Losing Track between two tracking corrections

In this subsection, we will discuss the PLT between any two consecutive tracking corrections where the initial error before the first correction is within . Let and be the slot indices of the two consecutive tracking slots. We also consider that the angular change between the start of slot and the start of slot is Lipschitz continuous in time.

As before, we denote the initial beam alignment error before the correction, by . Let denote the random correction as obtained from (13), the angular change and . A loss of track occurs either when the error immediately after correction goes outside the interval , i.e., , or the error goes outside the interval at an intermediate time , i.e., , but does not occur. Since the angular change is Lipschitz continuous, there exists a constant such that . Therefore, a necessary condition to is that , since . Denote . Thus, the probability of exiting the interval , either initially or during the update interval, over all possible initial errors, can then be upper bounded by

 Ja=supϵψ(t)∈[−BT,BT][P{Aϵψ(t)}+P{Bϵψ(t)(a)⋂¯¯¯¯¯¯¯¯¯¯¯¯Aϵψ(t)}], (15)

where denotes the complementary event to .

The PLT bound depends on , which reflects the maximum angular change during the update interval. (For fixed update interval, the parameter reflects the speed of angular change.) also depends on other algorithm parameters including , and the pilot sequence length . Moreover, the bound is tight in the sense that given , we can find an initial error such that the PLT is arbitrarily close to . In other words, the bound represents the worst case PLT starting from an arbitrary initial error.

To further understand the impact of mobility on PLT, we present the following lemma.

###### Lemma 1.

Under the fixed single-path model, is monotonic increasing for .

The proof of Lemma 1 follows immediately from the definitions of and and because the is being taken. This lemma shows that a larger caused by a higher mobility and/or a longer time duration between two updates (thus a lower tracking rate), will lead to a higher . Our numerical evaluations of show that for moderate or high values of , e.g., , is dominated by and that is sensitive to the choices of as well as the pilot sequence length . Thus it should be used to further determine these parameter choices.

Fig. 7 and Fig. 7 graph for two different pairs of , i.e., and , respectively. From both figures, it can be seen that a higher leads to a higher for the same . Also decays as the pilot sequence length increases. Moreover, by comparing Fig. 7 with Fig. 7, it can be seen that for the same , for is lower than that for . This means that for the same tracking overhead, parameter choice leads to lower PLT than , thus is a better choice.

Overall, through the MAE and PLT analysis above, it can be concluded that the choice of for the SC algorithm provide lower mean absolute error after tracking update and lower PLT. Therefore, in the simulations in Section V, we use this particular parameter combination. We note that the suggested parameter combination has taken into account the number of antennas through and hence is applicable to ULAs with any number of antennas. For each tracking update, it is required to compute (12), whose computational complexity is dominated by computing and given in (5) and is on the order of .

###### Remark 3.

The results presented in Fig. 7 show that can be reduced by 1) increasing with fixed and 2) decreasing with fixed. Since can be reduced by performing tracking updates more frequently, i.e., increasing the tracking overhead, and increasing also increases the tracking overhead, it can be concluded that generally decreases as the overhead increases. Increasing the overhead, however, will reduce the available time for data transmission. Thus, as expected, there are tradeoffs between the tracking overhead and the system performance.

In fact, there are also tradeoffs between the tracking rate, or equivalently , and the pilot length , since the same amount of overhead can be achieved by different combinations of and . Our analysis in Appendix A, assuming an idealised scenario where the angular change speed is constant and known and there is no fading, shows that it is important to control the tracking rate such that is neither too small nor too large. The results in Appendix A show that moderate , e.g., in the range of , requires a minimum amount of overhead to meet a performance target measured by the probability of losing track over slots. Setting the tracking rate very high or very low such that is very small or very large will increase the amount of overhead required. This is because for smaller , the required pilot length tends to decrease more slowly and hence the increased tracking rate (as a result to keep smaller) becomes the dominant factor of the overhead. This effect becomes obvious when . Very large such as requires very long pilots which pushes up the overhead, as shown in Table V in Appendix A. Furthermore, is about the maximum angular change that can be tracked reliably with reasonably low overhead.

Nevertheless, the tradeoff analysis in Appendix A shows the importance of adjusting the tracking rate such that is always in the right range, according to the time-varying angular change speed. For this reason, in Section IV-B, we present a method to set the tracking rate adaptively based on estimates of the angular change rate.

## Iv The Algorithm of Adaptive Tracking with Stochastic Control (ATSC)

In Section III, we have presented the stochastic control algorithm for beam tracking and discussed the parameter choices. For the complete ATSC framework as explained in Section II, there are two additional elements: (1) realignment and (2) adaptive tracking frequency. In what follows, we elaborate these elements in order.

### Iv-a Realignment

We begin by presenting the criteria of realignment. A realignment is needed when the path to be tracked is lost or blocked or disappears. In these circumstances, the SNR after applying the data beams may degrade significantly compared to the previous time slots when the path is on track. Hence, it is reasonable to initiate a realignment when the SNR after beamforming has dropped significantly compared to a reference level. To formally present our adopted criteria of realignment, let be the latest slot that the beams were updated (by either a tracking update or a realignment) before slot and be the set of slots prior to slot . A realignment is initiated at slot if

 Γ(t)

where is a threshold in dB and s are also measured in dB.

Once the realignment condition is triggered, the BS and the UE will attempt to re-discover the dominant path at slot via methods such as [15] or [13]. There can be possible refinements to these methods to better meet the need for realignment, which are left for future works.

It is expected that when the angle of the dominant path changes fast, tracking measurements are to be collected more frequently, and vice versa. With this purpose, the tracking rate is set adaptively according to the angular speed estimated based on beam angle updates. The algorithm works as follows.

Denote as the set of slots that tracking updates are performed, where is the number of slots per tracking interval. At each tracking slot , the BS and/or the UE angles are updated. The algorithm uses these updated angles to estimate the angular change rate per slot:

 αψ=Tf−2∑k=0|Ψ(t−ktf)−Ψ(t−(k+1)tf)|tf(Tf−1)  and  αϕ=Tf−2∑k=0|Φ(t−ktf)−Φ(t−(k+1)tf)|tf(Tf−1). (17)

Suppose the targeted angular change per tracking intervals are and at the BS and the UE, respectively. Then the expected number of slots per tracking interval, i.e., , can be updated as: and for the BS and the UE, respectively. As we assume that the BS and UE tracking are performed at the same rate and at the same slots, can be updated as:

 tf←max{min(⌊βTαψ⌋,⌊βRαϕ⌋),1}. (18)

Note that is calculated according to the targets and , which must be chosen to be consistent to the tracking capability of the SC algorithm and according to the beam width. As we have shown in Appendix A, angular change up to per tracking interval is about the limit that can be tracked reliably by the SC algorithm with reasonably low overhead and accurate information of the angular change rate. We therefore will consider in Section V-B and when testing the ATSC. We will also consider lower targeted values of and , which will give ATSC more tolerance to the errors of the angular speed estimates. We finally summarise ATSC in Table I.

## V Experiments and Discussions

In this section, we consider three experiments to evaluate the performance of the proposed ATSC framework for mmWave beam tracking. In the first experiment, we consider a single-sided beam tracking problem where only the BS has multiple antennas (hence requiring beam tracking). We use this experiment to examine the stochastic control (SC) algorithm for beam tracking as detailed in Section III, further verify our analysis and make comparisons to one state-of-the-art algorithm in the literature. In the second experiment, we consider more complicated scenarios where both the BS and the UE have multiple antennas, hence both requiring beam tracking, and also consider fading channels. We use this experiment to test the proposed ATSC framework (with adaptive tracking frequency and realignment) and to determine the tracking targets and . In the third experiment, we evaluate the performance of ATSC in a more realistic scenario where the mmWave channels are generated using Wireless Insite [20], a commercial ray-tracing simulator that is known for its high accuracy and has been widely used in the literature [30, 14, 23].

### V-a One-Sided Tracking with Fixed Single-Path Channel

In this experiment, BS has antennas and UE has antennas, thus only the BS angle is to be tracked. The UE is moving around the BS along a circle centred at the BS such that the SNR without beamforming at the UE is fixed at dB and the angle is changing at a constant speed such that , where is the constant angular speed. The channel has only one path with fixed channel gain.

For each trial of the experiment, we simulate slots, within each of which the angle is assumed fixed. Tracking measurements and beam updates are performed every 10 slots, e.g., at slot . In this case, the angular change per tracking interval is .

The initial error at the start of each trial, i.e, at slot 1, is assumed uniformly distributed in , where we remind the reader that is half of the ULA beam width. This initial error corresponds to the error after an initial beam alignment via scanning a fixed set of beams [13]. We repeat the trials for 10,000 times to collect summary statistics. The results of the proposed SC algorithm are compared to the tracking algorithm developed in [36], which also uses signals collected from two sampling beams to update the beam. Note that a perturbing distance of was suggested by [36] while the SC algorithm adopts based on the MAE and PLT analysis. The algorithm in [36] also has a different updating mechanism.

For each slot , the post-beamforming SNR is calculated as: , where is the variance of the effective noise. Let and be the optimal beams defined by the directions . Clearly, the maximum achievable post-beamforming SNR at slot is .

Fig. 8

presents the empirical cumulative distribution function (CDF) of the average post-beamforming SNR, i.e.,

, for the proposed SC algorithm and the one in [36] for different angular velocities. Pilot length is for both methods. Fig. 8 also presents the empirical CDF of the post-beamforming SNR upper bound (computed assuming no tracking error) for benchmarking. It can be seen that higher angular velocities lead to larger losses of beamforming gain with respect to the maximum, for both methods. It can also be seen that the proposed algorithm provides noticeably higher average SNR than the algorithm in [36] at all the three angular velocities. For instance, for (or equivalently ), the proposed algorithm provides a dB of gain over the algorithm in [36] ( dB vs. dB).

We emphasise that although the gain on the average SNR does not sound remarkable, it actually mitigates disruptive outages (due to significant losses of SNR) in the process of tracking. As a representative example, we present a snapshot of one of the 10,000 trials in Fig. 9, where . As can be seen from Fig. 9(a), the proposed algorithm maintains steady post-beamforming SNRs that are close to the upper bound. The tracking error is almost always within , as can be seen from Fig. 9(b). The algorithm in [36] incurred deep losses of the post-beamforming SNRs, which are disruptive and can lead to losses of data packets as well as possible need for realignment.

Fig. 10 shows the complimentary CDF of , which is the fraction of time that the tracked SNR is at least dB lower than the maximum SNR, as measured in a single run of slots. Here is the indicator function and . In Fig. 10, and . It can be seen that for the algorithm from [36], is significant for a good fraction of the simulation runs. For instance, about of the 10,000 runs have . This means that it is common to see occurrences for which the tracked SNR is lower than the maximum possible SNR by at least 3 dB. In contrast, for the proposed algorithm, is almost always zero, suggesting that such occurrences are very rare.

### V-B Two-Sided Search with Fading Channels

In this experiment, we examine the proposed ATSC framework in a two-sided beam tracking problem, where . Since , it is natural to consider that . For notational convenience, we denote and (the targeted angular change per tracking interval) from this subsection. The purpose of this experiment is to examine the impact of parameter on the overall performance of beam tracking.

In the simulations, the channel is assumed to have one dominant path and follow a Rician fading model [7]. The BS and UE angle of the dominant path change every slot with the same velocity : and . The average pre-beamforming SNR of the channel is set to dB and the channel is generated every slot following the time-varying angle and the Rician model. Two Rician -factors are considered, i.e., dB and dB, which correspond to LOS scenarios [1] and NLOS scenarios [1], respectively. The realignment threshold is set to dB, which is approximately the loss of beamforming gain if both the BS and UE have tracking errors around .

Fig. 11 presents the results of one typical simulation run (of 1,000 slots) in LOS with and . Fig. 11 (a) presents the SNR of the tracked channel (the red curve), the SNR upper bound achieved by the optimal directional beamforming (the black curve) and a baseline SNR (the blue curve) achieved by the best BS-UE beam pair from fixed codebooks that have and DFT beams. Fig. 11 (b) presents the number of slots per tracking interval, i.e., in (18). Fig. 11 (c) and (d) presents the directions of the true path and the tracked beam at Tx and Rx, respectively.

From Fig.