I Introduction
Millimeter wave (mmWave) communications have received great attention from both academia and industry thanks to the huge available spectrum resources at mmWave frequency bands and the resultant multigigabit data transmission rate [1]. However, to realize such an appealing performance, directional beamforming with a large antenna array is essential [2, 3]
, which requires accurate channel state information (CSI). Thus, mmWave systems usually consume a lot of pilot resources to estimate their CSI, incurring significant training overhead. This condition becomes even worse in timevarying scenarios where the timedemanding channel estimation process has to be invoked periodically to update the stale CSI. Consequently, developing an algorithm that can do channel estimation with low overhead is crucial for mmWave systems, especially when their channels are timevarying.
Since a typical mmWave channel consists of only a few dominant paths and each path can be characterized by its angle of departure (AoD), angle of arrival (AoA), and path gain coefficient, it is sufficient to only estimate these channel parameters rather than the large dimensional channel matrix [4]. Furthermore, as one transmission time interval (TTI) in mmWave networks is as short as 0.1 milliseconds [5], the channel parameters between two successive TTIs can be correlated if no abrupt events such as blockages occur, as reported by [6, 7]. This guarantees that the AoDs, AoAs, and path gain coefficients estimated in previous TTIs can be leveraged to accelerate the current beam training process^{1}^{1}1The AoA and AoD estimation process is often referred to as beam training in mmWave communications.[6]. Such prior information aided beam training is also known as beam tracking, which has been popular in the stateoftheart mmWave researches [8].
In particular, to leverage the sparse nature of mmWave channels [9], a lot of angular space based beam tracking algorithms have been reported, e.g., [10, 11, 12]. As shown in [10], by ignoring the grid quantization errors, the problem of estimating AoAAoD pairs in each beam tracking period can be formulated as a sparse recovery problem. By leveraging an approximate message passing algorithm, the indices of nonzero entries in the sparse virtual channel matrix are estimated, which uniquely reflect the current AoAAoD pairs. Moreover, to guarantee an appealing beam tracking performance at the low SNR range, several directional training beam sequence design approaches were also investigated in [10]. To incorporate the channel dynamics into training beam sequence design, the authors in [11] defined a weighted CramerRao lower bound, and the training beam sequence design problem in this literature was solved by selecting a set of beam codewords that can minimize their defined metric. Furthermore, the authors in [12]
proposed to transform the beam tracking problem into a partially observable Markov decision process (POMDP) problem. Within their POMDP framework, both optimal and suboptimal codebook based training beam sequence design policies were derived. Despite the good performance of those directional training beams in the cases when AoDs change smoothly, their performances will degrade heavily when AoDs change significantly.
To address the aforementioned issue, we develop a noncodebook based training beam sequence design approach for efficient beam tracking. Unlike the highly directional codebook based training beams, the beampatterns of the employed beamformers are more flexible and can be optimized to adapt to different beam tracking scenarios. To be more specific, we consider a dynamic mmWave multipleinput singleoutput (MISO) channel and impose a discrete Markov process assumption on the evolution of its AoD. To incorporate the AoD transition information into beam tracking, we propose to leverage the maximum a posteriori (MAP) criterion for AoD estimation in each beam tracking period. Because it is infeasible to derive an explicit form of the resultant tracking error probability (TEP), we turn to its upper bound, which possesses a closedform expression. We rely on this upper bound as the objective function to optimize the training beam sequence employed in each beam tracking period. Considering the complicated objective function and the unit modulus constraints imposed by analog phase shifters, we resort to the particle swarm algorithm (PSA) to solve the formulated optimization problem, earning a set of efficient training beams for AoD tracking^{2}^{2}2In this paper, we use the two terms AoD tracking and beam tracking interchangeably.. Simulation results validate the superiority of the proposed training beam sequence design approach, particularly in the cases when the AoD changes significantly between two consecutive beam tracking periods.
Notations: Bold uppercase letters (e.g., ), bold lowercase letters (e.g., ), and nonbold letters (e.g.,
) are respectively used to indicate matrices, vectors, and scalars.
, , , , , , , and denote the th row, th column, th entry, conjugate, determinant, transpose, conjugate transpose, and rank of , respectively. is the norm operator.is the identity matrix;
is the standard basis vector; and is the zero vector. represents a complex Gaussian vector with mean and covariance matrix . is the probability of event conditioned on event . indicates proportionality up to a constant scale factor. and are the real and imaginary parts of , respectively.Ii System Model
We consider a mmWave MISO system, where one base station (BS) equipped with antennas communicates with a singleantenna user^{3}^{3}3Motivated by [13], the proposed beam tracking approach can be directly extended to multiuser scenarios leveraging narrowband frequency tones. To be more specific, by assigning each radio frequency chain a unique frequency tone to individually serve one user, the multiuser beam tracking problem is decoupled into multiple singleuser beam tracking problems.. Referring to [9], since only a few scattering clusters exist between mmWave transceivers, the considered MISO channel can be thus modeled as [14]
(1) 
where denotes the total number of propagation paths and the two variables , respectively denote the gain coefficient and the normalized AoD of the th path, . is termed antenna array response vector. By assuming a uniform linear array at the BS, we can express as
The relationship between and the physical AoD, denoted by , , is given by
where is the spacing between two adjacent antenna elements at the transmitter, and is the carrier wavelength. Moreover, we assume that and , such that , and there exists a onetoone mapping between and .
Following [4, 12, 11], in this paper we assume that ’s are taken from a uniform grid of points^{4}^{4}4As reported in [4], [12], and [16], the grid quantization errors only lead to minor performance loss when ., given by
(2) 
where to guarantee a fine angular resolution. To reflect the dynamic scenario, we further assume that changes within following a discrete Markov process. An example of the AoD transition function between two successive TTIs is given by
(3) 
where , and the parameter characterizes the variation extent of between two TTIs. In particular, implies that changes slowly since the probability of a large difference between and is negligible when ; on the contrary, corresponds to the cases where can change significantly between two consecutive TTIs. Additionally, the parameter is a boundary, and is the normalization coefficient. Furthermore, as in [12] and [15], we assume that the complex path gain coefficient and it changes independently from one TTI to another. Until now, we have introduced the dynamic modeling of the timevarying AoDs, and in the following we elaborate on the employed beam training protocol.
(10)  
As shown in Fig. 1, we focus on one transmission frame which consists of TTIs. In the initial TTI, since no useful information about the AoDs can be leveraged, we assign channel uses for conventional beam training. While for the subsequent TTIs, since the AoDs estimated in the previous TTI and the AoD transition function can be leveraged, we only assign channel uses for beam tracking, where . If the welldesigned training beams can achieve a comparable performance against the conventional ones, more channel uses are therefore left for data communication, leading to a higher throughput. Last, since various conventional beam training algorithms have been investigated in the open literature, e.g., [11], we thus only consider the beam tracking periods in this paper.
Iii Training Beam Sequence Design
From this section, we focus on the singlepath channel model, i.e., , and introduce the proposed training beam sequence design approach. Note that the path index is dropped for brevity.
Iiia Formulated Sparse Recovery Problem
Considering the th beam training period, , if we denote the th pilot symbol by , and the corresponding transmit beamforming vector by , the received signal is then given by
(4) 
where is the additive Gaussian noise and
is the training signaltonoise ratio (SNR). We then introduce a beam codebook matrix
(5) 
and transform into its beamspace representation, given by
(6) 
where the vector has only one nonzero element whose location is uniquely determined by according to our previous ongrid assumption. By substituting (6) into (4), we rewrite the latter as
(7) 
Stacking the received pilot symbols into a vector , we then construct a sparse recovery problem, given by
(8) 
where is termed sensing matrix with , and .
IiiB Proposed MAP Estimation
In this section, we propose to use the MAP criterion to estimate the position of the nonzero entry in . Without loss of generality, we assume that the current AoD is , such that according to (6). Thus, we can recast (8) as
(9) 
We then compute the posterior probability of
, given by (10), where is omitted in since it is a constant in each entry of , and . To facilitate subsequent computations, we normalize the summation ofto one. Moreover, to incorporate the previous tracking errors, we compute the prior probability of
in a recursive fashion, given by(11) 
where is the transition probability from to , or equivalently, from to . According to the MAP criterion, the estimated location of the nonzero element in is given by
(12) 
According to [17], (12) provides the optimal estimation of AoD in each beam tracking period. Last, once AoD estimation is completed, the user feeds back to the BS for subsequent data communication and the next beam tracking period.
(14)  
(15)  
IiiC Beam Tracking Error Probability
Before introducing the proposed training beam sequence design approach, we first discuss the metric used for evaluating the performance of a particular set of training beams. In accordance with our assumption in (9) that the th entry in is nonzero, under the proposed MAP criterion in (12), the probability of successfully estimating the current AoD can be expressed as
Recall that . Taking each possible case of into consideration, the expected probability of successfully estimating is then given by
(13) 
The TEP in the th TTI is then given by (14), where is due to .
(18)  
IiiD A ClosedForm Upper Bound for TEP
Expressing (14) in an explicit form is challenging, and hence we turn to its upper bound, given by (15), where is due to the Boole’s inequality and . To derive an explicit expression for , we need to first determine the distribution of .
Recall that when
. By leveraging the eigenvalue decomposition of
, given bywe can rewrite as
and transform into
where . Denoting the eigenvalue decomposition of by , we can further transform into
where and it follows since
is an unitary matrix. Until now, we have successfully transformed
into(16)  
where is the th largest diagonal entry in . Furthermore, by rewriting as
(17) 
we can realize that , and consequently , which implies that has at most two nonzero diagonal entries. Moreover, since , both and
follow an exponential distribution with mean
. Thanks to this observation, we can derive a closedform expression for , as detailed below.Case 1: If , , we can express as (18), where .
Case 2: and ,
Case 3: and ,
Case 4: and ,
IiiE Formulated Optimization Problem
As shown in (10) and (11), the training beams used in the th beam tracking period also influence the th beam tracking period via . Nonetheless, we can still focus on one beam tracking period and optimize its associated training beams to minimize the TEP, since the more accurate the current beam tracking result is, the better the subsequent beam tracking result can be.
Therefore, we employ as the objective function and formulate the training beam sequence design problem as
(19)  
s.t.  
where (IIIE) accounts for the unit modulus constraint imposed by analog phase shifters [2]. Due to the complicated objective function and the highly nonconvex constraint (IIIE), we resort to PSA^{5}^{5}5
PSA is an evolutionary computation technique and it searches for optima by updating generations rather than by gradient. As such, PSA can be used to solve very difficult problems such as the ones with nondifferentiable objective functions. Despite the powerfulness of PSA, it cannot guarantee an optimal solution is found. For more details about PSA, refer to
[18]. to solve .Remark: The proposed beam tracking approach for the singlepath channel can be readily extended to multipath scenarios. Specifically, we can track the multiple paths in an iterative manner, and only one path is addressed within each iteration. Following the idea of successive interference cancellation, the contributions of the paths that have been found in the previous iterations should be subtracted from the received signals before finding new path.
Iv Numerical Results
In this section, we provide numerical results to evaluate the performance of the proposed beam tracking approach. In simulations, the BS is equipped with antennas and all these antennas are connected to a single radio frequency chain, leading to an analogonly architecture. Following [10] and [12], we assume that one transmission frame consists of TTIs and the AoD in the first TTI is accurately estimated via conventional beam training. The other parameters are defined as follows: , , and .
The performance of our proposed beam tracking approach is shown in Fig. 2, where four benchmark approaches are also provided for comparison: 1) MAP with two TEP based directional training beams, i.e., the two beam codewords in that minimize ; 2) Particle filtering (PF) estimator with two TEP based directional training beams; 3) MAP with two CRB based training beams [19]; and 4) Orthogonal matching pursuit (OMP) with directional training beams [11]. From this figure, we easily observe that the proposed TEP based training beams perform better than the CRB based training beams and the TEP based directional training beams. In particular, when , by leveraging MAP estimation and the proposed TEP based training beam sequence design, we only need to assign two channel uses for beam tracking. The resultant TEP performance is comparable to that of the beam cycling approach which consumes channel uses for conventional beam training at the beginning of each TTI.
The TEP performances of our proposed beam tracking approach in different timevarying scenarios are shown in Fig. 3, where the performances of the CRB based training beams and the TEP based directional training beams are also provided. From this figure, we immediately observe that our proposed TEP based training beam sequence design approach performs better than the other two benchmark schemes across the whole range of , which further validates the superiority of our proposed TEP based training beam sequence design approach. We can also observe from this figure that the performance gap between the TEP based training beams and the TEP based directional training beams becomes larger as increases. The reason is that directional training beams have narrow beamwidths and hence the maximum number of AoD candidate directions they can probe is limited compared to the training beams generated by . When is large, which means that the AoD can change significantly between two consecutive TTIs, since the directional training beams can cover only a few AoD candidate directions, the current AoD will be missed if it is located in the uncovered directions. In contrast, the training beams generated by are much more flexible. When the AoD changes significantly, the training beams output by are expected to have wide beamwidth in order to cover more AoD candidate directions, thereby significantly reducing the beam tracking error.
The performances of our proposed beam tracking approach under different SNR scenarios are provided in Fig. 4, where the performances of the TEP based directional training beams are also provided for comparison. From this figure, it is observed that our derived TEP upper bound decreases following a similar trend as the simulated TEP in the second TTI when increasing the training SNR from 10dB to 25dB, which validates the effectiveness of as the objective function to optimize the training beam sequence. We can also observe from this figure that our proposed TEP based training beams perform much better than the TEP based directional training beams and their performance gap becomes larger as SNR increases. Moreover, as shown in Fig. 4, the performance of the TEP based directional training beams has an “error floor”. As explained earlier, this is due to the narrow beamwidth of directional training beams. Last, for the proposed beam tracking approach, we also observe that its performance gap between the second and the last TTIs becomes narrower as the training SNR increases from 10dB to 25dB.
V Conclusions
In this paper, we considered a mmWave MISO channel and imposed a discrete Markov process assumption on the evolution of its timevarying AoD. To incorporate the AoD transition information into beam tracking design, we introduced the MAP criterion for AoD update in each beam tracking period. We derived a closedform upper bound for the TEP, and leveraged it as the objective function to optimize the training beam sequence used in each beam tracking period. Numerical results validated the good performance of the proposed beam tracking approach for sparse mmWave channels.
References
 [1] M. Xiao, S. Mumtaz, Y. Huang, et al, “Millimeter wave communications for future mobile networks,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 9, pp. 19091935, 2017.
 [2] R. W. Heath, N. G. Prelcic, S. Rangan, et al, “An overview of signal processing techniques for millimeter wave MIMO systems,” IEEE Journal of Sel. Topics in Signal Proc., vol. 10, no. 3, pp. 436453, 2016.
 [3] W. Roh, J. Seol, J. Park, et al, “Millimeterwave beamforming as an enabling technology for 5G cellular communications: theoretical feasibility and prototype results,” IEEE Communications Magazine, vol. 52, no. 2, pp. 106113, 2014.
 [4] A. Ahmed, O. E. Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” IEEE Journal of Sel. Topics in Signal Proc., vol. 8, no. 5, pp. 831846, 2014.
 [5] R. Ford, M. Zhang, M. Mezzavilla, et al, “Achieving ultralow latency in 5G millimeter wave cellular networks,” IEEE Communications Magazine, vol. 55, no. 3, pp. 196203, 2017.
 [6] A. Patra, L. Simic, and P. Mahonen, “Smart mmwave beam steering algorithm for fast link reestablishment under node mobility in 60 GHz indoor WLANs,” ACM International Symposium on Mobility Management and Wireless Access, pp. 5362, 2015.
 [7] J. Palacios, D. De Donno, and J. Widmer, “Tracking mmwave channel dynamics: fast beam training strategies under mobility,” IEEE Conference on Computer Communications, pp. 19, 2017.
 [8] Y. Wang, Z. Wei, and Z. Feng, “Beam training and tracking in mmWave communication: a survey,” arXiv preprint, available online: https://arxiv.org/abs/2205.10169, 2022.
 [9] M. R. Akdeniz, Y. Liu, M. K. Samimi, et al, “Millimeter wave channel modeling and cellular capacity evaluation,” IEEE Journal on Selected Areas in Communications, vol. 32, no. 6, pp. 11641179, 2014.
 [10] Q. Duan, T. Kim, H. Huang, et al, “AoD and AoA tracking with directional sounding beam design for millimeter wave MIMO systems,” IEEE Annual International Symposium on PIMRC, pp. 22712276, 2015.
 [11] S. H. Lim, S. Kim, B. Shim, and J. W. Choi, “Efficient beam training and sparse channel estimation for millimeter wave communications under mobility,” IEEE Transactions on Communications, vol. 68, no. 10, pp. 65836596, 2020.
 [12] J. Seo, Y. Sung, G. Lee, and D. Kim, “Training beam sequence design for millimeterwave MIMO systems: a POMDP framework,” IEEE Transactions on Signal Processing, vol. 64, no. 5, pp. 12281242, 2016.
 [13] L. Zhao, D. W. K. Ng, and J. Yuan, “Multiuser precoding and channel estimation for hybrid millimeter wave systems,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 7, pp. 15761590, 2017.
 [14] S. Noh, M. D. Zoltowski, and D. J. Love, “Multiresolution codebook and adaptive beamforming sequence design for millimeter wave beam alignment,“ IEEE Transactions on Wireless Communications, vol. 16, no. 9, pp. 56895701, 2017.
 [15] D. Zhang, A. Li, M. Shirvanimoghaddam, et al, “Codebookbased training beam sequence design for millimeterwave tracking systems,” IEEE Trans. Wireless Commun., vol. 18, no. 11, pp. 5333  5349, 2019.
 [16] D. Zhang, A. Li, C. Pradhan, et al, “Training beam sequence design for multiuser millimeter wave tracking systems,” IEEE Transactions on Communications, vol. 69, no. 10, pp. 69396955, 2021.
 [17] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for online nonlinear/nonGaussian Bayesian tracking,” IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 174188, 2002.

[18]
R. Poli, J. Kennedy, and T. Blackwell, “Particle swarm optimization,”
Swarm Intelligence vol. 1, no. 1, pp. 3357, 2007.  [19] N. Garcia, H. Wymeersch, and D. T. M. Slock, “Optimal precoders for tracking the AoD and AoA of a mmWave path,” IEEE Trans. Signal Processing, vol. 66, no. 21, pp. 57185729, 2018.