# Line-of-Sight MIMO for High Capacity Millimeter Wave Backhaul in FDD Systems

Wireless backhaul is considered to be the key part of the future wireless network with dense small cell traffic and high capacity demand. In this paper, we focus on the design of a high spectral efficiency line-of-sight (LoS) multiple-input multiple-output (MIMO) system for millimeter wave (mmWave) backhaul using dual-polarized frequency division duplex (FDD). High spectral efficiency is very challenging to achieve for the system due to various physical impairments such as phase noise (PHN), timing offset (TO) as well as the poor condition number of the LoS MIMO. In this paper, we propose a holistic solution containing TO compensation, PHN estimation, precoder/decorrelator optimization of the LoS MIMO for wireless backhaul, and the interleaving of each part. We show that the proposed solution has robust performance with end-to-end spectral efficiency of 60 bits/s/Hz for 8x8 MIMO.

## Authors

• 8 publications
• 1 publication
• 13 publications
10/25/2017

### A New Look at MIMO Capacity in the Millimeter Wave

In this paper, we present a new theoretical discovery that the multiple-...
09/30/2019

### Large Intelligent Surface for Positioning in Millimeter Wave MIMO Systems

Millimeter-wave (mmWave) multiple-input multiple-output (MIMO) system fo...
03/21/2018

### Statistical Properties and Variations of LOS MIMO Channels at Millimeter Wave Frequencies

Measurement results for millimeter wave LOS MIMO systems are presented w...
06/23/2021

### Wavenumber-Division Multiplexing in Line-of-Sight Holographic MIMO Communications

The ultimate performance of any wireless communication system is limited...
11/02/2017

### Single-Carrier Modulation versus OFDM for Millimeter-Wave Wireless MIMO

This paper presents results on the achievable spectral efficiency and on...
02/03/2021

### Analyzing the Impact of Molecular Re-Radiation on the MIMO Capacity in High-Frequency Bands

In this paper, we show how the absorption and re-radiation energy from m...
12/16/2017

### Joint Data-Aided Carrier Frequency Offset, Phase Offset, Amplitude and SNR Estimation for Millimeter-Wave MIMO Systems

This work is devoted to solve the problem of estimating the carrier freq...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## I Introduction

Dense small cells and femtocells have been proposed to enhance the spatial reuse and boost the capacity of hot spots in future wireless networks [1, 2]. As a result, future wireless networks may comprise a substantial number of small base stations, and the backhaul transmission will be a critical capacity and cost bottleneck. In this paper, we focus on designing a very high spectral efficiency mmWave backhaul transmission using MIMO technology for future wireless networks.

MIMO has been widely used in 3GPP long term evolution (LTE) and 5G systems as a key technique to enable the capacity requirement of the wireless access network. Leveraging the rich scattering in the non-line-of-sight (NLoS) fading channels, spatial multiplexing for MIMO systems has been widely studied in [3]. However, there are various technical challenges in adopting the MIMO technique for improving the spectral efficiency of the wireless backhaul. First, in the context of wireless backhaul, the propagation channel is primarily LoS due to the high carrier frequency together with the narrow beamwidths being used. With limited scattering in the propagation environment, the LoS MIMO channel responses can be highly correlated, leading to a rank deficient channel matrix. Nevertheless, by optimizing the antenna placements, the capacity and rank of the LoS MIMO channel can be improved with a large separation between the antennas in the arrays [4, 5]. However, it is costly to install arrays with large antenna-separation, especially for MIMO systems. This poses the technical challenge in adopting the MIMO technique for improving the spectral efficiency of the wireless backhaul since low antenna-separation arrays will lead to a poor condition number of the LoS MIMO channel and the spatial multiplexing benefit can be jeopardized.

The second challenge is brought by the existence of physical impairments in the system, such as TO and PHN. The presence of timing offsets among the transmit and receive antenna will bring severe inter-symbol interference (ISI), which will degrade the performance of the wireless backhaul. As a result, this poses a very stringent requirement for accurate timing synchronization. Additionally, PHN in MIMO systems will bring two penalties in the MIMO system, namely the demodulation penalty caused by phase distortion to signal constellation and the multi-access interference (MAI) penalty induced by the coherency loss of the precoder and decorrelator.111The precoder and decorrelator are used to mitigate interference between spatially multiplexed data streams. The precoder and decorrelator were designed based on the estimated CSI at the beginning of a frame and were fixed throughout the frame. However, due to PHN, there is an increasing mismatch between the precoder/decorrelator and the effective channel (incorporating the PHN) and hence, the MAI increases. These physical impairments usually exist in practical wireless backhaul, the effect of which is more severe in mmWave systems [6] due to the high carrier frequency. Unlike wireless access applications, the target spectral efficiency of wireless backhaul is very high. These physical impairments can be the performance bottleneck for wireless backhaul and pose a very stringent requirement for accurate estimation and compensation of the TO and PHN. Unfortunately, the spatial multiplexing in MIMO systems will cause severe MAI, which is a huge hurdle for the compensation of these impairments. Furthermore, dual-polarization is usually adopted in wireless MIMO backhaul [7] to overcome the space limitation, which allows two orthogonal streams from each dipole antenna to travel in the same bandwidth at the same time (one vertically (V mode) and one horizontally (H mode)). In such a dual-polarized system, high cross-polarization discrimination (XPD) of each dipole antenna is required to reduce the interference between cross-polar link transmission. However, high XPD together with severe MAI will lead to inaccurate cross-polar link measurements for the TO and PHN, making it more challenging to estimate and compensate these physical impairments.222The cross-polar link represents the data link between the V mode (H mode) at the transmitter and the H mode (V mode) at the receiver in a dual-polarized MIMO (See Fig.1), which is very weak in high XPD systems and will be corrupted by the MAI caused by the signal from other links. Thus, a novel and holistic solution considering the physical impairments for mmWave LoS MIMO backhaul with dual-polarization is required.

There are many existing works that considered different subsets of the aforementioned physical impairments [8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 6]. TO estimation has been well studied in [9, 10, 11]. In [9], the authors propose a maximum likelihood (ML) estimator to the TO in the MIMO system with an impractical assumption that the TOs are identical across all the antennas. [10, 11] propose estimation methods of different TOs over antennas. However, there are strict assumptions of TO in these works. Specifically, [10] assumes TOs are small and within one symbol time and [11] only considers TOs that are multiples of symbol time. In addition, these works only estimate the sum-offsets 333Sum-offsets are the effective TOs in the transmitter-receiver links. For example, in an 8x8 MIMO, we have altogether 16 unknown TOs for all the antennas, but these works only estimate the 64 combinations of the sum-offsets between the transmitter and receiver antennas. There are only 16 freedoms in these 64 sum-offsets, but such information is not exploited in these works. and the preamble used in this work may fail due to the severe MAI induced by the high XPD in the dual-polarized LoS MIMO channel.

PHN estimation algorithms in single-input single-output (SISO) systems are widely studied in [12, 13, 14, 19]. However, these methods cannot be applied to MIMO systems since the received signal at each antenna will be influenced by multiple PHNs in the transmitters, which needs to be jointly estimated. For MIMO systems, PHN estimation and compensation algorithms are proposed in [15, 16, 17] with data-aided and decision-directed sum-PHNs444Sum-PHNs are the effective PHNs in the transmitter-receiver links. For example, in an 8x8 MIMO, we have altogether 16 unknown PHNs for all the antennas, but these works only estimate the 64 combinations of the sum-PHNs between the transmitter and receiver antennas. There are only 16 freedoms in these 64 sum-PHNs, but such information is not exploited in these works. estimators. However, [15] assumes that the channel is perfectly known at the receiver and [16, 17] have ignored the effect of TOs at the transceiver. In addition, the high XPD in dual-polarized MIMO is not considered, hence, the sum-PHNs extracted from the estimated channel will have large estimation errors for transceiver links with a small amplitude (e.g., the cross-polar links). In these works, after estimating the sum-PHNs, the authors design the decorrelator to equalize the aggregated channel and sum-PHN matrix. These compensation schemes impose a heavy computational burden at the receiver since the PHN is varying over time and the decorrelator needs to be updated along with the variations of the PHN. In [6], the authors propose a method to estimate and compensate the per-antenna PHN for MIMO, but they do not track the time-varying PHN and they ignore the interplay with the precoder and decorrelator. Hence, existing works are not applicable when various impairments are considered in mmWave LoS MIMO. The precoder and decorrelator are key components in MIMO systems to achieve high spectral efficiency for wireless backhaul, since they enable stable multi-stream transmissions. There are a lot of existing works on MIMO precoder/decorrelator design [20, 21, 22]. [20] proposes a unified linear transceiver design framework, in which the optimal decorrelator is fixed as the Wiener filter and the design problem can be formulated as convex optimization problems in terms of precoder under different design criteria. In [21, 22], the authors consider the weighted MMSE optimization and propose low complexity algorithms based on alternative optimization. However, in all these works, the physical impairments such as TO and PHN have been ignored. This may be justified for wireless access applications but these physical impairments cannot be ignored for wireless backhaul applications due to the very high target spectral efficiency. Recently, many works have considered the physical impairments issue. Hardware impairments aware (HIA) MIMO transceivers are proposed in [23, 24, 25]. However, the effect of physical impairments is simply modeled as additive Gaussian noise to each antenna, which is an over-simplification of the impairments due to the TO and the PHN in the MIMO systems. In summary, these existing solutions cannot be applied in our case because of very different target spectral efficiency and the practical considerations of the LoS MIMO.

In this paper, we adopt a holistic approach and propose a practical solution for a dual-polarized LoS MIMO in mmWave backhaul addressing the aforementioned physical impairments. The solution is not a trivial combination of existing techniques as each component is inter-related. Based on the proposed solution, we can achieve very high spectral efficiency (e.g., 60 bits/s/Hz with 8x8 MIMO) and fully unleash the potential of the LoS MIMO in mmWave backhaul applications. The following summarizes our contributions.

• Decentralized Spatial Timing Estimation and Compensation: We propose a low complexity spatial timing estimator that has a similar complexity to the cross-correlation approach [8, 11] but it is capable of utilizing spatial information across different antennas to estimate the per-antenna TO. The proposed spatial TO estimator only requires local information and hence it can be implemented separately at the transmitter and receiver. To overcome the strong MAI induced by spatial multiplexing of dual-polarized LoS MIMO channels, we propose new preamble sequences with improved auto-correlation and cross-correlation rejection. Based on this, we propose a decentralized timing compensation scheme where the transmitter and receiver compensate for the TO based only on local information without any explicit signaling.

• Decentralized Phase Noise Estimation and Compensation with Decision Feedback. We propose a low complexity per-antenna PHN estimation and compensation scheme, which enables compensation for both the phase distortion to the received symbols and the MAI caused by the loss of coherence of the precoder and decorrelator due to the drifting of PHN.555Note that such per-antenna compensation is not possible if one uses conventional PHN estimators, which only estimate and compensate the effective sum-PHN. To reduce the pilot overhead, we adopt the decision feedback and regression-based fusion to enhance the PHN estimation quality. The proposed PHN estimator has low complexity and requires local information only. Based on this, we propose a decentralized PHN compensation scheme, which compensates the per-antenna PHN locally at the transmitter and the receiver.

• Robust MIMO Precoder and Decorrelator Design: We exploit the MIMO precoder and decorrelator to suppress the inter-symbol interference (ISI) and MAI induced by the physical impairments of the TO and the PHN. We show that the precoder and decorrelator can significantly alleviate the requirement of the TO compensation and PHN compensation to achieve very high spectral efficiency for mmWave backhaul applications. The design is formulated as a nonconvex optimization problem. By exploiting structures in the ISI and MAI, we transform the problem into a tractable form and propose a low complexity solution using alternative optimization techniques.

This paper is organized as follows. In Section II, we present the system model, including the mmWave dual-polarized LoS MIMO channel model, the TO model, and the PHN model, as well as the data path in the transmitter and receiver. In Section III, V and IV, the proposed timing synchronization, PHN estimation and compensation, as well as the procoder and decorrelator scheme, respectively, are illustrated. The numerical simulation results and the corresponding discussions are provided in Section VI. Finally, Section VII summarizes the whole work.

Notations:

In this paper, lowercase and upper bold face letters stand for column vectors and matrices, respectively. The operations

## Ii System Model

In this paper, we consider mmWave MIMO wireless backhaul in a dual-polarized FDD system. Both the transmitter and the receiver comprise an antenna array mounted on a pole with a primarily LoS channel in between. Each of the transmit (and receive) antennas has a local oscillator that is loosely synchronized to a master control unit. The illustration of the system is shown in Fig.1. We shall elaborate on each part of this system below.

### Ii-a LoS MIMO Channel Model

In mmWave backhaul, a terrestrial link usually exists. Therefore, we consider the Rummler model [26] in this work, which is an LoS propagation model with a single NLoS path caused by the terrestrial reflection between two fixed antenna towers. For a single-input-single-output (SISO) system, the impulse response of the Rummler model can be specified as

 h(t)=δ(t)+βej2πf0τdδ(t−τd), (1)

where the first term represents the LoS path and the second term represents the NLoS path, is the propagation delay associated with the difference in the propagation time between the LoS path and the NLoS path and is assumed to be within one symbol time. denotes the notch frequency, which will be anywhere in the spectral efficiency. We consider the minimum-phase case of the Rummler model (), where we set the channel gains , with denoting the notch depth, which relates the power of the LoS path to that of the NLoS paths.

Generalizing the SISO Rummler model to a dual-polarized 666In this paper, we consider the H mode and V mode of one dipole antenna as two different antennas. For example, in Fig.1, there are 4 dipole antennas at each side of the transmitter and the receiver, and the system is considered to be a MIMO system. MIMO system, the channel model is given by

 ~H(t) =HLoS(t)+HNLoS(t), =HLoS(t)+β⊙R⊙HLoS(t−τd) (2)

where and is the random magnitude attenuation and the random phase rotation matrix, respectively, caused by reflection.Since and are multiplied element-wisely to the LoS channel response, the expression for the NLoS propagation in (2) can represent any NLoS propagation. The LoS channel is highly deterministic [27] and can be modeled as

 HLoS=Hxp⊗JN2×M2⊙HA, (3)

where contains the cross-polar gains, is the all ones matrix and is the array response matrix between transmitter and receiver with and flat-panel dual-polarized MIMO antenna arrays. Note that in this paper, we assume spherical curvature of the propagating waves, thus the array response is given by

 [HA]i,j =e−j2πλdi,j,

where is the distance between the -th receiver antenna and the -th transmit antenna. A similar propagation assumption and the array response matrix can be found in [6], in which the dual-polarized array is not considered.

With dual-polarized MIMO antennas, XPD of the antennas has a significant influence on the system performance. In general, high antenna XPD is desired to leverage the benefits of dual-polarization [28, 29]. However, high XPD also causes very weak cross-polar transmission links, i.e., the link between the horizontal mode to the vertical mode of each dipole antenna. This will cause a great challenge for the estimation of the TO and PHN since the physical impairments are different on each antenna and the estimation for these physical impairments requires measurements from all the transceiver links. Moreover, the intensity of the MAI will increase as the number of antennas of the MIMO system increases, which will further distort the cross-polar measurements. For example, in an MIMO with a typical dB, the power of the MAI will be dB of the desired signal for a cross-polar link. We show this influence on TO estimation in Fig. 2 with the timing correlation metric in a cross-polar link using the traditional Zadoff–Chu (ZC) sequence. It is observed that as the number of antennas increases, the severe MAI will induce several peaks of similar intensity in the timing correlation metric. As a result, there might be large timing estimation errors due to the false peaks in the metric.

### Ii-B Timing offset Model

In mmWave MIMO wireless backhaul, the transmit antennas are not collocated and hence the clock of each transmit antenna/receive antenna is only roughly synchronized to a master clock. Let and denote the normalized TO of the clock at the -th transmit antenna and -th receive antenna, respectively, w.r.t. the symbol duration . Without loss of generality, we assume and TOs and are independent quasi-static random processes777 In practice, the drift in the oscillators is a slowly varying process with coherence time in the order of minutes or hours [30].uniformly distributed in , where is the maximum timing delay that can occur in the system. We denote and as the TO vector for the transmitter and receiver, respectively, and let .

### Ii-C Phase Noise Model

In this paper, we consider the independent phase noise on each transmit antenna and on each receive antenna. For free-running oscillators, the discrete time PHNs and for and can be modeled as a Wiener process [31, 15, 16], which are given by

 θtxj(n) =θtxj(n−1)+Δtxj(n), θrxi(n) =θrxi(n−1)+Δrxi(n). (4)

The terms and are random phase innovations for the oscillators at each sample, assumed to be white real Gaussian processes with and , respectively. and

stands for the variance of the innovations at the

-th and -th transmit and receive antennas, respectively, which are given by [16, 32]

 σ2Δtxj =2πctxjTs, σ2Δrxi =2πcrxiTs, (5)

where and denote the one-sided 3 dB bandwidth of the Lorentzian spectrum of the oscillators at the -th and -th transmit and receive antennas, respectively, and is the sampling time. We assume that and are known at the receiver since they are dependent on the oscillator properties.

### Ii-D Transmit and Receive Data Path

We consider a frame structure, illustrated in Fig. 3, in this paper. There is a preamble sequence with symbols at the beginning of each frame for timing synchronization and channel estimation. Specifically, since timing offset is a slowly varying process, we use the preamble of the first frame (initial frame) to perform timing synchronization, while preambles of the rest frames are used for channel estimation. Each frame consists of subframes, and within each subframe, there is a data section with symbols, denoted by , where is the transmitted data symbols at -th transmit antenna. In addition, there is a pilot sequence with length at the beginning of each subframe for the estimation and tracking of the PHN over the entire frame. To summarize, the first subframe of each frame has the length of symbols, while the other subframes are of length symbols. As a result, the overall pilot and preamble overhead is given by . In practice, the period of the preamble is comparable to the coherence time of the channel (which is ~ ms for fixed-point wireless backhaul applications [33]). The period of the pilot is more frequent as it has to track the variation of the PHN and . For a typical oscillator at GHz, the effective bandwidth of the power spectral density (PSD) of the PHN is Hz [34] and hence, the coherence time of the PHN is around ms. Consequently, a pilot density of pilots per frame would be sufficient. Hence, for a system with a symbol rate of M/s, the pilot and preamble overhead is around 5% for , , and , which is quite low.

Figure.LABEL:fig:Data_path illustrates the uplink and downlink datapath of the transmitter and receiver for the wireless backhaul between two base stations (BSs) A and B. We consider an FDD system, and the transmitter and the receiver of a site share a common oscillator. Hence, the PHN and the TO during downlink for BS A (BS B) as a receiver (transmitter) are identical to the PHN and the TO during uplink for BS A (BS B) as a transmitter (receiver), i.e., in Fig. LABEL:fig:Data_path we have , , and , , . To avoid the abuse of notation, we assume , , , in the following illustrations.

Let be the response of the pulse-shaping filters evaluated at . The equivalent channel for the -th transmit antenna and the -th receive antenna is given by

 hτi,j(t)=[~H(t)]i,j∗g(t−(τtxj+τrxi)T), (6)

where the superscript means the variable is parameterized by . Let denote the complex-valued symbol sequence of length transmitted by the -th transmitter, which can be a preamble sequence, pilot sequence or data. Suppose the received waveform is sampled at a rate of samples per symbol (i.e., ), the received signal of -th receive antenna at the -th sample is given by

 yi(n) (7) = M∑j=1L−1∑k=0ej[θtxj(n)+θrxi(n)]hτi,j(nTs−kT)uj(k)+vi(n),

where is the symbol index, and is the complex additive white Gaussian noise (AWGN) with zero mean and variance , i.e., Specifically, to show a symbol level signal model, the received signal at the -th sample for symbol index is given by:

 yi(kQ) =ej[θtxi(kQ)+θrxi(kQ)]PHN distortion⎧⎪ ⎪ ⎪⎨⎪ ⎪ ⎪⎩hτi,i(0)ui(k)desired signal +L−1∑k′=1;k′≠khτi,i((k−k′)T)ui(k′)ISI +M∑j=1;j≠iL−1∑k′=0hτi,j((k−k′)T)uj(k′)MAI⎫⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎬⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎭+vi(kQ)AWGN. (8)

From (8), we have the following observations. First, the PHN will introduce a random phase distortion on the received signal constellation during the demodulation process. Furthermore, the MAI and ISI of the spatially multiplexed streams will be mitigated by the precoder and the decorrelator, which are designed based on the estimated channel at the beginning of each frame and remain constant throughout the frame. While the channel will be quasi-static within a frame, the PHN process will be time-varying and hence the PHN distortion will induce loss of coherency of the precoder and decorrelator for symbols within a frame. Second, the impacts of the TO appear in both the ISI and the MAI terms, as illustrated in (8), which will increase both types of interference. These physical impairments may not be a significant performance bottleneck in regular wireless access applications. However, due to the very high spectral efficiency target in wireless backhaul applications, they can be a significant bottleneck.

## Iii Timing Offset Estimation and Compensation

We consider the TO issue in this section. Conventional correlation-based TO estimators correlate the preamble sequences with delayed replicas of the received samples at each receive antenna, and find the maximum correlation peak in the timing metric to estimate the sum-offsets in each transmitter-receiver link. For an MIMO system, the correlator can estimate sum-offsets for all the transmitter-receiver links, but there are only degrees of freedom in these sum-offsets as we only have one unknown TO for each individual antenna to be estimated. Hence, there are correlations among the sum-offsets, and this correlation can be further exploited to enable a more accurate TO estimation and compensation for each individual antenna.

Conventionally, the ZC sequence is used as preamble in LTE systems, serving as the primary synchronization signal (PSS) to extract timing information. The cross-correlation of the ZC sequence is low for traditional wireless MIMO applications due to the moderate MAI, thus the accurate estimations of the sum-offsets can be guaranteed. However, the cross-correlation of the ZC sequence is not sufficiently low to mitigate the MAI for a dual-polarized LoS MIMO channel with high XPD, especially for the cross-polar links, which experience severe MAI. Thus, the traditional ZC preamble is not enough to isolate MAI in cross-correlation, which hinders correlator-based timing synchronization from fully utilizing the spatial information. In addition, the high spectral efficiency of wireless backhaul poses a very stringent requirement for accurate timing synchronization, thus preamble sequences with a good auto-correlation property that is robust to ISI are also required. This motivates us to design preamble sequences with superior auto/cross-correlation properties for the TO estimation.

### Iii-a Preamble Sequence Design

Let denote the preamble sequence transmitted by the -th transmit antenna. The cross-correlation of two preamble sequences and is defined as

 ηj,j′(l) =Lt−1−l∑k=0aj(k+l)a∗j′(k)=η∗j′,j(−l), ∀l=0,...,Lt−1, (9)

where represents the -th lag. Note that (9) reduces to the auto-correlation of when . A satisfying set of preamble sequences should have a very low cross-correlation for all possible lags and a high auto-correlation only when , thus, the correlation peak occurs only when the two sequences are from the same antenna and are perfectly aligned. However, [35] shows that it is impossible to design such a set. Fortunately, it is feasible to achieve the required properties in a specific lag interval. Since the TO will not exceed a maximum value of in each transmit and receive antenna, we can seek preamble sequences with the required properties at the lag interval . Such a set of sequences can be obtained by solving the following optimization problems [36]:

 minUT M∑j=1M∑j′=1Lt−1∑l=1−Ltωlf∣∣ηj,j′(l)∣∣2−ω0L2tM s.t. ∣∣aj(k)∣∣=1,  k=0,...,Lt−1,  j=1,...,M, (10)

where , are non-negative weights assigned to different time lags and are defined in our problem as

 ω±l={1,0≤l≤⌈2τmax⌉,0,otherwise.

The unimodular constraint in (10) ensures the symbols in preamble sequences have a constant amplitude. The optimization problem (10

) is solved via the majorization-minimization (MM) algorithm and can be implemented efficiently for very long sequences via fast Fourier transform (FFT)

[6]. Fig. 5 shows the auto/cross-correlation of the proposed preamble sequence, compared with the ZC and Walsh sequence. It shows that the proposed sequences provide -70 dB isolation from MAI in cross-correlation and -70 dB isolation from ISI in auto-correlation at desired lags, which is sufficient for TO estimation for our channel model with high XPD.

### Iii-B Per-Antenna Timing Offset Estimation

To obtain the per-antenna TO, we first estimate the sum-offsets via finding the maximum correlation peak in the timing correlation metric. For a high resolution TO estimation, we consider upsampling at the received signal. Correlating the -fold upsampled version of the received signal at the -th receive antenna and the preamble from the -th transmit antenna, the timing correlation metric at the -th sample shift 888Sample shift can be regarded as the sample-level counterpart of lag in Section III-A. is given by

 Λi,j(sf)=∣∣ ∣∣Lt−1∑k=0a∗j(k)⋅yi(kQ+sf)∣∣ ∣∣2.

The sum-offset for the -th transmit antenna and the -th receive antenna, , is obtained by selecting the highest peak in the correlation metric, as given by

 ^τi,j=TsT⋅(argmaxsf=0,...,⌈2Qτmax⌉Λi,j(sf)).

Then, the per-antenna TO can be related to the sum-offsets by

 γ=¯INMτ, (11)

where and is given by

 ¯INM=[IN⊗1M1N⊗IM]. (12)

The column rank of is only , thus the transformation (11) is underdetermined and one additional constraint is required. The reference TO, i.e., without loss of generality, provides this additional constraint with which the per-antenna TO can be obtained by solving the following LS problem:

 ^τ= argminτ∥∥¯INMτ−^γ∥∥2 s.t. τ(1)=0. (13)

### Iii-C Per-Antenna Timing Offset Compensation

In our system, the TOs are compensated with the corresponding transmit and receive pulse shaping filters, specifically, with , where and represents the transmit and receive pulse shaping filters, respectively. The TOs are compensated using and at the -th transmit and -th receive antenna, respectively. Thus, the resulting equivalent channel after TO compensation will be

 hΔτi,j(t)=[~H(t)]i,j∗g(t−(Δτtxj+Δτrxi)T),

where and represents the residual TO at the -th transmit and -th receive antenna, respectively.

Since the TOs of BS A (BS B) are identical in the downlink and uplink transmission, the TO estimated during downlink (uplink) for BS A (BS B) as a receiver can also be used to compensate the TO during uplink (downlink) for BS A (BS B) as a transmitter. Thus, the per-antenna TO compensation can be implemented by only using the local information in each BS without feedback. For example, consider an FDD system between BS A (with antennas) and BS B (with antennas), as illustrated in Fig. LABEL:fig:Data_path. The per-antenna TO compensation scheme is described in Fig. 6. The received samples in BS B during uplink transmission are used to correlate with the preamble sequences transmitted by BS A to obtain the uplink sum-offsets , which is then used to obtain per antenna TO by solving the LS problem with constraint (13). The TOs of the antennas of BS B are then compensated by the first elements in , i.e., in the corresponding transmit (receive) pulse shaping filters for the downlink (uplink) transmission. The same compensation procedure is implemented for BS A using downlink measurements.

## Iv Robust MIMO Precoder and Decorrelator Design

In this section, we propose the precoder/decorrelator design for MAI and ISI suppression. After compensating the TO, there are two physical impairments that will hinder the performance of the precoder/decorrelator. One is the residual of the TO estimation, which is within one symbol time, i.e., and , and these residuals will introduce ISI. Another is the PHN, as illustrated in (8), which will introduce MAI. For the convenience of the following discussion, we define as the aggregate residual of the TO estimation and the PHN matrix with and , as the PHN of the -th symbol at the -th transmit antenna and the -th receive antenna. Since the operations of downlink and uplink are similar in this section, we omit the indicator for specifying the downlink or the uplink for the channel and the precoder/decorrelator. The channel state information (CSI) is assumed to be static, and the CSI drift caused by the PHN will be discussed in Section V.

### Iv-a Preliminary for Precoder/decorrelator Design

Before presenting the proposed precoder/decorrelator design, the channel estimation procedure is introduced in this subsection since the CSI is a fundamental preliminary for the precoder/decorrelator design. Due to the existence of the residual of the TO estimation, there is ISI at each symbol transmission. Hence, we need to estimate the CSI for several taps to support the precoder/decorrelator to suppress the ISI. We consider the channels within a finite length window with a symbol length. Using the discrete expression of the channel (6), the aggregate channel to be estimated is given by

 ¯H=[H[−W],...,H[0],...,H[W]]∈CN×(2W+1)M, (14)

where represents the principal channel corresponding to the current transmitted symbol, while is defined as the ISI channel with corresponding to the adjacent interference symbols. Each subchannel in is given by

 [H[w]]i,j=hΔτi,j[w]ej[Λ[k∗+w]]i,j,∀w∈{−W,...,0...,W}, (15)

where . is the reference symbol; in other words, we assume the principal channel and ISI channels for each symbol are identical to the principal channel and ISI channels for the -th symbol. Since the residual of the TO estimation is within one symbol time and the channel is assumed to be static, the input-output relationship at the -th symbol in the preamble for channel estimation can be expressed as

 y(k)=¯HUT(k)+ν[k] (16)

where is the reshaped preamble for channel estimation and is the AWGN vector at the -th symbol with variance . Using preamble symbols at the beginning of each frame (except the initial frame for timing) to estimate the CSI, the received signal is given by

 Y =¯H[UT(0),…UT(Lt−1)]+[ν[0],…,ν[Lt−1]] =¯H¯UT+¯ν.

Then, the CSI estimation is obtained by the LS solution

 ^¯H=argmin¯H∈CN×(2W+1)M∥∥Y−¯HUT∥∥2F=YUTH(UTUHT)−1, (17)

where .

### Iv-B Optimization Problem Formulation for Precoder/decorrelator Design

We use the estimated CSI to optimize the precoder/decorrelator. Consider a MIMO system, which can support parallel transmission streams. In order to suppress MAI and ISI, we apply a memoryless precoder, , at the transmitter, and a memory decorrelator, =[, at the receiver, where is the decorrelator at the -th tap. The received signal in the -th symbol time at the -th stream can be represented as

 rm(k)=~wHm(D+W)∑w′=−(D+W)~H[w′]Fs(k+w′)+~wHm~ν[k], (18)

where is the -th column of , and , is the aggregate channel and noise, respectively, after considering the memory effect at the decorrelator. is the transmitted symbol vector at the -th symbol time in the data section,999Note that signal model (18) can also be applied to pilot symbols by substituting with . and we assume and are i.i.d with zero mean unit and variance.

The existence of both ISI and MAI in Eq.(18

) cannot be handled by traditional solutions, such as singular value decomposition (SVD) or naive water filling due to the existence of both ISI and MAI. Furthermore, considering the practical maximum modulation level constraint (e.g., we assume that the maximum modulation level that can be supported in implementation is 4096-QAM), the design of an optimal precoder/decorrelator is challenging. To achieve a high spectrum efficiency and incorporate ISI, MAI, and the modulation constraint, we formulate the following max-sum-rate problem:

 maximizeF,~WNs∑m=1min{log2(1+SINRm),ϖ} s.tTr(FHF)≤P, (19)

where is the capacity under a given QAM modulation level, , with a small error (e.g., symbol error rate (SER) ). is the total transmit power, and is the signal to interference plus noise ratio (SINR) of the -th stream, which can be expressed as

### Iv-C Precoder/decorrelator Design via Alternative Optimization

Problem (19) is non-convex and non-smooth, hence is very challenging to solve. Inspired by [22], we introduce an auxiliary variable and use an alternative optimization scheme to find a simple solution to problem (19). Using the variable , problem (19) can be well approximated by

 minimizeF,~W,ΓTr(ΓNs∑m=1ITmE(W,F)Im)−logdet(Γ) s.tTr(FHF)≤P, (21) [Γ]m,m≤2ϖ,∀m,

where is the diagonal matrix with diagonal elements as the elements in the

-th column of the identity matrix, and

 E(~W,F) =(~WH~H[0]F−I)(FH~H[0]H~W−I) + ~WH((D+W)∑w′=−(D+W);w′≠0~H[w′]FFH~H[w′]H+σ2I)~W. (22)

The details of this approximation can be found in Appendix A.

We then present the detailed update rules using the alternative optimization method for problem (21). At the -th iteration of the optimization, we alternatively update and by the following 3 steps:

• Step 1: Update given and by

 ~W(titer)=B−1~H[0]F(titer−1), (23)

where

 B =~H[0]F(titer−1)(F(titer−1))H~HH[0] +(D+W)∑w′=−(D+W);w′≠0~H[w′]F(titer−1)(F(titer−1))H~H[w′]H (24) +σ2I.
• Step 2: Update given and by

 Γ(titer)=min{[E(~W(titer),F(titer−1))]−1m,m,2ϖ}. (25)
• Step 3: Update given and by solving

 minimizeFTr(Γ(titer)Ns∑m=1ITmE(~W(titer),F)Im) s.tTr(FHF)≤P. (26)

Problem (26) is convex, which can be efficiently solved by any solver for convex problems, e.g., CVX Matlab package [37].

These 3 update rules are obtained by the property that problem (21) becomes convex in terms of any individual variable in when fixing the other two. This property guarantees the sequence generated by the above alternative optimization converges to a stationary point of problem (21). To summarize, the precoder/decorrelator can be obtained via Algorithm 1.

## V Phase Noise Estimation and Compensation

After the precoder and decorrelator, the MAI and ISI is suppressed to enable high spectral efficiency transmission. However, the phase of the effective channel changes after the channel estimation stage due to the drifting of the PHN, and thus there will be accumulating MAI caused by the coherence loss in the precoder and decorrelator. As a result, the PHN needs to be tracked and compensated in the data transmission stage, which is the main target of this section.

### V-a Per-Antenna PHN Estimation based on Pilots

From equation (15) and (16), the channel estimation stage absorbs the initial PHN into the channel phase response. But as the PHN continues to drift, the effect of coherency loss will become dominant and introduce more MAI. To reset the phase error over one frame, we utilize the pilots inserted in a frame after the preamble to estimate the PHN increment from the preamble to each subframe. However, the presence of TO error and multipath creates non-negligible ISI, which deteriorates the PHN estimation quality. Though we can accurately estimate the sum-PHN in each ISI channel matrix as in the channel estimation stage, it requires a long pilot sequence to achieve high accuracy, which introduces significant overhead. In this section, we exploit the interference suppression capability of the precoder/decorrelator to improve the estimation accuracy.

Since the PHN varies very slowly within each pilot sequence, we assume that the PHN is constant during pilot sequence transmission in each subframe. We define the accumulated PHN increment from the preamble to the -th subframe as and for the -th transmit antenna and the -th receive antenna, where is the reference symbol we picked in the preamble for channel estimation (as elaborated in Eq.(15)). The collection of the accumulated PHN increments at the -th subframe from all transmit antennas and from all receive antennas are denoted by and . We assume that after the processing by the precoder/decorrelator, the ISI in the pilot transmissions is canceled, hence the received symbols at the -th pilot is given by

 R[q+1] = D∑d=−DW[d]HDrxΔϕ[q]H[−d]DtxΔϕ[q]Hq+1[d]FUP+V[q+1] = D∑d=−DW[d]HHq+1[d]X+V[q+1],

where

• is an diagonal matrix, where collects the PHN increments at the receiver;

• is an diagonal matrix, where collects the PHN increments at the transmitter;

• is the channel corresponding to the desired pilot signal for tap at the beginning of the -th subframe;

• denotes the transmitted signal after applying the precoder to the pilots and

• contains the AWGN noise.

To estimate of the PHN increments at , we use first order Taylor expansion to approximate and by