The advent of future wireless networks bearing new components in large numbers including eNodeBs, cloud servers, relays, smart grids, massive multiple-input multiple-output (MIMO), and big data nodes, and the recent advances both in software and hardware architectures surging the applicability of parallel and distributed computations  have brought innovative solutions and paradigm shifts in recent years[2, Chap. 10][3, 4, 5]. However, the distributed solutions based on conventional dual decomposition and other methods lack the effectiveness on the numerical stability, fast convergence rates , and scalability to high dimensional problems  compared to alternating direction method of multipliers (ADMM) which combines the strengths of dual decomposition and augmented Lagrangian methods .
ADMM studies in relay [9, 10, 11] and point-to-point [12, 13, 14, 15, 16, 7] networks are limited. In , an improved version of ADMM is proposed for power minimization under SINR constraints in multi-cluster relay networks with single antenna nodes. In , max-min SINR optimization is studied for decode-and-forward (DF) relay networks with single antenna transmitters and receivers under the constraints of total transmitter power, total relay power, and total number of relays. In , a distributed transmit power control algorithm via ADMM is proposed for a single relay aided network. ADMM in point-to-point networks is slightly more investigated in the literature. In , cloud radio access networks (C-RAN) with single antenna receivers are optimized for power minimization under SINR constraints. In , power minimization with the worst-case SINR constraints due to the channel state information (CSI) errors in multi-cell coordinated multiple-input single-output (MISO) networks is solved via semi-definite relaxation (SDR). In , power minimization problem with rate constraints in wireless sensors networks is studied. In , beamforming design for power minimization under SINR constraints is proposed in MISO downlink systems by solving second order cone problems (SOCP). In , max-min flow rate optimization problem is considered for software defined radio access networks (SD-RAN) with single antenna nodes under wired and wireless link constraints. In , a distributed power control algorithm is proposed for the utility maximization without SINR constraints in interference networks with single antenna nodes.
The three prominent features of future high performance wireless networks are multi-stream transmissions, multi-antenna nodes, and the line-of-sights between transmitters and receivers. The existence of direct links between transmitters and receivers is an immediate outcome of deploying a large number of intermediate nodes in the network to close the distances between transmitters and receivers. In C-RANs, SD-RANs, and wireless relay networks, the wireless intermediate nodes are called radio access units , base stations , and relays, respectively. Wireless relay networks can be regarded as the wireless communications parts of C-RANs and SD-RANs, which embody wired communications parts in their architectures as well. Although the three mentioned features are critical in practical systems, many studies on relay networks are based on communications settings with lesser features due to the difficulties that arise from the coexistence of all three features. Among many objective functions [18, Chap. 8], the mean square error (MSE) minimization problems are more tractable in relay networks with the three features. Nevertheless, the studies on MSE are still limited, and to the best of our knowledge, the MSE problem in a relay network with all the three features is studied only in .
The limited attributes of aforementioned researches on relay networks can diminish their applications in future practical systems. In this paper, we consider amplify-and-forward (AF) multi-relay interference networks with all the three mentioned attributes. The problem of interest in this network setting is to minimize the total power consumption of transmitters and relays with guaranteed quality of service (QoS) by distributed optimization of the transmit beamforming filters. The QoS metric chosen in this paper is stream SINR, which is interconnected to the data rate and bit error rate (BER) expressions. The problem is a member of non-convex quadratically constrained quadratic programming (QCQP) problems [21, 22], which is not directly amenable to distributed optimization due to the intricate stream SINR constraints. In particular, the SINR constraints are not in a linear form and also are not decoupled over streams for parallel and distributive implementation. We fit the problem into the ADMM framework, a potent tool for distributed optimization.
When designing a solution for the problem, the feasibility of problem, i.e., the SINR constraints (targets) must be jointly supported, must be assured in the first stage before solving the main problem in the second stage. There are two approaches to assure the success of the first stage, namely, deriving the feasibility conditions  and relaxing the initial conditions, e.g., searching for feasible SINR targets[24, 4] and reducing the number of users . The derivation of the feasibility condition is challenging even in simpler networks. In , an approximate condition is derived for a multi-relay network with a single transmitter and a receiver. The feasibility search, on the other hand, can be as costly as the solving the main problem in terms of the number of iterations and computational complexity per iteration.
In this paper, we propose to use random initializations of beamforming vectors to automatically determine feasible SINR targets with a high probability. All cases where the convergence is slow, fluctuant, and infeasible, i.e., the SINR targets are infeasible, make up less thanof the simulations presented in this paper. The elimination of these mentioned cases is beneficial in two important applications: (1) Accurate and extensive cross-analyses of crucial network parameters, and benchmarking with competitive schemes over these varying parameters can be executed in short times. (2) By weighting the auto assigned SINR targets with scalar variables, the feasibility search problem can be reduced to a simpler linear search problem as demonstrated in Section VII-F. The cross-analyses are accurate since no approximations are needed in contrast to the approximately derived feasibility conditions .
Relay beamforming design in the existence of direct links has been a long standing open problem due to the challenge in the expression of SINR metric in terms of relay filters as detailed in Section VI. In this paper, an SINR reformulation is proposed that gives good approximation when the direct channels and the effective channels between the transmitters and receivers are independent. The proposed distributed joint transmit and relay beamforming assures improved total power saving than that of only the distributed transmit beamforming optimization. However, since each relay serves all streams in the network, the complexity substantially increases.
The main contributions of this paper are summarized as follows:
A generic network model with multiple multi-antenna nodes at all sides, i.e., the transmitter, relay, and receiver sides, to carry out multi-stream transmissions in the presence of direct links is transformed into a compact matrix system model.
The proposed distributed ADMM algorithm achieves the optimal centralized solutions in the given generic network model. In addition, in terms of convergence rate, computational complexity, and message exchange load performance metrics, the proposed solution surpasses other optimal distributed algorithms.
By eliminating the feasibility of problem stage automatically, an extensive evaluation of the effects of crucial network parameters on the system performance metrics are attained that reveals new insights in this paper.
Analytical and numerical results demonstrate the lower computational complexity and message exchange load, higher convergence rate, i.e., lesser number of iterations, and finally the convergence of proposed distributed algorithm.
SINR approximation at the relay side is proposed to implement distributed joint transmit and relay beamforming optimization that further improves the total power saving at the cost of increased complexity.
The closest to our contribution in this paper is given in , where ADMM is also applied. The major difference between  and this paper is that a scalar power variable in a single relay network and a beamforming vector in a multi-relay aided network are optimized, respectively. Clearly, the extension in this paper is nontrivial. Further differences between  and this paper are discussed in later sections.
ADMM framework is applicable to many problems under mild conditions. The methodology is different than the distributed algorithms that are designed for particular problems. In , a distributed algorithm is proposed for interference alignment in signal space. To achieve the alignment in a distributed manner, each user minimizes the interference covariance matrix. Such distributed solutions that are designed for particular areas are in contrary to the distributed ADMM solutions that have broad application areas. In fact, ADMM solution for interference alignment is already exploited in .
The rest of the paper is organized as follows. The multi-stream transmission capable multi-relay interference network is introduced in Section II. In Section III, the transmit beamforming design problem for power minimization under stream SINR constraints is formulated. The distributed ADMM solution and benchmark distributed solutions are presented in Section IV. The attributes of distributed solutions including convergence, computational complexity, and message exchange load are studied in Section V. Distributed joint transmit and relay beamforming filter optimization via SINR approximation at the relay side is provided in Section VI. The numerical results and discussions are presented in Section VII, and finally, the paper is concluded with the summary of main results in Section VIII.
Ii System Model
Consider a -user two-hop MIMO multi-relay interference network aided by relays. The source (transmitter) and the destination (receiver) each has antennas while the relay has antennas as shown in Fig. 1. Each transmitter communicates with its corresponding receiver with the aid of all relays. Without loss of generality, transmitter and receiver pair can be called user. We assume that all relay nodes work in half-duplex mode. Thus the communication between the users is completed in two time slots and there are non-negligible direct links between all transmitters and receivers. In the first time slot, the transmitter transmits the signal vector , where and are the transmit beamforming matrix and symbol vector with and for , respectively. Here, is the number of streams of the user, i.e., the number of independent data streams to be transmitted between the transmitter and receiver pair. We assume
for sufficient degrees of freedom in signal detection. The transmitted signal from the user has a power constraint
where is the maximum power of the transmitter. The received signal at the relay and receiver in the first time slot is given by
respectively, where is the channel from the transmitter to the relay, and is the channel between the transmitter and the receiver. and are the complex additive white Gaussian noise (AWGN) at the relay and at the receiver in the first time slot with zero mean, and with the covariances and , respectively. In the second time slot, the received signal is precoded at the relay by the relay filter , . The relay transmit power is , where and is the covariance matrix of the received signal at and the maximum power of the relay, respectively. The relay transmit power can be explicitly written as
The received signal at the receiver in the second time slot is given by
where is the channel between the relay and the receiver.
Define the effective channel from transmitter to receiver through all relays as
Then, in (II) is rewritten as
To obtain the SINR expression, the received signal at a receiver is written in terms of the desired signal, interference signal, and noise summands. Thus, define the aggregate channel matrix and noise vector as
The list of channel notations used in the paper is given in Table I for convenience. Then, the aggregate received signal at receiver can be written as
For the sake of linear decoding complexity, the intra- and inter-user stream interferences are treated as noise in this paper. The receive filter is since the receive filters for the time slots and , and , respectively, are stacked in this matrix, i.e., . After applying the receive filter to the received signal , the SINR of the stream of the user is obtained as
where and is the receive beamforming vector for the stream of the user, i.e., column of the receiver filter (similarly , are the columns of the receiver filters , , respectively), is the power of the aggregate noise after the receive beamforming
is the covariance matrix of the aggregate noise, and the notation denotes that and/or .
Iii Problem Formulation
In this paper, the total power of transmitters and relays under SINR per stream constraints is minimized via the distributed ADMM algorithm  by optimizing all stream beamforming filters in parallel at each stream, aka processor, in the network. From (1) and (3), the total power, i.e., the sum of transmitter and relay powers, can be rewritten in terms of stream filters ( column of the transmitter filter ) as
Similarly, SINR of a stream can be rewritten as
where . The superscript index in indicates the transmitter, since is the aggregate channel between the transmitter and receiver. The simplifications of matrices are listed in Table II for convenience.
The problem formulation is given as
where and is the set of users and set of streams of the user, respectively.
|Channel from the transmitter to the receiver|
|Channel from the relay to the receiver|
|Channel from the transmitter to the relay|
|Effective channel from the transmitter to the receiver|
|through all relays|
|Aggregate effective channel from the transmitter to the|
|receiver, i.e., stacked matrix of and|
Iv Multi-Stream Beamforming Under Stream SINR Constraints
PIII is a non-convex quadratically constrained quadratic programming (QCQP) problem [21, 22]. To obtain the filters, PIII can be equivalently rewritten by using the fact that , where , and by rewriting the SINR constraint as a summation inequality rather than a division inequality. PIII can be rewritten as
The transmit beamforming covariance matrix constraint (13c) imposes the convex constraint that matrix belongs to the cone of symmetric and positive semi-definite matrices of dimension (denoted by ). Note that, since the covariance matrices in (13b), i.e., and , are Hermitian, , which means that the SINR inequality constraint (13b) is well defined. However, PII is still non-convex due to the last constraint (13d), hence SDR can be applied, i.e., the last constraint can be relaxed [21, 22]. Nonetheless, the resulting SDR of PII still cannot be solved distributively. To obtain at the stream via a parallel and distributed approach, both the objective function and constraints in PII must be separable with respect to each stream. The objective function of PII is separable; however, the reformulated SINR constraints (13b) are coupled. Moreover, since the SINR constraints are not linear, ADMM is not directly applicable to PII.
Iv-a Proposed ADMM Algorithm
Before presenting the proposed ADMM algorithm, we briefly review the main steps of ADMM. ADMM can solve the following convex problem
where , , , the functions and are convex, and are nonempty convex sets. Then the augmented Lagrangian for P() is given as
where is the Lagrange multiplier of the constraint (14b), re(.) is the real part operator, and is again the Lagrangian dual update step size. P() is solved by ADMM via three steps at each iteration as follows
Now, the SINR constraint (13b) is in a linear form via the constraints (18b), (18c), and (18d), and the coupling constraint (18b) is a simple linear constraint that is viable for the ADMM algorithm. The partial augmented Lagrangian for PIV-A can be written as
where is the Lagrange multiplier of the constraint (18b), and is a positive constant parameter for adjusting the convergence speed, i.e., Lagrangian dual update step size.
P() and PIV-A have the following correspondences
where and is a mapping function between and .
As mentioned earlier, in , total power minimization under SINR constraints problem is solved via a distributed power control algorithm in a simpler network. In contrast, in this work, the problem is solved via beamforming vectors in a generic network. Hence, the problem in this work is more challenging and also the results are more effective than , i.e., higher sum-SINRs can be achieved with lesser power consumptions. Another distinction between  and this paper is the difference between the augmented Lagrangian functions, where both of the proposed solutions are fundamentally based on. As seen in [11, P2], the auxiliary definitions [11, Eqs. (3c), (3d)] are augmented in the Lagrangian function [11, Eq. (4)] to obtain closed-form solutions, whereas in this work, the summation of auxiliary terms (18b) are augmented as seen in (IV-A). In , power control algorithm is proposed for a simpler relay network architecture. In other words, transmit power scalar variables are optimized in  as opposed to the transmit beamforming filters in this work. The summation of auxiliary terms in [11, Eq. (3b)] cannot be augmented in the Lagrangian function. Otherwise, the variable disappears in the Lagrangian differentiation process, hence closed-form solutions cannot be obtained. On the other hand, the auxiliary definitions (18c) and (18d) cannot be augmented due to the trace operator in this work. Therefore, in this work, (18b) is augmented in the Lagrangian function and the covariance matrix solutions are obtained via CVX. Resorting to CVX for the solutions of covariance matrices is a common practice in the literature [12, 9]. After obtaining the transmit covariance matrices , obtaining the transmit vectors is a well-known process , i.e., if the obtained covariance matrix is not rank-one, then additional rank-one approximate solution methods can be applied including the Gaussian randomization method. However, as observed by the numerical results in Section VII, the optimal matrices are always rank-one. This indicates that the proposed distributed optimal solution to PIV-A serves as a global optimal solution to PIII.
PIV-A is separable with respect to the streams, thus it requires solving problems of P, where is the total number of streams in the network. However, in order to apply the two-block ADMM algorithm which is better understood than -block ADMM algorithms in terms convergence , where , the variables can be divided into two groups and . Hence, PIV-A is separated into two simpler parts: P() and P. Therefore, ADMM can distributively solve PIV-A by solving simpler subproblems in parallel.
The Main Steps of ADMM
ADMM consists of sequential updates of primal variables , , and , and the dual variables  as presented below.
The smaller problem of is
The smaller problem of is
The updates of dual variables are given as
where is the conventional Lagrangian dual update step size.
Iv-C Feasibility of Problem
PIV-A can be infeasible for the given SINR targets . Therefore, the feasibility of PIV-A must be assured in the first stage before solving PIV-A in the second stage. There are basically two techniques in the literature: 1) the feasibility conditions are derived , and 2) the initial conditions are relaxed, e.g., the SINR targets are tested and the feasible targets are searched [24, 4]. Deriving the feasibility condition of PIV-A is challenging even for a simpler network and a simpler problem . In , power minimization under the worst stream SINR condition is studied for a multi-antenna relay network with a single transmitter and a receiver, and the direct links are neglected. Furthermore, in , the feasibility condition is derived based on the signal-to-interference ratio (SIR) instead of SINR. After assuming the targets are feasible based on the approximate feasibility condition derived from the SIR metric, the beamforming vectors are solved based on SINR. The approximate feasibility condition is more accurate in the high SNR regime, where noise can be neglected. On the other hand, testing the SINR targets and searching for the feasible SINR targets [24, 4] are as costly as solving problem PIV-A, i.e., both require high iteration numbers and high computational complexities per iteration, which severely impedes the cross-analysis due to the long simulation durations.
In this work, we adopt a new technique. We randomly initialize the transmit and relay beamforming vectors with full powers, i.e., , and , to determine feasible SINR targets with a high probability. The infeasible cases that make up a small portion of tests along with slow and fluctuant converging cases are filtered through step 11 and the detection window as detailed earlier. Hence, by testing randomly and automatically generated feasible SINR targets for any permutation of the network parameters, the cross-analysis is executed systematically and extensively within short simulation durations. Since cross-analysis at this comprehensive level has not been performed in the literature yet, many new insights are revealed in this paper. For all simulations, the receive filters are also randomly initialized but normalized to unity since including the receiver power as another parameter in the already large set of cross-varying network parameters substantially complicates the cross-analysis in Section VII.
In fact, the transmit and relay power constraints
respectively, where , can be incorporated into P. The Matlab script for CVX solution is given in Algorithm 2. However, instead of incorporating the power constraints into CVX as shown in Algorithm 2, checking the power constraints at step 11 of Algorithm 1 has the following advantages. Firstly, the CVX algorithm runs faster without the incorporated power constraints, particularly for networks with high number of relays and relay antennas. Secondly, if the SINR targets are infeasible, the algorithm starts fluctuating, i.e., different SINRs that do not meet SINR targets are achieved over the iterations while the power constraints are still assured by the constraints incorporated in Algorithm 2. Therefore, it is time-consuming to detect whether the fluctuation is due to infeasibility or it is a rare case where the algorithm still converges after the fluctuation. These two cases significantly hinder the simulation durations and executing extensive cross-analysis in Section VII becomes impractical. As seen in Section VII, in the worst case, our proposed distributed algorithm requires iterations on average. Therefore, instead of plugging the power constraints (IV-C) into the CVX optimization, letting the algorithm converge to the SINR targets and then checking the power constraints by step 11 of Algorithm 1 can swiftly determine the feasibility of the targets.
Iv-D Benchmark Distributed Algorithms
Iv-D1 ADMM with Bounded Guarantee (ADMM-BG)
The conventional dual decomposition method  for a distributed solution of PIV-A is not applicable since the dual of PIV-A can be unbounded . This can be demonstrated by considering to solve the dual problem
instead of solving the subproblem P() in step a) of our proposed algorithm presented earlier via CVX. The inner optimization problem of the above dual problem can be unbounded below given the dual variable so that , i.e., the solution can go to minus infinity. This problem can be avoided by the extra quadratic penalty term added in the ADMM scheme
and then introducing the slack variables [12, Eq. (29d)]