## I Introduction

Cooperative communication systems have been a long-lasting research topic during the past decades, thanks to its inherent advantages in boosting the achievable rate and communication reliability. Most of the existing cooperative techniques are based on amplify-and-forward (AF) relaying schemes, which make the relay simple, or decode-and-forward (DF) relaying schemes, which avoid noise amplification [1]. On the other hand, superposition coding (SC) has been extensively studied in the area of information theory to approach the channel capacity [2]. The embodiment of SC’s corresponding information-theoretic ideas in cooperative wireless systems was first proposed in [3], where the relay simultaneously transmits its own signal and the signal for which it acts as relay by superimposing them together. We note that most of existing designs for cooperative relay systems using SC are based on Gaussian input signals, whose actual implementation will require unaffordable encoding and decoding efforts [4]. Furthermore, the actual transmitted signals in practical wireless systems are drawn from finite-alphabet constellations, such as phase shift keying (PSK) modulation and quadrature amplitude modulation (QAM). Applying the results obtained from Gaussian input signals in practical systems using finite-alphabet signals may cause significant performance loss [5].

To the best of our knowledge, there is only a handful of work focused on the design of relay systems using SC and finite-alphabet signals. Optimal precoders for an AF half-duplex relay channel were designed to obtain the maximum coding gain in [6]

. In this work, the source had two signal vectors to transmit in each transmission block while the relay had no information of its own to transmit. During the first time slot, the source transmitted its first signal vector to both the destination and relay. In the second time slot, the source transmitted its second signal vector to the destination while the relay simply amplified and forwarded what it has received from the first time slot to the destination. The two signal vectors transmitted in the second time slot were considered to be received in a superposition manner at the destination. In

[7], an optimal superposition-coded relay scheme and a sub-optimal switched-power SC scheme were proposed by maximizing the equivalent squared minimum distance (ESMD) that determined the error performance. Moreover, the error performance of a high-rate bit-interleaved coded modulation with iterative decoding (BICM-ID) system using superposition modulation over a non-orthogonal amplify-and-forward (NAF) half-duplex single relay channel was studied in [8]. Although the above studies have made great efforts on the SC-based relay systems, only the source was considered to have information to transmit while the relay was only responsible for forwarding the source information. However, in many practical systems (e.g., wireless sensor networks), the relay can be a normal network node that has its own information to transmit while helping forward the source’s signal. As far as we know, there is no existing work that designs SC-based relay systems with both the source and relay having their own information to transmit using finite-alphabet constellations.Motivated by this gap, in this paper we concentrate on the signal design in a Gaussian AF half-duplex relay channel with SC being applied at the relay, which allows the source and relay to transmit their own information within two time slots. In our design, the relay superimposes its own information on the amplified version of the received noisy signal from the source before forwarding the superposed signal to the destination. For practical consideration, QAM constellations of not necessarily the same order are assumed to be adopted at the source and relay. We optimize the weight coefficients

of both the source and relay to maximize the minimum Euclidean distance of the composite constellation of the received signal at the destination, which dominates the error performance of the considered system in high signal-to-noise ratio (SNR) regimes. The formulated problem is a

mixed continuous-discrete problem that is in general non-trivial to resolve. To tackle it, we resort to the punched Farey sequence framework developed in [9] to strategically segment the entire feasible region of the formulated optimization problem into a finite number of sub-intervals and obtain the closed-form optimal solution of the formulated problem in each sub-interval. We then acquire the optimal solution to the formulated problem by comparing the solutions on each sub-interval. Simulation results demonstrate that our proposed SC-based scheme can achieve better average bit error rate (BER) performance than that using conventional time division multiple access (TDMA). Furthermore, the performance gain increases as the channel gain of the relay-destination link increases.## Ii System Model

We consider a Gaussian AF relay channel with SC being applied at the relay as depicted in Fig. 1, where both the source and relay have their own information to transmit to the destination. In our considered system, we assume no direct link between the source and destination. During each transmission block, it is assumed that two time slots with equal length are used. In the first time slot, the source transmits its information to the relay. The received signal at the relay can be given by

(1) |

where denotes the complex channel coefficient of the source-relay link, is the weight coefficient adjusting the transmitted signal at the source, and

is the additive zero-mean, circularly symmetric complex Gaussian (CSCG) noise with variance

, i.e., .In the second time slot, the relay amplifies the signal received from the source, then superimposes its own signal and forwards the superposed signal to the destination. The received signal at the destination can thus be expressed as

(2) |

where is the complex channel coefficient of the relay-destination link, and are the weight coefficients adjusting the superposed signal at the relay, and is the additive zero-mean CSCG noise on this link. It is assumed that the source knows perfect channel state information (CSI) of the source-relay link and the relay knows perfect CSI of the relay-destination link. The transmitted symbols are chosen randomly, independently and equally likely from the -ary square QAM constellation . Based on the knowledge of perfect CSI, we assume that completely offsets the phase rotation brought by to make sure there is no phase difference between the transmitted signals in and , and thus they can be superimposed at the relay by means of SC. Similarly, and are also assumed to completely offset the phase rotation brought by .

Although (1), (2) are in the form of complex baseband, the transmitting oscillators at the source and relay can only generate real sinusoids rather than complex signals.

This means that the considered two-dimensional QAM constellation actually could be split into two one-dimensional pulse amplitude modulation (PAM) constellations, which are called in-phase and quadrature branches [10], to be transmitted. Consequently, the complex Gaussian AF relay channel of the relay-destination link given in (2) can be decomposed into two real-scalar Gaussian AF relay channels. In this paper, we will consider the symbols whose in-phase and quadrature branches are symmetric, i.e., equal power allocation between the two branches. However, it is worth mentioning that our work can be readily extended to the unsymmetric case [11]. Because of the symmetry of the two sub-channels, we simply need to design one of the two real-scalar Gaussian AF relay channels with PAM constellation sets. Designing the constellation of the in-phase real-scalar Gaussian AF relay channel will be considered hereafter. The design of the quadrature branch is similar and thus is omitted for brevity.

Mathematically, we can rewrite (2) as

(3) | ||||

For the sake of notations, we assume that , , , , and , where is the real part of a complex number. Besides, , and are real non-negative weight coefficients controlling the minimum Euclidean distance of the PAM constellation sets during the actual transmission. Note that we do not need information-bearing symbol to replace because it does not contain any transmitted symbols . Now, the in-phase branch of (3) can be expressed as

(4) | ||||

From the given assumption that completely offsets the phase rotation brought by , we can write that , and is thus a real non-negative scalar itself, i.e., . As a result, (4) can be rewritten as

(5) | ||||

where and are independent and identically distributed (i.i.d.) real additive white Gaussian components because of the CSCG noise terms and . As a guarantee of generality, we assume that and , where and are and -ary square QAM constellations given by and respectively. Therefore, the information-bearing symbols and

are extracted from the standard PAM constellations with equal probability.

Consider that the individual average power constraints of the source and relay are and respectively. We then have the following constraints,

(6a) | |||

(6b) |

Recall that the symbols considered in this paper have equal power between their in-phase and quadrature branches. We thus have , i.e., the power constraint of the in-phase signal transmitted by the source should be

(7) |

For the terms on the left-hand side of (6b), we first have by noting . Considering that and are independent, we further have . Similarly, the second and third items of (6b) can be derived as and , respectively. Based on these derivations, we can rewrite (6b) as Therefore, the power constraint of the in-phase signal transmitted by the relay can be expressed as

(8) |

Jointly optimizing the weight coefficients , and to minimize the average error probability of the received signals is the focus of this paper. We will formally formulate and resolve this optimization problem in the following section for any given QAM constellation sizes of both transmitted signals, subject to the individual average power constraints of both the source and relay.

## Iii The Weight Coefficients Design for SC in Real-Scalar Gaussian AF Relay Channel

In this section, we will optimize the weight coefficients , and to minimize the error performance of the system. As the in-phase and quadrature branches are symmetric, the corresponding optimization problem of the quadrature component can be solved using exactly the same method and hence is omitted for brevity.

### Iii-a Problem Formulation

In our scheme, the transmitted signals from the source and relay are superimposed together using SC at the relay before they are transmitted to the destination. Furthermore, we adopt joint decoding by applying a coherent joint maximum likelihood (ML) detector at the destination to estimate the received signals in a symbol-by-symbol fashion. At the destination, the estimated signals can be expressed as

(9) |

According to the nearest neighbor approximation method for the ML receiver [10, Ch. 6.1.4], we know that the average error rate is dominated by the minimum Euclidean distance of the received constellation points at high SNRs. As a result, in this paper, we intend to work out the optimal value of to maximize the minimum Euclidean distance of the received constellation points so that the error rate is reduced in essence. For given pairs of in-phase signals and , the normalized Euclidean distance between the two received signals and at the destination in the noise free case is given by

(10) | ||||

As the information-bearing symbols and

are drawn from the standard PAM constellations as mentioned above, all of them are odd numbers. Hence, we can write that

and , where and with . In addition, is also defined here. According to the above definitions, we can deduce that from . To proceed, we define(11) | ||||

where and . In the following, the formal formulation of the max-min optimization problem is described.

Problem 1: Optimal design of weight coefficients for SC in real-scalar AF relay channel with PAM constellation: Devise the optimal value of which is subject to the power constraints in order to maximize the minimum Euclidean distance of the received signal constellation points, i.e.,

(12a) | ||||

(12b) | ||||

where (12b) is the power constraint derived from (7) and (8).

We can see that the inner optimization problem is on the discrete variables and , while the outer one is determined by the continuous ones . This means that Problem 1 is a mixed continuous-discrete optimization problem which is generally difficult to solve. As far as we know, only a handful of open literature, such as [12], [13], work on solving this kind of optimization problem while all of them only provide numerical solutions. In this paper, we apply the punched Farey sequence framework, which is developed from the original Farey sequence applied in [14], to methodically and optimally solve the formulated problem. By taking advantage of the punched Farey sequence, the entire feasible region of can be divided into a finite number of mutually exclusive sub-intervals. Within each of these sub-intervals, we achieve the closed-form optimal solution to the formulated problem. By doing this, we can acquire the overall maximum value of Problem 1 by taking the maximum value of the inner function among all the sub-intervals. We now first deal with the inner optimization problem in (12) given by:

Problem 2: Finding differential pairs with the minimum Euclidean distance:

(13) | ||||

Actually, we can observe that the essential problem to be solved in Problem 2 is to find rational approximations of the irrational numbers determining the minimum Euclidean distance, where the punched Farey sequence can be of great help. Therefore, in the following section, we will give the solution to Problem 2 by means of the punched Farey sequence. It is also worth pointing out that finding the closed-form solution to the optimal differential pairs is tricky because the channel coefficient which is crucial to the optimal solution, can vary across the whole positive real axis. Furthermore, the optimal weight coefficients cannot be determined beforehand for the inner optimization problem.

B. The Minimum Euclidean Distance of the Constellation Points of the Received Signal

In this section, we aim to solve Problem 2 to find the differential pairs having the minimum Euclidean distance. For this purpose, we first introduce the definition of the punched Farey sequence developed in [9].

Definition 1: Punched Farey sequence: The punched Farey sequence is the ascending sequence of irreducible fractions whose denominators are no greater than and numerators are no greater than .

According to this definition, the punched Farey sequence is a sequence of irreducible fractions with , and arranged in an increasing order, where denotes the greatest common divisor of non-negative integers . In addition, denotes the cardinality of . An example of the punched Farey sequence is given as follows:

Example 1: is the ordered sequence .

Some elementary properties, which are helpful to find the differential pairs having the minimum Euclidean distance, are presented with proof in [9]. Using these properties, the solution to Problem 2 can be given in the following proposition.

Proposition 1: For any , , , with , such that and , are successive in , we have 1) If , then ; 2) If , then .

The proof is omitted due to space limitation.

C. Optimal Solution to Problem 1

With the optimal solution to the inner problem given in Proposition 1, we are now ready to find the optimal solution to the outer optimization on of Problem 1. We observe that has a separate power constraint, while and have a joint power constraint. We thus first find the optimal solution of , which is given in the following lemma:

Lemma 1: The optimal value of within its average power constraint to maximize the minimum Euclidean distance of given in (13) should be chosen as:

(14) |

The proof is omitted due to space limitation.

To proceed, we first write the entire feasible region of Problem 1 in terms of as . According to the definition of the punched Farey sequence, we know that the punched Farey sequence , where , can divide the positive real axis into mutually exclusive sub-intervals. By denoting , where , we can solve Problem 1 by restricting . Concretely speaking, we intend to find the optimal such that

(15a) | ||||

(15b) | ||||

By applying the results obtained in Proposition 1, we can acquire the optimal solution to problem (15) given by the following theorem.

Theorem 1: The optimal solution to (15) is given as follows:

(16) | ||||

where , and .

The proof is omitted due to space limitation.

Summary: Based on Theorem 1, we can find for , which means the maximum value of the minimum Euclidean distance on each sub-interval determined by the punched Farey sequence can be obtained by simple algebraic operations. By comparing all these values, we can readily acquire the sub-interval where the overall maximum value of Problem 1 appears, and we finally find the optimal value of .

Note that the computational complexity of our proposed method is just . In the absence of our method, the exhaustive search approach is commonly used to solve Problem 1 with computational complexity as high as , where and are the discretization levels for the feasible ranges of and respectively. Obviously, our proposed method greatly reduces the computational complexity.

## Iv Simulation Results and Discussions

In this section, computer simulations are carried out to demonstrate the BER performance of our proposed SC-based design compared with that using TDMA for the transmission of the source and relay information. Without loss of generality, we assume that all channels are subject to Rayleigh fading such that , . During each transmission block, the source and relay each use one time slot to transmit information in our proposed SC-based scheme. In contrast, in TDMA-based scheme, the relay uses two time slots to separately forward the source’s information and transmit its own information, and the source still uses one. In terms of the transmission power, we assume that the source transmits at a power value of and 1 in SC-based scheme and TDMA-based scheme respectively while the instantaneous transmission power of the relay is and 1 in SC-based scheme and TDMA-based scheme respectively . These power constraint numbers are justified as follows: If the instantaneous transmission power of the relay in each time slot is assumed to be 1 in TDMA-based scheme, where a total of three time slots are used per transmission block, its average transmission power is equal to . Therefore, in SC-based scheme, the relay should transmit with the power of , which results in an average power of , to ensure the fairness of transmission power. The instantaneous transmission power of the source can be determined in the same way.

For the sake of comparison, we assume that, the number of transmission blocks of the TDMA-based scheme is two thirds of that transmitted in SC-based scheme over the same time period. Therefore, the constellation sizes of the TDMA-based scheme should be increased to - and -ary PAM constellations (i.e., - and -ary QAM constellations) to maintain the same data rate. Note that we should select values for and carefully because the constellation sizes in TDMA-based scheme are special.

We first study the average BER of both schemes against the SNR. As depicted in Fig. 2, we have the comparison in two cases, where the variances of the channel coefficients are and respectively. In both cases, we assume that the source adopts 256-QAM and the relay adopts 16-QAM in our proposed SC-based design. Correspondingly, 4096-QAM and 64-QAM should be adopted by the source and relay, respectively, in TDMA-based scheme. In both simulated cases, our proposed SC-based design outperforms the TDMA-based scheme in moderate and high SNR regimes. Specifically, the SNR gain of our proposed scheme over the TDMA-based scheme is approximately 2dB at the BER of when , while the performance gain is about 4dB when . We note that the performance gain increases as the channel gain of the relay-destination link increases.

As can be seen from Fig. 2, the BER performance gain of our proposed SC-based scheme is related to the relative channel gains of both links. Inspired by this observation, we now study the BER against the relative strength of the channels by fixing the variance of the source-relay link as and changing the variance of the relay-destination link, i.e., , which is represented in dB. As depicted in Fig. 3, in this simulation, we also consider two cases with the SNRs being 40dB and 50dB respectively. In this first case of SNR = 40dB, it is apparent that when is larger than 0dB (i.e., the variance of the relay-destination link is larger than 1), the BER performance of the SC-based scheme becomes better than that of the TDMA-based scheme, but the BER gain gradually reaches saturation. This is mainly because the error performance of both schemes is dominated by the source-relay link when the

relay-destination link becomes rather good. Since the source in our proposed SC-based scheme uses an instantaneous power value of to transmit with 256-QAM while that in TDMA-based scheme transmits with 4096-QAM using an instantaneous power value of 1, the minimum Euclidean distance between the constellation points in our scheme is much larger than that in TDMA-based scheme, which makes the proposed scheme essentially outperform the TDMA-based scheme in BER performance. In the second case of SNR = 50dB, the BER gain of our proposed SC-based scheme becomes even more significant, which is consistent with what we observed in Fig. 2.

## V Conclusions

In this paper, we developed a practical design framework for the Gaussian amplify-and-forward (AF) relay channel with superposition coding (SC) being applied at the relay, which allowed the source and relay to transmit their own information using practical quadrature amplitude modulation (QAM) constellations within two time slots. Specifically, to improve the error performance, we formulated a mixed continuous-discrete optimization problem to optimize the weight coefficients adopted at the source and relay to maximize the minimum Euclidean distance of the composite constellation of the received signal at the destination, subject to their individual average power constraints. By taking advantage of the punched Farey sequence and its unique properties, we attained the optimal solution to the formulated problem. Simulation results demonstrated the effectiveness of our proposed design based on SC over that using conventional time division multiple access (TDMA).

## References

- [1] J. N. Laneman, D. N. C. Tse and G. W. Wornell, “Cooperative diversity in wireless networks: Efficient protocols and outage behavior,” IEEE Trans. Inf. Theory, vol. 50, no. 12, pp. 3062-3080, Dec. 2004.
- [2] T. Cover and A. E. Gamal, “Capacity theorems for the relay channel,” IEEE Trans. Inf. Theory, vol. 25, no. 5, pp. 572-584, Sept. 1979.
- [3] E. G. Larsson and B. R. Vojcic, “Cooperative transmit diversity based on superposition modulation,” IEEE Commun. Lett., vol. 9, no. 9, pp. 778-780, Sept. 2005.
- [4] T. M. Cover, “Comments on broadcast channels,” IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2524-2530, Oct. 1998.
- [5] A. Lozano, A. M. Tulino and S. Verdu, “Optimum power allocation for parallel Gaussian channels with arbitrary input distributions,” IEEE Trans. Inf. Theory, vol. 52, no. 7, pp. 3033-3051, Jul. 2006.
- [6] Y. Ding, J. Zhang and K. M. Wong, “Optimal precoder for amplify-and-forward half-duplex relay system,” IEEE Trans. Wireless Commun., vol. 7, no. 8, pp. 2890-2895, Aug. 2008.
- [7] X. Jin and H. Kim, “Switched-power two-layer superposition coding in cooperative decode-forward relay systems,” IEEE Trans. Wireless Commun., vol. 15, no. 3, pp. 2193-2204, Mar. 2016.
- [8] L. J. Rodríguez, N. H. Tran and T. Le-Ngoc, “High-rate BICM-ID with superposition modulation over amplify-and-forward relay channels,” in 2010 25th Biennial Symposium on Commun., Kingston, ON, 2010, pp. 150-154.
- [9] Z. Dong, H. Chen, J. Zhang, L. Huang and B. Vucetic, “Uplink non-orthogonal multiple access with finite-alphabet inputs,” IEEE Trans. Wireless Commun., vol. 17, no. 9, pp. 5743-5758, Sept. 2018.
- [10] A. Goldsmith, Wireless Communications. Cambridge University Press, 2005.
- [11] V. R. Cadambe, S. A. Jafar and C. Wang, “Interference alignment with asymmetric complex signaling—Settling the Høst-Madsen–Nosratinia conjecture,” IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4552-4565, Sept. 2010.
- [12] J. Harshan and B. S. Rajan, “On two-user Gaussian multiple access channels with finite input constellations,” IEEE Trans. Inf. Theory, vol. 57, no. 3, pp. 1299-1327, Mar. 2011.
- [13] X. Xiao, Q. Huang and E. Viterbo, “Joint optimization scheme and sum constellation distribution for multi-user Gaussian multiple access channels with finite input constellations,” in 2016 Austral. Commun. Theory Workshop (AusCTW), Melbourne, VIC, 2016, pp. 130-135.
- [14] Z. Dong, H. Chen, J. Zhang and L. Huang, “On non-orthogonal multiple access with finite-alphabet inputs in Z-channels,” IEEE J. Sel. Areas Commun., vol. 35, no. 12, pp. 2829-2845, Dec. 2017.