I Introduction
The available spectrum is a scarce resource, and many new technologies to be incorporated into 5G aim at reusing the spectrum more efficiently to improve data rates and fairness. Traditionally, temporal, spectral, or spatial^{1}^{1}1Spatial separation of UEs with MIMO can be used with either OMA or NOMA. orthogonalization techniques, referred to as orthogonal multiple access (OMA), are used to avoid interference among users (UEs) in a cell. They allow only one UE per timefrequency resource block in a cell. A promising candidate for more efficient spectrum reuse in 5G is nonorthogonal multiple access (NOMA), which allows multiple UEs to share the same timefrequency resource block. The set of UEs being served by a base station (BS) via NOMA is referred to as the UE cluster. A UE cluster is served by having messages multiplexed either in the power domain or in the code domain. NOMA is therefore a special case of superposition coding [2]. Decoding techniques using successive interference cancellation (SIC) [3] for multipleaccess channels have been studied from an informationtheoretic perspective for several decades [4], and they were implemented on a software radio platform in [5]. The focus of our work is on NOMA where messages are superposed in the power domain. This form of NOMA allows multiple UEs to transmit/receive messages in the same timefrequency resource block by transmitting them at different power levels. SIC techniques are then used for decoding.
With NOMA come a number of challenges, including:

Determining the size of the UE cluster, i.e., the number of UEs to be served by a BS.

Determining which UEs to include in a cluster, referred to as UE clustering.

Ordering UEs within a cluster according to some measure of link quality.

The objective of the cluster prioritizing individual UE performance, total cluster performance, or a middle ground between the two.

Resource allocation (RA) for the UEs in a cluster.
Promising results for NOMA as an efficient spectrum reuse technique have been shown [6, 7]. In [8, 9] power allocation (PA) schemes are investigated for universal fairness by achieving identical rates for NOMA UEs. The idea of cooperative NOMA is investigated in [10, 11]. Most NOMA works order UEs based on either their distance from the transmitting BS [12, 9, 11, 13] or on the quality of the transmission channel [14, 15, 8, 10, 7]. A number of works in the literature focus on RA [15, 8, 9, 16, 17]. RA schemes for maximizing rates with constraints often focus on small NOMA clusters such as the twouser case [9, 16, 17], though works such as [15, 8] consider a general number of UEs in a NOMA cluster.
The works in [6, 7, 8, 9, 10, 11, 15, 16, 17] consider NOMA in a single cell and therefore do not account for intercell interference, denoted by , which has a drastic negative impact on the NOMA performance as shown in [18]. Stochastic geometry has succeeded in providing a unified mathematical paradigm to model large cellular networks and characterize their operation while accounting for intercell interference [19, 20, 21, 22]. Using stochastic geometrybased modeling, a large uplink NOMA network is studied in [13, 23], a large downlink NOMA network in [24, 23], and a qualitative study on NOMA in large networks is carried out in [18]. However, [24] does not take into account the SIC chain in the signaltointerferenceandnoise ratio (SINR) analysis, which overestimates coverage. In [23], twouser NOMA with fixed PA is studied. In the downlink, comparisons are made between random UE selection and selection such that the weaker UE’s channel quality is below a threshold and the stronger UE’s is greater than a second threshold. It ought to be mentioned that fixed RA does not allow the system to meet a defined cluster objective and makes a comparison with other schemes such as OMA unfair.
In this work we analytically study a large multicell downlink NOMA system that takes into account intercell interference and intracell interference, henceforth called intraference and denoted by , error propagation in the SIC chain, and the effects of imperfect SIC for a general number of UEs served by each BS (i.e., a general cluster size). We discuss all of the challenges enumerated above. Our goal is to analyze the performance of such a large network setup using stochastic geometry. We introduce and study three different models to show the impact of locationbased selection of NOMA UEs in a cluster, i.e., UE clustering, on performance. We analyze and compare the network performance using two ordering techniques, namely mean signal power (MSP) based ordering, which is equivalent to distancebased ordering, and instantaneous signaltointercellinterferenceandnoiseratio (ISNR) based ordering. In this context, the rate region for the twouser case is studied for both ordering techniques. To the best of our knowledge, an analytical work that compares both ordering techniques does not exist. We consider two main objectives: 1) maximizing the cell sum rate defined as the sum of the throughput of all the UEs in a NOMA cluster of the cell, subject to a threshold minimum throughput (TMT) constraint of on the individual UEs 2) maximizing the cell sum rate when all UEs in a cluster have identical throughput, i.e., maximizing the symmetric throughput. Accordingly, we formulate optimization problems and propose algorithms for intercell interferenceaware RA for both objectives. We show a significant reduction in the complexity of our proposed algorithms when compared to an exhaustive search. OMA is used to benchmark the gains attained by NOMA.
The contributions of this paper can be summarized as follows:

We propose three models for the clustering of UEs (i.e., UE selection), which are governed by two important principles: First, a UE should be served by its closest (or strongest) BS; conversely, a BS chooses its NOMA UE from among the UEs in its Voronoi cell. Second, only UEs with good channel conditions (on average) should be served using NOMA (i.e., sharing resource blocks). In contrast, using a standard Matern cluster process such as in [13] would lead to the unrealistic situation where UEs from another Voronoi cell may be part of a NOMA cluster.

From the rate region for the twouser case we show that contrary to the expected result UE ordering based on ISNR, which takes into account information about not only the path loss but also fading, intercell interference and noise, is not always superior to MSPbased ordering. We discuss how RA and intraference impact this finding.

We show that there exists an optimum NOMA cluster size that maximizes the constrained cell sum rate given the residual intraference (RI) factor .

We show the existence of a critical level of SIC that is necessary for NOMA to outperform OMA.
The rest of the paper is organized as follows: Section II describes the system model. The SINR analysis and relevant statistics are in Section III. In Section IV the two optimization problems are formulated, the rate region for the twouser scenario is discussed, and algorithms for solving the problems are proposed. Section V presents the results, and Section VI concludes the paper.
Notation:Vectors are denoted using bold text, denotes the Euclidean norm of the vector x, denotes a disk centered at x with radius , and denotes the sector of the disk centered at x with radius and angle ; when , we denote the halfdisk by .
denotes the Laplace transform (LT) of the PDF of the random variable
. The ordinary hypergeometric function is denoted by .Ii System Model
Iia NOMA System Model
We consider a downlink cellular network where BSs transmit with a total power budget of . Each BS serves UEs in one timefrequency resource block by multiplexing the signals for each UE with different power levels; here denotes the cluster size. The BSs use fixedrate transmissions, where the rate can be different for each UE, referred to as the transmission rate of the UE. Such transmissions lead to effective rates that are lower than the transmission rate; we refer to the effective rate of a UE as the throughput of the UE. The BSs are distributed according to a homogeneous Poisson point process (PPP) with intensity . To the network we add an extra BS at the origin o, which, under expectation over the PPP , becomes the typical BS serving UEs in the typical cell. In this work we study the performance of the typical cell. Note that since does not include the BS at o, is the set of the interfering BSs for the UEs in the typical cell. Denote by the distance between the BS at o and its nearest neighbor. Since is a PPP, the distance follows the distribution
(1) 
Consider a disk centered at the o with radius . We refer to this as the indisk as shown in Fig. 1. The indisk is the largest disk centered at the serving BS that fits inside the Voronoi cell. UEs outside of this disk are relatively far from their BS, have weaker channels and thus are better served in their own resource block (without sharing) or even using coordinated multipoint (CoMP) transmission if they are near the cell edge [25, 26]. These UEs are not discussed further in this work. We focus on UEs inside the indisk since they have good channel conditions, yet enough disparity among themselves, and thus can effectively be served using NOMA.
We consider three models for the clustering of UEs. Each model results in a Poisson cluster process with points distributed uniformly and independently in each cluster. Let x be the parent point, i.e., the BS, and the distance to its nearest neighbor ; for brevity, is denoted by . The points in the cluster are:

Model 1: distributed on the disk

Model 2: distributed on the halfdisk such that all points in have distance at least from

Model 3: distributed on the line segment , where is the line through x and
More compactly, let be the (closed) disk sector of angle whose curved boundary has midpoint . Then for Model 1, , for Model 2, , and for Model 3, . A realization of the cell at o, its indisk, and the surrounding cells are shown in Fig. 1; the sectors are shown shaded for Models 1 and 2.
For Model 1, the union of all the disks is the socalled Stienen model [27]. The area fraction covered by the Stienen model is 1/4. This means that if all users form a stationary point process, 1/4 of them are served using NOMA in Model 1 and 1/8 in Model 2 (and 0 in Model 3). More generally, for arbitrary , the area fraction is . Note that the NOMA UEs form a Poisson cluster process where a fixed number of daughter points are placed uniformly at random on disks of random radii. The radii are correlated since the indisks of two cells whose BSs are mutual nearest neighbors have the same radius and all disks are disjoint, but given the radii, the daughter points are placed independently. Hence, there are three important differences to (advantages over) Matern cluster processes: the number of daughter points is fixed, the disk radius is random, and the disks do not overlap.
Focusing on the typical cell, the link distance
between a UE uniformly distributed in the sector of the indisk
with and the BS at o, conditioned on , follows(2) 
Since for Models 1 and 2, the statistics of their link distances are according to (2). For Model 3, however, the sector becomes a line segment as . Consequently, , conditioned on , in Model 3 follows the distribution
(3) 
Remark 1: Given , the exact distance between the UE and the interferer nearest to o in Model 3 is .
Remark 2: As there is no interfering BS inside , a UE located at u in , for any , is away from the boundary of this disk. Hence, all three models guarantee that there is no interfering BS in .
It makes sense to employ NOMA for UEs that have good channel conditions and thus can afford to share resource blocks with other UEs rather than those that cannot. Accordingly, any user close to a cell edge is worse off than the cell center users, on average. As decreases, users are located in the indisk farther from any cell edge, particularly the edge closest to o, and consequently have better intercell interference conditions. In this context, Model 2 can be used as a technique to improve the performance by selecting UEs for NOMA operation, i.e., UE clustering, more efficiently based on their locations, and Model 3 can be viewed as a method to upper bound the achievable performance via UE clustering.
A Rayleigh fading environment is assumed such that the fading coefficients are i.i.d. with a unit mean exponential distribution. A power law path loss model is considered where the signal power decays at the rate
with distance , where denotes the path loss exponent and .SIC is employed for decoding NOMA UEs. According to the NOMA scheme, the PA and transmission rate are designed such that the strongest UE is able to decode the messages intended for all those UEs weaker than itself. This requires ordering of UEs based on the quality of the transmission link. We order UEs in such a way that the UE, referred to as , has the strongest transmission link. There are various ways to define what comprises the link quality. The link quality should include the effect of path loss (and therefore link distance), fading and intercell interference. The impact of the largescale path loss is more stable and dominant than the fading effect which varies on a much shorter time scale. Additionally, accounting for intercell interference and fading necessitates very high feedback overhead. Since for small values of the path loss dominates the channel relative to fading, considering the quality of a channel to be based on the distance between a UE and its BS is often assumed to be a reasonable approximation [12, 9, 11, 13, 28, 29]. The link quality can be determined by ordering the UEs of the typical cell from strongest to weakest according to descending

Mean signal power (MSP)^{2}^{2}2It should be noted that this ordering is based on the total unit power transmission received at the UE.: this ignores fading and therefore orders UEs based on descending . Equivalently it can be viewed as ordering based on ascending link distance .

Instantaneous signal power (ISP): this includes fading and therefore orders UEs based on descending .

Meanfading signaltointercellinterferenceandnoise ratio (MFSNR): this assumes channels with the mean fading gain of 1 in both the transmission from the serving BS and in the intercell interference and therefore orders UEs based on descending where and are the locations of the interfering BSs and UE, respectively and is the noise power.

Instantaneous signaltointercellinterferenceandnoise ratio (ISNR): this includes fading and therefore orders UEs based on descending .
Analyzing the SINR for ordering based on ISP and MFSNR is out of the scope of this work. Hence, we analyze and compare the following two schemes:

MSPbased: the UEs of the typical cell are indexed according to their ascending ordered distance ; the closest UE from o is referred to as , for .

ISNRbased: UEs of the typical cell are indexed with respect to their descending ordered ISNR ^{3}^{3}3Note that is equivalent to in (28). We use the notation for brevity and to differentiate between the context it is being used in.; hence, has the largest ISNR, for .
The power for the signal intended for is denoted by , hence .
To successfully decode its own message, must therefore be able to decode the messages intended for all UEs weaker than itself, i.e, . This is achieved by allocating higher powers and/or lower transmission rates to the data streams of the weaker UEs. Correspondingly, is not able to decode any of the streams sent to UEs stronger than itself, i.e., due to their smaller powers and/or higher transmission rates. Assuming perfect SIC, the intraference experienced at when decoding its own message, , is comprised of the powers from the messages intended for . Since in practice SIC is not perfect, our mathematical model additionally considers a fraction of RI from the UEs weaker than in in a fashion similar to [30]. When perfect SIC is assumed, , while corresponds to no SIC at all. Additionally, suffers from intercell interference, , arising from the power received from all the other BSs in the network, and noise power . For the NOMA network parameters are to be selected, namely transmission rates and powers. Note that MSPbased ordering of UEs is agnostic to intercell interference and fading; however, our RA (choice of the parameters) is not. For the case of ISNRbased ordering, both ordering and RA are intercell interference and fadingaware.
IiB OMA System Model
We compare our NOMA model with a traditional OMA network where only one UE is served by each BS in a single timefrequency resource block. We focus on time division multiple access (TDMA). For a fair comparison with the NOMA system, the BS serves UEs distributed uniformly at random in (part of) the indisk as in the NOMA setup according to the model being employed. Each TDMA message is transmitted using full power for a duration . Without loss of generality, a unit time duration is assumed for a NOMA transmission and therefore . Consequently, parameters are to be selected for the OMA network, namely transmission rates and fractions of the time slot. We compare both MSPbased UE ordering and ISNRbased ordering for the OMA model, too.
Iii SINR Analysis
Iiia SINR in NOMA Network
In the NOMA network, the SINR at of the message intended for in the typical cell for is
where , is the location of , () is the fading coefficient from the serving BS (interfering BS) located at o (x) to . The intraference experienced when decodes the message for is denoted by . We use to denote .
IiiB Laplace Transform of the Intercell Interference
We analyze the LT of the intercell interference at both the unordered UE and the UE ordered based on MSP. Upon taking the expectation over the BS PPP and the unordered UE’s (ordered UE’s) location, the UEs in the cell with the BS at o become the typical unordered UEs (typical ordered UEs, from to .).
Lemma 1: The LT of () at the typical unordered UE (ordered ) conditioned on () and , where (), in Model 1 is approximated as
(4)  
(5) 
(6)  
(7) 
Proof:
Let , where and are the locations of the interfering BSs and the UE, respectively. The fading coefficient from the interfering BS at x to the UE is . The intercell interference experienced at the UE is
(8) 
The first term of the LT accounts for the first term in (8) corresponding to the nonnearest interferers from o lying at a distance at least () from the unordered UE (ordered
). It is obtained from employing Slivnyak’s theorem, the probability generating functional of the PPP, and MGF of
. However, this does not include the BS at distance from o, which is accounted for by the second term in (8) using the MGF of . Denote by the distance between this interferer and the typical UE. Then using the MGF of , the exact expression for the LT of the second term in (8) is . For simplicity we approximate it using the approximate mean of this distance. Since the average position of the typical UE distributed uniformly in the indisk is o, . ∎Note: The first term of the LT of () is pessimistic since the interference guard zone in our model () is smaller than the actual one. For the second term, an exact evaluation (by simulation) shows that the difference between and is less than 3.2%.
In the case of Model 2 the distance between the UEs and the interferer closest to o is larger than in the case of Model 1. This corresponds to a change in the impact of the second term of (8). The LT of intercell interference changes accordingly.
Lemma 2: The LT of () at the typical unordered UE (ordered ) conditioned on () and , where (), in Model 2 is approximated as
(9)  
(10) 
(11)  
(12) 
Proof:
The proof follows according to that in Lemma 1. However, in the second term, . We use this approximation because a UE located in Model 2, i.e. in the halfdisk away from the interferer nearest to o, has ; consequently, we approximate the average position of a UE in this model and accordingly. An exact evaluation (by simulation) for Model 2 shows that the difference between and is less than 0.92%. ∎
In the case of Model 3 the distance between the UEs and the interferer closest to o is exactly . This too corresponds to a change in the impact of the second term of (8). The LT of intercell interference changes accordingly.
Lemma 3: The LT of () at the typical unordered UE (ordered ) distributed according to Model 3, conditioned on () and , where (), and , is approximated as
(13)  
(14) 
(15)  
(16) 
IiiC UE Ordering
Since the order of the UEs is known at the BS, we use order statistics for the PDFs of the link quality. These are derived using the distribution of the unordered link quality statistics and the theory of order statistics [31].
IiiC1 MSPBased Ordering
UEs are ordered based on the ascending ordered link distance . Hence, is the distance between the nearest UE, i.e., to its serving BS, given , for . Using the distribution of the unordered link distance conditioned on in (2) for Models 1 and 2 we have
(17) 
for , where .
Similarly, using the distribution of the unordered link distance conditioned on in (3) for Model 3 we have
(18) 
for .
Note that MSPbased ordering guarantees that the nearest interfering BS from is farther than .
IiiC2 IsNRBased Ordering
UEs are ordered based on descending ordered ISNR, . The unordered ISNR is .
Lemma 4: The CDF of the unordered ISNR conditioned on is approximated as
(19) 
where is approximated in Lemmas 1, 2, and 3 for Models 1, 2, and 3, respectively, and is given in (2) for Models 1 and 2 and in (3) for Model 3.
Proof:
By definition of ,
Here (a) follows from and (b) using the definition of the LT of conditioned on and . ∎
Corollary 1: The CDF of the ordered ISNR conditioned on is approximated as
(20) 
where is given in (19).
IiiD Coverage in NOMA Network
In order to decode its intended message, needs to decode the messages intended for all UEs weaker than itself. We use to denote the SINR threshold corresponding to the transmission rate associated with the message for . Coverage at is defined as the event
(21) 
where
Remark 3: We observe that the impact of the intraference is that of a reduction in the effective transmit power to ; without intraference, in (21) would be replaced by . This reduction and thus is dependent on the transmission rate of the message to be decoded.
We introduce the notion of NOMA necessary condition for coverage, which is coverage when only intraference, arising from NOMA UEs within a cell, is considered. By definition we can write the signaltointraference ratio (SR) of the message for at as
(22) 
From (22), the SR of the message for is independent of the UE (i.e., ) it is being decoded at; hence, it can be rewritten as . In order for the message of to satisfy the NOMA necessary condition for coverage, we require
(23) 
The above condition constrains the transmission rate for the message of to be less than a certain value that is a function of the power distribution among the NOMA UEs. If this condition is not satisfied, the message for cannot be decoded since is an upper bound on , . Consequently will be in outage as will not be positive. Note that for the particular case of with perfect SIC (i.e., ), there is no intraference and implying always satisfies the NOMA necessary condition for coverage when SIC is perfect; equivalently, when , . Hence, failing to satisfy the NOMA necessary condition for coverage guarantees outage for all UEs that need to decode that message simply because the transmission rate is too high for the given PA. This shows the importance of RA in terms of PA and transmission rate choice.
Using , in (21) can be rewritten compactly as
(24) 
Next, we derive the coverage probabilities for UEs using each ordering technique.
IiiD1 Coverage for UEs Ordered Based on MSP
Theorem 1: If , the coverage probability of the typical ordered based on MSP, is approximated as
(25) 
where is given in (1), in (17) for Models 1 and 2 and in (18) for Model 3, and is approximated in Lemmas 1, 2, and 3 for Models 1, 2, and 3, respectively.
Proof:
as . The inner expectation is the conditional LT of (given and ). From this we obtain (25). ∎
IiiD2 Coverage for UEs Ordered Based on ISNr
Theorem 2: If , the coverage probability of the typical ordered based on ISNR, is approximated as
(26) 
where is given in (1).
Proof:
(26) follows using . ∎
For a given SINR threshold , corresponding to a transmission (normalized) rate of , the throughput of the typical is
(27) 
The cell sum rate is .
IiiE OMA Network
The SINR for of the typical cell is
(28) 
where is the location of , () is the fading coefficient from the serving BS (interfering BS) located at o (x) to . Coverage at is defined as .
Lemma 5: In the OMA network, the coverage probability of the typical ordered based on MSP is approximated as
(29) 
where is given in (1), in (17) for Models 1 and 2 and (18) for Model 3, and is approximated in Lemmas 1, 2, and 3 for Models 1, 2, and 3, respectively.
Proof:
Using the exponential distribution of and the LT of conditioned on and we obtain (29). ∎
Lemma 6: In the OMA network, the coverage probability of the typical ordered based on ISNR is approximated as
(30) 
Proof:
(30) follows from . ∎
Denote by the fraction of the time slot allotted to . For a given SINR threshold and corresponding transmission (normalized) rate , the throughput of the typical is
(31) 
Iv NOMA Optimization
Iva Problem Formulation
Determining the optimization objective of the NOMA framework can be complicated. The objective of NOMA is to provide coverage to multiple UEs in the same timefrequency resource block. Naturally we are interested in maximizing the cell sum rate. It is well known that the cell sum rate is maximized by allocating all resources (power in the NOMA network) to the best UE [32]. However, this comes at the price of a complete loss of fairness among NOMA UEs, which is one of the main motivations behind serving multiple UEs in a NOMA fashion. Hence, we constrain the objective of maximizing cell sum rate to ensure multiple UEs are served. In addition to the power constraint we consider two kinds of constraints: 1) constraining resources so that each of the typical UEs achieves at least the threshold minimum throughput (TMT), 2) constraining resources so that the typical UEs achieve symmetric (identical) throughput. Formally stated, these objectives are:

 Maximum cell sum rate subject to the TMT :
Because this problem is nonconvex, an optimum solution, i.e., choice of and that result in the maximum constrained , cannot be found using standard optimization methods. However, from the rate region for static channels we know that a RA that results in all UEs achieving the TMT , while all of the remaining power being allocated to the nearest UE, i.e., , to maximize its throughput is the optimum solution for that problem. An example of this for the twouser case is presented in [16].

 Maximum symmetric throughput:
This is equivalent to maximizing the smallest UE throughput. Solving this results in a RA that achieves the largest symmetric throughput (universal fairness), i.e., . Since this problem is also nonconvex, an optimum solution cannot be found using standard optimization methods.
Remark 4: The maximum symmetric throughput is the largest TMT that can be supported.
Remark 5: Due to outage, the typical UEs that achieve the same throughput () do not necessarily have the same individual transmission rates (and corresponding ’s).
The same objectives hold for OMA networks. The constrained resource allocated to the UEs, however, is time for TDMA instead of power for NOMA, i.e., . The OMA UEs enjoy full power in their transmissions. Optimization over transmission rate is done similarly to NOMA.