I Introduction
UNMANNED aerial vehicles (UAVs) have recently gained significant interest in a broad range of applications. High mobility and flexible deployment are among the features allowing the UAVs to expand their scope of action [1, 2, 3, 4]. For effective operation, in many of the applications, the UAVs need to maintain a reliable internet connection during their flights. This connection is mainly provided by ground base stations (GBSs). However, due to locationdependant and timevarying characteristics of the communication channels, and the fact that the cellular networks are designed to serve terrestrial users, service is not available in all parts of the sky [5]. A costeffective solution to bring this connectivity to the aerial users is to design appropriate trajectories for the UAVs.
The trajectory optimization problem for cellularconnected UAVs has been extensively studied [5, 6, 7, 8, 9, 10]. In [5], the authors studied the trajectory design problem for a cellularconnected UAV, and based on graph theory and convex optimization, they proposed an algorithm to minimize the mission time of the UAV. This minimization is subject to a maximum tolerable outage duration. In [6], by using dynamic programming, the authors proposed a suboptimal trajectory design algorithm to minimize the flight time of a cellularconnected UAV. In [7], the authors proposed an algorithm to optimize the UAV trajectory with the goal of maximizing its energy efficiency. In [8], the authors minimized the propulsionpower consumption of a fixedwing UAV while a certain connectivity constraint on the instantaneous signal to interference plus noise ratio (SINR) is satisfied. In [9], the authors studied the interference management problem for the uplink communication of cellularconnected UAVs. The authors in [9] obtained the trajectory of the UAVs to achieve a tradeoff between maximizing energy efficiency and minimizing the interference caused on the terrestrial cellular network.
The studies in [5, 6, 7, 8, 9, 10] optimize the trajectory of the UAVs subject to a certain connectivity constraint on the received signal. However, due to the channel randomness, there is no guarantee to satisfy these certain constraints in practice. To take the channel randomness into account, we have to consider probabilistic connectivity constraints. These constraints can be defined in terms of the outage probability and give more reliability to the path planning algorithms. In addition to this concern, the proposed algorithms in [5, 6, 7, 8, 9, 10] require instantaneous channel state information. By considering a probabilistic connectivity metric, we can design algorithms that work without having this instantaneous information. To achieve this goal, the UAVs need to have a global model of the outage probability in the environment. Since each UAV has its specific task, and depending on the task, it flies over a particular area, it can not build the global model using its own experience. In contrast, the UAVs need to work together to build this global model.
To address the issues mentioned above, in this paper, we study the radio mapping and path planning problem for a cellularconnected UAV network. Since the onboard energy of the UAVs is limited, the UAVs can not fly for a long time. To prolong the lifetime of the UAVs, we minimize the flight time of each UAV, ensuring that a probabilistic connectivity constraint is satisfied throughout their flights. This problem is nonconvex and challenging to solve. We first reformulate the problem into a mathematically more tractable form. Then, to solve the reformulated problem, we propose a twostep algorithm. In the first step, by using Federated Learning (FL) [11, 12], the UAVs collaboratively build a global model of the outage probability in the environment. In this method, each UAV uses its data to update the global model locally. Accordingly, the UAVs do not need to share their collected information with a centralized node to do the training task, and the training process is executed in a distributed manner. As a result, the training process will be faster. In the second step, we use the resulting global model to design the trajectory of the UAVs. To achieve this goal, we have to ensure that the connectivity constraint is satisfied. We propose a path planning algorithm based on rapidlyexploring random trees. In our trajectory design algorithm, we do not need the instantaneous channel gains, which was considered in previous works [5, 6, 7, 8, 9, 10]. Moreover, our algorithm allows the UAVs to update the model based on their newly collected data. As a result, the model is trained based on online status of the network, which can remarkably increase the model’s accuracy. The simulation results show the effectiveness of this twostep approach for UAV networks.
The rest of the paper is organized as follows. In section II, we present the system model and problem formulation. In section III, we propose a twostep algorithm for radio mapping and path planning. Section IV presents the simulation results, and section V concludes the paper.
Ii System Model and Problem Formulation
We consider a cellular network, including GBSs and cellularconnected UAVs. The UAVs have their missions and fly from their initial locations towards their destinations. We use indices and to denote the GBSs and UAVs, respectively. The position of the th UAV at time is represented by , where without loss of generality we assume that the altitude of the UAVs is fixed throughout their flights. The initial and final position of the th UAV are shown by and , respectively. The velocity of the th UAV at time and its maximum speed are represented by and , respectively. We denote the position of the th GBS by . Moreover, the flight time of the th UAV is represented by .
The channel gain between the th UAV and the th GBS is expressed as
(1) 
where is the smallscale fading term and is the average pathloss (PL) between the th UAV and the th GBS at time . The average PL depends on the probability of having a line of sight (LoS) link between the UAV and the GBS. Let , where is the carrier frequency and is the light speed. The average PL is given by
(2) 
where and are additional losses for the LoS and NonLoS links, respectively, and is the probability of having a LoS link between the th UAV and the th GBS. This probability can be expressed as , where is the elevation angle between the th UAV and the th GBS at time , and and are environmentrelated parameters [13]. According to experimental measurements presented in[14], for moderate altitudes (less than 100 meters), . As a result, (2) is simplified to
The received signal to interference plus noise ratio (SINR) of the th UAV from the th GBS at time is given by
(3) 
where is the noise power and is the transmit power of the th GBS at time .
As discussed earlier, the UAVs need to maintain reliable communication links to the GBSs. This is essential to support the command and data flows between the UAVs and the cellular network. However, the quality of the link is highly affected by the channel randomness. To take the channel’s random characteristic into account, we first define the outage probability as the probability that the received SINR of the UAV from each GBS is less than a certain threshold , i.e.,
(4) 
Using this outage probability, we can define the connection metric. we say the th UAV is connected to the cellular network at time if the outage probability is less than a given threshold , i.e.,
(5) 
To have a reliable connection to the cellular network, the UAV is not allowed to loose its connection to the cellular network for more than a given time duration . In other words, the maximum continuous time interval that the UAVs can be disconnected from the cellular network is . To formulate this constraint, first we define function as
(6) 
This function gives the last time instance before that the th UAV is connected to the cellular network. Using (6), the reliable connection requirement of the th UAV is given by [15]
(7) 
It is worth mentioning that the value of is a design parameter and differs for different applications.
Iia Problem Formulation
The goal of each UAV is to minimize its flight time while its constraint for having a reliable connection to the cellular network is satisfied. If , the optimization problem of the th UAV can be expressed as
(8)  
s.t.  
In (8), constraint C1 shows the reliable connection requirement of the UAV. Constraint C2 states that the UAV’s velocity is limited to its maximum speed, and constraint C3 represents the initial and final location of the UAV. Problem (8) is nonconvex due to C1. Moreover, the outage probability used in depends on the network’s topology. Even if we consider a simple topology for the network, the UAVs do not have access to the outage probability in the environment. As a result, traditional optimization techniques can not be used to solve (8). In addition to these concerns, the UAV’s flight time, , is among the optimization variables. To solve (8), we have to find and the value of for all , which is a challenging task. In what follows, we reformulate the problem into a more tractable form. With this reformulation, we do not need to solve the continuoustime problem. Instead, we can find the solution of an equivalent discretetime problem.
IiB Problem Reformulation
The goal of each UAV is to solve its corresponding optimization problem. For the sake of brevity, we omit index from the problem and continue our discussion for the general case. To reformulate problem (8), we use the fact that any feasible solution must satisfy constraint C1. Hence, instead of solving the problem for all time instances, it is sufficient to consider the problem for a sequence of discrete time instances , , , , where , , and make sure that the UAV is connected to the cellular network at these time instances. In other words, if
(9) 
the maximum continuous time that the UAV is in outage will be limited to . Therefore, constraint C1 will be satisfied. To reformulate C2, we have , Since , the equivalent form of C2 is given by
(10) 
It can be shown that to minimize the flight time, the UAVs fly with their maximum speed, . Considering this fact and using a similar approach to what has been shown for C2, we can write the objective of (8) as
(11) 
Let , . Problem (8) is equivalent to the following discretetime optimization problem
(12)  
s.t.  
Iii Twostep Algorithm for Radio Mapping and Path Planning
To find the solution of the reformulated problem, we still need to know the outage probability. However, this information is not availbale to the UAVs. Hence, we need to obtain this information first. In what follows we propose a twostep algorithm to solve (12
). In this approach, in the first step, we estimate the outage probability. To achieve this goal, we use Federated Learning (FL) which allows the UAVs to collaborate to build a global model of the outage probability based on their locally collected data. Moreover, the UAVs do not need to share their collected data with a central node in this approach. In the second step, we use the derived model to find the solution of the problem ensuring that
is satisfied.Iiia Radio Mapping based on Federated Learning
To estimate (.), as depicted in Fig. (1), we consider a scenario where all UAVs can collaborate to build a global model of the outage probability in the environment. Let denote the data set of the th UAV and let . We denote and , where is the cardinality of set . The data set of the th UAV includes all pairs of , where is the 2D coordinate of a visited location by the UAV and
(13) 
is the label of this data point. To find a global model of the outage probability, we consider a neural network with parameter
. The output of this neural network for input is denoted by which gives the outage probability at location . To find an appropriate model for the outage probability, we have to solve the following problem(14) 
where
is the loss function corresponding to the
th data point. We consider crossentropy for the loss function as(15)  
where is the th data point. We can write the objective of (14) as
(16) 
If we define , then we have
(17) 
To minimize (17), instead of using a central approach which requires access to data of all UAVs, we use a distributed approach based on FL [11]. In this algorithm, we assume that the global model parameter, , is available to all GBSs. The GBSs send this parameter to the UAVs and the UAVs update their local model parameter as . The UAVs fly over the area and based on their received signals from the cellular network, they form their data sets . Using the collected data, each UAV performs
steps of the stochastic gradient descent (SGD) on its local parameter,
. In other words, in each step, the th UAV updates its local parameter as(18) 
where is the step size and is the gradient operator. The updated parameters, , are sent back from the UAVs to the GBSs. The GBSs act as aggregators and share this infromation in their network. By averaging the received local parameters, the GBSs evaluate a new parameter as
(19) 
The global model is updated using this new . This updated parameter is again sent to all UAVs to perform their local updates with their new data. This procedure is repeated until the neural network is trained. Algorithm 1 presents the FLbased radio mapping for outage probability estimation.
The advantage of this approach is that the network parameter can be locally updated. This does not require high computation resources. Moreover, the UAVs do not need to share their data sets with the cellular network. In addition to these benefits, the UAVs might fly over differnt areas of the network. Hence, they will not have good knowledge of the other parts of the network. This FLbased algorithm allows the UAVs to collaborate to build a global model based on their limited local experiences.
IiiB Path Planning based on RRT
After the first step, we have a model of the outage probability in the environment. Let denote the parameter of the trained global model. In this step, we have to solve the following optimization problem
(20)  
s.t.  
where is the output of the trained neural network for position . Problem (20) is still challenging to solve. The reason is that the feasible region corresponding to constraint 1 is not necessarily convex. Moreover, which is the number of discrete time steps is a variable of the problem. To overcome these difficulties, we use rapidlyexploring random trees (RRTs) [16]. RRTs are designed to search nonconvex spaces. In what follows, we describe our algorithm which works based on a modified version of RRTs called RRT [16]. RRT is an efficient algorithm which can find the shortest path between a pair of initial and final locations in a continuous space.
We use RRT to find the path that minimizes the UAV’s flight time while ensuring that it satisfies the requirement for having a reliable connection to the cellular network. In our algorithm, we use a tree to represent the path between the initial point to any feasible coordinate . The set of vertices, , is the set of all coordinates in the space that has been explored by the algorithm. The root of the tree is the initial point . Coordinate is called parent of if the UAV arrives in from and it does not visit any other vertex in between. According to this notation, is the set of all edges which connects a visited coordinate to its parent. Moreover, in our algorithm, we need to define a cost function for each visited coordinate. If the UAV passes the sequence of edges to reach , where , then the cost of coordinate is defined as
(21) 
We start our path planning algorithm from the root, , and iteratively add new vertices to the tree and update its structure including the edges and costs. In each iteration, to add a new vertex to the tree, we sample a random coordinate from the space. This random coordinate is denoted by . Then, the closest vertex of to is found and is shown as . To satisfy constraint C2, we have to ensure that the distance between the new sampled coordinate and is less than , i.e., . To meet this requirement, in case that this inequality is violated, we replace with a closer point denoted as , where
After satisfying C2, we have to make sure that C1 is also satisfied. If , then this new coordinate is out of outage and hence, it can be added to the tree. Otherwise, we have to repeat this iteration from the begining to find a coordinate satisfying both C1 and C2. Assuming , we add this point to the tree as a new vertex. To set the cost of this new vertex, we find all vertices of whose distance to is less than . From these vertices, we choose the one minimizing
(22) 
After finding , we add edge to and rewire the tree to update the parents of the vertices located in a distance less than to . Algorithm 2 presents the path planning procedure for the UAVs.
Iv Simulation Results
In this section, we present the simulation results to show the performance of the proposed algorithm. We consider a km km area as shown in Fig. (1(a)). The altitude of the UAVs is m. The transmit power of each GBS is
mW. For the fading term, we consider a Rayleigh random variable with parameter
. Other simulation parameters are: , dB, , GHz, , and . Moreover, we consider and s.To estimate the outage probability, we use a fullyconnected neural network. The neural network has three hidden layers. Each hidden layer consists of neurons. The last layer of the neural network (output layer) has two outputs. We use rectified linear unit (ReLU) activation functions for the hidden layers. For the output layer, we use a softmax activation function. The batch size of each UAV is .
Fig. (1(b)) shows the resulting coverage (1outage) probability in the environment. The number of collaborating UAVs is , the number of training rounds, , is , and the number of SGD updates in each round is . As can be seen, in locations close to the GBSs, we expect to have stronger signals, which lead to a lower outage probability. However, as the distance from the GBSs increases, due to the PL, the outage probability highly increases. It is worth mentioning that to obtain this model, the UAVs do not know the true location of the GBSs. The only information they have is their received signals in different locations. Fig. (1(b)) also shows that connectivity is not available in all parts of the sky. So, it is essential to design proper paths for the UAVs.
Fig. (3) shows the average loss for different numbers of training steps per round (). We observe that as the value of increases, the UAVs have more time to update their models using the same training data. Therefore, the loss will decrease.
Fig. (4) presents the average loss for different numbers of collaborating UAVs. We observe that as the number of collaborating UAVs increases, the loss function decreases. However, this effect is negligible compared to the impact of on the loss function. In fact, by using more UAVs, we get data from different areas of the environment, depending on the flight paths of the UAVs. To use this data, the UAVs need to have enough updates on their model. If the value of is small, increasing the number of collaborating UAVs does not significantly increase the global model’s accuracy.
Fig. (5) presents the trajectory of a UAV for three different pairs of initial and final locations. The values of in (a), (b), and (c) are , and , respectively. To obtain these paths, Algorithm 2 considers coordinates with . We observe that to satisfy the connectivity constraint, the UAV needs to take longer paths which increases its flight time.
V conslusion
In this paper, we studied the radio mapping and path planning problem for a UAV network. We minimized the UAVs’ flight time, ensuring that the UAVs satisfy a probabilistic connectivity constraint during their flights. To solve the problem, we proposed a twostep approach. In the first step, using FL, the UAVs build a global model of the outage probability in the environment based on their collected data. In the next step, using this learned model and rapidlyexploring random trees, we develop a pathplanning algorithm that satisfies the cellularconnectivity requirements.
References
 [1] Y. Zeng, Q. Wu, and R. Zhang, “Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond,” Proceedings of the IEEE, vol. 107, no. 12, pp. 2327–2375, 2019.
 [2] E. Vinogradov, H. Sallouha, S. D. Bast, M. M. Azari, and S. Pollin, “Tutorial on UAV: A Blue Sky View on Wireless Communication,” 2019. [Online]. Available: http://arxiv.org/abs/1901.02306
 [3] S. Hosseinalipour, A. Rahmati, and H. Dai, “Interference Avoidance Position Planning in Dualhop and Multihop UAV Relay Networks,” IEEE Transactions on Wireless Communications, pp. 1–1, 2020.

[4]
B. Khamidehi and E. S. Sousa, “Reinforcement LearningBased Trajectory Design for the Aerial Base Stations,” in
IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 2019, pp. 1–6.  [5] S. Zhang, Y. Zeng, and R. Zhang, “CellularEnabled UAV Communication: A ConnectivityConstrained Trajectory Optimization Perspective,” IEEE Transactions on Communications, vol. 67, no. 3, pp. 2580–2604, 2019.
 [6] E. Bulut and I. Guevenc, “Trajectory Optimization for CellularConnected UAVs with Disconnectivity Constraint,” in 2018 IEEE International Conference on Communications Workshops (ICC Workshops), 2018, pp. 1–6.
 [7] S. Zhang, H. Zhang, B. Di, and L. Song, “Joint Trajectory and Power Optimization for UAV Sensing Over Cellular Networks,” IEEE Communications Letters, vol. 22, no. 11, pp. 2382–2385, 2018.
 [8] B. Khamidehi and E. S. Sousa, “Power Efficient Trajectory Optimization for the CellularConnected Aerial Vehicles,” in IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 2019, pp. 1–6.
 [9] U. Challita, W. Saad, and C. Bettstetter, “Interference Management for CellularConnected UAVs: A Deep Reinforcement Learning Approach,” IEEE Transactions on Wireless Communications, vol. 18, no. 4, pp. 2125–2140, 2019.
 [10] S. Zhang and R. Zhang, “Radio Map Based Path Planning for CellularConnected UAV,” in IEEE Global Communications Conference (GLOBECOM), 2019, pp. 1–6.

[11]
H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas,
“Communicationefficient learning of deep networks from decentralized
data,” in
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS)
, 2017.  [12] J. Konecný, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated Optimization: Distributed Machine Learning for OnDevice Intelligence,” 2016. [Online]. Available: http://arxiv.org/abs/1610.02527
 [13] A. Rahmati, X. He, I. Guvenc, and H. Dai, “Dynamic MobilityAware Interference Avoidance for Aerial Base Stations in Cognitive Radio Networks,” in IEEE Conference on Computer Communications (INFOCOM 2019), 2019, pp. 595–603.
 [14] X. Lin, V. Yajnanarayana, S. D. Muruganathan, S. Gao, H. Asplund, H. Maattanen, M. Bergstrom, S. Euler, and Y. . E. Wang, “The sky is not the limit: Lte for unmanned aerial vehicles,” IEEE Communications Magazine, vol. 56, no. 4, pp. 204–210, 2018.
 [15] B. Khamidehi and E. S. Sousa, “A Double QLearning Approach for Navigation of Aerial Vehicles with Connectivity Constraint,” in IEEE International Conference on Communications (ICC), 2020, pp. 1–6.
 [16] S. Karaman and E. Frazzoli, “Samplingbased algorithms for optimal motion planning,” The International Journal of Robotics Research, vol. 30, no. 7, pp. 846–894, 2011.
Comments
There are no comments yet.