Swarms of unmanned aerial vehicles (UAVs) will play an important role in various services ranging from delivery of goods to monitoring  and . To deliver those services, UAV swarms will employ machine learning (ML) for executing various tasks such as consensus trajectory planning, target recognition, and localization. However, due to the high altitude and mobility of UAVs, continuous connections between UAVs and ground base stations (BSs) cannot be guaranteed. Hence, using centralized ML approaches to execute learning-related tasks will be challenging, particularly when transmitting a large volume of data over aerial links. Instead, a distributed learning approach would be more apropos . In particular, one can use federated learning (FL) to enable each UAV to perform distributed ML tasks without relying on any centralized BSs . In this case, UAVs do not need to send any raw data to BSs when training learning models.
In essence, FL allows each UAV in a swarm to train its learning model based on its own collected data, and it can use the intra-swarm network to share FL parameters related to the learned models with other UAVs. As the learning process proceeds, UAVs in the swarm can reach a consensus on their collective learning tasks, e.g., trajectory planning or target recognition. However, since the updates of the learning models in FL are transmitted over a wireless network, the FL convergence and task consensus for the UAV swarm will inevitably be affected by wireless factors such as transmission delay. Also, due to the high mobility of UAVs, other factors (like wind and mechanical vibrations) can increase the uncertainty of wireless channels by affecting the UAVs’ antenna angles which, in turn, will impact the FL convergence.
A number of recent works have investigated how wireless communication impacts FL [5, 6, 7]. For instance, in , the authors solve the joint learning, wireless resource allocation, and user selection problem to minimize the FL convergence time while optimizing the FL performance. Also, the work in  proposes a strategy for bandwidth allocation and device scheduling to improve the energy efficiency for networks implementing FL. Moreover,  studies the impact of different scheduling policies on the performance of FL. While interesting, none of these works in [5, 6, 7] considers the role of FL in a UAV swarm. Also, due to the high mobility of UAVs and their limited energy, the analysis in [5, 6, 7] cannot be directly applied for UAV swarms.
The main contribution of this paper is a novel framework for enabling FL within a swarm of wireless-connected UAVs. In particular, we first conduct a convergence analysis for FL to show how wireless factors within the UAV swarm impact the convergence of FL. We then determine the convergence round, defined as the minimum number of communication rounds needed to achieve FL convergence. Using this key insight, we formulate an optimization problem that jointly designs the power allocation and scheduling for the UAV swarm network to reduce the FL convergence round. In particular, due to the stringent energy limitations of UAVs, we consider the constraint of the energy consumed by learning, communications, and flying during FL convergence. We also take into account the delay constraint imposed by the control system to guarantee the stability of the UAV swarm. To solve the joint design problem, we use a sample average approximation approach from stochastic programming along with a dual method from convex optimization. To the best of our knowledge, this is the first work that implements FL for the UAV swarm, studies the impact of wireless factors on the convergence of FL, and optimizes the FL convergence by jointly designing power allocation and scheduling of the UAV network. Simulation results validate the convergence analysis of FL and show that the joint design can reduce the convergence round by as much as % compared with baselines without the joint design.
The rest of the paper is organized as follows. Section II presents the system model for the UAV swarm. Section III analyzes the FL convergence and shows the joint system design. Section IV provides simulation results, and conclusions are drawn in Section V.
Ii System Model
Consider a swarm of wirelessly connected autonomous UAVs flying at the same altitude, as shown in Fig. 3LABEL:sub@systemmodel1. The UAV swarm consists of a leader and a set of followers. Every follower keeps a target distance and speed with the leader. While flying, the UAV swarm collects data and performs FL for data analysis and inference tasks like trajectory planning and cooperative target recognition. Using FL, each follower uses its collected data to train a local FL model and send the parameters related to the learned model to the leading UAV in the uplink, as shown in Fig. 3LABEL:sub@systemmodel1. The leading UAV will integrate all received information to generate a global FL model, and, then, transmit the parameters of the global model to following UAVs over the downlink. Moreover, to guarantee that the followers fly with the same speed while keeping a safe distance, the leading UAV will also broadcast the target spacing information and its speed and heading direction.
Ii-a Federated learning model
In the learning model, we assume that UAV collects a set
of input data where each collected sample is represented by a vector, that captures the input features and is the number of collected samples. We also assume the input sample , , corresponds to a single output . The output vector is thereby for UAV . We define a vector as the parameters related to the local FL model that is trained by and at UAV . The convergence of the FL training processes requires each local learning vector to converge to a vector which solves the following problem:
where is the total number of the collected samples by all followers, and
captures the loss function when using learning vectorfor dataset . Note that, the loss function
plays a pivotal role in determining the FL performance, and the expression of the loss function is application-specific. For example, for a simple linear regression FL algorithm,.
To solve (1), the FL framework uses an iterative update scheme . In particular, the leading UAV will first generate an initial global FL model represented by vector and send the initial vector to all followers. Hence, in the first communication round, follower will first use for its own data to train the local model and, then, it sends the vector of the trained model to the leader. Next, the leading UAV will aggregate all received local FL vectors and update the global FL model vector which will be later transmitted to the followers. Each communication round will be followed by another round, and the same process will repeat among leader and followers in each round. In this case, as FL proceeds, the local and global models are sequentially updated, and the total loss for the updated global model with vector will continuously decrease . To identify whether the optimal solution is found for (1), one must analyze the convergence of the loss function to . That is, when the gap between the current loss and the minimal loss is below a threshold , the FL optimization problem is solved . Therefore, we can use the convergence of to to quantify the FL performance.
Moreover, for each communication round, we can divide the total time duration into two periods: Uplink and downlink transmission. In particular, to guarantee that the leading UAV has enough time to process all received models from its followers, all uplink transmissions should be completed within a target time , where is a scheduling parameter to schedule uplink-downlink traffic in time. Also, to receive the global FL model update from the leading UAV successfully, the time constraint for downlink transmissions is thereby . In this case, if the communication link between follower and leader fails to meet the time constraints and , the global FL model cannot use the corresponding FL model for the aggregation. At the same time, for the local FL model, the following UAV cannot use the recently updated global vector to train its local data. In other words, the transmission delay of the uplink and downlink links will impact the update of the global and local FL models thus having a major impact on FL convergence.
In addition, when training the global FL model, we can calculate the energy consumption for the UAV as , where captures the energy consumption coefficient depending on the computing system and is the number of computing cycles needed per data bit . is the frequency of the CPU clock of UAVs, and is the packet size of , transmitted from UAV , in bits. Similarly, we can determine the training energy consumption for follower as .
Ii-B Communication model
To minimize the interference from other UAVs located outside of the swarm, we assume that all UAVs use directional antennas, as shown in Fig. 3LABEL:sub@systemmodel1, However, as shown in Fig. 3LABEL:sub@systemmodel2
, due to the impact of wind, payload, and non-ideal mechanical and control systems, the angle of the UAVs will randomly fluctuate and deviate from the initial angle setting. Based on the central limit theorem, we model the angle deviation for each UAV as a Gaussian random variable. Moreover, we consider a squared cosine function to capture the antenna aperture of UAV when communicating with UAV as follows :
where is the initial angle setting for UAV when communicating with UAV ,
is the angle deviation with variance, and captures the antenna gain at the side lobes. Also, similar to , we can approximate (2) by using a sectionalized expression:
To reduce the interference over the uplink transmissions, we assume that uplinks do not share the wireless resource with each other. Hence, the transmission delay of the uplink between follower and leader can be calculated as
where is the bandwidth used by each subchannel in the uplink, is the transmission power of UAV with maximum power as , and is the path-loss exponent. is the channel gain of the Rician fading channel between UAVs and , and is the noise power spectral density. Note that, despite the use of directional antenna, the swarm still experiences uplink interference generated by UAVs located outside of the swarm. In particular, these interfering UAVs share the same channel resource and exist in the main lobe of the UAV , and we define as the set of UAVs that generates interference to the uplink from UAV to UAV .
Similarly, we can derive the transmission delay for the downlink from UAV to UAV as:
where is the downlink bandwidth, is the transmission power of UAV , and refers to the set of UAVs that will generate interference at the downlink.
Ii-C Control model
To guarantee constant speed and altitude and avoid collisions between UAVs within the swarm, the leading UAV will broadcast its speed and heading direction to the followers in the downlink. Here, the control system of each follower will use both its sensor data (e.g. location) and information received from the wireless links to coordinate its movement and achieve a target spacing and speed. Note that the target distance between the UAV leader and each follower is predefined such that there will be no collision between two nearby UAVs.
Similar to our previous work in 
, we can build a Cartesian coordinate system to capture the locations of UAVs in the swarm, and, then, we decompose the velocity of each UAV into two components, as shown in Fig.3LABEL:sub@systemmodel2. We can also define the control law of each UAV the same way as the one provided in . Since the transmission delay will have a negative impact on the stability control of the UAV swarm, we must consider the delay requirement imposed by the control system when designing the UAV network.
In addition, in order to fly with a constant speed and maintain a stable flying motion, each UAV must spend energy to overcome the gravity and the air drag forces due to the wind and forward motions. For a forward speed with as the maximum speed, the minimum flying power of UAV is , where is the induced velocity required for constant speed and given thrust with being the UAV mass and being the gravitational constant . Also, the induced velocity can be obtained by solving the following equation :
where and capture, respectively, the number and diameter of the UAV rotors, and is the air density. Moreover, we can further correct the theoretical minimum motion power consumption by the overall power efficiency of the UAV in order to obtain the actual power consumption as . Since the control of a UAV’s dynamic motion consumes the most energy , we must consider the flying energy consumption when designing the swarm of UAVs. In particular, the flying energy consumption can be calculated as during the flying time .
To guarantee the convergence of FL and the stable operation of the control system in the UAV swarm, we need to properly design the wireless communication network. At the same time, to guarantee that the energy spent on learning, communication, and flying will not exceed the energy limitation of each UAV, we need to consider the energy consumption during the FL convergence. Next, we first conduct the convergence analysis for the FL algorithm and derive the number of communication rounds needed to achieve the FL convergence. Then, we formulate an optimization problem that jointly designs the power allocation and scheduling policy to minimize the convergence round of FL while considering the delay requirement from the control system and energy consumption during the FL convergence.
Iii Convergence Analysis and Joint Design
Iii-a FL convergence analysis
In order to guarantee FL convergence, we assume that the following UAVs adopt a standard gradient descent method to update their local FL models . Thus, for following UAV , the local model at communication round is given by
where is the global FL model at communication round , is the learning rate, and . After the leading UAV collects local vectors , the global FL model can be updated:
However, for ensuring successful updates of both global and local FL models as shown in (7) and (8), the transmission delay of uplink and downlink should be within, respectively, and . Hence, after considering the impact of the transmission delays, we can rewrite the global FL model update as
With the aim of quantifying the convergence of FL, we use the notion of a convergence round, defined as the minimum number of communication rounds needed to achieve a target difference of the expected gap between current loss and the minimal loss, i.e., . Moreover, to determine the convergence round, we make the following two standard assumptions: Function is continuously differentiable, and the gradient of is uniformly Lipschitz continuous with positive parameter . We also consider the function to be strongly convex with positive parameter , and these exists constants and , meeting . Given the above assumptions, we can derive the convergence round.
To realize an expected convergence of under an accuracy threshold , i.e., , the convergence round is given by:
where is the ceiling function, and captures the convergence speed given as follows
Due to the space limitation, the proof is included in Appendix -A. ∎
As shown in Theorem 1, the convergence performance of FL depends on the transmission delay of both uplink and downlink in the network. In particular, to increase the convergence speed, we need to maximize the probability that both uplink and downlink meet the corresponding delay requirements of FL. Thus, Theorem 1 provides a concrete characterization of the interplay between wireless communications and FL performance in a UAV swarm.
For the stability analysis of the control system, we will follow the method provided by our previous work in . That is, we first build the augmented error state vector. Then, we use Lyapunov-Razumikhin theorem to derive the control system delay requirements , for downlink that can guarantee the stability of the UAV swarm.
Iii-B Problem formulation and solution concept
Here, we formulate an optimization problem to minimize the convergence round by jointly designing the power allocation and scheduling for the UAV network, as follows:
where vector . Constraint (13) guarantees that the probability of total energy consumption for the leading UAV being less than a threshold will be greater than . Similarly to (13), constraint (14) represents the constraint on energy consumption of each follower . Constraint (15) guarantees that the UAV communication network is reliable to support the stability of the swarm with probability . Constraints (16) and (17) ensure that the optimization variables, i.e., the transmission power, scheduling parameter, and velocity, are chosen within reasonable ranges. Note that, in the optimization problem, we also optimize the operation speed of the UAV swarm to minimize the motion energy consumption and relax the energy constraints in (13) and (14).
Since both exponent and base in the logarithm function (10) are less than , minimizing the logarithm function in (12) is equivalent to minimizing the base for the constant exponent. Also, according to (11), we can simplify (12) as
We observe that, after simplifications, both objective function and constraints are represented by probability terms. In this case, directly deriving the probability terms will be challenging since it requires multidimensional integrations. Also, as the optimization problem is not convex, employing convex approximations to simplify the optimization problem will be impossible. Instead, we use a sample average approximation approach where the probability terms in the objective function and constraints are replaced by an empirical distribution found by random samples . In particular, we first generate independent samples of the random parameters, i.e., wireless channel gains and angle deviations, and we calculate the corresponding transmission delay and convergence round. Then, we can reformulate the optimization problem as
where the indicator function , once ; otherwise, we have
. Due to the presence of the indicator function, the reformulated problem is non-smooth. To obtain a smooth problem, we can further replace the indicator functions with modified sigmoid functions, i.e.,, where determines how quickly the modified sigmoid function changes near . To obtain a sub-optimal solution to the reformulated optimization problem with the indicator functions replaced by the modified sigmoid functions, we can use the dual method . In particular, the Lagrangian function is
where vector is the vector of Lagrangian multipliers, and the dual objective function can be defined as . The corresponding dual optimization problem is
where the terms are expressed by optimized variables . The proof of subgradients is similar to the one provided in , and is omitted here. Thereby, we can solve the problem in (24) by either the subgradient method or the ellipsoid method, and their complexities are, respectively, and with accuracy . Then, the sub-optimal solution of can be obtained by solving dual objective function . In particular, similar to , we use the iterative method to sequentially derive the sub-optimal value of each element in (the details are omitted here due to space limitations). Note that, we assume that all these steps of solving the optimization problem are done by a central unit (e.g., cloud or BS), before the swarm starts training their learning models in FL. In particular, there is no need for the central unit to collect any information from UAVs, since all samples of wireless channel gains and antenna deviations are randomly generated by the central unit itself. Also, since the number of UAVs in the swarm is usually small, the complexity of using sample average approximation and dual approach will be low. As a result, the central unit can readily obtain the sub-optimal solution to the joint design problem and later send the power allocation and scheduling parameters to UAVs in the swarm.
|Number of followers|
|Transmission power threshold||W|
|Maximum speed||m/s |
|Energy consumption efficient|||
|Number of cycles needed per bit|||
|Frequency of the CPU||cycle/s|
|Time for each communication round||s|
|Side lobe gain , path loss exponent||dB,|
|Noise spectral density||dBm/Hz|
|Packet size and||kB|
|Number of rotors and the diameter||, m |
|Power efficiency and density of the air||%, kg/ |
|Number of samples , Energy limits||, J|
Iv Simulation Results and Analysis
For our simulations, we first validate the theoretical analysis in Theorem 1. Then, we show the impact of angle deviations on the convergence of FL, and we compare our joint design with baseline schemes that optimize power allocation and scheduling separately. In particular, we consider two baselines. The first baseline is a system with optimized power allocation (same power allocation in the joint design) and randomized scheduling parameters. The second baseline is a system with optimized scheduling (same scheduling used by the joint design) and randomized power allocation. We also assume equal uplink and downlink bandwidths, i.e., MHz, and equal angle deviation variance for each UAV, i.e., . All simulation parameters are summarized in Table I.
Fig. 4 shows the convergence round versus the difference threshold . Note that, in Fig. 4, we choose the range of based on the value of , and the range of will be varied for different settings of data and initial global FL model and the accuracy requirement. As observed from Fig. 4, the theoretical analysis derived in Theorem 1 is aligned with the simulation results with less than % difference, thus corroborating the validity of Theorem 1. Moreover, Fig. 4 shows that, when the difference threshold increases, the convergence round decreases. This is because, with a larger difference threshold, the requirement of convergence becomes less stringent. In this case, FL requires fewer communication rounds to converge.
Fig. 5 shows the convergence round when the variance of angle deviations changes. From Fig. 5, we observe that, when the variance of angle deviations increases, FL needs more communication rounds to converge. This is due to the fact that, when the angle deviation variance increases, the antennas at transmitter and receiver in the network will be less aligned, leading to a drop in the antenna gains’ product between transmitter and receiver in (4) and (5). As a result, the transmission delay of wireless links will increase, and the probability of meeting the delay requirements, i.e., , decreases. Therefore, more communication rounds are needed to achieve the FL convergence. Moreover, as shown in Fig. 5, when the bandwidth allocated to uplink and downlink increases, the FL algorithm requires fewer communication rounds to achieve convergence. This stems from the fact that, a large bandwidth improves the probability of meeting the delay requirements, yielding a fast FL convergence.
Fig. 6 compares our proposed joint power allocation and scheduling design with the baselines without a joint design. It is shown that, for the same network setting, the convergence round for a network with joint design is always less than its counterparts of baselines. In particular, when the bandwidth is MHz, the system with a joint design reduces the convergence round by as much as % compared with the baseline system with optimized scheduling and randomized power allocation design. Moreover, as shown in Fig. 6, when the bandwidth assigned to uplink and downlink increases, the performance gap between the system with the proposed joint design and the baselines decreases. That is because, as we increase the bandwidth, it becomes more probable for all three systems to meet the delay constraints at uplink and downlink. Therefore, the impact of communications delay on the FL convergence will be minimized.
In this paper, we have studied the possibility of implementing FL over a swarm of UAVs. In particular, we have carried out a convergence analysis to study the impact of wireless factors, such as transmission delay and antenna angle deviations, on the convergence of FL. Using the derived insight, we have jointly designed the power allocation and scheduling policy for the UAV swarm to optimize the convergence performance of FL while guaranteeing the stability of control system and controlling the energy consumption. Simulation results have corroborated the convergence analysis of FL and showed the merits of the proposed joint design.
-a Proof of Theorem 1
According to the assumptions about function made in Section III, we know that function is continuously differentiable, and the gradient of is uniformly Lipschitz continuous, i.e., for some positive parameter , ; the function is strongly convex with positive parameter : . If is twice-continuously differentiable, these two assumptions are equivalent to . Also, following a standard assumption in stochastic optimization, we consider that there exists constants and , meeting .
In this case, since the global FL model is the aggregation of all local FL models, the global FL model without the impact of the transmission delay can be given as
After taking into account the impact of transmission delays, we can rewrite the global FL model update as
where . Based on the assumption on the uniform Lipschitz continuity and strong convexity, we can have the following inequalities:
Since , we can simplify (-A) when the learning rate is as
To find a lower bound on the norm of , we can minimize both sides of (-A) with respect . The minimal value of the left side of (-A) is achieved when , and the minimal value of the right side of (-A) is realized when . Particularly, we have
When replacing with in (30), we can obtain a lower bound for the norm of as
According to , when , we can achieve the strong expected linear convergence, i.e.,
According to the strong expected linear convergence requirement, we know that the convergence rate satisfies . By using the results in , we have the following inequality:
Therefore, for the convergence rate , we have
We can further determine the convergence round needed to achieve a target difference threshold, i.e., , as follows:
where in (a), we use the fact that . Since the convergence round must be integral, we can have the results in Theorem 1.
-  M. Mozaffari, W. Saad, M. Bennis, Y. Nam, and M. Debbah, “A tutorial on UAVs for wireless networks: Applications, challenges, and open problems,” IEEE Communications Surveys Tutorials, vol. 21, no. 3, pp. 2334–2360, Third Quarter, 2019.
-  M. Mozaffari, A. Kasgari, W. Saad, M. Bennis, and M. Debbah, “Beyond 5G with UAVs: Foundations of a 3D wireless cellular network,” IEEE Transactions on Wireless Communications, vol. 18, no. 1, pp. 357–372, Jan 2019.
-  J. Park, S. Samarakoon, M. Bennis, and M. Debbah, “Wireless network intelligence at the edge,” Proceeding of the IEEE, to appear, 2019.
-  J. Konecný, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527, 2016.
-  M. Chen, H. V. Poor, W. Saad, and S. Cui, “Convergence time optimization for federated learning over wireless networks,” arXiv preprint arXiv:2001.07845.
-  Q. Zeng, Y. Du, K. K. Leung, and K. Huang, “Energy-efficient radio resource allocation for federated edge learning,” arXiv preprint arXiv:1907.06040, 2019.
-  H. H. Yang, Z. Liu, T. Q. S. Quek, and H. V. Poor, “Scheduling policies for federated learning in wireless networks,” IEEE Transactions on Communications, to appear, 2019.
-  N. H. Tran, W. Bao, A. Zomaya, N. Minh N.H., and C. S. Hong, “Federated learning over wireless networks: Optimization model design and analysis,” in Proc. of IEEE International Conference on Computer Communications, Paris, France, Apr. 2019.
-  Y. Pan, C. Pan, Z. Yang, and M. Chen, “Resource allocation for D2D communications underlaying a NOMA-based cellular network,” IEEE Wireless Communications Letters, vol. 7, no. 1, pp. 130–133, Feb. 2018.
-  M. T. Dabiri, H. Safi, S. Parsaeefard, and W. Saad, “Analytical channel models for millimeter wave UAV networks under hovering fluctuations,” arXiv preprint arXiv:1905.01477, 2019.
-  ITU-R, Mathematical models for radiodetermination radar systems antenna patterns for use in interference analyses, Recommendation ITU-R M.1851-1, Jan. 2018.
-  T. Zeng, M. Mozaffari, O. Semiari, W. Saad, M. Bennis, and M. Debbah, “Wireless communications and control for swarms of cellular-connected UAVs,” in Proc. of IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, Oct. 2018.
-  J. K. Stolaroff, C. Samaras, E. R. O’Neill, A. S. Mitchell, and D. Ceperley, “Energy use and life cycle greenhouse gas emissions of drones for commercial package delivery,” Nature Communications, vol. 9, no. 409, pp. 1–13, Feb. 2018.
-  D. P. Bertsekas and J. N. Tsitsiklis, Neuro-dynamic programming. Athena Scientific Belmont, MA, 1996, vol. 5.
-  B. K. Pagnoncelli, S. Ahmed, and A. Shapiro, “Sample average approximation method for chance constrained programming: theory and applications,” Journal of optimization theory and applications, vol. 142, no. 2, pp. 399–416, 2009.
-  W. Yu and R. Lui, “Dual methods for nonconvex spectrum optimization of multicarrier systems,” IEEE Transactions on Communications, vol. 54, no. 7, pp. 1310–1322, July 2006.
-  Y. Nesterov, Lectures on convex optimization. Springer, 2018, vol. 137.
-  F. Zhou, Y. Wu, R. Q. Hu, and Y. Qian, “Computation rate maximization in UAV-enabled wireless-powered mobile-edge computing systems,” IEEE Journal on Selected Areas in Communications, vol. 36, no. 9, pp. 1927–1941, Sep. 2018.
-  M. P. Friedlander and M. Schmidt, “Hybrid deterministic-stochastic methods for data fitting,” SIAM Journal on Scientific Computing, vol. 34, no. 3, pp. A1380–A1405, 2012.
-  M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” arXiv preprint arXiv:1909.07972, 2019.