Coordination strategies for unmanned aerial vehicles (UAVs) has been an active area of research especially in the realm of multi-agent systems over the past decade [1, 2, 3], having numerous applications, including agriculture, aerial surveying, fire detection, disaster management, weather monitoring, and commercial product delivery. Recent analyses of the commercial UAV market show that their use is expected to grow manyfold over the coming years, as aerial drones are becoming a household product, used for recreational and industrial purposes alike111http://www.businessinsider.com/commercial-uav-market-analysis-2017-8. These advancements suggest an urgent need to explore designs for airspace safety systems that can regulate the traffic and usage of UAVs in a large scale. Similarly, UAVs already play a major role in military engagement scenarios, and their use as part of swarm tactics (encirclement, coordinated attack, search and rescue, perimeter defense) promises to change future battlefield operations. It should therefore come as no surprise that a great amount of work has been devoted over the past decade to study coordination strategies of multi-agent UAV problems . To this end, UAV coordination strategies that formulate the problem as a multi-player pursuit-evasion (PE) game offer solutions that address many of the challenges involving multi-agent systems such as of collision avoidance, surveillance and target acquisition [5, 6].
The literature for multi-pursuer multi-evader (MPME) problems is actually limited. In most cases, some form of heuristic is introduced in order to make the problem tractable. Ge et al.  proposed three approaches, which include hierarchical decomposition, moving horizon hierarchical decomposition, and cooperative control. Li et al.  also explored a hierarchical approach, while Jin and Qu  proposed a heuristic task allocation algorithm. Extensions to the MPME problem includes problems with incomplete information , nonlinear dynamics , and a mix of continuous and discrete observations . However, finding scalable algorithms which can be implemented in real-life MPME scenarios is still an open problem [13, 14].
This work aims at extending current solution techniques for MPME games involving large teams of UAVs by developing implementable, scalable solutions based on a decomposition of the original MPME problem to a sequence of simpler multi-pursuer single-evader (MPSE) problems. A major enabler for this decomposition is a new result that allows us to characterize each pursuer as active or redundant for each evader. Only the relevant pursuers participate in the MPSE pursuit of each evader. The identification and classification of each pursuer as active or redundant makes use of the classical tool of the Apollonius circle . Previously, the proposed MPME formulation was analyzed with the pursuit and the evading teams being homogeneous . In this paper, we generalize the results in Ref.  to include heterogeneous teams of agents with guarantees on finite-time capture. The videos for the simulation results discussed in this paper can be found in the web222https://youtu.be/KTH9lmdUdRs.
Ii Motivation and Problem Setup
Consider a group of agents (pursuers) guarding a given area of interest. The objective of the agents is to pursue and intercept (where typically ) intruders (or evaders) that may be detected in this area. Some of the relevant questions that arise while solving this problem include: a)Which pursuer(s) should go after which evader(s)? b) How many pursuers should chase each intruder (evader) in order to capture it in the shortest time possible?
Obtaining the answers to the previous questions in their most general form is elusive at this point. Addressing them involves solving a multi-player dynamic game, eventually demanding the solution to a high-dimensional partial differential equation, with the dimensionality increasing as the number of players (). In order to proceed and mitigate this problem, the following assumptions are made in this work.
The pursuers are all faster compared to the evaders.
The pursuers know the instantaneous positions and the velocities of all the evaders, and each pursuer follows a constant bearing strategy with respect to the evader to which it is assigned.
The rationale behind these assumptions is as follows. Under assumption A1, a constant bearing strategy guarantees capture. Furthermore, constant bearing strategy has been implemented successfully in various aerial defense systems .
Formally, we address a pursuit-evasion problem in the Euclidean plane that involves pursuers and evaders. The pursuers’ objective is to capture all the evaders. Capture occurs when one or more pursuers enter the capture zone of an evader (assumed here to be a disk of radius centered at the instantaneous position of the evader). Let denote the pursuer and let denote the evader. Let also denote the set of pursuers and, similarly, let denote the set of evaders. The equations of motion of all the agents are given below
where , and denote the positions of pursuer , and evader , respectively, and and denote the heading angles (control inputs) for the pursuers and the evaders, respectively. In (1) and (2), and are the speeds of and , which are assumed to be constant with (A1).
A potential approach to solve complicated MPME problems is a dynamic “divide and conquer” approach, where the pursuers are divided into several groups corresponding to the evader they pursue at each instant of time. In essence, such divide and conquer strategies formulate the original MPME problem as a sequence of several (simpler) MPSE problems . This approach leads to decentralized (although likely suboptimal) solutions. By analyzing the associated MPSE problems, one may arrive at an efficient dynamic task-allocation algorithm of pursuers to evaders. To this end, we first present an approach for task allocation in MPSE problems using Apollonius circles in Section III.
A schematic of the proposed PE problem with one evader and multiple pursuers, that follow a constant bearing strategy–henceforth referred to as the MPSE problem–is shown in Fig. 1. Since the pursuers are assumed to be following a constant bearing strategy, the problem can be analyzed by tracking the relative distances between the pursuers and the evader. In this regard, the dynamics can be written in the form,
where is the relative distance between pursuer and the evader, and is the corresponding line of sight (LoS) angle. For the MPSE problem, we drop the subscripts for the evader’s speed , and will use instead of (when required) to denote the single evader in this setting. Furthermore, in the MPSE problem, we indicate the pursuers using the subscripts directly and the set . Note that in the case of a constant bearing strategy the bearing angle between a pursuer and the evader remains constant until the time of capture. Using this fact, the instantaneous heading of pursuer (, ) can be obtained from the relation,
which is a function of the instantaneous heading of the evader . The above relation has two possible solutions for each , given , and the solution for which is chosen.
Iii Task Allocation in Multi-Pursuer Single-Evader Problems
In this section, we address task allocation in the MPSE problem by dynamically categorizing the pursuers into active and redundant. The proofs for the lemmas presented in this section can be found in Ref. . First, the formal definitions for time-of-capture and the corresponding capturing pursuer set, that are used to define active and redundant pursuers, in the MPSE problem are provided below.
For a given evading strategy, the time-to-capture is the minimum time so that there is at least one pursuer in the capture zone of the evader, and the capturing pursuer set is the set of pursuers that are in the capture zone of the evader at .
The time-to-capture is always finite since the pursuers follow a constant bearing strategy. Note that at the time of capture, one or more pursuers can be in the capture zone of the evader. Therefore, , where card represents the cardinality of the set. Now, the following definitions establish the notions of active and redundant pursuers.
If there exists an evading strategy for which , then is an active pursuer. Otherwise, is a redundant pursuer.
Given the instantaneous positions of the pursuers and the evader, it is of interest to find a condition to verify whether a pursuer is active or redundant. To this end, Apollonius circles are employed in this work. The Apollonius circle for a pursuer-evader pair is the locus of points where capture occurs, for all possible initial headings of a non-maneuvering evader, given the initial positions of the pursuer/evader pair and assuming that the pursuer follows a constant bearing strategy, see Fig. 2. For the MPSE problem, the Apollonius circle of the pair - is denoted as . It has its center at and radius , where (speed ratio) . The Apollonius circles evolve in time as the players move, but the time dependencies will be dropped for the sake of brevity. Let be the closest point to the evader on the Apollonius circle where collision occurs when the evader goes head-on with the pursuer, as shown in Fig. 2. Therefore the distance of from the evader is . Next, we define Apollonius boundary and analyze its properties to identify the active and the redundant pursuers.
Given the positions all the pursuers and the evader at time , the Apollonius boundary around the evader at time is the set of points
where denotes the set of points on the line segment with endpoints (position of ) and at time .
In other words, the Apollonius boundary is the set of points that belong to the union of all the instantaneous Apollonius circles corresponding to the evaders and, in addition, each such point is the closest to the evader along its respective line-of-sight originating from the evader. Note that the Apollonius boundary evolves with time as the Apollonius circles also evolve with time. The subscript will be dropped for the rest of this section, since we are dealing with the MPSE problem.
Iii-a Apollonius circle based Active Pursuer Check
The algorithm that is developed to identify active/redundant pursuers in this work is based on the following conjecture.
Given the positions of all the players in an MPSE problem at time , and assuming that the pursuers follow a constant bearing strategy, pursuer is active at time if , and is redundant otherwise.
The conjecture implies that a pursuer is active at time if and only if its corresponding Apollonius circle is part of the Apollonius boundary at that instant. The conjecture is inspired from the fact that the region in which the capture point lies in is bounded by the instantaneous Apollonius circle for any strategy of the evader. Note that if a pursuer is active at time , it need not remain active for all . But if a pursuer is redundant at time , it will remain redundant for all . The following lemmas based on Conjecture III.4 provide simple checks to determine whether a pursuer is active or redundant.
 Given the positions of the players in the MPSE problem at time , pursuer is the only active pursuer if and only if the conditions
 Given the positions of the players in the MPSE problem at time , and that the Apollonius circle intersects at least one of the other Apollonius circles. Then, pursuer is an active pursuer if and only if there exists such that , where is the set of intersection points between and the rest of the Apollonius circles.
The set of intersection points can be obtained analytically given the instantaneous positions of all the players . The above two lemmas can be used to verify if a pursuer is active or redundant. In this regard, Algorithm 1 below, named Apollonius circle based Active Pursuer Check (AAPC), can be employed to check the status of each pursuer. The time complexity of the algorithm is of order , since the maximum number of intersections between any two circles is two. Note that by dynamically allocating the task of capturing the evader using AAPC (where at every instant the active pursuers keep pursuing the evader while the redundant pursuers do not react), the pursuers as a group will be able to capture the evader in minimum time. Furthermore, if a pursuer becomes redundant at any point of time , it remains redundant after that (i.e., till capture occurs).
Iii-B Numerical Simulations
In this section simulations of pursuer allocation using AAPC involving five pursuers and one evader are presented. There are three different pursuers with speeds , and the speed of the evader is set to . The radius of capture is chosen as . The evader follows a form of blind evasion strategy with switching times that are predefined . At each switching time the evader randomly chooses a heading from a set of allowable headings. The allowable headings set that is specific to the example showcased in this work is .
Fig. 4(a) shows the initial positions of all the players along with the corresponding Apollonius circles. The triangle denotes the initial position of the evader and the square markers denote the initial positions of the pursuers. It can be observed that at the initial time, the pursuers identified with the colors red, magenta, green, and blue are the active pursuers, as their corresponding Apollonius circles are part of the Apollonius boundary. Fig. 4(b) shows the trajectories of all the players. It can be seen that the green pursuer finally captures the evader, and the rest of the three pursuers, which are initially active, become redundant as time progresses. The cyan pursuer, which is redundant at the initial time, does not move at all.
Iv Apollonius Allocation Algorithm
In this section, the AAPC is extended to solve MPME problems. Given the positions of all the players at some instant of time, the set of evaders for which a pursuer is active can be obtained using AAPC. Note that at a given time instant, a pursuer can be classified as active by more than one evader or no evader whatsoever. In the case where a pursuer is classified as active for more than one evader, one can break the tie by assigning the pursuer to the evader that can be captured in minimum possible time i.e.,.
The aforementioned criterion for breaking a tie is equivalent to choosing the nearest evader when the teams are assumed to be homogeneous .
Using this criterion, the following algorithm can be used for pursuer allocation in MPME problems.
Apollonius Allocation (A2) Algorithm: At a given time instant , let be the set of evaders that are yet to be captured, and let be the set of evaders that have already been captured. Note that . Given the current positions of all the players, let be the initial allocation function that maps each evader (in ) to its set of active pursuers obtained by considering the positions of all the pursuers. That is, for a given , is a subset of . Furthermore, denotes the set of all the active (or assigned) pursuers according to the initial allocation function . Given the initial allocation function , let now be the dual function defined by . In other words, maps each pursuer to the set of the evaders to which it is allotted as per . Next, we define the final allocation function and the intermediate allocation function as follows.
If card, for all , then let . Otherwise, let be defined as . The function maps each evader to a set of pursuers in accordance to the mapping , such that each active pursuer is assigned to the nearest evader among its assigned ones. Note that can be an empty set for some , i.e., an evader can end up be unassigned as per .
Let be the set of unassigned pursuers. Now for each evader , find the active pursuers considering the positions of the pursuers that are only in the set , and obtain an updated allocation function and its corresponding dual .
Repeat steps (a) and (b), by replacing and with and , respectively, until is obtained.
Note that in step (a) of the algorithm, ties with multiple assignments of the same pursuer are broken using distance as the metric. Furthermore, if each pursuer is assigned to only one evader or if it remains unassigned, then the initial allocation function is also the final one. In any other case, once the intermediate allocation function is obtained in step (b) of the algorithm, the set of unassigned pursuers according to is obtained. In step (b), an updated allocation function is obtained by checking for active pursuers among the set of unassigned pursuers coupled with the pursuers assigned as per , for each evader. Because one of the unassigned pursuers (in the set ) can become active to the evaders that have lost one or more pursuers during the tie break in step (a). With and its corresponding dual , steps (a) and (b) are repeated until each pursuer has only one (or none) assignment. Once an evader is captured, it is removed from the set and added to the set .
The above algorithm is run at every time instant to obtain the allocation function , given the players’ current positions, until all the evaders are captured, i.e., until is empty. The algorithm provides a potentially sub-optimal solution, but it is scalable for any number of pursuers and evaders. The algorithm guarantees capture of all evaders as is shown in Theorem IV.4 below. In order to prove this theorem, several preparatory results are needed.
Given the instantaneous positions of the players in the MPME problem at time , the current shortest time (CST) is defined by .
At a given time instant , , where , and is the final allocation function of A2 algorithm.
From Definition IV.1 it is understood that represents the minimum possible time taken to capture an agent in the evading team by an agent in the pursuing team, when the corresponding evader goes head-on with the pursuer. Therefore, if it can be shown that the Apollonius boundary around the evader contains part of the Apollonius circle (i.e., , ), then the pursuer will be assigned to the evader , even when there is a tie. Since the Apollonius circle denotes the capture points for a non-maneuvering evader, the point , with the length of the line segment , is the closest capture point to evader along the corresponding line of sight. Therefore , and hence, , .
Assuming the pursuers are assigned to the evaders using A2, at any given time , CST will converge to zero in finite time, and hence at least one evader will be captured in finite time.
From Lemma IV.2, pursuer (corresponding to the CST) is always assigned to evader . Since all the pursuers are faster compared to the evaders, and since they follow a constant bearing strategy, ddt , for all . Furthermore, as the initial CST is finite, the CST converges to zero in finite time. Hence, capture of one evader is guaranteed in finite time.
The A2 algorithm guarantees capture of all the evaders in finite time.
The result immediately follows from Lemmas IV.2 and IV.3. Note that the CST is updated (from zero) every time a capture occurs, and the captured evader is removed from the list of participating players. Also, the number of evaders are finite.
Fig. 5 demonstrates the performance of A2 algorithm for 10 pursuers and 5 evaders. For the sake of simulation, the evading is assumed to be homogeneous. The simulation parameters remain the same as in Section III-B. In Fig. 5, the red triangles indicate the current positions of the evaders that are not captured and the magenta ones are the evaders that are captured. The blue squares indicate the current positions of the active pursuers and the cyan ones indicate the redundant pursuers. In all three plots, the Voronoi partition of the domain with the evaders as generators is also included for reference.
Under the assumption that the pursuers are faster than the evader(s), and that they follow a constant bearing strategy, workable solutions for multi-pursuer single-evader (MPSE) and multi-pursuer multi-evader (MPME) problems are provided. A dynamic allocation algorithm for the pursuers that is independent of the evader’s strategy has been proposed to solve the MPSE problem. The proposed algorithm is based on the notion of active/redundant pursuers, and employs the concept of Apollonius circles. The algorithm is further extended to solve MPME problems for any number of pursuers and evaders. These algorithms ensure capture of all the evaders either in an MPSE or an MPME setting in finite time. Several extensions of this work are possible. For example, the computational requirements can be reduced by having an estimate of when the assignment can change to avoid unnecessary calculations at every time instant. It would also be very interesting to extend the notion of Apollonius circles to account for turn-radius constraints for all the players, but this would probably end up being a major challenge.
-  Y. Huang, S. J. Thomson, W. C. Hoffmann, Y. Lan, and B. K. Fritz, “Development and prospect of unmanned aerial vehicle technologies for agricultural production management,” International Journal of Agricultural and Biological Engineering, vol. 6, no. 3, pp. 1–10, 2013.
-  F. Nex and F. Remondino, “UAV for 3D mapping applications: a review,” Applied Geomatics, vol. 6, no. 1, pp. 1–15, 2014.
-  C. Yuan, Y. Zhang, and Z. Liu, “A survey on technologies for automatic forest fire monitoring, detection, and fighting using unmanned aerial vehicles and remote sensing techniques,” Canadian Journal of Forest Research, vol. 45, no. 7, pp. 783–792, 2015.
-  Y. Rizk, M. Awad, and E. W. Tunstel, “Decision making in multiagent systems: A survey,” IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 3, pp. 514–529, 2018.
-  T. Mylvaganam, M. Sassano, and A. Astolfi, “A constructive differential game approach to collision avoidance in multi-agent systems,” in Proceedings of the American Control Conference, Portland, OR, 2014, pp. 311–316.
-  J. Las Fargeas, P. Kabamba, and A. Girard, “Cooperative surveillance and pursuit using unmanned aerial vehicles and unattended ground sensors,” Sensors, vol. 15, no. 1, pp. 1365–1388, 2015.
-  J. Ge, L. Tang, J. Reimann, and G. Vachtsevanos, “Suboptimal approaches to multiplayer pursuit-evasion differential games,” in AIAA Guidance, Navigation, and Control Conference and Exhibit, Keystone, CO, 2006, AIAA Paper 2006-6786.
-  D. Li, J. B. Cruz, G. Chen, C. Kwan, and M.-H. Chang, “A hierarchical approach to multi-player pursuit-evasion differential games,” in Proceedings of the 44th IEEE Conference on Decision and Control and European Control Conference, Seville, Spain, 2005, pp. 5674–5679.
-  S. Jin and Z. Qu, “A heuristic task scheduling for multi-pursuer multi-evader games,” in International Conference on Information and Automation, Shenzhen, China, 2011, pp. 528–533.
-  A. Antoniades, H. J. Kim, and S. Sastry, “Pursuit-evasion strategies for teams of multiple agents with incomplete information,” in Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, HI, 2003, pp. 756–761.
D. M. Stipanović, A. Melikyan, and N. Hovakimyan, “Guaranteed strategies
for nonlinear multi-player pursuit-evasion games,”
International Game Theory Review, vol. 12, no. 01, pp. 1–17, 2010.
-  ——, “Some sufficient conditions for multi-player pursuit-evasion games with continuous and discrete observations,” in Advances in Dynamic Games and Their Applications. Springer, 2009, pp. 1–13.
-  M. A. Vieira, R. Govindan, and G. S. Sukhatme, “Scalable and practical pursuit-evasion with networked robots,” Intelligent Service Robotics, vol. 2, no. 4, p. 247, 2009.
-  A. Pierson, Z. Wang, and M. Schwager, “Intercepting rogue robots: An algorithm for capturing multiple evaders with multiple pursuers,” IEEE Robotics and Automation Letters, vol. 2, no. 2, pp. 530–537, 2017.
-  R. Isaacs, Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization. Mineola, NY: Dover Publications, Inc., 1999, Chapter 6.
-  V. R. Makkapati and P. Tsiotras, “Optimal evading strategies and task allocation in multi-player pursuit–evasion problems,” Dynamic Games and Applications, vol. 9, no. 4, pp. 1168–1187, 2019.
-  N. A. Shneydor, Missile Guidance and Pursuit: Kinematics, Dynamics and Control. Horwood Publishing Limited, 1998, Chapters 3, 4.
-  M. V. Ramana and M. Kothari, “Pursuit-evasion games of high speed evader,” Journal of Intelligent & Robotic Systems, vol. 85, no. 2, pp. 293–306, 2017.
-  E. W. Weisstein. Circle-circle intersection. From MathWorld–A Wolfram Web Resource. [Online]. Available: http://mathworld.wolfram.com/Circle-CircleIntersection.html
-  R. W. Morgan and J. L. Riel, “Blind evasion by random switching maneuvers,” in Proceedings of the American Control Conference, Boston, MA, 2016, pp. 3126–3131.