Pursuit-evasion is a game played between two opposing sides, the pursuer(s) and evader(s). The goal of the pursuers is to capture the evaders, while the evaders try to stay free for as long as they can. There exists many variations on the problem, including different numbers of players on each side, definition of capture, pursuer and evader capabilities, and shape and size of the game area, as described in the survey by Chung et al. . The different variations have been motivated by e.g., search and rescue strategies, security robots clearing a building or for understanding the movements of animal predators searching for prey.
Caging is the act of creating a connected perimeter, in two or three dimensions, around a given target such that the target cannot leave the area without breaching the perimeter, but is still free to move within it . While the cage can be physical, like in grasping , it can also be based on sensory detection where the target is simply detected but not obstructed when it tries to escape. In this paper we consider the detection case. This problem can be found in areas such as wildlife surveillance, escorting, security and herding .
In this paper we consider the following pursuit evasion scenario, where the evader is either an aquatic mammal such as a whale, or an intruding submarine or AUV (See Figure 2. Initially there is a first sighting of the evader at a given location. After the sighting, the evader might move, and an uncertainty region of possible locations is growing over time. To contain the evader a large so-called containing cage (see Section III) is formed, possibly making use of terrain such as islands and shallower parts of the water volume. This is done by discretizing the problem into a graph, letting edge costs correspond to the area of the cage segment between the edge on the surface, and the seabed right underneath it, see Figure 2. Pursuers are then sent to create this containing cage, as in Figure 2. This cage is kept until a second sighting is reported. A new uncertainty region is now growing around the sighting, but the pursuers are much closer this time, and the possibility for a spherical so-called capturing cage are explored (see Section IV). If such a cage can be formed with the available pursuers, before the uncertainty region grows beyond it, the containing cage is replaced by a spherical capturing cage, as shown in Figure 2. Finally, this spherical cage is uniformly shrunk until capture, i.e., accurate spotting of the evader by the pursuers own sensors.
The main contribution of this paper is a two-step planning algorithm for underwater pursuit evasion that guarantees capture under computed requirements. The algorithm takes into account the bounded sensing range and movement speed of the pursuers. To the best of our knowledge, no such algorithm is described in the literature.
The agents only need minimal computation on board. The initial large area cage is computed before the operation and the secondary spherical cage formations are computed and stored before the operation. Given the new information after the first cage is established, a suitable secondary cage is simply chosen from the available ones.
The outline of this paper is as follows, first, in Section II we describe related work. Then, in Section III we describe the first algorithm that creates the large area cage while minimizing the number of agents needed. Following this, in Section IV we describe the generation of the spherical cage that eventually captures the evader. In the following Section V we show the overall working of the system to show how the two parts come together, followed by conclusions in Section VI.
Ii Related Work
The vast majority of the work on pursuit evasion has considered 2D environments, as described in recent surveys, [5, 13], and 3D pursuit evasion has largely been an open problem. In , it is noted that most authors use a 2D single-layer representation of the world.
In a recent work , the authors present a control framework for encircling a moving target in 3D. Encircling is an adjacent method to caging, where an incomplete cage is in constant motion and the pursuers speed compared to the evader speed makes the cage possible. The pursuers create a circle on a plane around the evader and use that plane as their basis of movement. While the 2D plane rotates and moves in 3D, the encircling agents are always on that plane thus the evader is never completely caged. In their proposed future work, the authors mention encircling on multiple planes. Encircling on multiple planes is different than caging in 3D in the sense that agents on different planes do not have to keep a maximum distance from each other, they only need to keep the distance in the plane they are part of. A similar encirclement problem is addressed in  with a control method that is fully distributed. They consider the case where any agent only knows the position of the two neighboring agents on the circle. Both of these works rely on the evader being slow and large enough that the encirclement can catch it before it fully escapes. In this paper, we do not rely on the evader being large or slow for capture, once the cage is complete, capture is guaranteed regardless of the physical properties of the evader.
The authors of  present a method of control in 3D to cage a target using multiple agents. In their work they assume that a 3D shape around the evader is given. This shape is not required to be circles on planes unlike . The focus of the work is on keeping this given formation while allowing movement rather than choosing the formation. The methods described in  can thus be used in combination with the methods described in this paper where the methods in this paper generate the shapes required.
In , the authors consider the problem of pursuit-evasion inside a closed and fully observable 2D polygonal space. They assume the pursuers always know the position of the evader and vice versa. They use identical pursuers and evaders in terms of movement capabilities. Their main contribution is that they show that three pursuers are always enough and sometimes necessary to guarantee capture for arbitrary polygonal worlds. The biggest difference between this paper and ours is the knowledge both the pursuers and the evaders have of each other. The assumption of continuous knowledge of evader position is very unlikely to hold in an underwater setting since communication ranges and bandwidth are very limited. The authors of  also consider the problem in 2D.
The work done in  use experiments to evaluate the capability of acoustic communication underwater. They show that recent advances in acoustic underwater communications enables a dynamic multi-agent system such as the one described in this paper. Communication is useful for our work in the sense that the pursuers in our work need to communicate in order to coordinate the phases of operation, and overall performance is thus enhanced by more reliable and frequent communications. The work done in  shows that the system we suggest is indeed realizable.
In this paper, we go beyond the works described above and propose a two-step algorithm for 3D pursuit-evasion. Our method creates a plan that first restricts the evader to a known volume using the minimum number of agents, followed by the creation of a secondary cage that shrinks the volume to guarantee the capture of the evader. We make minimal assumptions about the land in which the operation takes place and we incorporate the sensing models of the agents into the plan.
Iii Minimal Surface Area Prismatic Containing Cage
Iii-a Problem Formulation
In order to formally specify this problem, we first need some definitions.
Depthmap and Freespace
A depthmap is a measurement of the seafloor depth at every point such that any point has depth . . The Freespace is the set of all positions that the pursuers and evaders can occupy,
A contaminated area is a connected subset of which we are interested in containing completely by a cage.
A cage is a single closed loop curve in . The area under the cage is defined as
By construction, separates into two disjoint sets: and . See Figure 2.
Minimum area cage
Find an optimum cage in such that is minimum and the set contains and does not intersect . Our formal problem definition is as follows:
Intuitively, the less wall area the cage has, the fewer number of AUVs will be required to completely cover the cage, given that each AUV is equipped with a finite-range sensor. Distributing circular sensor footprints over a given wall shape corresponds to an instance of the so-called circle packing problem . Here, we note that the number of AUVs needed is proportional to the wall area.
is not a spiral-like shape
has at least two half-lines that start from and go through where both half-lines intersect exactly once.
We explain why we make this assumption later in the paper. This assumption will allow us to devise a method that is computationally efficient.
Iii-B Proposed Approach
In order to solve Problem 1, we discretize the space in the following way:
Consider a graph that is embedded in the plane. Let furthermore the vertices be placed in a grid with distance between non-diagonal neighbors and with edges to the eight closest neighbours.
Thus, , and . Let the boundary vertices
Define a cost function as .
is a discrete graph representation of where the edge costs carry the depth information from (See Figure 2). Each edge represent a parallelogram that starts at the surface and extends down to the seabed. A collection of these parallelograms create a closed surface that separate into and while at the same time separating into two disjoint graphs and . Similarly, the contaminated area is also represented by a sub-graph . With these new sub-graphs available we define the following discrete version of Problem 1.
Minimum surface area discrete cage
Find a discrete closed path , with and that minimizes the surface area such that the graph is composed of two non-connected sub-graphs and . Formally:
A set of solutions can be found in Figure 3.
Pick a vertex in the contaminated set and draw a half-line in some direction. Let be the set of edges that intersect and be the graph that remains after is removed from . Given an optimal closed path , if (a single edge, by Assumption 1) then the optimal open path has the cost:
|And this cost is close to the optimum closed path cost:|
Since was chosen arbitrarily, the selection of the edge does not change the optimality of the produced open path and vice-versa.
Proof by contradiction: Given and are both optimum then , thus the edge must also be the optimum edge. If there exists another edge such that , then there exists another closed path with cost where . Which means has a lower cost than , which is contradictory to being the optimum. Thus a lower cost edge can not exist.
The definition of allows us to separate Problem 2 into two parts. The first part is finding the opening itself, this can be done arbitrarily since the selection of the edge does not change the cost difference between the closed path and the open path as long as Assumption 1 holds.
The second part of the problem is to find a minimal cost connecting path from to under the constraint that it does not intersect with the contaminated set . This is a well-known path planning problem and can be solved with the A* algorithm.
In Algorithm 1 we show the full algorithm that solves this problem. The naive method to solve Problem 2 is to linearly iterate through all and all combinations and running an optimum path finding algorithm such as A* on each, then choosing the minimum cost one. However, this is inefficient. We can make this search more efficient by utilizing multi-start multi-end path finding, binary search and Assumption 1. The following parts explains the details of the algorithm.
Iii-B1 Binary Search on (Line 6)
The first speed-up we get is from eliminating the need for linear search on each . In order to eliminate large numbers of pairs, we first divide into two parts and . For each part, we run the multi-start multi-end A* algorithm where the starting points are on one side and ending points are on the other side of each . This effectively relaxes the constraint and thus gives us a lower-bound cost for all of the paths that have adjacent to . We can compare this lower-bound cost to an arbitrary reference cost . If we can remove from the search, effectively eliminating the need to run A* for all pairs of adjacent to
. As a reasonable estimate for, we choose the closest pair to . This procedure is effectively binary search for the lowest cost on and can be repeated until the minimum cost path on is found.
Iii-B2 Non-spiral Optimum Cage
Given that cuts exactly one edge of the optimum closed path , we need to find only one to find the corresponding but since the shape of is not known, we can not be sure that the we used cuts only one edge. Unless all are searched, it is not possible to be certain that the problem is solved. However, if we can find some number of s that all yield the same closed path, then we can be reasonably sure that the found path is the optimal one. Unless the non-spiral assumption (Assumption 1) breaks, randomly choosing s will eventually lead to the two s we assume exists. This randomized approach allows us to skip a linear search over the half-lines.
In order to test our algorithm, we have generated random depthmaps and random contaminated areas. The depthmaps were generated using smoothed semi-structured noise and the contaminated areas were generated with smoothed random rectangles. The depthmaps were truncated to remove values smaller than such that any point with value corresponds to an island. See Figure 3 for some of the results. The graph was generated such that every pixel in the map is one vertex in .
As can be seen from Figure 3, the cages found often take the longer path that goes through the island parts of the depthmap. Since the islands are essentially zero cost for the cage, meaning we need not put AUVs on islands, using islands might reduce the cost of the operation considerably. In other cases where islands are not present or are too far away from the contamination, our method finds the optimal cage closer to the contaminated area.
Iv Maximum Volume Spherical Capturing Cage
Iv-a Problem Formulation
We formalize the main problem of this section as follows.
Spherical and conical sensors
We assume the AUVs are equipped with sensors such that they can sense a volume of water in the shape of a cone, with opening angle and radius . Note that if the cone becomes a sphere.
Free space, spherical sensors
Given an evader with position with maximum speed that is detected at time in position , and a set of pursuers with positions with maximum speed and sensor range .
Find a set of poses and speed limit such that we can guarantee capture of , in terms of at some point in time being within sensor range of at least one pursuer .
Free space, conical sensors
This problem is the same as Problem 3, but with conical sensors, .
Note that the conical sensor problem can be transformed to the spherical sensor problem by introducing a circular movement pattern to each AUV. This can be achieved by moving each AUV in a spherical pattern such that the conical sensor scans a spherical volume. This movement can also abstract away under-actuated AUVs that can not stand still in water. The necessary margins can then be added to the sensing model to complete the abstraction.
Iv-B Proposed Approach
Informally, the proposed solution for Problem 3 and 4 is composed of three steps. First we compute the maximal cage that can be created, given the number of pursuers and their sensor range . Then we compute the time needed to reach that cage formation. Finally, we compute the size of the contaminated volume, the volume in which the evader must be, at that time. If the contaminated volume is inside the maximal cage when the pursuers establish it, we can guarantee capture at some point by gradually shrinking the cage.
To formalize the approaches we need some definitions.
We extend our previous definition (Definition 2) of a contaminated area to a contaminated volume. Let the contaminated set be given by a sphere of radius , centered at .
Note that, since the evader was observed at time at , we have by construction, that is, the evader must be inside the contaminated set.
Sensor footprint radius
Let be the maximal radius of the circle that is given by intersecting the sensor cone (sphere) with another sphere , when the cone is pointed towards the center of . Note that if is much larger than and is large, the sensor footprint radius will be close to the sensing radius , and for small , , see Figure 5.
Note that this definition lets us generalize our sensor model to any shape that has a circular footprint on a sphere. As long as a circular footprint exists and can be found, other sensor models are also usable. For simplicity, we refer to as our sensor range in the rest of the section.
Given a sensor range , we say that the pursuer positions form a Caging Formation if we can create a closed surface made of triangles with pursuers at the vertices, such that every point of a triangle is within range of at least one .
Note that if an evader starts within a caging formation, it cannot escape without passing one of the triangles of the closed surface, and thereby coming within sensor range of one of the pursuers. Thus, to solve the problems we need to create a caging formation at some time instant such that the closed surface of the formation contains the contaminated set .
The creation of a proper caging formation is divided into three parts. First, we find a good formation for pursuers. Second, we scale the formation so that it is a maximal caging formation given the sensor ranges . Finally, we need to assign agents to all the positions in the formation.
Iv-B1 Spherical Sensors
Here we address Problem 3 using Algorithm 2 and 3. The algorithms are explained in detail below, but we first note that in the pseudo-code, returns the edges of the triangulated convex hull, returns the maximum length of the given line segments, and optimizes the orientation of the cage.
Generic formation for N pursuers (Algorithm 2 lines 1-2)
Since the sphere is the surface of given area that maximizes the enclosed volume, it is reasonable to look for caging formations where the pursuers are positioned on the surface of a sphere.
The problem of regular distribution of points around a sphere is well studied, and known as the Thomson Problem, . It was originally proposed in terms of trying to minimize the electric potential energy of a system of electric charges moving on a sphere. These charges push each other with a force proportional to where is the distance between the charges and .
It has also been shown, that apart from the so-called Platonic solids (PS)  with it is not possible to find a single maximal distance such that for all given they are the closest neighbors, . Platonic solids are regular polyhedra such that every face is the same shape and size thus all edges are the same length.
Inspired by the Thomson Problem we simulate electric charges on a sphere of unit radius in order to find a good caging formation. The relative positions of the points will then be scaled according to given real distances. Since the formation is positioned on the surface a sphere, scaling around is trivial.
Caging formation for N pursuers with given spherical sensor range (Algorithm 2 lines 3-7)
Given a generic formation for pursuers on the surface of a sphere we must now scale it so that it is a caging formation with respect to the sensor range .
If we assume that the formation is made up of equilateral triangles, we have the situation depicted in Figure 4. As can be seen, to make sure every part of the triangle is within sensing range, the pursuer distances must be smaller than . Thus, given and a generic formation for agents on the unit sphere, we can scale the formation so that , as illustrated in Figure 4. Note however, that if all triangles will not be equilateral.
We use the Delaunay triangulation  on the convex hull of the vertices  to get the triangulation of the closed surface, and the corresponding edges. Now there will be some more overlap in the caging, compared to the Platonic solids, but the same scaling will still guarantee a complete cage. This can also be seen from Figure 4 where it is clear that no gap will appear by shortening one or more of the edges.
Assigning the N pursuers to formation positions (Algorithm 3 lines 1-3)
Assuming the previous parts are solved, we have a formation configuration that is some distance away from the starting positions of pursuers and pursuers that we want to move to the vertices of the formation, in such a way that the overall time to completion is minimized. Before doing so, it is possible to further optimize the formation with respect to by considering the rotation around . This can be done using any optimization method of choice. After the rotation, the problem is now reduced to an instance of the linear bottleneck assignment problem .
Pursuer trajectories and required speed (Algorithm 3 lines 4-5)
To finally put everything together we need to move the pursuers from their initial locations at to their respective formation positions . Get them to wait there until all pursuers have reached their , and then shrink the cage by simultaneously moving them towards . The requirement on the pursuer speed is that they reach fast enough so that the contaminated volume is enclosed. This corresponds to where is the maximal travel distance, is the evader speed, and is the radius of the maximal caging formation, as computed previously.
is its normal vector.is a line.
Iv-B2 Extension to Conical Sensors
As noted above, the case of conical sensors is very similar to the spherical case, when the sensor radius is computed as described in Definition 7. The difference is that it is the positions of the center of the sensor footprint that is computed, and that the actual AUVs need to be positioned at the proper distance from the sphere, on a line through the center of the sphere and the center of the footprint. Thus, the distance from needs to be , where , is the positive root of , and . see Figure 5.
In order for the conical sensors to be able to create cage edges when close to the surface or the seabed, we introduce an assumption.
The sensing cone is able to create an edge such that when the agent is located at the intersection of a plane and the cage sphere , the cone covers until the plane, thus needs to hold.
In order to test our algorithms, we have generated caging formations for a fixed set of parameters where we could calculate the optimal inter-agent distances analytically and compared our results. We have tested cases for both Problem 3 and Problem 4. Since our method starts in a randomized state, we have simulated many runs of the same problems to see the variations in the outcomes.
Iv-C1 Spherical Sensors
As can be seen from the final formations in Figure 1, when , the configurations are indeed close approximations of Platonic solids. Looking at Table I we also see that their mean distances are very close to the analytical results, with very small standard deviations.
we also see that their mean distances are very close to the analytical results, with very small standard deviations. For, the formations contain faces that are not equilateral triangles and thus the distances vary much more.
|N||MD Mean||MD Std.||PS Edges||FR Mean||FR Std.|
The fact that the distances are the same for all triangles of the Platonic solids make them very efficient for caging purposes, as the formation can be scaled up to minimize the overlap in sensor footprints. Looking at the radius of the final cage (FR) in Table I and Figure 6 we see that it makes significant jumps in size for the Platonic solids. It is even the case that spreading five AUVs across the sphere makes a less efficient cage than using four. Looking at Figure 6 we see that the irregular increase in FR continues also for larger numbers of AUVs even though the effect is not as strong as for the smaller numbers.
V Tying It All Together
In this section we will show how the methods explained in Section III and Section IV are used to solve the original problem of pursuit evasion in 3D. The first thing to note is that both of the methods can also be used separately where applicable.
In order to paint a clearer picture, we will use the scenario from Section I. In this scenario, we have sightings of evaders near some coast. These sightings constitute a set of points inside the known area . Given and , we use Algorithm 1 in order to find a cage that contains . Points in that have depth smaller than are removed, as we need not cover where there is land. Using circle-fitting  on the remaining areas, we generate the points where the AUVs need to be positioned in order to cover the walls of the minimum area prismatic cage. We have now guaranteed that the evaders can not leave the volume enclosed by . The next step is to ascertain the exact location of the evaders. To do so, we need to get the AUVs into close range.
In order to reduce the computation and communication requirements during operation, we tabulate the solutions of Algorithm 2 for all where is the number of available AUVs from . This tabulation allows the AUVs to choose a formation with center point of the spherical cage (that they can only acquire during the operation). During operation, will be measured once is established and an appropriate spherical cage will be chosen from the table. At this point, the initial positions for Algorithm 3 are the ones generated by Algorithm 1, is estimated on-line and an appropriate spherical cage is chosen from the table.
The full algorithm is given in Algorithm 4. Note that is arbitrary and only affects the circle packing overlap amount. and max are known quantities of the AUVs and the evader. The function circlePack() takes as arguments a cage , sensor footprint radius and depthmap and returns the positions of the AUVs in order to cover the walls between and . The functions minimalAreaCage, maxVolumeSphericalCage and requiredSpeedForCaging are Algorithms 1, 2 and 3 respectively. The function DeployAUVs moves the AUVs from their initial position to and measureC() measures an uncertain location of the evader.
In this paper we have shown the generation of a minimal area cage that contains the evader to a known volume. Followed by this containing cage, we have shown how we can use the AUVs available in order to create the maximum volume capturing spherical cage. Using these two methods, we have shown how a two-step plan can guarantee capture of an evader. Our methods rely on little information about the evaders state. They cage all possible locations of the evader at once and then shrink it in order to capture it. Due to this way of capture, it is suitable to use where continuous sensing of the evader is not possible. An example of such a scenario was discussed.
The methods explained require little computation during the mission, most of the computation can be done offline and then re-used as needed. Communication between vehicles is only required when moving between the two steps.
In the future, we would like to take into account the uncertainties in the underwater domain. Localization and sensing uncertainties, together with sparser available maps make the task of capturing an evader more challenging.
This work was supported by Stiftelsen for Strategisk Forskning (SSF) through the Swedish Maritime Robotics Center (SMaRC) (IRC15-0046).
-  M. Aranda, G. López-Nicolás, C. Sagüés, and M. M. Zavlanos. Three-dimensional multirobot formation control for target enclosing. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 357–362, Sept 2014.
-  C. Bradford Barber, David P. Dobkin, and Hannu Huhdanpaa. The quickhull algorithm for convex hulls. ACM Trans. Math. Softw., 22(4):469–483, December 1996.
-  Deepak Bhadauria, Kyle Klein, Volkan Isler, and Subhash Suri. Capturing an evader in polygonal environments with obstacles: The full visibility case. The International Journal of Robotics Research, 31(10):1176–1189, 2012.
-  Rainer E Burkard, Mauro Dell’Amico, and Silvano Martello. Assignment problems, revised reprint, volume 125. Siam, 2009.
-  Timothy H. Chung, Geoffrey A. Hollinger, and Volkan Isler. Search and pursuit-evasion in mobile robotics. Autonomous Robots, 31(4):299, Jul 2011.
-  B Delaunay. Sur la sphere vide. Bulletin de l’Académie des Sciences de l’URSS, 6:793–800, 1934.
-  Jonathan Fink, M Ani Hsieh, and Vijay Kumar. Multi-robot manipulation via caging in environments with obstacles. In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages 1471–1476. IEEE, 2008.
-  Antonio Franchi, Paolo Stegagno, and Giuseppe Oriolo. Decentralized multi-robot encirclement of a 3d target with guaranteed collision avoidance. Autonomous Robots, 40(2):245–265, Feb 2016.
-  J.J. Thomson F.R.S. Xxiv. on the structure of the atom: an investigation of the stability and periods of oscillation of a number of corpuscles arranged at equal intervals around the circumference of a circle; with application of the results to the theory of atomic structure. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 7(39):237–265, 1904.
-  Mhand Hifi and Rym M’hallah. A literature review on circle and sphere packing problems: models and methodologies. Advances in Operations Research, 2009.
-  Anthony Pugh. Polyhedra: a visual approach. Univ of California Press, 1976.
-  Brooks Reed, Josh Leighton, Milica Stojanovic, and Franz Hover. Multi-vehicle Dynamic Pursuit Using Underwater Acoustics, pages 79–94. Springer International Publishing, Cham, 2016.
-  Cyril Robin and Simon Lacroix. Multi-robot target detection and tracking: taxonomy and survey. Autonomous Robots, 40(4):729–760, Apr 2016.
-  Alberto Rodriguez, Matthew T Mason, and Steve Ferry. From caging to grasping. The International Journal of Robotics Research, 31(7):886–900, 2012.
-  Anastasiia Varava, Kaiyu Hang, Danica Kragic, and Florian T Pokorny. Herding by caging: a topological approach towards guiding moving agents via mobile robots. In Proceedings of Robotics: Science and Systems, 2017.
-  David J. Wales and Sidika Ulker. Structure and dynamics of spherical crystals characterized for the thomson problem. Phys. Rev. B, 74:212101, Dec 2006.
-  W. Yao, Z. Zeng, X. Wang, H. Lu, and Z. Zheng. Distributed encirclement control with arbitrary spacing for multiple anonymous mobile robots. In 36th Chinese Control Conference (CCC), pages 8800–8805, July 2017.