Multi-vehicle coordination has been attracting increasing attention from researchers as it provides much enhanced capability for applications such as search and rescue or mapping. Cooperation between multiple vehicles can lead to faster and more effective missions. For example, a cooperating UAV can fly above obstacles and travel faster than a UGV, whilst providing sensor coverage for a much wider area than that accessible to the UGV. Meanwhile the UGV can inspect the environment more closely and precisely than the UAV, which flies at altitude and may have a partially blocked view. Supporting ground transportation task is a potential application of the UAV-UGV cooperation [7, 8]. In this task, a UAV can be fitted with a camera to capture the visual pose of UGVs moving on the ground, helping to localize the UGVs and guide them along a safe path. In summary, the cooperation of UAVs-UGVs is beneficial in many autonomous missions.
There are two control strategies for a multi-vehicle system: centralized or decentralized. In the former approach, a ground station (GS) collects all required data (posture, velocity, acceleration) and transmits proper control signals to each agent. However, centralized control depends on perfect communication which can be an issue if data dropouts occur due to overloaded networks or when communication range constraints are challenged [1, 2]. With decentralized control, a local controller is designed for each robot, and the control signals are provided by only using local information about agents and their neighbors [4, 5]. This approach significantly reduces the amount of transmission data, time delay and is more robust against communication failures. Therefore, we base our consensus control approach on a decentralized strategy where the agents know only their own velocities, accelerations, Euler angles, and their relative position from other agents.
Within the application of multi-vehicle systems, forming and maintaining a prescribed formation is a real challenge. A large body of work related to multi-robot systems can be found in the literature [12, 13, 14, 16]. More consensus results are shown in [13, 14, 19, 18]. These papers pay much attention to formation stabilization problems, but the internal stability of each vehicle based on these cooperative methodologies is not sufficiently addressed. Furthermore, as seen in [22, 20, 21, 14], agents in a cooperative network are often treated as second-order systems to solve the formation tracking problems. However, the position and velocity tracking loops of the UAV or the UGV are actually represented by a higher order transfer function. For example, the closed-loop velocity control of a quadrotor leads to at least a fourth-order model [10, 11]. Using an improvement in NI consensus theory, our method can solve this problem by considering simultaneously the stability of the formation control and dynamics of agents.
More and more researchers have realized that formation control problems can be handled using consensus-based approaches. For example, the relevant works in [3, 30] proposed time invariant formation controllers for multiple UAVs. However, in most practical tasks, multi-robot systems should also be able to vary their formation over time so that they can make progress towards the goal position while avoiding obstacles in an unknown environment.
The shortcoming of using a fixed formation has been overcome recently by a few researchers. In [13, 14], a time-varying formation controller was tested and validated on five quadrotors. For this method to work, each agent must be able to sense the absolute position and orientation of its neighbor. As a result, time delays or data drop during inter-agent communication may occur as the number of agents increases dramatically. Similarly, a time-varying formation control approach for a UAV based on a virtual leader structure is illustrated in , where the follower vehicles maintain a relative position from a simulated leader represented by an idealized trajectory. Compared to these methodologies [13, 14, 15], our slave agents only utilize measurements of the relative distances from their master and each other, such that no data need to be exchanged during the concensus process.
One of the practical applications of time-varying formation control approaches for avoiding unexpected obstacles, has been considered in [15, 29, 28]. Once a potential collision between the robots and an obstacle is predicted, the robots in  will automatically produce a queuing formation for safely avoiding the corresponding obstacle. However, this strategy cannot be used to avoid multiple obstructions. This issue was solved in . Whenever an obstacle is detected inside the repulsion zone, an escape angle is determined by the robot orientation relative to the real target and the disposition of ultrasonic sensors in the ring, in relation to the axis of robot movement. It is pointed out from the simulation results that the stability of the obstacle avoidance algorithm was not examined. Besides, the control output command pairs based on the behavioral control approach, including turn left/turn right and forward/backward, coincide in some cases since the working area of each control layer is not adaptive. A better formation control strategy for obstacle avoidance was developed by Dai, Y., et al  in 2015. In this work, a safe path for the leader was planned by a Geometric Obstacle Avoidance Control Method (GOACM) where the slaves are moved into a safe formation relative to the leader. Unfortunately, because of sudden changes in the velocity setpoints and control saturation while switching into a safe triangle formation, this method suffers from oscillations in robot position. Moreover, this method is based on the real surface boundary of an obstacle to formulate the rotation angle, making it not applicable for practical situations where obstacles often have arbitrarily complicated shapes.
This paper researches time-varying formation control approaches for multi-robot systems based on the Negative Imaginary (NI) consensus systems theory. Since it was first introduced in 2010 , NI systems theory is becoming popular in many high technology applications, especially in nano-positioning control , vibration control of flexible structures  and multi-DOF robotic arms. According to  and , closed-loop multi-systems can achieve high robustness with respect to external disturbances when certain conditions of NI theory are met in the plant and controllers. However, in practice, the cooperative architecture of this theory shows four disadvantages, which is proven by our simulation as shown in Fig. 1: (1) the NI-consensus-based formation control architecture is not developed in this literature, (2) all agents are controlled to reach one unique rendezvous point, resulting in position consensus at the same point which would lead to collision, (3) the asymmetrical consensus equations between master and slave can cause practical difficulties in tuning the gain values of the SNI controllers, and (4) the input setpoints of each agent such as position/velocity set points are not mentioned in the overall network plant of this theory. To handle these problems, we have previously proposed and tested a new NI architecture for multi-UAV operation . Nevertheless, our previous paper presented only the experimental results of a homogeneous UAV system with an invariant formation and did not mention any potential application for the NI formation control protocol.
Therefore, the significant contributions of this paper compared to our earlier results are four folds: (1) providing an improved formation control protocol via a more effective strategy using next-position prediction of the master and relative distance between agents; (2) simultaneously guaranteeing both stability of distributed time-variant formation and obstacle avoidance control, relative distance modification and velocity control loops via the NI stability criteria; (3) developing a distributed time-varying formation controller for multi-vehicles system using NI systems theory; (4) presenting a new obstacle avoidance approach by varying the formation shape; and (5) considering necessary and sufficient conditions for all obstacle avoidance situations and proposing effective solutions. An additional contribution of our work is introducing a practical strategy for UAV-UGV coordination since not many recent studies consider a consensus methodology for these two different types of unmanned robots.
In order to test our new methodology, three indoor experiments implementing our enhanced NI-systems cooperative approach are conducted using both UAV and UGV platforms. The first test is carried out using two UGVs and one UAV. The autonomous vehicles must maintain a small invariant triangular pattern while the UGV (master) follows a pre-defined rectangular trajectory. The purpose of this test is to prove the effectiveness and stabilization of our proposed method when two types of robots coordinate to perform an indoor exploration task. The second test, which is more complicated than the first one, is used to show the obstacle avoidance capability of our varying formation method in case the master (UAV) predicts a collision with an obstacle and the master (UGV) predicts a collision with multiple obstacles on either side of its desired path. The final test involves three UGVs with randomly located obstacles.
Moreover, in outdoor cooperative scenarios, where the UAV and UGVs are tasked to explore uncertain environment, situations may occur when an absolute position sensor (like GPS) is only available to one vehicle at a time. Because the absolute position navigation may not be available to all vehicles (e.g. GPS interference caused by urban canyon effect), we assume the other vehicles are only able to measure their relative distance using sensors such as machine vision sensors or radio beacons. Therefore, the relative distances between the autonomous robots are used as key components in our cooperative method during indoor experiments. This information is simulated and measured by a Vicon Motion Capture system. Finally, due to the sharp interior angle vertexes of a rectangle (generally being 90 degree), yaw angle consensus among the UGVs is also considered in this paper. This additional angle consensus allows the UGVs to perform smoother movement following a trajectory with sharp corners like a rectangle by rotating simultaneously around the vertical (Z) axis, especially when the master UGV reaches a vertex of the rectangle.
The rest of this paper is organized as follows: Section II introduces the hardware/software configuration and architecture design for the formation experiment. Preliminary graph theory and the NI theory is reviewed in Section III. The dynamic models of the UAV and UGV are analyzed in Section IV. The formation tracking control architecture for an autonomous team (UAVs and UGVs) is studied in Section V. Stability of the whole system is verified in Section VI. Simulation results from implementing the NI formation control architecture in a team of multi-vehicles are shown in Section VII. The geometrical approach for obstacle avoidance is illustrated in Section VIII. Finally, experimental results are presented in Section IX, followed by conclusions drawn in Section X.
Throughout this paper, for simplicity of notation, we let 1
represent a column vector of sizen with all elements being 1. Let represent the Kronecker product and [P(s), P(s)] denote the positive feedback interconnection between transfer functions P(s) and P(s).
Ii Experimental Apparatus
Ii-a Hardware/Software Configuration
One UAV (F450 quadrotor platform), three UGVs (Pioneer P3-AT and P3-DX) and four obstacles (plastic packing boxes) are used to conduct our experiments. The dimension of each packing box is 36cmW36cmD36cmH. The size of each UAV or UGV is approximately 30cmW30cmD
. The position and orientation of each robot are broadcasted continuously from a ground station (GS) where the movements of the UAV and UGVs in three dimensions are analyzed by the Vicon Motion Capture System software that is installed in the station facilities. The master UAV/UGV utilizes this information to correct its trajectory relative to the next waypoint while the two slaves (UGVs or UAV) predict the next position of the master from its estimated velocity and recalculate the relative distance between the master and the slave before adjusting its position to preserve the determined formation in real time. The information exchange protocol between the GS and the robots is achieved using a WiFi network and the Robot Operating System (ROS) framework.
Ii-B Overall Architecture
Our control architecture is separated into two layers (Fig. 2): UGV-Leader layer and Leader-Follower layer. The functionality and the control design of each layer will be described in the next sections.
Iii Preliminaries of graph theory and Negative-Imaginary theory
Iii-a The Basics of Graph Theory
Graph theory can be used to effectively describe the inter-agent communication including N unmanned robots. Each robot is represented as a vertex. Let G = (,) denote a directed graph with N vertexes and L edges (NL matrix), such that = (v, v,…, v) and (v,v): v,v , v v mathematically describe the finite vertexes set and the ordered edges set, respectively. Suppose that each edge of G is assigned an orientation, which is arbitrary but fixed. The (vertex-edge) incidence matrix of G, denoted by Q(i), is a NL matrix defined as follows:
Note that the incidence matrix has a column sum equal to zero, which is assumed from the fact that every edge must have exactly one head and one tail. Extending this graph theory, given an arbitrary oriented set of the edge , the graph Laplacian, , is defined as L(G)= Q(G)Q(G).
For convenience in interpreting further the UAV-UGV cooperative theory, some mathematical information graph matrices are newly described.
Iii-A1 The consensus matrix
Let the NL matrix Q represent a consensus matrix which corresponds to a graph (,) that has N vertexes and L is the number of communication orientations. This NL matrix indicates which robot at vertex v in will send its consensus information to the adjacent vertexes v in .
Iii-A2 The reference matrix
The (NL) reference matrix Q = diagR describes which node or UAV in the group will directly follow to the reference trajectories:
Iii-B NI Systems Theory
The definition of an SNI system was firstly introduced in . According to Lemma III.1, its Nyquist plot should start from an arbitrary point on the real axis, curve below this axis and end at the center point (0,0). Similarly, with a less strict criteria as compared to an SNI system, the definition of an NI system is presented in Lemma III.2.
Lemma III.1 (Petersen, I.R. and Lanzon, A.,2010)
For a single input/single output (SISO) case, a transfer function P(s) is called strictly negative imaginary (SNI) if its Nyquist plot is located strictly below the real axis for all positive frequencies in s domain; i.e, P(s) SNI if j(P(j)-P(j))0 for all 0.
Lemma III.2 (Petersen, I.R. and Lanzon, A.,2010)
P(s) is negative imaginary (NI) if its Nyquist plot for 0 is contained in the lower half of the complex plane. This Nyquist plot can touch, but not cross the real axis; i.e, P(s) NI if j(P(j)-P(j))0
when a SNI/NI plant system is achieved, the internal stability of a positive feedback interconnection between this plant and its NI/SNI controller is illustrated as in Lemma III.3.
Lemma III.3 (Petersen, I.R. and Lanzon, A.,2010)
Considering the case in which an SNI/NI plant M(s) and an NI/SNI controller N(s) such that M()N() = 0 and N() ≥ 0 are interconnected by a positive feedback, it follows that the corresponding loop transfer function L(s) = M(s)N(s) is internally stable if the DC gain condition at zero frequency M(0)N(0) 1 is satisfied.
Since the (-1,0) critical point is never enclosed by the Nyquist plot of a SNI or NI system, it is assumed that any SNI/NI system has a robust stability.
The major concept of this theory can be extended further to MIMO and cooperative control applications  and . Given a group of SNI/NI systems or a multi-input multi-output (MIMO) system, the collaboration of SNI/NI agents is achieved when these systems are connected to their SNI controllers via a cooperative information graph. The SNI/NI-agents cooperation is formulated as follows:
where and are the input and output of the overall network plant, respectively. is the group of SNI/NI consensus controllers for the multiple NI/SNI systems, indicated as . Thus P is a column vector of size M which contains a set of m coordinated outputs.
Based on the consensus theory of Wang [20, 21], an NI-systems consensus architecture for the multi-UAV systems was proposed in . A reference matrix, consensus matrix as well as an offset distance between the UAVs are added to the original structure to solve existing formation and consensus problems as shown in Fig. 3. The overall formulas for the whole cooperative structure are given as:
where r denotes the reference position on the plane of the master, X is the reference matrix, and x corresponds to the relative position between the slave UAVs and the master UAV in the configured formation. u is the velocity set point input of each UAV on x and y axes while the output y is the current position of each UAV. and are the input and output of overall network plant. e is the error between the desired position of the master and its current one, while e is the error between the desired relative position and the current one among the UAVs. is the group of SNI consensus and tracking controllers for the group of UAVs.
The simple formation experiment conducted in this paper shows that consensus based on NI-systems cooperative control between UAVs is guaranteed, but the position response of the remaining UAVs is somewhat slow. This small delay in the position response causes an unsatisfactory formation shape which is also discussed in the conclusion of the Section VII.
In an effort to improve the UAV-UGV formation performance based on the consensus algorithm, we will outline an innovative architecture in the next sections.
Iv Dynamic Modeling for Velocity Control of The UAV and The UGV
First, the linear velocities of the UAV/UGV in the x and y axes are stabilized by using a PID controller, which is tuned to satisfy the closed-loop SNI system requirement. The proposed velocity control structure is illustrated in Fig. 4.
During experimental tests, all relevant data of the master and the slaves (position, linear velocity, acceleration, Euler angles) were automatically collected by the ground station to serve the purpose of finding the closed-loop velocity transfer functions in the lateral and longitudinal directions. In the next step, this data was analyzed using MATLAB to determine the closed-loop transfer functions between the velocity set point input (velxsp/velysp) and the absolute position output (posx/posy) for the UAV/UGV on the x and y axes (Table I, II). We use an ARMAX model to represent the performance of these transfer functions. The overall dynamic equations for the UAV and the UGV in the z domain have a general form as follows:
where u(k) is the system input, y(k) is the system output, n is the system delay, k is the present time and e(k) is the disturbance in the system.
Based on the transfer functions found, the Nyquist diagrams of the UAV and UGVs are plotted to identify if the transfer functions are SNI. As seen in Fig. 5, the linear velocity transfer functions for both the UAV and the UGVs satisfy the SNI conditions.
Similarly, the yaw rate transfer functions for UGVs are also shown to be SNI systems using their Nyquist plots.
V Architecture design
In this section, an architecture for the NI-system formation and consensus control is newly designed. Firstly, all of the SNI velocity transfer functions for the x and y axes, found in the previous section, are combined to form an SNI plant. The inputs of this plant are the velocity set points of the SNI systems (V, V) while its outputs correspond to the current locations of the vehicles on the x and y axes as shown in Fig. 6. In the next step, the network matrix ([Q Q]I) is appended to the output of plant in order to calculate the relative distances between the robots and the position error between the master and the slaves (dx;dy;dx;dy). Similarly, the matrix ([Q Q]I) is appended to the input of the plant to determine the desired velocity values for each robot via the NI consensus controller gains K, the desired relative distance between robots (X, Y) and the next position prediction of the master (p, p). This control architecture is referred to as the overall network plant for consensus (Fig. 7).
The distance between the current position of the master and its next position (p, p) in Fig. 8 is predicted from its current velocity on the horizontal plane. The corresponding equations are as follows:
where V and V are the master horizontal and vertical velocities, respectively. This data is sent continuously to every slave agent. Here, dt represents the sample time for the whole system.
Finally, the overall network plant is then combined with the remaining parts of the consensus controller, including the NI controllers, the relative positions and the future position prediction, to create a new formation consensus control method as shown in Fig. 8.
where n is the number of agents participating in formation transformation. are the relative distances between the old and new formation on the x and y axes. t is the desired time interval to reach a new formation.
As a result, the consensus and reference gain should be adaptive to obtain the given velocity.
Multi-agent systems can obtain a time-varying formation tracking on the x-y plane if for any given bounded initial states, the following condition holds:
where dest is the expected traveling time to achieve a new formation. p, are expressed as the current and desired position of each robot in the varying formation.
In the case where = , it is assumed from Eq. (14) that - . Therefore, this theorem becomes the definition of targets pursuing problems intended for multi-agent systems.
Based on this new strategy, the multi-vehicles under the control of the adaptive NI controllers and the information network topology will be guaranteed not only to maintain a desired spatial pattern but also successfully modify their formation over time.
Vi Stability proof of the enhanced NI formation control architecture
We assume that M(s) and N(s) are the SNI velocity transfer function and its NI controller for one slave UAV on the x axis respectively. As presented in Table II and Fig. 5, the M(s) transfer function (TF1) is an SNI system. Unfortunately, the position prediction term is only an NI system (TF2) as shown in Lemma III.2. It is outlined in Lemma VI.1 that the whole structure is still SNI if two systems are linked together by a positive connection. As presented in the Fig. 7, we may conclude that the complete plant (TF3) is SNI as shown in Fig. 10.
Lemma VI.1 (Petersen, I.R. and Lanzon, A.,2010)
A positive connection between an NI plant and an SNI plant results in an SNI structure.
Lemma VI.2 (Ghallab, et al., 2017)
Lyapunov-based stability proof verifies that the results given in Lemma III.3 for the internal stability of feedback interconnections of NI systems is still correct.
Furthermore, it is mentioned in Lemma III.3 that the condition of internal stability for the whole structure with the positive feedback interconnection is M(0)N(0) 1. The DC gain of M(0) equals to 28.58 while that of N(s) is a constant gain of -0.7 as presented in Test 2. It is seen that the DC gain condition of M(s)N(s) is guaranteed to be less than 1 due to the negative constant gain value in N(s) controller. As a result, M(s) under the control of N(s) is internally stable. Moreover, a Nyquist plot of whole architecture is drawn, and the result of stability is the same as in Lemma III.3 since its figure never encircles the critical point (1j0) as shown in Fig. 11.
Similarly, the stability of the Y axis SNI velocity transfer function and its NI controller as well as that of whole NI formation control structure are proved by the same approach.
Vii Simulation Results
To validate the effectiveness of the proposed architecture and controllers, we simulated four consensus schemes. The NI controller used for the master UAV will receive reference points from the rectangular path planner and convert them into linear velocity set points. During the time the master navigates to each next waypoint, the slaves maintain its relative distance from the predicted future position of the master. Case 1 shows a complicated consensus and formation for the combination of one UAV and two UGVs with the time-varying formation while the yaw angle consensus is also presented in Case 2 to solve the position consensus problem at each vertex of the rectangular trajectory. Thanks to this approach, UGVs can make the position consensus better, especially at the sharp corner of two trajectories which has not been mentioned in recent studies  . The overall equations describing the consensus and formation control for one master UAV and two slave UGVs, using the enhanced NI systems theory are summarized as follows:
In order to generate a rotation at the sharp corner for UGVs, the yaw angle of the follower is controlled to follow the yaw angle of the leader (Eq. 20) instead of being determined by the position vectors on the x and y axes during the consensus, avoiding and returning processes (Eq. 21 and Eq. 22). Meanwhile, the position consensus is completely turned off during the turns.
where Kr and Kc are the reference and consensus gain values of the NI controllers for velocity control on the x and y axes and the angular rate control around the z axis. Let and represent the angular rate set point of the UGVs. d and d are the relative positions between the leader and the follower on the x and y axes while is the relative angle between the leader UGV and the follower UGVs.
The output equation of the SNI plant is:
where P(s) is the SNI velocity transfer function matrix of the two UAVs on the x and y axes. n is the number of agents.
Case 1: The distances between robots on the x and y axes are varied at each vertex of the master’s rectangular trajectory. X = [X, X]. X is chosen as [100,50], [50,50], [100,50], [100,0] and X is [-100,50], [-50,50], [-100,50], [-100,0]. The communication topology is given in Fig. 13. The X parameter will be assigned with the waypoints of the rectangular trajectory. The four vertexes of this rectangle in the x-y coordinate plane are [0, 100], [0, 0], [80, 0], [80, 100]. Desired time interval for formation variation is 2 seconds. Therefore, the initial values for NI controllers are selected as (-0.002, -0.5, -0.5, -0.5, -0.5, -0.5). As seen in Fig. 12, UAV, UGV and UGV well maintains the varied formation over time under the control of NI controllers. The UGVs obtain the newly desired relative distance in approximately 2 seconds.
Case 2: The communication topology is given in Fig. 13. The NI controllers are the constant gains (-0.066, -0.02), X=(0, 0) degree, Q = [1; -1], Q= [0; -1], Q= [1; 0], the X term is [90; 90] degree. These parameter settings are used to synchronize the yaw angle for the UGVs. The corresponding consensus result is depicted in Fig. 14.
The simulation results have verified that our proposed algorithm can overcome the problem existing in time-varying formation control. Once the expected relative distances on the x and y axes are able to be altered at each vertex of the rectangular trajectory, our robots will take approximately 2 seconds to reach the new distance offset and to stabilize around this value. Additionally, the consensus of robots is performed successfully at each vertex of rectangle owing to our yaw angle consensus approach.
Viii The Geometry Method to Avoid Obstacles
After verifying the feasibility of our NI varying formation control method in a simulation environment, it is necessary to indicate a new obstacle avoidance algorithm for the two avoiding cases: one obstacle alone and two obstacles in opposite sides.
Viii-a Identification of obstacles and robots
Each robot is assumed to be equipped with a sensor (e.g. RGB-Depth camera) to determine the real obstacle boundaries. In a cluttered environment, the shape of real obstructions is intricate. Therefore, other studies, which compute the escape angle via actual border of the obstructions, often show the results within acceptable limits. Similarly, other approaches are implemented in a simple environment in which only some obstacles of simple shape exist. We solve this problem via two steps: (1) finding the center point of the real boundary parts within the Field of View (FOV) of the camera by using integral formula and geometric decomposition, and (2) generating a virtual circle that surrounds the recognized obstruction. The radius of this circle is equal to the distance between the center point and the furthest point of the observed obstacle boundary. By using the virtual circles to cover the obstacle boundaries, any computation based on the circle border line is more accurate. Each time the robot goes through the identified area, the calculating process will be repeated.
In Fig. 15, the light-green colored regions which have polygon shapes are the overlapping ones between the camera’s FOV and the top sections of the obstacle. Moreover, OC1 and OC2 are the geometric centers of the two polygons mentioned above and are also the centers of virtual circles which represent the viewable obstacle regions detected by the camera.
Occasionally, the obstacle boundary within the observation view is a discontinuity, which results in small spaces being created between the obstacles. It would be unacceptable if the avoiding algorithm drives our robot to pass through these narrow gaps. To overcome this obstruction, we group or separate these obstacles via the following conditions:
where dis12 is the distance between the two obstacle centers. r1 and r2 are the radii of the obstacle circle 1 and 2 respectively. d is the diameter of the robot circumscribed circle.
As presented in Fig. 16, if the distance between the two centers of OC1 and OC2 is less than the diameter of the robot’s virtual circle plus the two radii, a grouping is executed; otherwise, the separation is applied. The center coordinate and radius of the grouped circle OC12 is computed as shown in Fig. 17.
These computations can be extended for multi-obstacles situations if there are more than two obstacles whose locations satisfy the grouping condition. However, in case of using a laser rangefinder sensor; e.g., 3D LiDAR, which brings powerful 360-degree and 6-meter distance sensing capabilities, the obstacle circle radius and the number of grouped obstacles must be limited. The primary reason is that the larger area of the obstacle boundary this sensor can recognize, the smaller the space between obstacles which the robots can bypass is.
Finally, by considering the size of each robot in our obstacle avoidance approach, each robot is also geometrically circumscribed by a virtual circle. Once the robot’s virtual circle intersects with the obstacle circle, a potential collision is detected.
Viii-B Avoiding one obstacle
We obtain the necessary and sufficient conditions of this mode as follows:
where SP1 and SP2 denote the coordinates of two projection points projected from the center line of two opposite obstacles. SD1 and SD2 illustrate the intersection points between the robot virtual circles and the center lines of two opposite robots. FOV is the Field of View of a camera. OC1 and OC2 present the center location of each obstacle while SC1 and SC2 present the center location of each robot.
After the requirements of the one-obstacle avoidance mode are determined, two strategies to transform the robots’ formation will be designed. When one of the slaves predicts a potential collision with the obstacle (Strategy 1) via the distance between the obstacle and the robot, this robot will calculate the updated relative distance from the master position and will execute the avoiding process with the following steps: (1) calculating an intersection point A between the obstacle circle and OA3; (2) finding a projected point SP1 on SC1-SC2 line segment; (3) recomputing the relative distance disSP1MC1 between the master and the corresponding slave; (4) the slave travels into the safe place; and (5) after avoiding process, the three robots reach the destination. Similarly, when the master detects a potential collision with the obstacle (Strategy 2), the methods for determining the projected point SP1 are the same as those mentioned in Strategy 1. However, in the third step (3), the SP1 coordinate is used as the position reference for the master while the slaves attempt to maintain their former orientation by storing its last position on the horizontal axis. All processes for two strategies are depicted as in Fig. 18.
Viii-C Avoiding two facing obstacles
Regarding the evasion for obstacles located at two opposite sides, there are three situations which are mathematically illustrated as below:
where D1 is the master robot’s virtual circle diameter
In Case 1 (1), the three robots are able to maintain the configured formation and move to the target without being blocked by any obstruction due to a large clearance between two obstacles. In Case 2 (2) and Case 3 (3), the three robots are forced to alter their formation pattern while there are two opposite obstacles blocking their path and the relative distance between the master and the obstacles is 1 meter. The former will modify the distance between robots while the latter will shift from the original formation to the queuing formation. In the scope of this paper, we will concentrate on solving the problems arising from the first and second cases.
When the conditions of Case 1 are met, no action is required by the robots. However, when the conditions of Case 2 are satisfied, our robots will be guided following five steps as shown in Fig. 19: (1) creating the centers line SC1-SC2 between the two slaves; (2) finding the two intersection points A and B between the obstacle’s boundary circles and the connecting line OC1-OC2; (3) determining their two projected points SP1 and SP2 on the SC1-SC2 line; (4) computing the new desired relative distance that the two slaves must follow while the master robot is driven to the middle point C of OC1-OC2; and (5) the former formation pattern will be restored right after the master robot moves out of FOV of the master’s camera.
Ix Experimental Results
Three experiments have been presented to illustrate the main results of this research. Test 1 introduces the ability of invariant formation control via the NI consensus strategy applied to multi-vehicle systems. In contrast, Test 2 demonstrates the transformation of robots’ time-variant formation pattern with the main purpose of keeping away from obstacles in two various situations: one obstacle and two group of obstacles in parallel. The final test is conducted for an overall situation in which obstacles are placed randomly.
Test 1: Test 1 aims at validating the efficiency of both consensus and formation control for multi-robot systems, including two UGVs and one UAV. The master UGV (system 1 in Fig. 20) will receive the waypoints on the rectangular trajectory path while the slaves, involving one UGV (system3) and one UAV (system2), follow the master and maintain a relative distance, determined by the pre-defined triangular formation. The geometrical distance between the master and each slave is 100 cm. This term is configured by the X parameter in the enhanced NI formation control diagram. The communication topology is given in Fig. 20. The NI controllers are the constant gains (-0.0028, -0.0028, -0.7, -0.7, -0.0069, -0.0069), X=(100, -100, 100, 100, 0, 0) cm. The X parameter is the location matrix for the four vertexes of the rectangle. The coordinates of four vertexes are [100, -150], [100, 10], [-150, 10], [-150, -150]. Moreover, the yaw angles of the two UGVs are also synchronized with each other using the same approach. The NI controllers used for yaw angle consensus are the constant gains (-0.0056, -0.0107). Similarly, X=(0, 0) degrees. The X parameter is referred to as the desired angle setpoints set and is calculated using the rotation matrix and the next vertex location. All experimental results are shown in Fig. 21 and Fig. 22.
For more details, Fig. 23 illustrates graphically a triangle shape-shifting in space produced by virtual connections made by the central poses of the two UGVs and the UAV (the different line types and colors) based on the enhanced NI formation control strategy with varying time steps.
As seen in Fig. 21 and Fig. 23, good formation and consensus for a group of UAV and UGVs are guaranteed. Based on the desired relative distance information with the master position, the two slaves, including the UAV and the UGV, together with the master UGV, have generated an inverted-V-shaped formation (green-colored lines) and moved along the rectangular paths (green, blue and black-colored lines). The maximum error of position is approximately 15 cm for the UAV and 7 cm for the UGVs.
Moreover, response time of systems under the control of the enhanced NI formation architecture is improved dramatically. In order to provide more clear proof of this improvement, the formation experiments using our former NI architecture for three UGVs  are conducted in a similar way as those in Test 1. Fig. 24 and Fig. 25 show the performances of the three UGVs over time. Compared with the old architecture in stabilizing the desired distance of 100 cm between the UGVs, the actual relative distance between the UGVs in Fig. 24 is up to 95 cm while that in Fig. 25 is only 30 cm. This verifies that the enhanced NI architecture for formation control is better than the ordinary one.
Finally, the rectangular trajectory of the two UGVs in Fig. 21 is extremely smooth, especially at the four vertexes. Such performance is achieved thanks to strong yaw angle consensus as shown in Fig. 14 and Fig. 22.
Test 2: Two avoidance schemes mentioned in Section VIII are implemented separately in an indoor environment using the UAV-UGV/UGVs systems. These systems are integrated in our NI time-varying formation control protocol. In experiment 1, the master UAV attempts to evade the obstacle by changing its waypoints. Meanwhile, the two slave UGVs maintain their pre-defined formation. Different from the first test, experiment 2 concentrates on how the three UGVs can overcome the two groups of obstacles located at two opposite sides.
All settings of two experiments are as per those of Test 1. The radius of the robot circumscribed circles is 32 cm, and that of the obstacle’s boundary circles is 35 cm. The FOV of the camera is roughly 2.2 m. The coordinate of the destination point in the experiment 1 is [-100, 170] while it is [-100, 220] in the experiment 2.
Test 3: In the final experiment, an uncertain environment, containing two opposite-side obstacles and a single obstacle, is explored by a team of 3 UGVs. The target is assigned at the coordinate [-100, 240]. The task of this team is to preserve the triangle-formation as best as possible while avoiding unexpected obstacles and moving to the target simultaneously.
Fig. 26 shows the significant results of the straight-line shaped formation and the master-obstacle collision avoidance. The initial objectives of this experiment are achieved. The straight-line shaped formation is preserved perfectly by the two UGVs even though the leader must avoid the obstacle. Similarly, Fig. 27 indicates that the three UGVs are driven appropriately according to the first strategy of avoiding two groups of opposite obstacles (Section VIII) while maintaining their triangular formation.
Following the success of Test 2, Test 3 is conducted without any constraints. The three UGVs easily pass through any obstacles on their path and reach the target despite the two facing arrays of multiple obstacles and a single obstacle are placed randomly in indoor environment.
In each test, the position consensus errors made by the robots are approximately 10 cm on the x and y axes. Since there are not any oscillations occurring in the real robot trajectories, it is pointed out that our formation methodology is better than that of authors in . Besides, our approach also pays attention to the size of each robot. It is noted that the distance between the real avoiding path of all examined robots and the obstacle circle OC12 is roughly 31 cm with respect to the desired radius of 32 cm as shown in Fig. 26 to Fig. 28. Moreover, the formation regrouping takes place smoothly via the robots’ steering behavior. The boundary of the restoration trajectory is the total of the robot radius and the border line of the obstacle’s virtual circle. Finally, thanks to the relative distance calculation via the boundary of the obstacle’s virtual circle, the safe path generated is better and smoother than that of the traditional GOACM method which entirely depends on the real boundary of the complex-shaped obstacle.
As seen in Fig. 21 and Fig. 26, there is no problem when the master is either the UAV or the UGV although two systems have very different dynamics. The UAV’s outer-loop velocity controller sends attitude commands to the inner-loop attitude controller which commands the motors driving the rotors. In contrast, the UGV’s velocity controller controls the velocity of four wheels directly. This control is achieved by converting the formation control into the velocity stabilization of each robot on the x and y axes via the NI-systems control algorithm.
A time-varying formation control scheme based on the enhanced NI systems consensus theory for two popular types of mobile robots (UAVs-UGVs) has been introduced in this paper. In this consensus problem, the master corrects its state in space upon receiving waypoints from the path planner, while the slaves predict the next position of the master and will respond to its movement with an offset distance that is determined by the formation planner. In the obstacle avoidance problem, comprehensive situations are considered, and the experiments in simulation and real tests present the positive results of the proposed method compared to the recent studies.
Some position errors at the sharp corners are experienced by the UAV as shown in Fig. 21. The distance reference with the leader UGV when this leader rotates at a sharp corner and the lack of a real velocity sensor are the major causes for these errors. In future, to handle these problems, the measured distance between the master UGV and the slave UAV should be neglected during the rotational motion of the UGVs. To deal with the second issue, the UAV platform should be equipped with a velocity sensor (e.g. optical flow based sensor) to measure the real velocities on the x and y axes.
In future work, other applications generated by formation control architecture such as distributed consensus protocols, leaderless formation or group control of UAVs-UGVs using NI theory will also be studied; notably, the robot alignment, the V-shape formation control problem or the actual formation of cooperative groups.
-  Milutinovic, D., Lima, P. (2006). Modeling and optimal centralized control of a large-size robotic population. IEEE Transactions on Robotics, 22(6), 1280-1285.
-  Huang, M., Caines, P. E., Malhamé, R. P. (2012). Social optima in mean field LQG control: centralized and decentralized strategies. IEEE Transactions on Automatic Control, 57(7), 1736-1751.
-  Turpin, M., Michael, N., Kumar, V. (2012, May). Decentralized formation control with variable shapes for aerial robots. In Robotics and Automation (ICRA), 2012 IEEE International Conference on (pp. 23-30). IEEE.
-  Sahraei, B. R., Shabaninia, F. (2010, August). A robust H∞ control design for swarm formation control of multi-agent systems: A decentralized adaptive fuzzy approach. In Resilient Control Systems (ISRCS), 2010 3rd International Symposium on (pp. 79-84). IEEE.
-  de Lope, J., Maravall, D., Quiñonez, Y. (2013). Response threshold models and stochastic learning automata for self-coordination of heterogeneous multi-task distribution in multi-robot systems. Robotics and Autonomous Systems, 61(7), 714-720.
-  Parker, L. E. (2008). Multiple mobile robot systems. In Springer Handbook of Robotics (pp. 921-941). Springer Berlin Heidelberg.
-  Forster, C., Pizzoli, M., Scaramuzza, D. (2013, November). Air-ground localization and map augmentation using monocular dense reconstruction. In Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on (pp. 3971-3978). IEEE.
Vidal, R., Shakernia, O., Kim, H. J., Shim, D. H.,Sastry, S. (2002). Probabilistic pursuit-evasion games: theory, implementation, and experimental evaluation. IEEE transactions on Robotics and Automation, 18(5), 662-669.
-  Michael, N., Shen, S., Mohta, K., Mulgaonkar, Y., Kumar, V., Nagatani, K., Ohno, K. (2012). Collaborative mapping of an earth quakedamaged building via ground and aerial robots. Journal of Field Robotics, 29(5), 832-841.
-  Li, J., Li, Y. (2011, August). Dynamic analysis and PID control for a quadrotor. In Mechatronics and Automation (ICMA), 2011 International Conference on (pp. 573-578). IEEE.
-  Zhang, X., Li, X., Wang, K., Lu, Y. (2014). A survey of modelling and identification of quadrotor robot. In Abstract and Applied Analysis (Vol. 2014). Hindawi.
-  Yan, Z., Jouandeau, N., Cherif, A. A. (2013). A survey and analysis of multi-robot coordination. International Journal of Advanced Robotic Systems, 10(12), 399.
-  Dong, X., Zhou, Y., Ren, Z., Zhong, Y. (2017). Time-varying formation tracking for second-order multi-agent systems subjected to switching topologies with application to quadrotor formation flying. IEEE Transactions on Industrial Electronics, 64(6), 5014-5024
-  Dong, X., Yu, B., Shi, Z., Zhong, Y. (2015). Time-varying formation control for unmanned aerial vehicles: Theories and applications. IEEE Transactions on Control Systems Technology, 23(1), 340-348.
-  Rahimi, R., Abdollahi, F., Naqshi, K. (2014). Time-varying formation control of a collaborative heterogeneous multi agent system. Robotics and autonomous systems, 62(12), 1799-1805.
-  Su, Y., Hong, Y., Huang, J. (2013). A general result on the robust cooperative output regulation for linear uncertain multi-agent systems. IEEE Transactions on Automatic Control, 58(5), 1275-1279.
-  Beard, R. W., Lawton, J., Hadaegh, F. Y. (2001). A coordination architecture for spacecraft formation control. IEEE Transactions on control systems technology, 9(6), 777-790.
-  Seo, J., Kim, Y., Kim, S., Tsourdos, A. (2012). Consensus-based reconfigurable controller design for unmanned aerial vehicle formation flight. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 226(7), 817-829.
-  Qu, Y., Zhu, X., Zhang, Y. M. (2012, October). Cooperative control for UAV formation flight based on decentralized consensus algorithm. In International Conference on Intelligent Robotics and Applications (pp. 357-366). Springer, Berlin, Heidelberg.
-  Wang, J., Lanzon, A., Petersen, I. R. (2015). Robust cooperative control of multiple heterogeneous Negative-Imaginary systems. Automatica, 61, 64-72.
-  Wang, J., Lanzon, A., Petersen, I. R. (2015). Robust output feedback consensus for networked negative-imaginary systems. IEEE Transactions on Automatic Control, 60(9), 2547-2552.
-  Dong, X., Zhou, Y., Ren, Z., Zhong, Y. (2017). Time-varying formation tracking for second-order multi-agent systems subjected to switching topologies with application to quadrotor formation flying. IEEE Transactions on Industrial Electronics, 64(6), 5014-5024.
-  Petersen, I. R., Lanzon, A. (2010). Feedback control of negative-imaginary systems. IEEE Control Systems, 30(5), 54-72.
-  Mabrok, M. A., Kallapur, A. G., Petersen, I. R., Lanzon, A. (2014). Spectral conditions for negative imaginary systems with applications to nanopositioning. IEEE/ASME Transactions on Mechatronics, 19(3), 895-903.
-  Ghallab, A. G., Mabrok, M. A., Petersen, I. R. (2017). Lyapunov-based Stability of Feedback Interconnections of Negative Imaginary Systems. IFAC-PapersOnLine, 50(1), 3424-3428.
-  Abdullahi, A. M., Mohamed, Z., Abidin, M. Z., Akmeliawati, R., Husain, A. R., Bature, A. A. (2015, May). LMI-based state feedback controller design for vibration control of a negative imaginary system. In Control Conference (ASCC), 2015 10th Asian (pp. 1-6). IEEE.
-  Mercado, D. A., Castro, R., Lozano, R. (2013, July). Quadrotors flight formation control using a leader-follower approach. In Control Conference (ECC), 2013 European (pp. 3858-3863). IEEE.
-  Dai, Y., Kim, Y., Wee, S., Lee, D., Lee, S. (2015). A switching formation strategy for obstacle avoidance of a multi-robot system based on robot priority model. ISA transactions, 56, 123-134.
-  Lee, G., Chwa, D. (2018). Decentralized behavior-based formation control of multiple robots considering obstacle avoidance. Intelligent Service Robotics, 11(1), 127-138.
-  Tran, V. P., Garratt, M., Petersen, I. R. (2017, December). Formation control of multi-UAVs using negative-imaginary systems theory. In Control Conference (ASCC), 2017 11th Asian (pp. 2031-2036). IEEE.