Unmanned aerial vehicles, especially multicopters, have been used in a myriad of applications over the last decade, such as environment mapping, asset monitoring, risk assessment, sports broadcasting, wind-turbine inspection, and their applications continue to grow [7, 3, 2, 29, 30, 27]
. In its most common form, a multicopter has four propellers. By controlling the spin rates of the four propellers, a force along a body-fixed axis and moments about three linearly independent body-fixed axes can be independently applied to affect desired translational as well as rotational motions. However, due to the nonlinear and unstable nature of the quadcopter dynamics, precise control of a quadcopter is a well-recognized challenging problem.
Nonlinear techniques such as feedback-linearization  and back-stepping  have been applied to deal with the nonlinearities and to construct stabilizing controllers. However, these techniques require accurate plant models at all operating conditions . Adaptive techniques have also been investigated to reduce the need for an accurate model [37, 11], however, they also require a sufficiently accurate plant model to construct stabilizing controllers. Iterative learning control is used learn the pitch and roll controller in the closed-loop system in 
. Reinforcement learning control is used to learn low-level controllers in case of multiple actuator failure in. L1 adaptive control is used to improve the stability margins of a stable control loop in a quadcopter flight control system in . Fuzzy control was used in the position control in . Bidirectional brain emotional learning was used to improve trajectory tracking and handle payload uncertainties in . Model reference adaptive control was used in the attitude controller in  to counteract model uncertainties. Adaptive twisting sliding mode control was used in the attitude controller in  to resolve the chattering issues in standard sliding mode control. Robust fixed point transformation based adaptive control was used in 
to improve quadcopter stability in the presence of parameter uncertainties and external disturbances. Adaptive particle swarm optimization was used in a PD controller (APSO-PD) in to improve the rise time, settling time, overshoot, and peak time of a standard PSO controller. Robust adaptive control based on backstepping is used in   to provide robust quadcopter altitude and attitude tracking after payload change. Immersion and invariance based adaptive backstepping control is used in  to remedy quadcopter attitude instabilities due to disturbance torques or parameter uncertainties. Adaptive fault tolerant control based on the adaptive minimum projection method is used to provide quadcopter stability in the event of actuator failure in . Balanced control is used in  to improve quadcopter performance by switching between fuzzy adaptive PID and optimized PID midflight. L1 adaptive control is used in  to stabilize a fixed-wing pitch controller, and in  to counteract fixed-wing split drag rudder damage. Adaptive control is also used in 
to resolve instabilities due to trailing vortices in fixed-wing formation flight. Online adaptive model parameter estimation is used in the velocity controller in to mitigate error due to model parameter uncertainty. However, most of these control techniques focus on optimizing a part of the control system, assuming that the rest of the control system is sufficiently good.
Typical quadcopter autopilots are, however, based on cascaded controllers, which consist of an inner loop to stabilize the dynamics, and an outer loop to track position commands. Traditionally, all controllers in the autopilot are constructed using manually tuned PID control laws. In fact, widely used open-source autopilots such as PX4 and ArduPilot contain finely-tuned PID control laws for many commercially available multicopter configurations[23, 4]. These autopilots cannot guarantee stability and thus have a fixed operational envelope, which is usually unknown. Moreover, autopilots tuned for a specific geometry and inertia properties do not perform well in the case where these properties vary with time, such as, in the case of unknown suspended payload, hardware alteration, and dynamic environmental changes. Such variations invariably degrade the performance of the autopilot.
With these motivations in mind, this paper develops an adaptive autotuner for multicopters with unknown dynamics. In particular, all of the fixed-gain control laws of an autopilot are replaced by adaptive control laws that optimize the gain by flying a single learning trajectory. Specifically, the adaptive controllers are updated by the retrospective cost adaptive control (RCAC) algorithm . The learning trajectory is designed such that all possible motions of the quadcopter are excited and thus nonzero control outputs are required from all control laws, which ensures that all of the controller gains are adaptively updated.
The contribution of the work presented in this paper is the development of an adaptive autotuner that does not require any prior knowledge of the system dynamics, and instead uses a simple learning trajectory to adapt the autopilot gains; and its numerical and experimental demonstration. The performance of the autotuner is demonstrated by flying a test trajectory and comparing its performance with a finely-tuned autopilot. The improvements due to the autotuning process are demonstrated through simulations and flight tests.
The paper is organized as follows. Section II briefly presents the control system architecture implemented in the PX4 autopilot. Section III explains the setup of the adaptive autotuner. Section IV describes the RCAC algorithm used to tune the control parameters. Section V presents simulation and flight test results showing the adaptation of the autotuner and comparing the performance of the PX4 autopilot using the stock and autotuned control parameters. Finally, section VI concludes the paper with a summary and future research directions.
Ii Quadcopter Autopilot
This section reviews the quadcopter autopilot considered in this work. This autopilot is based on the control architecture implemented in the PX4 autopilot. The notation used in this paper is described in more detail in .
The control system consists of a mission planner and two nested loops as shown in Figure 1. The mission planner generates the position, velocity, and azimuth setpoints from the user-defined waypoints using the guidance law described below. The outer loop consists of the position controller
, whose inputs are the position and velocity errors, defined as the difference between the setpoints and the measurements. The output of the position controller is the thrust vector setpoint. Note that the thrust vector is expressed in terms of the Earth-fixed frame. The inner loop consists of theattitude controller, whose inputs are the thrust vector setpoint, the azimuth and azimuth-rate setpoints, as well as the attitude and the angular rate measured in the body-fixed frame. The output of the attitude controller is the moment vector setpoint in the body-fixed frame. The magnitude of the thrust vector setpoint and the moment vector setpoint uniquely determine the required rotation rates of the four propellers.
The mission planner uses a guidance law described in Appendix A to generate the position and the azimuth setpoints. Using a user-specified maximum velocity and maximum acceleration , the guidance law generates a trajectory that consists of a constant acceleration phase, a cruise phase, and a constant deceleration phase. A similar guidance law is used to generate azimuth setpoints, given a user-specified maximum angular velocity and maximum angular acceleration
The position controller consists of two cascaded linear controllers as shown in Figure 2. The first controller consists of three proportional controllers. The second controller consists of three decoupled PID controllers and a velocity setpoint feedforward controller, and yields the thrust vector setpoint.
The force vector setpoint along with the azimuth setpoint are used to calculate the attitude setpoint, which is represented as a quaternion. The attitude controller consists of two cascaded controllers and as shown in Figure 3. The first controller is an almost globally stabilizing controller  that consists of three proportional gains. The second controller consists of three PID controllers and an angular rate setpoint feedforward controller, and yields the moment vector setpoint.
The control system implemented in the PX4 autopilot thus consists of 27 gains. In particular, the outer loop includes three gains in and nine gains in ; and the inner loop includes three gains in and 12 gains in In standard practice, these 27 gains are manually tuned and require considerable expertise.
Iii Adaptive Autotuner
The adaptive autotuner is constructed by replacing the fixed-gain controllers in the autopilot described in the previous section with adaptive controllers that are updated by retrospective cost optimization. In particular, each fixed-gain controller described in the previous section is replaced by an adaptive controller parameterized with the same structure. Specifically, denoting the th component of the input and the output of at step by and the velocity setpoint
where is a error-normalization function, and the gains are updated by the RCAC algorithm described in the next section. The functions used in this work are given in Table II. Similarly,
where, for , the entry
The controllers and are similarly parameterized by the gains and
The adaptive gains and are updated in a learning trajectory, which consists of flying through the waypoints described in Table I, parameterized by and . Note that the waypoints are denoted by coordinates which correspond to the three components of the position vector and azimuth of the multicopter in the East()-North()-Up() (ENU) coordinate frame.
|2||Move along direction|
|3||Move along direction|
|4||Move along direction|
|5||Move along direction|
|6||Move along direction|
|7||Move along direction|
|8||Move along direction|
|9||Move along direction|
The gains and obtained at the end of the learning trajectory are the autotuned gains and the autopilot implemented with the autotuned gains is the autotuned autopilot.
Iv RCAC Algorithm
Consider a SISO PID controller with a feedforward term
where and are time-varying gains to be optimized, is an error variable, is the feedforward signal, and, for all ,
Note that the integrator state is computed recursively using . For all , the control law can be written as
where the regressor and the controller gains are
Note that the P, PI, or PID controllers can be parameterized by appropriately defining and Various MIMO controller parameterizations are shown in .
To determine the controller gains , let , and consider the retrospective performance variable defined by
where The sign of is the sign of the leading numerator coefficient of the transfer function from to Furthermore, define the retrospective cost function by
where is the initial vector of PID gains and is positive definite.
See  ∎
Finally, the control is given by
V Experimental Results
This section describes the experimental results obtained in the simulation environment and in the physical flight tests. In both the simulation and the physical flight tests, the autotuner is used to tune the 27 controller gains described in Section II using the learning trajectory described in Section III
. Note that in the autotuning mode, all of the gains in both loops are initialized at zero. The hyperparametersand the error-normalization function used in the RCAC algorithm are shown in Table II. Furthermore, in all four controllers.
V-a Simulation Flight Tests
First, the autotuner is tested in a simulation environment, where the quadcopter is simulated using jMAVSim. In the learning trajectory, m, and m. The guidance law generates the setpoints using m/s, m/, rad/s, 0.5 rad/
Figure 4 shows the response of the quadcopter in the learning trajectory. Note that the response improves as RCAC re-optimizes the controllers continuously in real time. Figure 5 shows the gains adapted by the RCAC algorithm for all four controllers. The gains at the end of the learning trajectory are saved, are the autotuned gains and constitute the autotuned autopilot.
Next, the quadcopter is commanded to follow a test trajectory, whose waypoints are generated using a second-order Hilbert curve, first with the default autopilot, and next with the autotuned autopilot.
The default autopilot gains and the actuator constraints in PX4 are specified in the
Note that the results in this paper are based on PX4 version V1.11.3.
Figure 6 shows the response of the quadcopter obtained with the autotuned autopilot and the default autopilot.
Note that the autotuned autopilot outperforms the default autopilot in terms of position tracking error.
In order to quantify and compare the performance of the autotuned autopilot with the default autopilot, a position-tracking cost variable defined by
where is the position error and is the total flight time in seconds, is computed for each test. Table III shows the cost default autopilot cost and the autotuned autopilot cost Note that the autotuned autopilot is relatively 38 better than the default autopilot in simulation.
Next, the sensitivity of the autotuning process to changes in the physical parameters of the quadcopter is investigated. To do so, the mass of the quadcopter is scaled in the dynamic model simulated by jMAVSim. Specifically, the mass of the quadcopter is multiplied by For each value of the autopilot gains are tuned using the autotuner. Note that the RCAC hyperparameters are not changed. The performance of the autotuned and the default autpilots is compared for each value of by flying the test trajectory defined by the second-order Hilbert curve. Figure 7 shows the cost computed for the position controller in each flight flown with the autotuned and the default autopilot. Note that the autotuned autopilot is re-tuned for each value of whereas default autopilot is fixed for all values of
V-B Physical Flight Tests
Next, the autotuner is tested in physical flight tests conducted in the M-Air facility at the University of Michigan, Ann Arbor with the Holybro X500 quadcopter frame. The M-Air is equipped with a motion capture system that allows high-precision position and attitude measurements.
In the learning trajectory, m, m, and m. The guidance law generates the setpoints using m/s, m/, rad/s, 0.5 rad/ Figure 8 shows the response of the quadcopter in the learning trajectory. Figure 9 shows the gains adapted by the RCAC algorithm for all four controllers.
Next, the quadcopter is commanded to follow a test trajectory with the autotuned autopilot and the default autopilot in the M-Air facility. The waypoints for the test trajectory are generated using a second-order Hilbert curve. Figure 10 shows the response of the quadcopter obtained with the autotuned autopilot and the default autopilot during the physical flight tests. Table III shows the position-tracking cost computed for the test trajectory flown during the physical flight tests. Note that the autotuned autopilot is relatively 32 better than the default autopilot.
Vi Conclusions and Future Work
This paper presented an adaptive autotuner that can automatically tune a multicopter autopilot without using any modeling information about the multicopter. The adaptive autotuner, implemented in the PX4 flight stack, consists of adaptive PID controllers that are updated by the retrospective cost adaptive control algorithm by flying a single learning trajectory.
The adaptive autotuner was validated in a simulation environment, and its performance was compared against the default autopilot by scaling the mass of the quadcopter. The adaptive autotuner outperformed the default autotuner in all cases without any change in the hyperparameters of the adaptive algorithm. Next, using the same hyperparameters used for the simulation, the adaptive autotuner was used to tune the autopilot for the X500 Holybro quadcopter in the M-Air facility, where the adaptive autotuner outperformed the default autotuner in the test trajectory.
Future work will focus on using the adaptive autopilot to improve the performance of the default autopilot flying a multicopter with an unknown suspended payload and under various actuator failure conditions.
Let denote the current position, denote the desired position, and denote the distance between the current and the desired position, that is, Let and The desired trajectory consists of an acceleration phase, a cruise phase, and a deceleration phase along the straight line between and
Let the setpoints be given by the guidance law
where is calculated as shown below. Using the fact that and where is the total flight time, it follows that and Note that is the distance between and
Let denote the time to reach the speed at maximum acceleration Thus, and
First, consider the case where In this case, the quadcopter cruises at for where denotes the time at which deceleration phase starts, and is calculated as shown below. Note that
Since the quadcopter takes seconds to decelerate from to speed at constant deceleration the total flight time is and
It follows from (19) that and thus, the total flight time The distance is thus given by
Next, consider the case where In this case, the quadcopter velocity does not reach , that is, there is no cruise phase. Let denote the time instant at which maximum velocity is reached. Thus, and Assuming that the quadcopter decelerates at the same rate to rest, it follows that and thus and The distance is thus given by
-  (2018) Black-box identification and iterative learning control for quadcopter. In 2018 6th International Conference on Control Engineering & Information Technology (CEIT), pp. 1–5. External Links: Cited by: §I.
-  (2015) Nonlinear controller of quadcopters for agricultural monitoring. In International Symposium on Visual Computing, pp. 476–487. Cited by: §I.
-  (2017) Multicopter platform prototype for environmental monitoring. Journal of Cleaner Production 155, pp. 204–211. Cited by: §I.
-  ArduPilot. Note: https://ardupilot.org/ardupilot/ Cited by: §I.
-  (2019) Projection modification based robust adaptive backstepping control for multipurpose quadcopter UAV. IEEE Access 7, pp. 154121–154130. External Links: Cited by: §I.
-  (2004) Real-time stabilization and tracking of a four-rotor mini rotorcraft. IEEE Transactions on control systems technology 12 (4), pp. 510–516. Cited by: §I.
-  (2016) Development of a multicopter-carried whole air sampling apparatus and its applications in environmental studies. Chemosphere 144, pp. 484–492. Cited by: §I.
-  (2011) Rigid-Body Attitude Control. IEEE Control Systems Magazine 31 (3), pp. 30–51. External Links: Cited by: §II.
-  (2017) Novel method for quadcopter controlling using nonlinear adaptive control based on robust fixed point transformation phenomena. In 2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 000289–000294. External Links: Cited by: §I.
-  (2020) Reinforcement learning based flight controller capable of controlling a quadcopter with four, three and two working motors. In 2020 20th International Conference on Control, Automation and Systems (ICCAS), pp. 161–166. External Links: Cited by: §I.
-  (2013) Adaptive control of quadrotor uavs: a design trade study with flight evaluations. IEEE Transactions on Control Systems Technology 21 (4), pp. 1400–1406. Cited by: §I.
-  (2005) Backstepping-based flight control with adaptive function approximation. Journal of Guidance, Control, and Dynamics 28 (6), pp. 1089–1102. Cited by: §I.
-  (2020-07) Adaptive Control of MIMO Systems Using Sparsely Parameterized Controllers. In 2020 American Control Conference (ACC), Vol. , pp. 5340–5345. External Links: Cited by: §IV.
-  (2021) Experimental implementation of an adaptive digital autopilot. In 2021 American Control Conference (ACC), Vol. , pp. 3737–3742. External Links: Cited by: §II.
-  (2018) Adaptive parameter estimation and velocity control of uav systems. In 2018 37th Chinese Control Conference (CCC), pp. 9866–9871. External Links: Cited by: §I.
-  (2017) Adaptive twisting sliding mode control for quadrotor unmanned aerial vehicles. In 2017 11th Asian Control Conference (ASCC), pp. 671–676. External Links: Cited by: §I.
-  (2020) Research on uav balance control based on expert-fuzzy adaptive PID. In 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA), pp. 787–789. External Links: Cited by: §I.
-  (2019-06) Recursive Least Squares for Real-Time Implementation. IEEE Control Systems Magazine 39 (3), pp. 82–85. External Links: Cited by: §IV.
-  (2019) Adaptive digital PID control of first-order-lag-plus-dead-time dynamics with sensor, actuator, and feedback nonlinearities. Advanced Control for Applications 1 (1), pp. e20. External Links: Cited by: §I, §IV.
-  (2018) Coping with quadcopter payload variation via adaptive robust control. In 2018 IEEE International Multidisciplinary Conference on Engineering Technology (IMCET), pp. 1–6. External Links: Cited by: §I.
-  (2009) Feedback linearization vs. adaptive sliding mode control for a quadrotor helicopter. International Journal of control, Automation and systems 7 (3), pp. 419–428. Cited by: §I.
-  (2018) Lateral control reconfiguration of tailless flying-wing UAV based on L1 adaptive control method. In 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), pp. 1–7. External Links: Cited by: §I.
-  (2015) PX4: a node-based multithreaded open source robotics framework for deeply embedded platforms. In 2015 IEEE international conference on robotics and automation (ICRA), pp. 6235–6240. Cited by: §I.
-  (2021) Real-time adaptive intelligent control system for quadcopter UAV with payload uncertainties. IEEE Transactions on Industrial Electronics. External Links: Cited by: §I.
-  (2017) Immersion and invariance based adaptive control of aerial robot in presence of inertia uncertainty. In 2017 IEEE 4th International Conference on Knowledge-Based Engineering and Innovation (KBEI), pp. 0959–0964. External Links: Cited by: §I.
-  (2017) Integration of model reference adaptive control (MRAC) with PX4 firmware for quadcopters. In 2017 24th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), pp. 1–6. External Links: Cited by: §I.
-  (2017) Multispectral imaging system with uav integration capabilities for crop analysis. In 2017 First IEEE International Symposium of Geoscience and Remote Sensing (GRSS-CHILE), pp. 1–4. Cited by: §I.
-  (2017-10) Retrospective Cost Adaptive Control: Pole Placement, Frequency Response, and Connections with LQG Control. IEEE Control System Magazine 37, pp. 28–69. External Links: Cited by: §IV.
-  (2016) Multicopter unmanned aerial vehicle for automated inspection of wind turbines. In 2016 24th Mediterranean Conference on Control and Automation (MED), pp. 244–249. Cited by: §I.
-  (2015) Autonomous visual navigation of unmanned aerial vehicle for wind turbine inspection. In 2015 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 998–1007. Cited by: §I.
-  (2018) Adaptive fault tolerant control of quadcopter by using minimum projection method. In IECON 2018-44th Annual Conference of the IEEE Industrial Electronics Society, pp. 2201–2206. External Links: Cited by: §I.
-  (2016) Modeling and design optimization for quadcopter control system using L1 adaptive control. In 2016 IEEE 7th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pp. 1–5. External Links: Cited by: §I.
-  (2020) Fuzzy self-tuning of strictly negative-imaginary controllers for trajectory tracking of a quadcopter unmanned aerial vehicle. IEEE Transactions on Industrial Electronics 68 (6), pp. 5036–5045. External Links: Cited by: §I.
-  (2018) Optimal PD tracking control of a quadcopter drone using adaptive PSO algorithm. In 2018 International Conference on Computer, Control, Informatics and its Applications (IC3INA), pp. 146–151. External Links: Cited by: §I.
-  (2018) Adaptive control for UAV close formation flight against disturbances. In 2018 3rd International Conference on Robotics and Automation Engineering (ICRAE), pp. 196–201. External Links: Cited by: §I.
-  (2020) Pitch angle control of UAV based on L1 adaptive control law. In 2020 35th Youth Academic Annual Conference of Chinese Association of Automation (YAC), pp. 68–72. External Links: Cited by: §I.
-  (2014) Adaptive trajectory tracking control of output constrained multi-rotors systems. IET Control Theory & Applications 8 (13), pp. 1163–1174. Cited by: §I.