Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts

07/27/2021

∙

We explore the reinforcement learning approach to designing controllers by extensively discussing the case of a quadcopter attitude controller. We provide all details allowing to reproduce our approach, starting with a model of the dynamics of a crazyflie 2.0 under various nominal and non-nominal conditions, including partial motor failures and wind gusts. We develop a robust form of a signal temporal logic to quantitatively evaluate the vehicle's behavior and measure the performance of controllers. The paper thoroughly describes the choices in training algorithms, neural net architecture, hyperparameters, observation space in view of the different performance metrics we have introduced. We discuss the robustness of the obtained controllers, both to partial loss of power for one rotor and to wind gusts and finish by drawing conclusions on practical controller design by reinforcement learning.

READ FULL TEXT

Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts

Accelerate Reinforcement Learning with PID Controllers in the Pendulum Simulations

Mid-flight Propeller Failure Detection and Control of Propeller-deficient Quadcopter using Reinforcement Learning

Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

A Reinforcement Learning Approach for Robust Supervisory Control of UAVs Under Disturbances

Improving Wind Resistance Performance of Cascaded PID Controlled Quadcopters using Residual Reinforcement Learning

Worrisome Properties of Neural Network Controllers and Their Symbolic Representations

Deep Residual Reinforcement Learning based Autonomous Blimp Control

Reinforcement Learning with Formal Performance Metrics for Quadcopter Attitude Control under Non-nominal Contexts

Related Research

Accelerate Reinforcement Learning with PID Controllers in the Pendulum Simulations

Mid-flight Propeller Failure Detection and Control of Propeller-deficient Quadcopter using Reinforcement Learning

Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

A Reinforcement Learning Approach for Robust Supervisory Control of UAVs Under Disturbances

Improving Wind Resistance Performance of Cascaded PID Controlled Quadcopters using Residual Reinforcement Learning

Worrisome Properties of Neural Network Controllers and Their Symbolic Representations

Deep Residual Reinforcement Learning based Autonomous Blimp Control