Deep Drone Racing: From Simulation to Reality with Domain Randomization

by   Antonio Loquercio, et al.

Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges for robotics, which still limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art path-planning and control system with the perceptual awareness of a convolutional neural network (CNN). The CNN directly maps raw images to a desired waypoint and speed. Given the CNN output, the planner generates a short minimum-jerk trajectory segment that is tracked by a model-based controller to actuate the drone towards the waypoint. The resulting modular system has several desirable features: (i) it can run fully on-board, (ii) it does not require globally consistent state estimation, and (iii) it is both platform and domain independent. We extensively test the precision and robustness of our system, both in simulation and on a physical platform. In both domains, our method significantly outperforms the prior state of the art. In order to understand the limits of our approach, we additionally compare against professional human drone pilots with different skill levels.


page 1

page 5

page 7

page 9

page 10

page 11

page 12

page 14


Deep Drone Racing: Learning Agile Flight in Dynamic Environments

Autonomous agile flight brings up fundamental challenges in robotics, su...

Deep Drone Acrobatics

Performing acrobatic maneuvers with quadrotors is extremely challenging....

Vision-Based Autonomous Drone Control using Supervised Learning in Simulation

Limited power and computational resources, absence of high-end sensor eq...

Robust Navigation for Racing Drones based on Imitation Learning and Modularization

This paper presents a vision-based modularized drone racing navigation s...

Directing Cinematographic Drones

Quadrotor drones equipped with high quality cameras have rapidely raised...

Fast-Racing: An Open-source Strong Baseline for SE(3) Planning in Autonomous Drone Racing

With the autonomy of aerial robots advances in recent years, autonomous ...

ATG-PVD: Ticketing Parking Violations on A Drone

In this paper, we introduce a novel suspect-and-investigate framework, w...