DeepAI AI Chat
Log In Sign Up

Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents

01/07/2022
by   Christian Pfeiffer, et al.
4

Humans race drones faster than neural networks trained for end-to-end autonomous flight. This may be related to the ability of human pilots to select task-relevant visual information effectively. This work investigates whether neural networks capable of imitating human eye gaze behavior and attention can improve neural network performance for the challenging task of vision-based autonomous drone racing. We hypothesize that gaze-based attention prediction can be an efficient mechanism for visual information selection and decision making in a simulator-based drone racing task. We test this hypothesis using eye gaze and flight trajectory data from 18 human drone pilots to train a visual attention prediction model. We then use this visual attention prediction model to train an end-to-end controller for vision-based autonomous drone racing using imitation learning. We compare the drone racing performance of the attention-prediction controller to those using raw image inputs and image-based abstractions (i.e., feature tracks). Our results show that attention-prediction based controllers outperform the baselines and are able to complete a challenging race track consistently with up to 88 visual attention-prediction and feature-track based models showed better generalization performance than image-based models when evaluated on hold-out reference trajectories. Our results demonstrate that human visual attention prediction improves the performance of autonomous vision-based drone racing agents and provides an essential step towards vision-based, fast, and agile autonomous flight that eventually can reach and even exceed human performances.

READ FULL TEXT

page 4

page 5

page 6

03/08/2021

Human-Piloted Drone Racing: Visual Processing and Control

Humans race drones faster than algorithms, despite being limited to a fi...
08/08/2019

Learning Vision-based Flight in Drone Swarms by Imitation

Decentralized drone swarms deployed today either rely on sharing of posi...
12/21/2022

Cooperative Flight Control Using Visual-Attention – Air-Guardian

The cooperation of a human pilot with an autonomous agent during flight ...
10/26/2022

Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing

Autonomous drones can operate in remote and unstructured environments, e...
09/03/2018

Learning Vision-based Cohesive Flight in Drone Swarms

This paper presents a data-driven approach to learning vision-based coll...
04/17/2019

Gaze Training by Modulated Dropout Improves Imitation Learning

Imitation learning by behavioral cloning is a prevalent method which has...
04/18/2019

Learning a Controller Fusion Network by Online Trajectory Filtering for Vision-based UAV Racing

Autonomous UAV racing has recently emerged as an interesting research pr...