Human-Piloted Drone Racing: Visual Processing and Control

03/08/2021
by   Christian Pfeiffer, et al.
0

Humans race drones faster than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A better understanding of the ability of human pilots of selecting appropriate motor commands from highly dynamic visual information may provide key insights for solving current challenges in vision-based autonomous navigation. This paper investigates the relationship between human eye movements, control behavior, and flight performance in a drone racing task. We collected a multimodal dataset from 21 experienced drone pilots using a highly realistic drone racing simulator, also used to recruit professional pilots. Our results show task-specific improvements in drone racing performance over time. In particular, we found that eye gaze tracks future waypoints (i.e., gates), with first fixations occurring on average 1.5 seconds and 16 meters before reaching the gate. Moreover, human pilots consistently looked at the inside of the future flight path for lateral (i.e., left and right turns) and vertical maneuvers (i.e., ascending and descending). Finally, we found a strong correlation between pilots eye movements and the commanded direction of quadrotor flight, with an average visual-motor response latency of 220 ms. These results highlight the importance of coordinated eye movements in human-piloted drone racing. We make our dataset publicly available.

READ FULL TEXT

page 1

page 6

research
01/07/2022

Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents

Humans race drones faster than neural networks trained for end-to-end au...
research
09/15/2021

Expertise Affects Drone Racing Performance

First-person view drone racing has become a popular televised sport. How...
research
05/15/2018

Characterizing the Temporal Dynamics of Information in Visually Guided Predictive Control Using LSTM Recurrent Neural Networks

Theories for visually guided action account for online control in the pr...
research
06/09/2021

Using Eye Tracker To Evaluate Cockpit Design – A Flight Simulation Study

This paper investigates applications of eye tracking in transport aircra...
research
10/26/2022

The eyes and hearts of UAV pilots: observations of physiological responses in real-life scenarios

The drone industry is diversifying and the number of pilots increases ra...
research
07/22/2013

Understanding Humans' Strategies in Maze Solving

Navigating through a visual maze relies on the strategic use of eye move...
research
12/10/2021

DronePose: The identification, segmentation, and orientation detection of drones via neural networks

The growing ubiquity of drones has raised concerns over the ability of t...

Please sign up or login with your details

Forgot password? Click here to reset