DeepAI AI Chat
Log In Sign Up

Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography

by   Mirko Gschwindt, et al.
Aarhus Universitet
Technische Universität München
Nanyang Technological University
Carnegie Mellon University

Aerial filming is becoming more and more popular thanks to the recent advances in drone technology. It invites many intriguing, unsolved problems at the intersection of aesthetical and scientific challenges. In this work, we propose an intelligent agent which supervises motion planning of a filming drone based on aesthetical values of video shots using deep reinforcement learning. Unlike the current state-of-the-art approaches which mostly require explicit guidance by a human expert, our drone learns how to make favorable shot type selections by experience. We propose a learning scheme which exploits aesthetical features of retrospective shots in order to extract a desirable policy for better prospective shots. We train our agent in realistic AirSim simulations using both hand-crafted and human reward functions. We deploy the same agent on a real DJI M210 drone in order to test generalization capability of our approach to real world conditions. To evaluate the success of our approach in the end, we conduct a comprehensive user study in which participants rate the shots taken using our method and write comments about them.


page 1

page 5


Long-Term Planning with Deep Reinforcement Learning on Autonomous Drones

In this paper, we study a long-term planning scenario that is based on d...

Human Motion Capture Using a Drone

Current motion capture (MoCap) systems generally require markers and mul...

Drone Shadow Tracking

Aerial videos taken by a drone not too far above the surface may contain...

AirCapRL: Autonomous Aerial Human Motion Capture using Deep Reinforcement Learning

In this letter, we introduce a deep reinforcement learning (RL) based mu...

TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos

Drone-to-drone detection using visual feed has crucial applications like...

Learning to Learn in Simulation

Deep learning often requires the manual collection and annotation of a t...