Continuous Deep Hierarchical Reinforcement Learning for Ground-Air Swarm Shepherding

by   Hung The Nguyen, et al.

The control and guidance of multi-robots (swarm) is a non-trivial problem due to the complexity inherent in the coupled interaction among the group. Whether the swarm is cooperative or non cooperative, lessons could be learnt from sheepdogs herding sheep. Biomimicry of shepherding offers computational methods for swarm control with the potential to generalize and scale in different environments. However, learning to shepherd is complex due to the large search space that a machine learner is faced with. We present a deep hierarchical reinforcement learning approach for shepherding, whereby an unmanned aerial vehicle (UAV) learns to act as an Aerial sheepdog to control and guide a swarm of unmanned ground vehicles (UGVs). The approach extends our previous work on machine education to decompose the search space into hierarchically organized curriculum. Each lesson in the curriculum is learnt by a deep reinforcement learning model. The hierarchy is formed by fusing the outputs of the model. The approach is demonstrated first in a high-fidelity robotic-operating-system (ROS)-based simulation environment, then with physical UGVs and a UAV in an in-door testing facility. We investigate the ability of the method to generalize as the models move from simulation to the real-world and as the models move from one scale to another.


page 1

page 12


Machine Teaching in Hierarchical Genetic Reinforcement Learning: Curriculum Design of Reward Functions for Swarm Shepherding

The design of reward functions in reinforcement learning is a human skil...

Transparent Machine Education of Neural Networks for Swarm Shepherding Using Curriculum Design

Swarm control is a difficult problem due to the need to guide a large nu...

UAV Swarm Deployment and Trajectory for 3D Area Coverage via Reinforcement Learning

Unmanned aerial vehicles (UAVs) are recognized as promising technologies...

An Efficient Approach with Dynamic Multi-Swarm of UAVs for Forest Firefighting

In this paper, the Multi-Swarm Cooperative Information-driven search and...

Learning Unmanned Aerial Vehicle Control for Autonomous Target Following

While deep reinforcement learning (RL) methods have achieved unprecedent...

Towards Reliable UAV Swarm Communication in D2D-Enhanced Cellular Network

In the existing cellular networks, it remains a challenging problem to c...

Graphical Games for UAV Swarm Control Under Time-Varying Communication Networks

We propose a unified framework for coordinating Unmanned Aerial Vehicle ...

Please sign up or login with your details

Forgot password? Click here to reset