Learning When to Switch: Composing Controllers to Traverse a Sequence of Terrain Artifacts

by   Brendan Tidd, et al.

Legged robots often use separate control policies that are highly engineered for traversing difficult terrain such as stairs, gaps, and steps, where switching between policies is only possible when the robot is in a region that is common to adjacent controllers. Deep Reinforcement Learning (DRL) is a promising alternative to hand-crafted control design, though typically requires the full set of test conditions to be known before training. DRL policies can result in complex (often unrealistic) behaviours that have few or no overlapping regions between adjacent policies, making it difficult to switch behaviours. In this work we develop multiple DRL policies with Curriculum Learning (CL), each that can traverse a single respective terrain condition, while ensuring an overlap between policies. We then train a network for each destination policy that estimates the likelihood of successfully switching from any other policy. We evaluate our switching method on a previously unseen combination of terrain artifacts and show that it performs better than heuristic methods. While our method is trained on individual terrain types, it performs comparably to a Deep Q Network trained on the full set of terrain conditions. This approach allows the development of separate policies in constrained conditions with embedded prior knowledge about each behaviour, that is scalable to any number of behaviours, and prepares DRL methods for applications in the real world



page 1

page 4


Learning Setup Policies: Reliable Transition Between Locomotion Behaviours

Dynamic platforms that operate over manyunique terrain conditions typica...

Multi-Agent Deep Reinforcement Learning for Request Dispatching in Distributed-Controller Software-Defined Networking

Recently, distributed controller architectures have been quickly gaining...

A Deep Reinforcement Learning Environment for Particle Robot Navigation and Object Manipulation

Particle robots are novel biologically-inspired robotic systems where lo...

On Training Flexible Robots using Deep Reinforcement Learning

The use of robotics in controlled environments has flourished over the l...

Learning to Switch Between Machines and Humans

Reinforcement learning algorithms have been mostly developed and evaluat...

Large-scale traffic signal control using machine learning: some traffic flow considerations

This paper uses supervised learning, random search and deep reinforcemen...

Towards Scalable Verification of RL-Driven Systems

Deep neural networks (DNNs) have gained significant popularity in recent...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.