Emergence of Locomotion Behaviours in Rich Environments

by   Nicolas Heess, et al.

The reinforcement learning paradigm allows, in principle, for complex behaviours to be learned directly from simple reward signals. In practice, however, it is common to carefully hand-design the reward function to encourage a particular solution, or to derive it from demonstration data. In this paper explore how a rich environment can help to promote the learning of complex behavior. Specifically, we train agents in diverse environmental contexts, and find that this encourages the emergence of robust behaviours that perform well across a suite of tasks. We demonstrate this principle for locomotion -- behaviours that are known for their sensitivity to the choice of reward. We train several simulated bodies on a diverse set of challenging terrains and obstacles, using a simple reward function based on forward progress. Using a novel scalable variant of policy gradient reinforcement learning, our agents learn to run, jump, crouch and turn as required by the environment without explicit reward-based guidance. A visual depiction of highlights of the learned behavior can be viewed following https://youtu.be/hx_bgoTF7bs .


page 6

page 7

page 8


Information Theoretically Aided Reinforcement Learning for Embodied Agents

Reinforcement learning for embodied agents is a challenging problem. The...

Internally Rewarded Reinforcement Learning

We study a class of reinforcement learning problems where the reward sig...

Curious Exploration and Return-based Memory Restoration for Deep Reinforcement Learning

Reward engineering and designing an incentive reward function are non-tr...

Dynamical Distance Learning for Unsupervised and Semi-Supervised Skill Discovery

Reinforcement learning requires manual specification of a reward functio...

Predator-prey survival pressure is sufficient to evolve swarming behaviors

The comprehension of how local interactions arise in global collective b...

Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition

We study the problem of realizing the full spectrum of bipedal locomotio...

Simplifying Reward Design through Divide-and-Conquer

Designing a good reward function is essential to robot planning and rein...

Code Repositories


Pytorch implementation of Distributed Proximal Policy Optimization: https://arxiv.org/abs/1707.02286

view repo

Please sign up or login with your details

Forgot password? Click here to reset