Teaching a Robot to Walk Using Reinforcement Learning

by   Jack Dibachi, et al.
Stanford University

Classical control techniques such as PID and LQR have been used effectively in maintaining a system state, but these techniques become more difficult to implement when the model dynamics increase in complexity and sensitivity. For adaptive robotic locomotion tasks with several degrees of freedom, this task becomes infeasible with classical control techniques. Instead, reinforcement learning can train optimal walking policies with ease. We apply deep Q-learning and augmented random search (ARS) to teach a simulated two-dimensional bipedal robot how to walk using the OpenAI Gym BipedalWalker-v3 environment. Deep Q-learning did not yield a high reward policy, often prematurely converging to suboptimal local maxima likely due to the coarsely discretized action space. ARS, however, resulted in a better trained robot, and produced an optimal policy which officially "solves" the BipedalWalker-v3 problem. Various naive policies, including a random policy, a manually encoded inch forward policy, and a stay still policy, were used as benchmarks to evaluate the proficiency of the learning algorithm results.


Iterative Reinforcement Learning Based Design of Dynamic Locomotion Skills for Cassie

Deep reinforcement learning (DRL) is a promising approach for developing...

PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations

Evolution Strategy (ES) algorithms have shown promising results in train...

Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

In this article, we show that learned policies can be applied to solve l...

Sim2Real Transfer for Reinforcement Learning without Dynamics Randomization

In this work we show how to use the Operational Space Control framework ...

A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning

Deep reinforcement learning is a promising approach to learning policies...

Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning

In this work, we present and study a training set-up that achieves fast ...

Computing Complexity-aware Plans Using Kolmogorov Complexity

In this paper, we introduce complexity-aware planning for finite-horizon...

Please sign up or login with your details

Forgot password? Click here to reset