DeepAI AI Chat
Log In Sign Up

Feedback Control For Cassie With Deep Reinforcement Learning

by   Zhaoming Xie, et al.
The University of British Columbia
Oregon State University

Bipedal locomotion skills are challenging to develop. Control strategies often use local linearization of the dynamics in conjunction with reduced-order abstractions to yield tractable solutions. In these model-based control strategies, the controller is often not fully aware of many details, including torque limits, joint limits, and other non-linearities that are necessarily excluded from the control computations for simplicity. Deep reinforcement learning (DRL) offers a promising model-free approach for controlling bipedal locomotion which can more fully exploit the dynamics. However, current results in the machine learning literature are often based on ad-hoc simulation models that are not based on corresponding hardware. Thus it remains unclear how well DRL will succeed on realizable bipedal robots. In this paper, we demonstrate the effectiveness of DRL using a realistic model of Cassie, a bipedal robot. By formulating a feedback control problem as finding the optimal policy for a Markov Decision Process, we are able to learn robust walking controllers that imitate a reference motion with DRL. Controllers for different walking speeds are learned by imitating simple time-scaled versions of the original reference motion. Controller robustness is demonstrated through several challenging tests, including sensory delay, walking blindly on irregular terrain and unexpected pushes at the pelvis. We also show we can interpolate between individual policies and that robustness can be improved with an interpolated policy.


page 1

page 5

page 6


Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

Developing robust walking controllers for bipedal robots is a challengin...

Understanding the Stability of Deep Control Policies for Biped Locomotion

Achieving stability and robustness is the primary goal of biped locomoti...

DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning

Bipedal walking is one of the most difficult but exciting challenges in ...

Learning Control Policies for Fall prevention and safety in bipedal locomotion

The ability to recover from an unexpected external perturbation is a fun...

Learning Bipedal Walking for Humanoids with Current Feedback

Recent advances in deep reinforcement learning (RL) based techniques com...

Learning Symmetric and Low-energy Locomotion

Learning locomotion skills is a challenging problem. To generate realist...

Learning Symmetry and Low-energy Locomotion

Learning locomotion skills is a challenging problem. To generate realist...