DeepAI AI Chat
Log In Sign Up

Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking

03/15/2022
by   Helei Duan, et al.
Oregon State University
0

Recently, work on reinforcement learning (RL) for bipedal robots has successfully learned controllers for a variety of dynamic gaits with robust sim-to-real demonstrations. In order to maintain balance, the learned controllers have full freedom of where to place the feet, resulting in highly robust gaits. In the real world however, the environment will often impose constraints on the feasible footstep locations, typically identified by perception systems. Unfortunately, most demonstrated RL controllers on bipedal robots do not allow for specifying and responding to such constraints. This missing control interface greatly limits the real-world application of current RL controllers. In this paper, we aim to maintain the robust and dynamic nature of learned gaits while also respecting footstep constraints imposed externally. We develop an RL formulation for training dynamic gait controllers that can respond to specified touchdown locations. We then successfully demonstrate simulation and sim-to-real performance on the bipedal robot Cassie. In addition, we use supervised learning to induce a transition model for accurately predicting the next touchdown locations that the controller can achieve given the robot's proprioceptive observations. This model paves the way for integrating the learned controller into a full-order robot locomotion planner that robustly satisfies both balance and environmental constraints.

READ FULL TEXT

page 1

page 4

page 6

05/03/2022

Learning Dynamic Bipedal Walking Across Stepping Stones

In this work, we propose a learning approach for 3D dynamic bipedal walk...
04/15/2020

Improving Input-Output Linearizing Controllers for Bipedal Robots via Reinforcement Learning

The main drawbacks of input-output linearizing controllers are the need ...
04/02/2021

How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?

The difficulty of optimal control problems has classically been characte...
07/26/2022

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Deep reinforcement learning (RL) based controllers for legged robots hav...
12/10/2021

Zero-Shot Uncertainty-Aware Deployment of Simulation Trained Policies on Real-World Robots

While deep reinforcement learning (RL) agents have demonstrated incredib...
08/23/2018

Decentralized Control of a Hexapod Robot Using a Wireless Time Synchronized Network

Robots and control systems rely upon precise timing of sensors and actua...
08/17/2021

Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning

Simulation to real (Sim-to-Real) is an attractive approach to construct ...