DeepAI AI Chat
Log In Sign Up

Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning

by   Guillermo A. Castillo, et al.
Southern University of Science & Technology
The Ohio State University

This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the hybrid nature of the walking dynamics and the well-established hybrid zero dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, short training time, and less dependence on prior knowledge. We demonstrate the effectiveness of the proposed method on Cassie, a challenging 3D bipedal robot. The proposed solution produces stable limit walking cycles that can track various walking speed in different directions. Surprisingly, without specifically trained with disturbances to achieve robustness, it also performs robustly against various adversarial forces applied to the torso towards both the forward and the backward directions.


Reinforcement Learning Meets Hybrid Zero Dynamics: A Case Study for RABBIT

The design of feedback controllers for bipedal robots is challenging due...

Learning Linear Policies for Robust Bipedal Locomotion on Terrains with Varying Slopes

In this paper, with a view toward deployment of light-weight control fra...

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Deep reinforcement learning (RL) based controllers for legged robots hav...

Natural Multicontact Walking for Robotic Assistive Devices via Musculoskeletal Models and Hybrid Zero Dynamics

Generating provably stable walking gaits that yield natural locomotion w...

Reinforcement Learning with Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion

Recently reinforcement learning (RL) has emerged as a promising approach...

Excavation Reinforcement Learning Using Geometric Representation

Excavation of irregular rigid objects in clutter, such as fragmented roc...

Beyond Basins of Attraction: Evaluating Robustness of Natural Dynamics

It is commonly accepted that properly designing a system to exhibit favo...