Language to Rewards for Robotic Skill Synthesis

by   Wenhao Yu, et al.

Large language models (LLMs) have demonstrated exciting progress in acquiring diverse new capabilities through in-context learning, ranging from logical reasoning to code-writing. Robotics researchers have also explored using LLMs to advance the capabilities of robotic control. However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot. On the other hand, reward functions are shown to be flexible representations that can be optimized for control policies to achieve diverse tasks, while their semantic richness makes them suitable to be specified by LLMs. In this work, we introduce a new paradigm that harnesses this realization by utilizing LLMs to define reward parameters that can be optimized and accomplish variety of robotic tasks. Using reward as the intermediate interface generated by LLMs, we can effectively bridge the gap between high-level language instructions or corrections to low-level robot actions. Meanwhile, combining this with a real-time optimizer, MuJoCo MPC, empowers an interactive behavior creation experience where users can immediately observe the results and provide feedback to the system. To systematically evaluate the performance of our proposed method, we designed a total of 17 tasks for a simulated quadruped robot and a dexterous manipulator robot. We demonstrate that our proposed method reliably tackles 90 using primitive skills as the interface with Code-as-policies achieves 50 the tasks. We further validated our method on a real robot arm where complex manipulation skills such as non-prehensile pushing emerge through our interactive system.


page 6

page 8

page 14

page 25

page 26

page 27

page 28

page 30


Scaling simulation-to-real transfer by learning composable robot skills

We present a novel solution to the problem of simulation-to-real transfe...

A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy

We designed a modular robotic control stack that provides a customizable...

Towards Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and Generative Models

To enable robots to achieve high level objectives, engineers typically w...

Learning Adaptive Language Interfaces through Decomposition

Our goal is to create an interactive natural language interface that eff...

Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Models

In recent years, much progress has been made in learning robotic manipul...

Chat with the Environment: Interactive Multimodal Perception using Large Language Models

Programming robot behaviour in a complex world faces challenges on multi...

AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation

We propose a novel framework for learning high-level cognitive capabilit...

Please sign up or login with your details

Forgot password? Click here to reset