Log In Sign Up

Robot Program Parameter Inference via Differentiable Shadow Program Inversion

by   Benjamin Alt, et al.

Challenging manipulation tasks can be solved effectively by combining individual robot skills, which must be parameterized for the concrete physical environment and task at hand. This is time-consuming and difficult for human programmers, particularly for force-controlled skills. To this end, we present Shadow Program Inversion (SPI), a novel approach to infer optimal skill parameters directly from data. SPI leverages unsupervised learning to train an auxiliary differentiable program representation ("shadow program") and realizes parameter inference via gradient-based model inversion. Our method enables the use of efficient first-order optimizers to infer optimal parameters for originally non-differentiable skills, including many skill variants currently used in production. SPI zero-shot generalizes across task objectives, meaning that shadow programs do not need to be retrained to infer parameters for different task variants. We evaluate our methods on three different robots and skill frameworks in industrial and household scenarios. Code and examples are available at


page 1

page 4

page 5

page 6


Learning to Sequence and Blend Robot Skills via Differentiable Optimization

In contrast to humans and animals who naturally execute seamless motions...

Geometric Task Networks: Learning Efficient and Explainable Skill Coordination for Object Manipulation

Complex manipulation tasks can contain various execution branches of pri...

Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control

Robots need to learn skills that can not only generalize across similar ...

Learning Forceful Manipulation Skills from Multi-modal Human Demonstrations

Learning from Demonstration (LfD) provides an intuitive and fast approac...

SQRP: Sensing Quality-aware Robot Programming System for Non-expert Programmers

Robot programming typically makes use of a set of mechanical skills that...

Reversing Imperative Parallel Programs

We propose an approach and a subsequent extension for reversing imperati...