Bundled Gradients through Contact via Randomized Smoothing

09/11/2021
by   H. J. Terry Suh, et al.
0

The empirical success of derivative-free methods in reinforcement learning for planning through contact seems at odds with the perceived fragility of classical gradient-based optimization methods in these domains. What is causing this gap, and how might we use the answer to improve gradient-based methods? We believe a stochastic formulation of dynamics is one crucial ingredient. We use tools from randomized smoothing to analyze sampling-based approximations of the gradient, and formalize such approximations through the gradient bundle. We show that using the gradient bundle in lieu of the gradient mitigates fast-changing gradients of non-smooth contact dynamics modeled by the implicit time-stepping, or the penalty method. Finally, we apply the gradient bundle to optimal control using iLQR, introducing a novel algorithm which improves convergence over using exact gradients. Combining our algorithm with a convex implicit time-stepping formulation of contact, we show that we can tractably tackle planning-through-contact problems in manipulation.

READ FULL TEXT

page 1

page 6

research
06/22/2022

Global Planning for Contact-Rich Manipulation via Local Smoothing of Quasi-dynamic Contact Models

The empirical success of Reinforcement Learning (RL) in the setting of c...
research
10/18/2017

Nonsmooth optimal value and policy functions for mechanical systems subject to unilateral constraints

State-of-the-art approaches to optimal control of contact-rich robot dyn...
research
10/24/2018

Contact-Implicit Trajectory Optimization Based on a Variable Smooth Contact Model and Successive Convexification

In this paper, we propose a contact-implicit trajectory optimization (CI...
research
12/11/2018

KF-LAX: Kronecker-factored curvature estimation for control variate optimization in reinforcement learning

A key challenge for gradient based optimization methods in model-free re...
research
05/15/2020

Sobolev Gradients for the Möbius Energy

Aiming at optimizing the shape of closed embedded curves within prescrib...
research
09/23/2020

ContactNets: Learning of Discontinuous Contact Dynamics with Smooth, Implicit Representations

Common methods for learning robot dynamics assume motion is continuous, ...
research
06/23/2022

Augmenting differentiable physics with randomized smoothing

In the past few years, following the differentiable programming paradigm...

Please sign up or login with your details

Forgot password? Click here to reset