DeepAI AI Chat
Log In Sign Up

Discretizing Dynamics for Maximum Likelihood Constraint Inference

by   Kaylene C. Stocking, et al.

Maximum likelihood constraint inference is a powerful technique for identifying unmodeled constraints that affect the behavior of a demonstrator acting under a known objective function. However, it was originally formulated only for discrete state-action spaces. Continuous dynamics are more useful for modeling many real-world systems of interest, including the movements of humans and robots. We present a method to generate a tabular state-action space that approximates continuous dynamics and can be used for constraint inference on demonstrations that obey the true system dynamics. We then demonstrate accurate constraint inference on nonlinear pendulum systems with 2- and 4-dimensional state spaces, and show that performance is robust to a range of hyperparameters. The demonstrations are not required to be fully optimal with respect to the objective, and the most likely constraints can be identified even when demonstrations cover only a small portion of the state space. For these reasons, the proposed approach may be especially useful for inferring constraints on human demonstrators, which has important applications in human-robot interaction and biomechanical medicine.


Maximum Likelihood Constraint Inference from Stochastic Demonstrations

When an expert operates a perilous dynamic system, ideal constraint info...

Learning Constraints from Demonstrations

We extend the learning from demonstration paradigm by providing a method...

Joint Estimation of Expertise and Reward Preferences From Human Demonstrations

When a robot learns from human examples, most approaches assume that the...

Constraint Inference in Control Tasks from Expert Demonstrations via Inverse Optimization

Inferring unknown constraints is a challenging and crucial problem in ma...

Continuous Control with Action Quantization from Demonstrations

In Reinforcement Learning (RL), discrete actions, as opposed to continuo...

Learning Constraints from Locally-Optimal Demonstrations under Cost Function Uncertainty

We present an algorithm for learning parametric constraints from locally...

Computational-level Analysis of Constraint Compliance for General Intelligence

Human behavior is conditioned by codes and norms that constrain action. ...