Model-based Reinforcement Learning from Signal Temporal Logic Specifications

11/10/2020
by   Parv Kapoor, et al.
0

Techniques based on Reinforcement Learning (RL) are increasingly being used to design control policies for robotic systems. RL fundamentally relies on state-based reward functions to encode desired behavior of the robot and bad reward functions are prone to exploitation by the learning agent, leading to behavior that is undesirable in the best case and critically dangerous in the worst. On the other hand, designing good reward functions for complex tasks is a challenging problem. In this paper, we propose expressing desired high-level robot behavior using a formal specification language known as Signal Temporal Logic (STL) as an alternative to reward/cost functions. We use STL specifications in conjunction with model-based learning to design model predictive controllers that try to optimize the satisfaction of the STL specification over a finite time horizon. The proposed algorithm is empirically evaluated on simulations of robotic system such as a pick-and-place robotic arm, and adaptive cruise control for autonomous vehicles.

READ FULL TEXT
research
02/04/2022

Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectives

Reinforcement learning (RL) is a popular approach for robotic path plann...
research
12/11/2016

Reinforcement Learning With Temporal Logic Rewards

Reinforcement learning (RL) depends critically on the choice of reward f...
research
09/16/2021

Automated Testing with Temporal Logic Specifications for Robotic Controllers using Adaptive Experiment Design

Many robot control scenarios involve assessing system robustness against...
research
09/10/2023

Signal Temporal Logic Neural Predictive Control

Ensuring safety and meeting temporal specifications are critical challen...
research
06/25/2021

Non-Parametric Neuro-Adaptive Control Subject to Task Specifications

We develop a learning-based algorithm for the control of robotic systems...
research
09/07/2021

Safe-Critical Modular Deep Reinforcement Learning with Temporal Logic through Gaussian Processes and Control Barrier Functions

Reinforcement learning (RL) is a promising approach and has limited succ...
research
01/26/2020

Tractable Reinforcement Learning of Signal Temporal Logic Objectives

Signal temporal logic (STL) is an expressive language to specify time-bo...

Please sign up or login with your details

Forgot password? Click here to reset