Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specifications

10/03/2022
by   Mingyu Cai, et al.
0

This paper explores continuous-time control synthesis for target-driven navigation to satisfy complex high-level tasks expressed as linear temporal logic (LTL). We propose a model-free framework using deep reinforcement learning (DRL) where the underlying dynamic system is unknown (an opaque box). Unlike prior work, this paper considers scenarios where the given LTL specification might be infeasible and therefore cannot be accomplished globally. Instead of modifying the given LTL formula, we provide a general DRL-based approach to satisfy it with minimal violation. to the stochastic policy. I'm not repeating this comment everywhere that says "guarantees" but there are multiple places. To do this, we transform a previously multi-objective DRL problem, which requires simultaneous automata satisfaction and minimum violation cost, into a single objective. By guiding the DRL agent with a sampling-based path planning algorithm for the potentially infeasible LTL task, the proposed approach mitigates the myopic tendencies of DRL, which are often an issue when learning general LTL tasks that can have long or infinite horizons. This is achieved by decomposing an infeasible LTL formula into several reach-avoid sub-tasks with shorter horizons, which can be trained in a modular DRL architecture. Furthermore, we overcome the challenge of the exploration process for DRL in complex and cluttered environments by using path planners to design rewards that are dense in the configuration space. The benefits of the presented approach are demonstrated through testing on various complex nonlinear systems and compared with state-of-the-art baselines. The Video demonstration can be found on YouTube Channel:<https://youtu.be/jBhx6Nv224E>.

READ FULL TEXT

page 1

page 7

research
01/28/2022

Overcoming Exploration: Deep Reinforcement Learning in Complex Environments from Temporal Logic Specifications

We present a Deep Reinforcement Learning (DRL) algorithm for a task-guid...
research
03/02/2022

Model-free Neural Lyapunov Control for Safe Robot Navigation

Model-free Deep Reinforcement Learning (DRL) controllers have demonstrat...
research
02/04/2022

Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectives

Reinforcement learning (RL) is a popular approach for robotic path plann...
research
10/18/2021

In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specifications

We address the problem of building agents whose goal is to satisfy out-o...
research
07/17/2020

Hierarchical Deep Reinforcement Learning Approach for Multi-Objective Scheduling With Varying Queue Sizes

Multi-objective task scheduling (MOTS) is the task scheduling while opti...
research
04/20/2023

TempoRL: laser pulse temporal shape optimization with Deep Reinforcement Learning

High Power Laser's (HPL) optimal performance is essential for the succes...
research
06/12/2020

Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learning

This paper presents a neuro-symbolic agent that combines deep reinforcem...

Please sign up or login with your details

Forgot password? Click here to reset