Overcoming Exploration: Deep Reinforcement Learning in Complex Environments from Temporal Logic Specifications

01/28/2022

∙

We present a Deep Reinforcement Learning (DRL) algorithm for a task-guided robot with unknown continuous-time dynamics deployed in a large-scale complex environment. Linear Temporal Logic (LTL) is applied to express a rich robotic specification. To overcome the environmental challenge, we propose a novel path planning-guided reward scheme that is dense over the state space, and crucially, robust to infeasibility of computed geometric paths due to the unknown robot dynamics. To facilitate LTL satisfaction, our approach decomposes the LTL mission into sub-tasks that are solved using distributed DRL, where the sub-tasks are trained in parallel, using Deep Policy Gradient algorithms. Our framework is shown to significantly improve performance (effectiveness, efficiency) and exploration of robots tasked with complex missions in large-scale complex environments.

READ FULL TEXT

Overcoming Exploration: Deep Reinforcement Learning in Complex Environments from Temporal Logic Specifications

Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specifications

Reinforcement Learning for Signal Temporal Logic using Funnel-Based Approach

Creativity in Robot Manipulation with Deep Reinforcement Learning

Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functions

Reactive Temporal Logic Planning for Multiple Robots in Unknown Occupancy Grid Maps

Safe-Critical Modular Deep Reinforcement Learning with Temporal Logic through Gaussian Processes and Control Barrier Functions

Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning

Overcoming Exploration: Deep Reinforcement Learning in Complex Environments from Temporal Logic Specifications

Related Research

Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specifications

Reinforcement Learning for Signal Temporal Logic using Funnel-Based Approach

Creativity in Robot Manipulation with Deep Reinforcement Learning

Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functions

Reactive Temporal Logic Planning for Multiple Robots in Unknown Occupancy Grid Maps

Safe-Critical Modular Deep Reinforcement Learning with Temporal Logic through Gaussian Processes and Control Barrier Functions

Abstract Demonstrations and Adaptive Exploration for Efficient and Stable Multi-step Sparse Reward Reinforcement Learning