Inverse Reinforcement Learning of Autonomous Behaviors Encoded as Weighted Finite Automata

by   Tianyu Wang, et al.

This paper presents a method for learning logical task specifications and cost functions from demonstrations. Linear temporal logic (LTL) formulas are widely used to express complex objectives and constraints for autonomous systems. Yet, such specifications may be challenging to construct by hand. Instead, we consider demonstrated task executions, whose temporal logic structure and transition costs need to be inferred by an autonomous agent. We employ a spectral learning approach to extract a weighted finite automaton (WFA), approximating the unknown logic structure of the task. Thereafter, we define a product between the WFA for high-level task guidance and a Labeled Markov decision process (L-MDP) for low-level control and optimize a cost function that matches the demonstrator's behavior. We demonstrate that our method is capable of generalizing the execution of the inferred task specification to new environment configurations.


page 1

page 7


Planning With Uncertain Specifications (PUnS)

Reward engineering is crucial to high performance in reinforcement learn...

Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees

Reinforcement Learning (RL) has emerged as an efficient method of choice...

Environment-Independent Task Specifications via GLTL

We propose a new task-specification language for Markov decision process...

Interpretable Apprenticeship Learning with Temporal Logic Specifications

Recent work has addressed using formulas in linear temporal logic (LTL) ...

Reinforcement Learning of Control Policy for Linear Temporal Logic Specifications Using Limit-Deterministic Generalized Büchi Automata

This letter proposes a novel reinforcement learning method for the synth...

Synthesis of Static Test Environments for Observing Sequence-like Behaviors in Autonomous Systems

In this paper, we investigate formal test-case generation for high-level...

Explaining Multi-stage Tasks by Learning Temporal Logic Formulas from Suboptimal Demonstrations

We present a method for learning multi-stage tasks from demonstrations b...