
Transfer from Multiple MDPs
Transfer reinforcement learning (RL) methods leverage on the experience ...
read it

Importance Weighted Transfer of Samples in Reinforcement Learning
We consider the transfer of experience samples (i.e., tuples < s, a, s',...
read it

Accelerating Reinforcement Learning by Composing Solutions of Automatically Identified Subtasks
This paper discusses a system that accelerates reinforcement learning by...
read it

Induction and Exploitation of Subgoal Automata for Reinforcement Learning
In this paper we present ISA, an approach for learning and exploiting su...
read it

Induction of Subgoal Automata for Reinforcement Learning
In this work we present ISA, a novel approach for learning and exploitin...
read it

TimeVariant Variational Transfer for Value Functions
In most transfer learning approaches to reinforcement learning (RL) the ...
read it

A Taxonomy of Similarity Metrics for Markov Decision Processes
Although the notion of task similarity is potentially interesting in a w...
read it
Transfer of Temporal Logic Formulas in Reinforcement Learning
Transferring highlevel knowledge from a source task to a target task is an effective way to expedite reinforcement learning (RL). For example, propositional logic and firstorder logic have been used as representations of such knowledge. We study the transfer of knowledge between tasks in which the timing of the events matters. We call such tasks temporal tasks. We concretize similarity between temporal tasks through a notion of logical transferability, and develop a transfer learning approach between different yet similar temporal tasks. We first propose an inference technique to extract metric interval temporal logic (MITL) formulas in sequential disjunctive normal form from labeled trajectories collected in RL of the two tasks. If logical transferability is identified through this inference, we construct a timed automaton for each sequential conjunctive subformula of the inferred MITL formulas from both tasks. We perform RL on the extended state which includes the locations and clock valuations of the timed automata for the source task. We then establish mappings between the corresponding components (clocks, locations, etc.) of the timed automata from the two tasks, and transfer the extended Qfunctions based on the established mappings. Finally, we perform RL on the extended state for the target task, starting with the transferred extended Qfunctions. Our results in two case studies show, depending on how similar the source task and the target task are, that the sampling efficiency for the target task can be improved by up to one order of magnitude by performing RL in the extended state space, and further improved by up to another order of magnitude using the transferred extended Qfunctions.
READ FULL TEXT
Comments
There are no comments yet.