A Taxonomy of Similarity Metrics for Markov Decision Processes

03/08/2021
by   Álvaro Visús, et al.
0

Although the notion of task similarity is potentially interesting in a wide range of areas such as curriculum learning or automated planning, it has mostly been tied to transfer learning. Transfer is based on the idea of reusing the knowledge acquired in the learning of a set of source tasks to a new learning process in a target task, assuming that the target and source tasks are close enough. In recent years, transfer learning has succeeded in making Reinforcement Learning (RL) algorithms more efficient (e.g., by reducing the number of samples needed to achieve the (near-)optimal performance). Transfer in RL is based on the core concept of similarity: whenever the tasks are similar, the transferred knowledge can be reused to solve the target task and significantly improve the learning performance. Therefore, the selection of good metrics to measure these similarities is a critical aspect when building transfer RL algorithms, especially when this knowledge is transferred from simulation to the real world. In the literature, there are many metrics to measure the similarity between MDPs, hence, many definitions of similarity or its complement distance have been considered. In this paper, we propose a categorization of these metrics and analyze the definitions of similarity proposed so far, taking into account such categorization. We also follow this taxonomy to survey the existing literature, as well as suggesting future directions for the construction of new metrics.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/09/2022

Transferred Q-learning

We consider Q-learning with knowledge transfer, using samples from a tar...
research
11/12/2020

A partition-based similarity for classification distributions

Herein we define a measure of similarity between classification distribu...
research
07/27/2022

Structural Similarity for Improved Transfer in Reinforcement Learning

Transfer learning is an increasingly common approach for developing perf...
research
08/31/2011

Transfer from Multiple MDPs

Transfer reinforcement learning (RL) methods leverage on the experience ...
research
02/02/2021

Metrics and continuity in reinforcement learning

In most practical applications of reinforcement learning, it is untenabl...
research
03/29/2020

Experience Selection Using Dynamics Similarity for Efficient Multi-Source Transfer Learning Between Robots

In the robotics literature, different knowledge transfer approaches have...
research
05/31/2021

Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning

The idea of transfer in reinforcement learning (TRL) is intriguing: bein...

Please sign up or login with your details

Forgot password? Click here to reset