DeepAI
Log In Sign Up

MULTIPOLAR: Multi-Source Policy Aggregation for Transfer Reinforcement Learning between Diverse Environmental Dynamics

09/28/2019
by   Mohammadamin Barekatain, et al.
6

Transfer reinforcement learning (RL) aims at improving learning efficiency of an agent by exploiting knowledge from other source agents trained on relevant tasks. However, it remains challenging to transfer knowledge between different environmental dynamics without having access to the source environments. In this work, we explore a new challenge in transfer RL, where only a set of source policies collected under unknown diverse dynamics is available for learning a target task efficiently. To address this problem, the proposed approach, MULTI-source POLicy AggRegation (MULTIPOLAR), comprises two key techniques. We learn to aggregate the actions provided by the source policies adaptively to maximize the target task performance. Meanwhile, we learn an auxiliary network that predicts residuals around the aggregated actions, which ensures the target policy's expressiveness even when some of the source policies perform poorly. We demonstrated the effectiveness of MULTIPOLAR through an extensive experimental evaluation across six simulated environments ranging from classic control problems to challenging robotics simulations, under both continuous and discrete action spaces.

READ FULL TEXT

page 2

page 15

page 16

02/19/2020

Efficient Deep Reinforcement Learning through Policy Transfer

Transfer Learning (TL) has shown great potential to accelerate Reinforce...
01/22/2018

Cross-Domain Transfer in Reinforcement Learning using Target Apprentice

In this paper, we present a new approach to Transfer Learning (TL) in Re...
10/27/2021

Transfer learning with causal counterfactual reasoning in Decision Transformers

The ability to adapt to changes in environmental contingencies is an imp...
05/12/2018

Adversarial Task Transfer from Preference

Task transfer is extremely important for reinforcement learning, since i...
10/17/2019

Single Episode Policy Transfer in Reinforcement Learning

Transfer and adaptation to new unknown environmental dynamics is a key c...
02/29/2020

Contextual Policy Reuse using Deep Mixture Models

Reinforcement learning methods that consider the context, or current sta...
08/01/2017

Deep Transfer in Reinforcement Learning by Language Grounding

In this paper, we explore the utilization of natural language to drive t...

Code Repositories

multipolar

Multi-Source Policy Aggregation for Transfer Reinforcement Learning between Diverse Environmental Dynamics


view repo