DeepAI
Log In Sign Up

Accelerating Reinforcement Learning through Implicit Imitation

06/03/2011
by   C. Boutilier, et al.
0

Imitation can be viewed as a means of enhancing learning in multiagent environments. It augments an agent's ability to learn useful behaviors by making intelligent use of the knowledge implicit in behaviors demonstrated by cooperative teachers or other more experienced agents. We propose and study a formal model of implicit imitation that can accelerate reinforcement learning dramatically in certain cases. Roughly, by observing a mentor, a reinforcement-learning agent can extract information about its own capabilities in, and the relative value of, unvisited parts of the state space. We study two specific instantiations of this model, one in which the learning agent and the mentor have identical abilities, and one designed to deal with agents and mentors with different action sets. We illustrate the benefits of implicit imitation by integrating it with prioritized sweeping, and demonstrating improved performance and convergence through observation of single and multiple mentors. Though we make some stringent assumptions regarding observability and possible interactions, we briefly comment on extensions of the model that relax these restricitions.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/24/2020

BabyAI 1.1

The BabyAI platform is designed to measure the sample efficiency of trai...
10/01/2018

Interactive Agent Modeling by Learning to Probe

The ability of modeling the other agents, such as understanding their in...
09/25/2019

Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems

Many tasks in practice require the collaboration of multiple agents thro...
07/23/2020

Bridging the Imitation Gap by Adaptive Insubordination

Why do agents often obtain better reinforcement learning policies when i...
09/29/2021

Information-Bottleneck-Based Behavior Representation Learning for Multi-agent Reinforcement learning

In multi-agent deep reinforcement learning, extracting sufficient and co...
01/28/2019

CLIC: Curriculum Learning and Imitation for feature Control in non-rewarding environments

In this paper, we propose an unsupervised reinforcement learning agent c...
04/30/2020

Towards Embodied Scene Description

Embodiment is an important characteristic for all intelligent agents (cr...