DeepAI AI Chat
Log In Sign Up

Evolutionary Selective Imitation: Interpretable Agents by Imitation Learning Without a Demonstrator

by   Roy Eliya, et al.

We propose a new method for training an agent via an evolutionary strategy (ES), in which we iteratively improve a set of samples to imitate: Starting with a random set, in every iteration we replace a subset of the samples with samples from the best trajectories discovered so far. The evaluation procedure for this set is to train, via supervised learning, a randomly initialised neural network (NN) to imitate the set and then execute the acquired policy against the environment. Our method is thus an ES based on a fitness function that expresses the effectiveness of imitating an evolving data subset. This is in contrast to other ES techniques that iterate over the weights of the policy directly. By observing the samples that the agent selects for learning, it is possible to interpret and evaluate the evolving strategy of the agent more explicitly than in NN learning. In our experiments, we trained an agent to solve the OpenAI Gym environment Bipedalwalker-v3 by imitating an evolutionarily selected set of only 25 samples with a NN with only a few thousand parameters. We further test our method on the Procgen game Plunder and show here as well that the proposed method is an interpretable, small, robust and effective alternative to other ES or policy gradient methods.


page 1

page 2

page 3

page 4


Imitation Learning for Generalizable Self-driving Policy with Sim-to-real Transfer

Imitation Learning uses the demonstrations of an expert to uncover the o...

Imitation Learning by State-Only Distribution Matching

Imitation Learning from observation describes policy learning in a simil...

Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Games

Recent research on vulnerabilities of deep reinforcement learning (RL) h...

Evaluation Function Approximation for Scrabble

The current state-of-the-art Scrabble agents are not learning-based but ...

A Ranking Game for Imitation Learning

We propose a new framework for imitation learning - treating imitation a...

Modelling Agent Policies with Interpretable Imitation Learning

As we deploy autonomous agents in safety-critical domains, it becomes im...

A Privacy-preserving Distributed Training Framework for Cooperative Multi-agent Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) sometimes needs a large amount of data...