DeepAI AI Chat
Log In Sign Up

Outcome-Guided Counterfactuals for Reinforcement Learning Agents from a Jointly Trained Generative Latent Space

07/15/2022
by   Eric Yeh, et al.
0

We present a novel generative method for producing unseen and plausible counterfactual examples for reinforcement learning (RL) agents based upon outcome variables that characterize agent behavior. Our approach uses a variational autoencoder to train a latent space that jointly encodes information about the observations and outcome variables pertaining to an agent's behavior. Counterfactuals are generated using traversals in this latent space, via gradient-driven updates as well as latent interpolations against cases drawn from a pool of examples. These include updates to raise the likelihood of generated examples, which improves the plausibility of generated counterfactuals. From experiments in three RL environments, we show that these methods produce counterfactuals that are more plausible and proximal to their queries compared to purely outcome-driven or case-based baselines. Finally, we show that a latent jointly trained to reconstruct both the input observations and behavioral outcome variables produces higher-quality counterfactuals over latents trained solely to reconstruct the observation inputs.

READ FULL TEXT

page 3

page 6

06/03/2022

Reinforcement Learning with Neural Radiance Fields

It is a long-standing problem to find effective representations for trai...
06/06/2019

DeepMDP: Learning Continuous Latent Space Models for Representation Learning

Many reinforcement learning (RL) tasks provide the agent with high-dimen...
06/17/2019

Visual Navigation by Generating Next Expected Observations

We propose a novel approach to visual navigation in unknown environments...
11/08/2017

LatentPoison - Adversarial Attacks On The Latent Space

Robustness and security of machine learning (ML) systems are intertwined...
10/03/2022

Interpretable Option Discovery using Deep Q-Learning and Variational Autoencoders

Deep Reinforcement Learning (RL) is unquestionably a robust framework to...
09/30/2022

Holographic-(V)AE: an end-to-end SO(3)-Equivariant (Variational) Autoencoder in Fourier Space

Group-equivariant neural networks have emerged as a data-efficient appro...
09/29/2022

Learning Parsimonious Dynamics for Generalization in Reinforcement Learning

Humans are skillful navigators: We aptly maneuver through new places, re...