GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning

02/22/2021
by   Guy Tennenholtz, et al.
0

Offline reinforcement learning approaches can generally be divided to proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. We impose a latent representation of states and actions and leverage its intrinsic Riemannian geometry to measure distance of latent samples to the data. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data. We integrate our metrics in a model-based offline optimization framework, in which proximity and uncertainty can be carefully controlled. We illustrate the geodesics on a simple grid-like environment, depicting its natural inherent topology. Finally, we analyze our approach and improve upon contemporary offline RL benchmarks.

READ FULL TEXT

page 6

page 13

research
09/05/2023

Model-based Offline Policy Optimization with Adversarial Network

Model-based offline reinforcement learning (RL), which builds a supervis...
research
06/03/2022

Offline Reinforcement Learning with Causal Structured World Models

Model-based methods have recently shown promising for offline reinforcem...
research
04/10/2023

Uncertainty-driven Trajectory Truncation for Model-based Offline Reinforcement Learning

Equipped with the trained environmental dynamics, model-based offline re...
research
06/13/2023

A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning

Offline reinforcement learning (RL) provides a promising solution to lea...
research
11/06/2022

Wall Street Tree Search: Risk-Aware Planning for Offline Reinforcement Learning

Offline reinforcement-learning (RL) algorithms learn to make decisions u...
research
10/08/2021

Revisiting Design Choices in Model-Based Offline Reinforcement Learning

Offline reinforcement learning enables agents to leverage large pre-coll...

Please sign up or login with your details

Forgot password? Click here to reset