A Validation Tool for Designing Reinforcement Learning Environments

12/10/2021
by   Ruiyang Xu, et al.
0

Reinforcement learning (RL) has gained increasing attraction in the academia and tech industry with launches to a variety of impactful applications and products. Although research is being actively conducted on many fronts (e.g., offline RL, performance, etc.), many RL practitioners face a challenge that has been largely ignored: determine whether a designed Markov Decision Process (MDP) is valid and meaningful. This study proposes a heuristic-based feature analysis method to validate whether an MDP is well formulated. We believe an MDP suitable for applying RL should contain a set of state features that are both sensitive to actions and predictive in rewards. We tested our method in constructed environments showing that our approach can identify certain invalid environment formulations. As far as we know, performing validity analysis for RL problem formulation is a novel direction. We envision that our tool will serve as a motivational example to help practitioners apply RL in real-world problems more easily.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/17/2020

REALab: An Embedded Perspective on Tampering

This paper describes REALab, a platform for embedded agency research in ...
research
09/17/2019

!MDP Playground: Meta-Features in Reinforcement Learning

Reinforcement Learning (RL) algorithms usually assume their environment ...
research
04/16/2021

Towards Standardizing Reinforcement Learning Approaches for Stochastic Production Scheduling

Recent years have seen a rise in interest in terms of using machine lear...
research
07/21/2023

JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning

In this paper, we present JoinGym, an efficient and lightweight query op...
research
06/02/2022

Incrementality Bidding via Reinforcement Learning under Mixed and Delayed Rewards

Incrementality, which is used to measure the causal effect of showing an...
research
04/28/2021

A Reinforcement Learning Environment for Polyhedral Optimizations

The polyhedral model allows a structured way of defining semantics-prese...
research
11/29/2019

When Blockchain Meets AI: Optimal Mining Strategy Achieved By Machine Learning

This work applies reinforcement learning (RL) from the AI machine learni...

Please sign up or login with your details

Forgot password? Click here to reset