DeepAI AI Chat
Log In Sign Up

NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty

03/23/2022
by   Jonathan Balloch, et al.
Georgia Institute of Technology
11

A robust body of reinforcement learning techniques have been developed to solve complex sequential decision making problems. However, these methods assume that train and evaluation tasks come from similarly or identically distributed environments. This assumption does not hold in real life where small novel changes to the environment can make a previously learned policy fail or introduce simpler solutions that might never be found. To that end we explore the concept of novelty, defined in this work as the sudden change to the mechanics or properties of environment. We provide an ontology of for novelties most relevant to sequential decision making, which distinguishes between novelties that affect objects versus actions, unary properties versus non-unary relations, and the distribution of solutions to a task. We introduce NovGrid, a novelty generation framework built on MiniGrid, acting as a toolkit for rapidly developing and evaluating novelty-adaptation-enabled reinforcement learning techniques. Along with the core NovGrid we provide exemplar novelties aligned with our ontology and instantiate them as novelty templates that can be applied to many MiniGrid-compliant environments. Finally, we present a set of metrics built into our framework for the evaluation of novelty-adaptation-enabled machine-learning techniques, and show characteristics of a baseline RL model using these metrics.

READ FULL TEXT
01/16/2023

Neuro-Symbolic World Models for Adapting to Open World Novelty

Open-world novelty–a sudden change in the mechanics or properties of an ...
01/16/2014

Non-Deterministic Policies in Markovian Decision Processes

Markovian processes have long been used to model stochastic environments...
02/23/2023

Characterizing Novelty in the Military Domain

A critical factor in utilizing agents with Artificial Intelligence (AI) ...
01/19/2023

Effective Diversity in Unsupervised Environment Design

Agent decision making using Reinforcement Learning (RL) heavily relies o...
06/22/2020

Ecological Reinforcement Learning

Much of the current work on reinforcement learning studies episodic sett...
06/02/2000

Novelty Detection on a Mobile Robot Using Habituation

In this paper a novelty filter is introduced which allows a robot operat...
06/05/2021

Empirically Evaluating Creative Arc Negotiation for Improvisational Decision-making

Action selection from many options with few constraints is crucial for i...