DeepAI AI Chat
Log In Sign Up

NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty

by   Jonathan Balloch, et al.
Georgia Institute of Technology

A robust body of reinforcement learning techniques have been developed to solve complex sequential decision making problems. However, these methods assume that train and evaluation tasks come from similarly or identically distributed environments. This assumption does not hold in real life where small novel changes to the environment can make a previously learned policy fail or introduce simpler solutions that might never be found. To that end we explore the concept of novelty, defined in this work as the sudden change to the mechanics or properties of environment. We provide an ontology of for novelties most relevant to sequential decision making, which distinguishes between novelties that affect objects versus actions, unary properties versus non-unary relations, and the distribution of solutions to a task. We introduce NovGrid, a novelty generation framework built on MiniGrid, acting as a toolkit for rapidly developing and evaluating novelty-adaptation-enabled reinforcement learning techniques. Along with the core NovGrid we provide exemplar novelties aligned with our ontology and instantiate them as novelty templates that can be applied to many MiniGrid-compliant environments. Finally, we present a set of metrics built into our framework for the evaluation of novelty-adaptation-enabled machine-learning techniques, and show characteristics of a baseline RL model using these metrics.


Neuro-Symbolic World Models for Adapting to Open World Novelty

Open-world novelty–a sudden change in the mechanics or properties of an ...

Non-Deterministic Policies in Markovian Decision Processes

Markovian processes have long been used to model stochastic environments...

Characterizing Novelty in the Military Domain

A critical factor in utilizing agents with Artificial Intelligence (AI) ...

Effective Diversity in Unsupervised Environment Design

Agent decision making using Reinforcement Learning (RL) heavily relies o...

Ecological Reinforcement Learning

Much of the current work on reinforcement learning studies episodic sett...

Novelty Detection on a Mobile Robot Using Habituation

In this paper a novelty filter is introduced which allows a robot operat...

Empirically Evaluating Creative Arc Negotiation for Improvisational Decision-making

Action selection from many options with few constraints is crucial for i...