Learning to Operate in Open Worlds by Adapting Planning Models

03/24/2023
by   Wiktor Piotrowski, et al.
0

Planning agents are ill-equipped to act in novel situations in which their domain model no longer accurately represents the world. We introduce an approach for such agents operating in open worlds that detects the presence of novelties and effectively adapts their domain models and consequent action selection. It uses observations of action execution and measures their divergence from what is expected, according to the environment model, to infer existence of a novelty. Then, it revises the model through a heuristics-guided search over model changes. We report empirical evaluations on the CartPole problem, a standard Reinforcement Learning (RL) benchmark. The results show that our approach can deal with a class of novelties very quickly and in an interpretable fashion.

READ FULL TEXT

page 1

page 2

page 3

research
06/09/2023

A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds

Model-based reasoning agents are ill-equipped to act in novel situations...
research
10/10/2011

Learning Symbolic Models of Stochastic Domains

In this article, we work towards the goal of developing agents that can ...
research
04/26/2018

Action Categorization for Computationally Improved Task Learning and Planning

This paper explores the problem of task learning and planning, contribut...
research
01/16/2023

Neuro-Symbolic World Models for Adapting to Open World Novelty

Open-world novelty–a sudden change in the mechanics or properties of an ...
research
06/24/2022

RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments

We propose RAPid-Learn: Learning to Recover and Plan Again, a hybrid pla...
research
05/26/2020

Efficient Use of heuristics for accelerating XCS-based Policy Learning in Markov Games

In Markov games, playing against non-stationary opponents with learning ...
research
04/15/2020

Bootstrapped model learning and error correction for planning with uncertainty in model-based RL

Having access to a forward model enables the use of planning algorithms ...

Please sign up or login with your details

Forgot password? Click here to reset