Learning Generalizable Behavior via Visual Rewrite Rules

12/09/2021
by   Yiheng Xie, et al.
8

Though deep reinforcement learning agents have achieved unprecedented success in recent years, their learned policies can be brittle, failing to generalize to even slight modifications of their environments or unfamiliar situations. The black-box nature of the neural network learning dynamics makes it impossible to audit trained deep agents and recover from such failures. In this paper, we propose a novel representation and learning approach to capture environment dynamics without using neural networks. It originates from the observation that, in games designed for people, the effect of an action can often be perceived in the form of local changes in consecutive visual observations. Our algorithm is designed to extract such vision-based changes and condense them into a set of action-dependent descriptive rules, which we call ”visual rewrite rules” (VRRs). We also present preliminary results from a VRR agent that can explore, expand its rule set, and solve a game via planning with its learned VRR world model. In several classical games, our non-deep agent demonstrates superior performance, extreme sample efficiency, and robust generalization ability compared with several mainstream deep agents.

READ FULL TEXT

page 2

page 4

page 6

page 7

research
02/04/2019

Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning

The rapid pace of research in Deep Reinforcement Learning has been drive...
research
09/07/2023

Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments

Understanding the interactions of agents trained with deep reinforcement...
research
07/05/2023

Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning

Text-based reinforcement learning agents have predominantly been neural ...
research
01/20/2018

Learning model-based strategies in simple environments with hierarchical q-networks

Recent advances in deep learning have allowed artificial agents to rival...
research
02/12/2022

Neural NID Rules

Abstract object properties and their relations are deeply rooted in huma...
research
04/16/2021

Evolving and Merging Hebbian Learning Rules: Increasing Generalization by Decreasing the Number of Rules

Generalization to out-of-distribution (OOD) circumstances after training...

Please sign up or login with your details

Forgot password? Click here to reset