Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning

by   Pedro A. Tsividis, et al.

Reinforcement learning (RL) studies how an agent comes to achieve reward in an environment through interactions over time. Recent advances in machine RL have surpassed human expertise at the world's oldest board games and many classic video games, but they require vast quantities of experience to learn successfully – none of today's algorithms account for the human ability to learn so many different tasks, so quickly. Here we propose a new approach to this challenge based on a particularly strong form of model-based RL which we call Theory-Based Reinforcement Learning, because it uses human-like intuitive theories – rich, abstract, causal models of physical objects, intentional agents, and their interactions – to explore and model an environment, and plan effectively to achieve task goals. We instantiate the approach in a video game playing agent called EMPA (the Exploring, Modeling, and Planning Agent), which performs Bayesian inference to learn probabilistic generative models expressed as programs for a game-engine simulator, and runs internal simulations over these models to support efficient object-based, relational exploration and heuristic planning. EMPA closely matches human learning efficiency on a suite of 90 challenging Atari-style video games, learning new games in just minutes of game play and generalizing robustly to new game situations and new levels. The model also captures fine-grained structure in people's exploration trajectories and learning dynamics. Its design and behavior suggest a way forward for building more general human-like AI systems.


page 6

page 16


Beating the World's Best at Super Smash Bros. with Deep Reinforcement Learning

There has been a recent explosion in the capabilities of game-playing ar...

Strategic Object Oriented Reinforcement Learning

Humans learn to play video games significantly faster than state-of-the-...

Hierarchial Reinforcement Learning in StarCraft II with Human Expertise in Subgoals Selection

This work is inspired by recent advances in hierarchical reinforcement l...

Learning model-based strategies in simple environments with hierarchical q-networks

Recent advances in deep learning have allowed artificial agents to rival...

Physically Embedded Planning Problems: New Challenges for Reinforcement Learning

Recent work in deep reinforcement learning (RL) has produced algorithms ...

Neural Game Engine: Accurate learning ofgeneralizable forward models from pixels

Access to a fast and easily copied forward model of a game is essential ...

Neural Game Engine: Accurate learning of generalizable forward models from pixels

Access to a fast and easily copied forward model of a game is essential ...

Please sign up or login with your details

Forgot password? Click here to reset