Representational efficiency outweighs action efficiency in human program induction

07/18/2018
by   Sophia Sanborn, et al.
0

The importance of hierarchically structured representations for tractable planning has long been acknowledged. However, the questions of how people discover such abstractions and how to define a set of optimal abstractions remain open. This problem has been explored in cognitive science in the problem solving literature and in computer science in hierarchical reinforcement learning. Here, we emphasize an algorithmic perspective on learning hierarchical representations in which the objective is to efficiently encode the structure of the problem, or, equivalently, to learn an algorithm with minimal length. We introduce a novel problem-solving paradigm that links problem solving and program induction under the Markov Decision Process (MDP) framework. Using this task, we target the question of whether humans discover hierarchical solutions by maximizing efficiency in number of actions they generate or by minimizing the complexity of the resulting representation and find evidence for the primacy of representational efficiency.

READ FULL TEXT
research
10/23/2021

Map Induction: Compositional spatial submap learning for efficient exploration in novel environments

Humans are expert explorers. Understanding the computational cognitive m...
research
05/28/2019

Planning with State Abstractions for Non-Markovian Task Specifications

Often times, we specify tasks for a robot using temporal language that c...
research
07/29/2019

Action Grammars: A Cognitive Model for Learning Temporal Abstractions

Hierarchical Reinforcement Learning algorithms have successfully been ap...
research
03/10/2016

Hierarchical Linearly-Solvable Markov Decision Problems

We present a hierarchical reinforcement learning framework that formulat...
research
06/08/2021

Learning Markov State Abstractions for Deep Reinforcement Learning

The fundamental assumption of reinforcement learning in Markov decision ...
research
02/02/2022

Using Deep Learning to Bootstrap Abstractions for Hierarchical Robot Planning

This paper addresses the problem of learning abstractions that boost rob...
research
07/07/2020

Strong Generalization and Efficiency in Neural Programs

We study the problem of learning efficient algorithms that strongly gene...

Please sign up or login with your details

Forgot password? Click here to reset