Learning to select computations

11/18/2017
by   Falk Lieder, et al.
0

Efficient use of limited computational resources is essential to intelligence. Selecting computations optimally according to rational metareasoning would achieve this, but rational metareasoning is computationally intractable. Inspired by psychology and neuroscience, we propose the first learning algorithm for approximating the optimal selection of computations. We derive a general, sample-efficient reinforcement learning algorithm for learning to select computations from the insight that the value of computation lies between the myopic value of computation and the value of perfect information. We evaluate the performance of our method against two state-of-the-art methods for approximate metareasoning--the meta-greedy heuristic and the blinkered policy--on three increasingly difficult metareasoning problems: metareasoning about when to terminate computation, metareasoning about how to choose between multiple actions, and metareasoning about planning. Across all three domains, our method achieved near-optimal performance and significantly outperformed the meta-greedy heuristic. The blinkered policy performed on par with our method in metareasoning about decision-making, but it is not directly applicable to metareasoning about planning where our method outperformed both the meta-greedy heuristic and a generalization of the blinkered policy. Our results are a step towards building self-improving AI systems that can learn to make optimal use of their limited computational resources to efficiently solve complex problems in real-time.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/27/2010

Rational Value of Information Estimation for Measurement Selection

Computing value of information (VOI) is a crucial task in various aspect...
research
01/31/2021

Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning

To make good decisions in the real world people need efficient planning ...
research
09/10/2021

Automated Machine Learning, Bounded Rationality, and Rational Metareasoning

The notion of bounded rationality originated from the insight that perfe...
research
01/30/2013

Flexible and Approximate Computation through State-Space Reduction

In the real world, insufficient information, limited computation resourc...
research
03/01/2022

A Theory of Abstraction in Reinforcement Learning

Reinforcement learning defines the problem facing agents that learn to m...
research
07/22/2021

A reinforcement learning approach to resource allocation in genomic selection

Genomic selection (GS) is a technique that plant breeders use to select ...
research
01/03/2022

Have I done enough planning or should I plan more?

People's decisions about how to allocate their limited computational res...

Please sign up or login with your details

Forgot password? Click here to reset