Value-of-Information based Arbitration between Model-based and Model-free Control

12/08/2019
by   Krishn Bera, et al.
0

There have been numerous attempts in explaining the general learning behaviours using model-based and model-free methods. While the model-based control is flexible yet computationally expensive in planning, the model-free control is quick but inflexible. The model-based control is therefore immune from reward devaluation and contingency degradation. Multiple arbitration schemes have been suggested to achieve the data efficiency and computational efficiency of model-based and model-free control respectively. In this context, we propose a quantitative 'value of information' based arbitration between both the controllers in order to establish a general computational framework for skill learning. The interacting model-based and model-free reinforcement learning processes are arbitrated using an uncertainty-based value of information. We further show that our algorithm performs better than Q-learning as well as Q-learning with experience replay.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/25/2018

Temporal Difference Models: Model-Free Deep RL for Model-Based Control

Model-free reinforcement learning (RL) is a powerful, general tool for l...
research
07/11/2020

Control as Hybrid Inference

The field of reinforcement learning can be split into model-based and mo...
research
01/31/2019

Successor Features Support Model-based and Model-free Reinforcement Learning

One key challenge in reinforcement learning is the ability to generalize...
research
03/15/2020

Robot Playing Kendama with Model-Based and Model-Free Reinforcement Learning

Several model-based and model-free methods have been proposed for the ro...
research
11/03/2020

Goal recognition via model-based and model-free techniques

Goal recognition aims at predicting human intentions from a trace of obs...
research
08/12/2017

Energy saving for building heating via a simple and efficient model-free control design: First steps with computer simulations

The model-based control of building heating systems for energy saving en...
research
02/25/2022

Behaviorally Grounded Model-Based and Model Free Cost Reduction in a Simulated Multi-Echelon Supply Chain

Amplification and phase shift in ordering signals, commonly referred to ...

Please sign up or login with your details

Forgot password? Click here to reset