From internal models toward metacognitive AI

09/27/2021
by   Mitsuo Kawato, et al.
0

In several papers published in Biological Cybernetics in the 1980s and 1990s, Kawato and colleagues proposed computational models explaining how internal models are acquired in the cerebellum. These models were later supported by neurophysiological experiments using monkeys and neuroimaging experiments involving humans. These early studies influenced neuroscience from basic, sensory-motor control to higher cognitive functions. One of the most perplexing enigmas related to internal models is to understand the neural mechanisms that enable animals to learn large-dimensional problems with so few trials. Consciousness and metacognition – the ability to monitor one's own thoughts, may be part of the solution to this enigma. Based on literature reviews of the past 20 years, here we propose a computational neuroscience model of metacognition. The model comprises a modular hierarchical reinforcement-learning architecture of parallel and layered, generative-inverse model pairs. In the prefrontal cortex, a distributed executive network called the "cognitive reality monitoring network" (CRMN) orchestrates conscious involvement of generative-inverse model pairs in perception and action. Based on mismatches between computations by generative and inverse models, as well as reward prediction errors, CRMN computes a "responsibility signal" that gates selection and learning of pairs in perception, action, and reinforcement learning. A high responsibility signal is given to the pairs that best capture the external world, that are competent in movements (small mismatch), and that are capable of reinforcement learning (small reward prediction error). CRMN selects pairs with higher responsibility signals as objects of metacognition, and consciousness is determined by the entropy of responsibility signals across all pairs.

READ FULL TEXT
research
06/02/2018

Internal Model from Observations for Reward Shaping

Reinforcement learning methods require careful design involving a reward...
research
06/07/2021

A Computational Model of Representation Learning in the Brain Cortex, Integrating Unsupervised and Reinforcement Learning

A common view on the brain learning processes proposes that the three cl...
research
05/23/2017

Reinforcement Learning with a Corrupted Reward Channel

No real-world reward function is perfect. Sensory errors and software bu...
research
05/03/2021

Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference

Recent advances in reinforcement learning have inspired increasing inter...
research
04/22/2021

A learning gap between neuroscience and reinforcement learning

Historically, artificial intelligence has drawn much inspiration from ne...
research
11/01/2016

Detecting Affordances by Visuomotor Simulation

The term "affordance" denotes the behavioral meaning of objects. We prop...
research
08/16/2003

Controlled hierarchical filtering: Model of neocortical sensory processing

A model of sensory information processing is presented. The model assume...

Please sign up or login with your details

Forgot password? Click here to reset