Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture

by   Pawel Ladosz, et al.
University of California, Irvine
Loughborough University
HRL Laboratories, LLC
University of Warwick

This paper introduces the modulated Hebbian plus Q network architecture (MOHQA) for solving challenging partially observable Markov decision processes (POMDPs) deep reinforcement learning problems with sparse rewards and confounding observations. The proposed architecture combines a deep Q-network (DQN), and a modulated Hebbian network with neural eligibility traces (MOHN). Bio-inspired neural traces are used to bridge temporal delays between actions and rewards. The purpose is to discover distal cause-effect relationships where confounding observations and sparse rewards cause standard RL algorithms to fail. Each of the two modules of the network (DQN and MOHN) is responsible for different aspects of learning. DQN learns low level features and control, while MOHN contributes to the high-level decisions by bridging rewards with past actions. The strength of the approach is to support a DQN standard framework when temporal difference errors are difficult to compute due to non-observable states. The system is tested on a set of generalized decision making problems encoded as decision tree graphs that deliver delayed rewards after key decision points and confounding observations. The simulations show that the proposed approach helps solve problems that are currently challenging for state-of-the-art deep reinforcement learning algorithms.


Deep Hierarchical Reinforcement Learning Algorithm in Partially Observable Markov Decision Processes

In recent years, reinforcement learning has achieved many remarkable suc...

Deep Reinforcement Fuzzing

Fuzzing is the process of finding security vulnerabilities in input-proc...

Revisiting State Augmentation methods for Reinforcement Learning with Stochastic Delays

Several real-world scenarios, such as remote control and sensing, are co...

On Improving Deep Reinforcement Learning for POMDPs

Deep Reinforcement Learning (RL) recently emerged as one of the most com...

Introducing Neuromodulation in Deep Neural Networks to Learn Adaptive Behaviours

In this paper, we propose a new deep neural network architecture, called...

Deep Reinforcement Learning for Efficient Measurement of Quantum Devices

Deep reinforcement learning is an emerging machine learning approach whi...

Please sign up or login with your details

Forgot password? Click here to reset