DeepAI AI Chat
Log In Sign Up

CT-DQN: Control-Tutored Deep Reinforcement Learning

by   Francesco De Lellis, et al.
University of Salerno
Pavol Jozef Safarik University
University of Naples Federico II

One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approximate model of the system, without any assumption about the knowledge of the system's dynamics. There is no expectation that it will be able to achieve the control objective if used stand-alone. During learning, the tutor occasionally suggests an action, thus partially guiding exploration. We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing. We demonstrate that CT-DQN is able to achieve better or equivalent data efficiency with respect to the classic function approximation solutions.


page 1

page 2

page 3

page 4


Off-Policy Deep Reinforcement Learning without Exploration

Reinforcement learning traditionally considers the task of balancing exp...

The Difficulty of Passive Learning in Deep Reinforcement Learning

Learning to act from observational data without active environmental int...

Contention Window Optimization in IEEE 802.11ax Networks with Deep Reinforcement Learning

The proper setting of contention window (CW) values has a significant im...

Intelligent Parameter Tuning in Optimization-based Iterative CT Reconstruction via Deep Reinforcement Learning

A number of image-processing problems can be formulated as optimization ...

Smart Magnetic Microrobots Learn to Swim with Deep Reinforcement Learning

Swimming microrobots are increasingly developed with complex materials a...

Computational Flight Control: A Domain-Knowledge-Aided Deep Reinforcement Learning Approach

This papers aims to examine the potential of using the emerging deep rei...

Contextual Exploration Using a Linear Approximation Method Based on Satisficing

Deep reinforcement learning has enabled human-level or even super-human ...