Thinking While Moving: Deep Reinforcement Learning with Concurrent Control

by   Ted Xiao, et al.
The Team at X

We study reinforcement learning in settings where sampling an action from the policy must be done concurrently with the time evolution of the controlled system, such as when a robot must decide on the next action while still performing the previous action. Much like a person or an animal, the robot must think and move at the same time, deciding on its next action before the previous one has completed. In order to develop an algorithmic framework for such concurrent control problems, we start with a continuous-time formulation of the Bellman equations, and then discretize them in a way that is aware of system delays. We instantiate this new class of approximate dynamic programming methods via a simple architectural extension to existing value-based deep reinforcement learning algorithms. We evaluate our methods on simulated benchmark tasks and a large-scale robotic grasping task where the robot must "think while moving".


page 7

page 17


Using Deep Reinforcement Learning for the Continuous Control of Robotic Arms

Deep reinforcement learning enables algorithms to learn complex behavior...

A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform

With the development of industry, drones are appearing in various field....

Performing Deep Recurrent Double Q-Learning for Atari Games

Currently, many applications in Machine Learning are based on define new...

Characterising the Robustness of Reinforcement Learning for Continuous Control using Disturbance Injection

In this study, we leverage the deliberate and systematic fault-injection...

Coinbot: Intelligent Robotic Coin Bag Manipulation Using Deep Reinforcement Learning And Machine Teaching

Given the laborious difficulty of moving heavy bags of physical currency...

Please sign up or login with your details

Forgot password? Click here to reset