PoPS: Policy Pruning and Shrinking for Deep Reinforcement Learning

by   Dor Livne, et al.

The recent success of deep neural networks (DNNs) for function approximation in reinforcement learning has triggered the development of Deep Reinforcement Learning (DRL) algorithms in various fields, such as robotics, computer games, natural language processing, computer vision, sensing systems, and wireless networking. Unfortunately, DNNs suffer from high computational cost and memory consumption, which limits the use of DRL algorithms in systems with limited hardware resources. In recent years, pruning algorithms have demonstrated considerable success in reducing the redundancy of DNNs in classification tasks. However, existing algorithms suffer from a significant performance reduction in the DRL domain. In this paper, we develop the first effective solution to the performance reduction problem of pruning in the DRL domain, and establish a working algorithm, named Policy Pruning and Shrinking (PoPS), to train DRL models with strong performance while achieving a compact representation of the DNN. The framework is based on a novel iterative policy pruning and shrinking method that leverages the power of transfer learning when training the DRL model. We present an extensive experimental study that demonstrates the strong performance of PoPS using the popular Cartpole, Lunar Lander, Pong, and Pacman environments. Finally, we develop an open source software for the benefit of researchers and developers in related fields.


page 9

page 11


The State of Sparse Training in Deep Reinforcement Learning

The use of sparse neural networks has seen rapid growth in recent years,...

Deep Reinforcement Learning for Clinical Decision Support: A Brief Survey

Owe to the recent advancements in Artificial Intelligence especially dee...

Deep Reinforcement One-Shot Learning for Artificially Intelligent Classification Systems

In recent years there has been a sharp rise in networking applications, ...

GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning

Deep reinforcement learning (DRL) has shown remarkable success in sequen...

Deep Reinforcement Learning with Decorrelation

Learning an effective representation for high-dimensional data is a chal...

BBReach: Tight and Scalable Black-Box Reachability Analysis of Deep Reinforcement Learning Systems

Reachability analysis is a promising technique to automatically prove or...

Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees

In an ever expanding set of research and application areas, deep neural ...

Please sign up or login with your details

Forgot password? Click here to reset