Log In Sign Up

PixelRL: Fully Convolutional Network with Reinforcement Learning for Image Processing

by   Ryosuke Furuta, et al.

This paper tackles a new problem setting: reinforcement learning with pixel-wise rewards (pixelRL) for image processing. After the introduction of the deep Q-network, deep RL has been achieving great success. However, the applications of deep reinforcement learning (RL) for image processing are still limited. Therefore, we extend deep RL to pixelRL for various image processing applications. In pixelRL, each pixel has an agent, and the agent changes the pixel value by taking an action. We also propose an effective learning method for pixelRL that significantly improves the performance by considering not only the future states of the own pixel but also those of the neighbor pixels. The proposed method can be applied to some image processing tasks that require pixel-wise manipulations, where deep RL has never been applied. Besides, it is possible to visualize what kind of operation is employed for each pixel at each iteration, which would help us understand why and how such an operation is chosen. We also believe that our technology can enhance the explainability and interpretability of the deep neural networks. In addition, because the operations executed at each pixels are visualized, we can change or modify the operations if necessary. We apply the proposed method to a variety of image processing tasks: image denoising, image restoration, local color enhancement, and saliency-driven image editing. Our experimental results demonstrate that the proposed method achieves comparable or better performance, compared with the state-of-the-art methods based on supervised learning. The source code is available on


page 8

page 9

page 10

page 11

page 12

page 13

page 14

page 15


Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing

This paper tackles a new problem setting: reinforcement learning with pi...

Guided Collaborative Training for Pixel-wise Semi-Supervised Learning

We investigate the generalization of semi-supervised learning (SSL) to d...

Unpaired Image Enhancement Featuring Reinforcement-Learning-Controlled Image Editing Software

This paper tackles unpaired image enhancement, a task of learning a mapp...

Efficient Deep Image Denoising via Class Specific Convolution

Deep neural networks have been widely used in image denoising during the...

A Machine Learning Imaging Core using Separable FIR-IIR Filters

We propose fixed-function neural network hardware that is designed to pe...

Pixel VQ-VAEs for Improved Pixel Art Representation

Machine learning has had a great deal of success in image processing. Ho...

Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor

Pixel binning is considered one of the most prominent solutions to tackl...