Searching for Controllable Image Restoration Networks

by   Heewon Kim, et al.

Diverse user preferences over images have recently led to a great amount of interest in controlling the imagery effects for image restoration tasks. However, existing methods require separate inference through the entire network per each output, which hinders users from readily comparing multiple imagery effects due to long latency. To this end, we propose a novel framework based on a neural architecture search technique that enables efficient generation of multiple imagery effects via two stages of pruning: task-agnostic and task-specific pruning. Specifically, task-specific pruning learns to adaptively remove the irrelevant network parameters for each task, while task-agnostic pruning learns to find an efficient architecture by sharing the early layers of the network across different tasks. Since the shared layers allow for feature reuse, only a single inference of the task-agnostic layers is needed to generate multiple imagery effects from the input image. Using the proposed task-agnostic and task-specific pruning schemes together significantly reduces the FLOPs and the actual latency of inference compared to the baseline. We reduce 95.7 latency 73.0


page 7

page 11

page 13

page 14

page 15

page 16

page 17

page 18


Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Search

We propose three novel pruning techniques to improve the cost and result...

TAPE: Task-Agnostic Prior Embedding for Image Restoration

Learning an generalized prior for natural image restoration is an import...

MIME: Adapting a Single Neural Network for Multi-task Inference with Memory-efficient Dynamic Pruning

Recent years have seen a paradigm shift towards multi-task learning. Thi...

Parameter-Efficient Transfer Learning with Diff Pruning

While task-specific finetuning of pretrained networks has led to signifi...

CFSNet: Toward a Controllable Feature Space for Image Restoration

Deep learning methods have witnessed the great progress in image restora...

Multi-Dimension Modulation for Image Restoration with Dynamic Controllable Residual Learning

Based on the great success of deterministic learning, to interactively c...

Deep Network Interpolation for Continuous Imagery Effect Transition

Deep convolutional neural network has demonstrated its capability of lea...