PIP: Positional-encoding Image Prior

11/25/2022
by   Nimrod Shabtay, et al.
0

In Deep Image Prior (DIP), a Convolutional Neural Network (CNN) is fitted to map a latent space to a degraded (e.g. noisy) image but in the process learns to reconstruct the clean image. This phenomenon is attributed to CNN's internal image-prior. We revisit the DIP framework, examining it from the perspective of a neural implicit representation. Motivated by this perspective, we replace the random or learned latent with Fourier-Features (Positional Encoding). We show that thanks to the Fourier features properties, we can replace the convolution layers with simple pixel-level MLPs. We name this scheme “Positional Encoding Image Prior" (PIP) and exhibit that it performs very similarly to DIP on various image-reconstruction tasks with much less parameters required. Additionally, we demonstrate that PIP can be easily extended to videos, where 3D-DIP struggles and suffers from instability. Code and additional examples for all tasks, including videos, are available on the project page https://nimrodshabtay.github.io/PIP/

READ FULL TEXT

page 1

page 5

page 7

page 8

page 9

page 12

research
08/08/2020

Using PSPNet and UNet to analyze the internal parameter relationship and visualization of the convolutional neural network

Convolutional neural network(CNN) has achieved great success in many fie...
research
01/11/2019

FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction

The basic principles in designing convolutional neural network (CNN) str...
research
08/24/2021

NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction

Image reconstruction is an inverse problem that solves for a computation...
research
07/06/2020

Joint Frequency- and Image-Space Learning for Fourier Imaging

We propose a neural network layer structure that combines frequency and ...
research
08/21/2023

Pixel Adaptive Deep Unfolding Transformer for Hyperspectral Image Reconstruction

Hyperspectral Image (HSI) reconstruction has made gratifying progress wi...
research
03/09/2018

Learning a Discriminative Prior for Blind Image Deblurring

We present an effective blind image deblurring method based on a data-dr...

Please sign up or login with your details

Forgot password? Click here to reset