PS-NeRV: Patch-wise Stylized Neural Representations for Videos

08/07/2022
by   Yunpeng Bai, et al.
0

We study how to represent a video with implicit neural representations (INRs). Classical INRs methods generally utilize MLPs to map input coordinates to output pixels. While some recent works have tried to directly reconstruct the whole image with CNNs. However, we argue that both the above pixel-wise and image-wise strategies are not favorable to video data. Instead, we propose a patch-wise solution, PS-NeRV, which represents videos as a function of patches and the corresponding patch coordinate. It naturally inherits the advantages of image-wise methods, and achieves excellent reconstruction performance with fast decoding speed. The whole method includes conventional modules, like positional embedding, MLPs and CNNs, while also introduces AdaIN to enhance intermediate features. These simple yet essential changes could help the network easily fit high-frequency details. Extensive experiments have demonstrated its effectiveness in several video-related tasks, such as video compression and video inpainting.

READ FULL TEXT

page 6

page 7

page 8

research
10/26/2021

NeRV: Neural Representations for Videos

We propose a novel neural representation for videos (NeRV) which encodes...
research
12/23/2022

FFNeRV: Flow-Guided Frame-Wise Neural Representations for Videos

Neural fields, also known as coordinate-based or implicit neural represe...
research
04/05/2023

HNeRV: A Hybrid Neural Representation for Videos

Implicit neural representations store videos as neural networks and have...
research
09/17/2022

MiNL: Micro-images based Neural Representation for Light Fields

Traditional representations for light fields can be separated into two t...
research
10/26/2022

SINCO: A Novel structural regularizer for image compression using implicit neural representations

Implicit neural representations (INR) have been recently proposed as dee...
research
11/09/2020

Detecting Outliers with Foreign Patch Interpolation

In medical imaging, outliers can contain hypo/hyper-intensities, minor d...
research
11/17/2019

Fast Color Constancy with Patch-wise Bright Pixels

In this paper, a learning-free color constancy algorithm called the Patc...

Please sign up or login with your details

Forgot password? Click here to reset