Old Photo Restoration via Deep Latent Space Translation

09/14/2020
by   Ziyu Wan, et al.
55

We propose to restore old photos that suffer from severe degradation through a deep learning approach. Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize. Therefore, we propose a novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs. Specifically, we train two variational autoencoders (VAEs) to respectively transform old photos and clean photos into two latent spaces. And the translation between these two latent spaces is learned with synthetic paired data. This translation generalizes well to real photos because the domain gap is closed in the compact latent space. Besides, to address multiple degradations mixed in one old photo, we design a global branch with apartial nonlocal block targeting to the structured defects, such as scratches and dust spots, and a local branch targeting to the unstructured defects, such as noises and blurriness. Two branches are fused in the latent space, leading to improved capability to restore old photos from multiple defects. Furthermore, we apply another face refinement network to recover fine details of faces in the old photos, thus ultimately generating photos with enhanced perceptual quality. With comprehensive experiments, the proposed pipeline demonstrates superior performance over state-of-the-art methods as well as existing commercial tools in terms of visual quality for old photos restoration.

READ FULL TEXT

Authors

page 2

page 8

page 10

page 11

page 12

page 13

04/20/2020

Bringing Old Photos Back to Life

We propose to restore old photos that suffer from severe degradation thr...
12/22/2020

Time-Travel Rephotography

Many historical people are captured only in old, faded, black and white ...
05/04/2022

Pik-Fix: Restoring and Colorizing Old Photos

Restoring and inpainting the visual memories that are present, but often...
02/11/2021

Deep Photo Scan: Semi-supervised learning for dealing with the real-world degradation in smartphone photo scanning

Physical photographs now can be conveniently scanned by smartphones and ...
02/05/2022

ROMNet: Renovate the Old Memories

Renovating the memories in old photos is an intriguing research topic in...
03/31/2022

Bringing Old Films Back to Life

We present a learning-based framework, recurrent transformer network (RT...

Code Repositories

Bringing-Old-Photos-Back-to-Life

Bringing Old Photo Back to Life (CVPR 2020 oral)


view repo

Image-Restoration

Image Restoration using ML


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.