Neural Rerendering in the Wild

04/08/2019
by   Moustafa Meshry, et al.
0

We explore total scene capture -- recording, modeling, and rerendering a scene under varying appearance such as season and time of day. Starting from internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud. For each photo, we render the scene points into a deep framebuffer, and train a neural network to learn the mapping of these initial renderings to the actual photos. This rerendering network also takes as input a latent appearance vector and a semantic mask indicating the location of transient objects like pedestrians. The model is evaluated on several datasets of publicly available images spanning a broad range of illumination conditions. We create short videos demonstrating realistic manipulation of the image viewpoint, appearance, and semantic labeling. We also compare results with prior work on scene reconstruction from internet photos.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 5

page 6

page 7

page 8

page 9

page 11

page 12

page 13

09/13/2018

Head Reconstruction from Internet Photos

3D face reconstruction from Internet photos has recently produced exciti...
11/14/2019

CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator

Instance based photo cartoonization is one of the challenging image styl...
11/10/2015

3D Time-lapse Reconstruction from Internet Photos

Given an Internet photo collection of a landmark, we compute a 3D time-l...
08/22/2018

Manipulating Attributes of Natural Scenes via Hallucination

In this study, we explore building a two-stage framework for enabling us...
06/09/2021

Plan2Scene: Converting Floorplans to 3D Scenes

We address the task of converting a floorplan and a set of associated ph...
07/30/2020

Crowdsampling the Plenoptic Function

Many popular tourist landmarks are captured in a multitude of online, pu...
09/14/2019

Selfie Drone Stick: A Natural Interface for Quadcopter Photography

A physical selfie stick extends the user's reach, enabling the creation ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.