Rendering Synthetic Objects into Legacy Photographs

12/24/2019
by   Kevin Karsch, et al.
0

We propose a method to realistically insert synthetic objects into existing photographs without requiring access to the scene or any additional scene measurements. With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene. We demonstrate in a user study that synthetic images produced by our method are confusable with real scenes, even for people who believe they are good at telling the difference. Further, our study shows that our method is competitive with other insertion methods while requiring less scene information. We also collected new illumination and reflectance datasets; renderings produced by our system compare well to ground truth. Our system has applications in the movie and gaming industry, as well as home decorating and user content creation, among others.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 9

page 11

research
07/06/2023

PSDR-Room: Single Photo to Scene using Differentiable Rendering

A 3D digital scene contains many components: lights, materials and geome...
research
12/08/2021

SIRfyN: Single Image Relighting from your Neighbors

We show how to relight a scene, depicted in a single image, such that (a...
research
03/18/2020

Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination

We present a deep learning solution for estimating the incident illumina...
research
03/15/2022

Active Exploration for Neural Global Illumination of Variable Scenes

Neural rendering algorithms introduce a fundamentally new approach for p...
research
10/12/2020

Cut-and-Paste Neural Rendering

Cut-and-paste methods take an object from one image and insert it into a...
research
11/04/2020

Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding

For many fundamental scene understanding tasks, it is difficult or impos...
research
11/27/2022

3D Scene Creation and Rendering via Rough Meshes: A Lighting Transfer Avenue

This paper studies how to flexibly integrate reconstructed 3D models int...

Please sign up or login with your details

Forgot password? Click here to reset