3D Scene Creation and Rendering via Rough Meshes: A Lighting Transfer Avenue

by   Yujie Li, et al.

This paper studies how to flexibly integrate reconstructed 3D models into practical 3D modeling pipelines such as 3D scene creation and rendering. Due to the technical difficulty, one can only obtain rough 3D models (R3DMs) for most real objects using existing 3D reconstruction techniques. As a result, physically-based rendering (PBR) would render low-quality images or videos for scenes that are constructed by R3DMs. One promising solution would be representing real-world objects as Neural Fields such as NeRFs, which are able to generate photo-realistic renderings of an object under desired viewpoints. However, a drawback is that the synthesized views through Neural Fields Rendering (NFR) cannot reflect the simulated lighting details on R3DMs in PBR pipelines, especially when object interactions in the 3D scene creation cause local shadows. To solve this dilemma, we propose a lighting transfer network (LighTNet) to bridge NFR and PBR, such that they can benefit from each other. LighTNet reasons about a simplified image composition model, remedies the uneven surface issue caused by R3DMs, and is empowered by several perceptual-motivated constraints and a new Lab angle loss which enhances the contrast between lighting strength and colors. Comparisons demonstrate that LighTNet is superior in synthesizing impressive lighting, and is promising in pushing NFR further in practical 3D modeling workflows. Project page: https://3d-front-future.github.io/LighTNet .


page 2

page 6

page 7

page 8

page 14

page 16

page 17

page 18


ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting

Recent advances in neural rendering have shown great potential for recon...

LANe: Lighting-Aware Neural Fields for Compositional Scene Synthesis

Neural fields have recently enjoyed great success in representing and re...

IBL-NeRF: Image-Based Lighting Formulation of Neural Radiance Fields

We propose IBL-NeRF, which decomposes the neural radiance fields (NeRF) ...

Photorealistic Image Synthesis for Object Instance Detection

We present an approach to synthesize highly photorealistic images of 3D ...

Rendering Synthetic Objects into Legacy Photographs

We propose a method to realistically insert synthetic objects into exist...

RobustNeRF: Ignoring Distractors with Robust Losses

Neural radiance fields (NeRF) excel at synthesizing new views given mult...

Non-Linear Phase-Shifting of Haar Wavelets for Run-Time All-Frequency Lighting

This paper focuses on real-time all-frequency image-based rendering usin...

Please sign up or login with your details

Forgot password? Click here to reset