Dr.Bokeh: DiffeRentiable Occlusion-aware Bokeh Rendering

by   Yichen Sheng, et al.
Purdue University
Nanyang Technological University
berkeley college

Bokeh is widely used in photography to draw attention to the subject while effectively isolating distractions in the background. Computational methods simulate bokeh effects without relying on a physical camera lens. However, in the realm of digital bokeh synthesis, the two main challenges for bokeh synthesis are color bleeding and partial occlusion at object boundaries. Our primary goal is to overcome these two major challenges using physics principles that define bokeh formation. To achieve this, we propose a novel and accurate filtering-based bokeh rendering equation and a physically-based occlusion-aware bokeh renderer, dubbed Dr.Bokeh, which addresses the aforementioned challenges during the rendering stage without the need of post-processing or data-driven approaches. Our rendering algorithm first preprocesses the input RGBD to obtain a layered scene representation. Dr.Bokeh then takes the layered representation and user-defined lens parameters to render photo-realistic lens blur. By softening non-differentiable operations, we make Dr.Bokeh differentiable such that it can be plugged into a machine-learning framework. We perform quantitative and qualitative evaluations on synthetic and real-world images to validate the effectiveness of the rendering quality and the differentiability of our method. We show Dr.Bokeh not only outperforms state-of-the-art bokeh rendering algorithms in terms of photo-realism but also improves the depth quality from depth-from-defocus.


page 1

page 3

page 9

page 10

page 11

page 12


Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors

We consider the problem of realistic bokeh rendering from a single all-i...

MPIB: An MPI-Based Bokeh Rendering Framework for Realistic Partial Occlusion Effects

Partial occlusion effects are a phenomenon that blurry objects near a ca...

Dr.Jit: A Just-In-Time Compiler for Differentiable Rendering

Dr.Jit is a new just-in-time compiler for physically based rendering and...

Occlusion-Aware Video Deblurring with a New Layered Blur Model

We present a deblurring method for scenes with occluding objects using a...

Unsupervised Layered Image Decomposition into Object Prototypes

We present an unsupervised learning framework for decomposing images int...

Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization

Differentiable rendering (DR) enables various computer graphics and comp...

BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fields

Neural Radiance Fields (NeRF) have received considerable attention recen...

Please sign up or login with your details

Forgot password? Click here to reset