Adaptive Mask-based Pyramid Network for Realistic Bokeh Rendering

by   Konstantinos Georgiadis, et al.

Bokeh effect highlights an object (or any part of the image) while blurring the rest of the image, and creates a visually pleasant artistic effect. Due to the sensor-based limitations on mobile devices, machine learning (ML) based bokeh rendering has gained attention as a reliable alternative. In this paper, we focus on several improvements in ML-based bokeh rendering; i) on-device performance with high-resolution images, ii) ability to guide bokeh generation with user-editable masks and iii) ability to produce varying blur strength. To this end, we propose Adaptive Mask-based Pyramid Network (AMPN), which is formed of a Mask-Guided Bokeh Generator (MGBG) block and a Laplacian Pyramid Refinement (LPR) block. MGBG consists of two lightweight networks stacked to each other to generate the bokeh effect, and LPR refines and upsamples the output of MGBG to produce the high-resolution bokeh image. We achieve i) via our lightweight, mobile-friendly design choices, ii) via the stacked-network design of MGBG and the weakly-supervised mask prediction scheme and iii) via manually or automatically editing the intensity values of the mask that guide the bokeh generation. In addition to these features, our results show that AMPN produces competitive or better results compared to existing methods on the EBB! dataset, while being faster and smaller than the alternatives.


page 3

page 11

page 12

page 13


Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors

We consider the problem of realistic bokeh rendering from a single all-i...

SEMI-PointRend: Improved Semiconductor Wafer Defect Classification and Segmentation as Rendering

In this study, we applied the PointRend (Point-based Rendering) method t...

Mask Embedding in conditional GAN for Guided Synthesis of High Resolution Images

Recent advancements in conditional Generative Adversarial Networks (cGAN...

Inductive Guided Filter: Real-time Deep Image Matting with Weakly Annotated Masks on Mobile Devices

Recently, significant progress has been achieved in deep image matting. ...

Structure First Detail Next: Image Inpainting with Pyramid Generator

Recent deep generative models have achieved promising performance in ima...

Sky Optimization: Semantically aware image processing of skies in low-light photography

The sky is a major component of the appearance of a photograph, and its ...

Automatic Image Blending Algorithm Based on SAM and DINO

The field of image blending has gained popularity in recent years for it...

Please sign up or login with your details

Forgot password? Click here to reset