Physics-Based Rendering for Improving Robustness to Rain

08/27/2019
by   Shirsendu Sukanta Halder, et al.
9

To improve the robustness to rain, we present a physically-based rain rendering pipeline for realistically inserting rain into clear weather images. Our rendering relies on a physical particle simulator, an estimation of the scene lighting and an accurate rain photometric modeling to augment images with arbitrary amount of realistic rain or fog. We validate our rendering with a user study, proving our rain is judged 40 state-of-the-art. Using our generated weather augmented Kitti and Cityscapes dataset, we conduct a thorough evaluation of deep object detection and semantic segmentation algorithms and show that their performance decreases in degraded weather, on the order of 15 segmentation. Furthermore, we show refining existing networks with our augmented images improves the robustness of both object detection and semantic segmentation algorithms. We experiment on nuScenes and measure an improvement of 15 original rainy performance. Augmented databases and code are available on the project page.

READ FULL TEXT

page 1

page 5

page 7

research
09/06/2020

Rain rendering for evaluating and improving robustness to bad weather

Rain fills the atmosphere with water particles, which breaks the common ...
research
08/11/2021

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

This work addresses the challenging task of LiDAR-based 3D object detect...
research
08/25/2017

Semantic Foggy Scene Understanding with Synthetic Data

This work addresses the problem of semantic foggy scene understanding (S...
research
06/16/2021

Robustness of Object Detectors in Degrading Weather Conditions

State-of-the-art object detection systems for autonomous driving achieve...
research
04/16/2019

Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds

Accurate detection of 3D objects is a fundamental problem in computer vi...
research
05/03/2023

ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering

Vision in adverse weather conditions, whether it be snow, rain, or fog i...
research
12/26/2017

Segmenting Sky Pixels in Images

Outdoor scene parsing models are often trained on ideal datasets and pro...

Please sign up or login with your details

Forgot password? Click here to reset