WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models

by   Aboli Marathe, et al.

The open road poses many challenges to autonomous perception, including poor visibility from extreme weather conditions. Models trained on good-weather datasets frequently fail at detection in these out-of-distribution settings. To aid adversarial robustness in perception, we introduce WEDGE (WEather images by DALL-E GEneration): a synthetic dataset generated with a vision-language generative model via prompting. WEDGE consists of 3360 images in 16 extreme weather conditions manually annotated with 16513 bounding boxes, supporting research in the tasks of weather classification and 2D object detection. We have analyzed WEDGE from research standpoints, verifying its effectiveness for extreme-weather autonomous perception. We establish baseline performance for classification and detection with 53.87 importantly, WEDGE can be used to fine-tune state-of-the-art detectors, improving SOTA performance on real-world weather benchmarks (such as DAWN) by 4.48 AP for well-generated classes like trucks. WEDGE has been collected under OpenAI's terms of use and is released for public use under the CC BY-NC-SA 4.0 license. The repository for this work and dataset is available at


page 1

page 3

page 5

page 8

page 9


aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception

Autonomous driving is a popular research area within the computer vision...

t-RAIN: Robust generalization under weather-aliasing label shift attacks

In the classical supervised learning settings, classifiers are fit with ...

Weather and Light Level Classification for Autonomous Driving: Dataset, Baseline and Active Learning

Autonomous driving is rapidly advancing, and Level 2 functions are becom...

Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditions

Advances in perception for self-driving cars have accelerated in recent ...

In Rain or Shine: Understanding and Overcoming Dataset Bias for Improving Robustness Against Weather Corruptions for Autonomous Vehicles

Several popular computer vision (CV) datasets, specifically employed for...

Why current rain denoising models fail on CycleGAN created rain images in autonomous driving

One of the main tasks of an autonomous agent in a vehicle is to correctl...

Vision-Based Autonomous Navigation for Unmanned Surface Vessel in Extreme Marine Conditions

Visual perception is an important component for autonomous navigation of...

Please sign up or login with your details

Forgot password? Click here to reset