RestoreX-AI: A Contrastive Approach towards Guiding Image Restoration via Explainable AI Systems

by   Aboli Marathe, et al.

Modern applications such as self-driving cars and drones rely heavily upon robust object detection techniques. However, weather corruptions can hinder the object detectability and pose a serious threat to their navigation and reliability. Thus, there is a need for efficient denoising, deraining, and restoration techniques. Generative adversarial networks and transformers have been widely adopted for image restoration. However, the training of these methods is often unstable and time-consuming. Furthermore, when used for object detection (OD), the output images generated by these methods may provide unsatisfactory results despite image clarity. In this work, we propose a contrastive approach towards mitigating this problem, by evaluating images generated by restoration models during and post training. This approach leverages OD scores combined with attention maps for predicting the usefulness of restored images for the OD task. We conduct experiments using two novel use-cases of conditional GANs and two transformer methods that probe the robustness of the proposed approach on multi-weather corruptions in the OD task. Our approach achieves an averaged 178 percent increase in mAP between the input and restored images under adverse weather conditions like dust tornadoes and snowfall. We report unique cases where greater denoising does not improve OD performance and conversely where noisy generated images demonstrate good results. We conclude the need for explainability frameworks to bridge the gap between human and machine perception, especially in the context of robust object detection for autonomous vehicles.


page 3

page 6

page 7

page 12

page 14

page 15

page 16


Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models

Image restoration under adverse weather conditions has been of significa...

Object Detection under Rainy Conditions for Autonomous Vehicles

Advanced automotive active-safety systems, in general, and autonomous ve...

Road scenes analysis in adverse weather conditions by polarization-encoded images and adapted deep learning

Object detection in road scenes is necessary to develop both autonomous ...

GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions

Image restoration in adverse weather conditions is a difficult task in c...

Channel Boosting Feature Ensemble for Radar-based Object Detection

Autonomous vehicles are conceived to provide safe and secure services by...

Contrastive View Design Strategies to Enhance Robustness to Domain Shifts in Downstream Object Detection

Contrastive learning has emerged as a competitive pretraining method for...

Why current rain denoising models fail on CycleGAN created rain images in autonomous driving

One of the main tasks of an autonomous agent in a vehicle is to correctl...

Please sign up or login with your details

Forgot password? Click here to reset