TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions

Removing adverse weather conditions like rain, fog, and snow from images is an important problem in many applications. Most methods proposed in the literature have been designed to deal with just removing one type of degradation. Recently, a CNN-based method using neural architecture search (All-in-One) was proposed to remove all the weather conditions at once. However, it has a large number of parameters as it uses multiple encoders to cater to each weather removal task and still has scope for improvement in its performance. In this work, we focus on developing an efficient solution for the all adverse weather removal problem. To this end, we propose TransWeather, a transformer-based end-to-end model with just a single encoder and a decoder that can restore an image degraded by any weather condition. Specifically, we utilize a novel transformer encoder using intra-patch transformer blocks to enhance attention inside the patches to effectively remove smaller weather degradations. We also introduce a transformer decoder with learnable weather type embeddings to adjust to the weather degradation at hand. TransWeather achieves significant improvements across multiple test datasets over both All-in-One network as well as methods fine-tuned for specific tasks. In particular, TransWeather pushes the current state-of-the-art by +6.34 PSNR on the Test1 (rain+fog) dataset, +4.93 PSNR on the SnowTest100K-L dataset and +3.11 PSNR on the RainDrop test dataset. TransWeather is also validated on real world test images and found to be more effective than previous methods. Implementation code and pre-trained weights can be accessed at https://github.com/jeya-maria-jose/TransWeather .

READ FULL TEXT

page 1

page 4

page 7

page 8

page 9

research
08/28/2023

MetaWeather: Few-Shot Weather-Degraded Image Restoration via Degradation Pattern Matching

Real-world vision tasks frequently suffer from the appearance of adverse...
research
05/29/2023

GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions

Image restoration in adverse weather conditions is a difficult task in c...
research
04/19/2022

Towards Efficient Single Image Dehazing and Desnowing

Removing adverse weather conditions like rain, fog, and snow from images...
research
06/15/2023

Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal

Image restoration under adverse weather conditions (e.g., rain, snow and...
research
07/29/2022

Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models

Image restoration under adverse weather conditions has been of significa...
research
03/24/2023

MoWE: Mixture of Weather Experts for Multiple Adverse Weather Removal

Currently, most adverse weather removal tasks are handled independently,...
research
05/17/2023

Restoring Images Captured in Arbitrary Hybrid Adverse Weather Conditions in One Go

Adverse conditions typically suffer from stochastic hybrid weather degra...

Please sign up or login with your details

Forgot password? Click here to reset