NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results

1 Introduction

Image denoising is a fundamental and active research area (e.g., [tai2017memnet, zhang2017beyond, zhang2018ffdnet, gu2019brief]

) with a long-standing history in computer vision (

e.g., [kuan1985adaptive, liu2008automatic]). A primary goal of image denoising is to remove or correct for noise in an image, either for aesthetic purposes, or to help improve other downstream tasks. For many years, researchers have primarily relied on synthetic noisy image for developing and evaluating image denoisers, especially the additive white Gaussian noise (AWGN)—e.g., [BuadesCVPR05, Dabov07imagedenoising, zhang2017beyond]. Recently, more focus has been given to evaluating image denoisers on real noisy images [abdelhamed2018high, plotz2017benchmarking, abdelhamed2019ntire]. To this end, we have proposed this challenge as a means to evaluate and benchmark image denoisers on real noisy images.

This challenge is a new version of the Smartphone Image Denoising Dataset (SIDD) benchmark [abdelhamed2018high] with a newly collected validation and testing datasets, hence named SIDD+

. The original SIDD consisted of thousands of real noisy images with estimated ground-truth, in both raw sensor data (rawRGB) and standard RGB (sRGB) color spaces. Hence, in this challenge, we provide two tracks for benchmarking image denoisers in both rawRGB and sRGB color spaces. We present more details on both tracks in the next section.

2 The Challenge

This challenge is one of the NTIRE 2020 associated challenges on: deblurring [nah2020ntire], nonhomogeneous dehazing [ancuti2020ntire]

, perceptual extreme super-resolution

[zhang2020ntire], video quality mapping [fuoli2020ntire], real image denoising [abdelhamed2020ntire], real-world super-resolution [lugmayr2020ntire], spectral reconstruction from RGB image [arad2020ntire] and demoireing [yuan2020demoireing].

The NTIRE 2020 Real Image Denoising Challenge is an extension of the previous NTIRE 2019 challenge [abdelhamed2019ntire]. Both challenges aimed to gauge and advance the state-of-the-art in image denoising. The focus of the challenge is on evaluating image denoisers on real, rather than synthetic, noisy images. In the following, we present some details about the new dataset used in this version of the challenge and how the challenge is designed.

2.1 Dataset

The SIDD dataset [abdelhamed2018high] was used for providing training images for the challenge. The SIDD dataset consists of thousands of real noisy images and their corresponding ground truth, from ten different scenes, captured repeatedly with five different smartphone cameras under different lighting conditions and ISO levels. The ISO levels ranged from 50 to 10,000. The images are provided in both rawRGB and sRGB color spaces.

For validation and testing, we collected a new dataset of 2048 images following a similar procedure to the one used in generating the SIDD validation and testing datasets.

2.2 Challenge Design and Tracks

Tracks

We provide two tracks to benchmark the proposed image denoisers based on the two different color spaces: the rawRGB and the sRGB. Images in the rawRGB format represent minimally processed images obtained directly from the camera’s sensor. These images are in a sensor-dependent color space where the R, G, and B values are related to the sensor’s color filter array’s spectral sensitivity to incoming visible light. Images in the sRGB format represent the camera’s rawRGB image that have been processed by the in-camera image processing pipeline to map the sensor-dependent RGB colors to a device-independent color space, namely standard RGB (sRGB). Different camera models apply their own proprietary photo-finishing routines, including several nonlinear color manipulations, to modify the rawRGB values to appear visually appealing (see [Hakki2016] for more details). We note that the provided sRGB images are not compressed and therefore do not exhibit compression artifacts. Denoising a rawRGB would typically represent a denoising module applied within the in-camera image processing pipeline. Denoising an sRGB image would represent a denoising module applied after the in-camera color manipulation. As found in recent works [abdelhamed2019ntire, abdelhamed2018high, plotz2017benchmarking], image denoisers tend to perform better in the rawRGB color space than in the sRGB color space. However, rawRGB images are far less common than sRGB images which are easily saved in common formats, such as JPEG and PNG. Since the SIDD dataset contains both rawRGB and sRGB versions of the same image, we found it feasible to provide a separate track for denoising in each color space. Both tracks follow similar data preparation, evaluation, and competition timeline, as discussed next.

Data preparation

The provided training data was the SIDD-Medium dataset that consists of 320 noisy images in both rawRGB and sRGB space with corresponding ground truth and metadata. Each noisy or ground truth image is a 2D array of normalized rawRGB values (mosaiced color filter array) in the range $[0, 1]$ in single-precision floating point format saved as Matlab .mat files. The metadata files contained dictionaries of Tiff tags for the rawRGB images, saved as .mat files.

We collected a new validation and testing datasets following a similar procedure to the one used in SIDD [abdelhamed2018high], and hence, we named the new dataset SIDD+.

The SIDD+ validation set consists of $1024$ noisy image blocks (i.e., croppings) form both rawRGB and sRGB images, each block is $256 \times 256$ pixels. The blocks are taken from $32$ images, $32$ blocks from each image ( $32 \times 32 = 1024$ ). All image blocks are combined in a single 4D array of shape $[1024, 256, 256]$ where each consecutive 32 images belong to the same image, for example, the first 32 images belong to the first image, and so on. The blocks have the same number format as the training data. Similarly, the SIDD+ testing set consists of $1024$ noisy image blocks from a different set of images, but following the same format as the validation set. Image metadata files were also provided for all $64$ images from which the validation and testing data were extracted. All newly created validation and testing datasets are publicly available.

We also provided the simulated camera pipeline used to render rawRGB images into sRGB for the SIDD dataset ¹¹1https://github.com/AbdoKamel/simple-camera-pipeline. The provided pipeline offers a set of processing stages similar to an on-board camera pipeline. Such stages include: black level subtraction, active area cropping, white balance, color space transformation, and global tone mapping.

Evaluation

The evaluation is based on the comparison of the restored clean (denoised) images with the ground-truth images. For this we use the standard peak signal-to-noise ratio (PSNR) and, complementary, the structural similarity (SSIM) index

[wang2004image] as often employed in the literature. Implementations are found in most of the image processing toolboxes. We report the average results over all image blocks provided.

For submitting the results, participants were asked to provide the denoised image blocks in a multidimensional array shaped in the same way as the input data (i.e., $[1024, 256, 256]$ ). In addition, participants were asked to provide additional information: the algorithm’s runtime per mega pixel (in seconds); whether the algorithm employs CPU or GPU at runtime; and whether extra metadata is used as inputs to the algorithm.

At the final stage of the challenge, the participants were asked to submit fact sheets to provide information about the teams and to describe their methods.

Timeline

The challenge timeline was performed in two stages. The validation stage started on December 20, 2019. The final testing stage started on March 16, 2020. Each participant was allowed a maximum of 20 and 3 submissions during the validation and testing phases, respectively. The challenge ended on March 26, 2020.

3 Challenge Results

From approximately $250$ registered participants in each track, $22$ teams entered in the final phase and submitted results, codes/executables, and factsheets. Tables 1 and 2 report the final test results, in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) index [wang2004image], for the rawRGB and sRGB tracks, respectively. The tables show the method ranks based on each measure in subscripts. We present the self-reported runtimes and major details provided in the factsheets submitted by participants. Figures 1 and 2 show a 2D visualization of PSNR and SSIM values for all methods in both rawRGB and sRGB tracks, respectively. For combined visualization, both figures are overlaid in Figure 3. The methods are briefly described in section 4 and team members are listed in Appendix A.

Figure 1: Combined PSNR and SSIM values of method from the rawRGB track.

Figure 2: Combined PSNR and SSIM values of method from the sRGB track.

Team	Username	PSNR	SSIM	Runtime (s/Mpixel)	CPU/GPU (at runtime)	Platform	Ensemble	Loss
Baidu Research Vision 1	zhihongp	${57.44}_{(1)}$	${0.99789}_{(2)}$	$5.76$	Tesla V100	PaddlePaddle, PyTorch	flip/transpose ( $\times 8$ )	$L_{1}$
HITVPC&HUAWEI 1	hitvpc_huawei	${57.43}_{(2)}$	${0.99788}_{(3)}$	$?$	GTX 1080 Ti	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$
Eraser 1	Songsaris	${57.33}_{(3)}$	${0.99788}_{(5)}$	$36.50$	TITAN V	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$
Samsung_SLSI_MSL	Samsung_SLSI_MSL-2	${57.29}_{(4)}$	${0.99790}_{(1)}$	$50$	Tesla V100	PyTorch	flip/transpose ( $\times 8$ ), models ( $\times 3$ )	$L_{1}$
Tyan 1	Tyan	${57.23}_{(5)}$	${0.99788}_{(6)}$	$0.38$	GTX 1080 Ti	TensorFlow	flip/rotate ( $\times 8$ ), model snapshots ( $\times 3$ )	$L_{1}$
NJU-IITJ	Sora	${57.22}_{(6)}$	${0.99784}_{(9)}$	$3.5$	Tesla V100	PyTorch	models ( $\times 8$ )	$L_{1}$
Panda	panda_ynn	${57.20}_{(7)}$	${0.99784}_{(8)}$	$2.72$	GTX 2080 Ti	TensorFlow	flip/rotate ( $\times 8$ ), model snapshots ( $\times 3$ )	$L_{1}$
BOE-IOT-AIBD	eastworld	${57.19}_{(8)}$	${0.99784}_{(7)}$	$0.61$	Tesla P100	TensorFlow	None	$L_{1}$
TCL Research Europe 1	tcl-research-team	${57.11}_{(9)}$	${0.99788}_{(10)}$	$?$	RTX 2080 Ti	TensorFlow	flip/rotate ( $\times 8$ ), models ( $\times 3 - 5$ )	$L_{1}$
Eraser 3	BumjunPark	${57.03}_{(10)}$	${0.99779}_{(4)}$	$0.31$	?	PyTorch	?	$L_{1}$
EWHA-AIBI 1	jaayeon	${57.01}_{(11)}$	${0.99781}_{(12)}$	$55$	Tesla V100	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$
ZJU231	qiushizai	${56.72}_{(12)}$	${0.99752}_{(11)}$	$0.17$	GTX 1080 Ti	PyTorch	self ensemble	$L_{1}, D C T$
NoahDn	matteomaggioni	${56.47}_{(13)}$	${0.99749}_{(14)}$	$3.54$	Tesla V100	TensorFlow	flip/rotate ( $\times 8$ )	$L_{1}$
Dahua_isp	-	${56.20}_{(14)}$	${0.99749}_{(13)}$	$?$	GTX 2080	PyTorch	?	$?$

Table 1: Results and rankings of methods submitted to the rawRGB denoising track.

Team	Username	PSNR	SSIM	Runtime (s/Mpixel)	CPU/GPU (at runtime)	Platform	Ensemble	Loss
Eraser 2	Songsaris	${33.22}_{(1)}$	${0.9596}_{(1)}$	$103.92$	TITAN V	PyTorch	flip/rotate/RGB shuffle ( $\times 48$ )	$L_{1}$
Alpha	q935970314	${33.12}_{(2)}$	${0.9578}_{(3)}$	$6.72$	RTX 2080 Ti	PyTorch	flip/rotate ( $\times 8$ )	Charbonnier
HITVPC&HUAWEI 2	hitvpc_huawei	${33.01}_{(3)}$	${0.9590}_{(2)}$	$?$	GTX 1080 Ti	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$
ADDBlock	BONG	${32.80}_{(4)}$	${0.9565}_{(5)}$	$76.80$	Titan XP	PyTorch	flip/rotate ( $\times 8$ ), models ( $\times 4$ )	$L_{1}$
UIUC_IFP	Self-Worker	${32.69}_{(5)}$	${0.9572}_{(4)}$	$0.61$	Tesla V100 ( $\times 2$ )	PyTorch	flip/rotate ( $\times 8$ ), models( $\times 3$ )	$L_{1}$
Baidu Research Vision 2	zhihongp	${32.30}_{(6)}$	${0.9532}_{(6)}$	$9.28$	Tesla V100 ( $\times 8$ )	PaddlePaddle, PyTorch	flip/transpose ( $\times 8$ )	$L_{1}$
Rainbow	JiKun63	${32.24}_{(7)}$	${0.9410}_{(11)}$	$2.41$	RTX 2080Ti	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$ /Laplace gradient
TCL Research Europe 2	tcl-research-team	${32.23}_{(8)}$	${0.9467}_{(9)}$	$?$	RTX 2080 Ti	TensorFlow	flip/rotate ( $\times 8$ ), models ( $\times 3 - 5$ )	$L_{1}$
LDResNet	SJKim	${32.09}_{(9)}$	${0.9507}_{(7)}$	$17.85$	GTX 1080	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$
Eraser 4	BumjunPark	${32.06}_{(10)}$	${0.9484}_{(8)}$	$?$	?	PyTorch	?	$L_{1}$
STAIR	dark_1im1ess	${31.67}_{(11)}$	${0.9281}_{(14)}$	$0$	1.86	Titan TITAN RTX ( $\times 2$ )	?	$L_{1}$
Couger AI 2	priyakansal	${31.61}_{(12)}$	${0.9383}_{(12)}$	$0.23$	GTX 1080	Keras/Tensorflow	None	MSE/SSIM
EWHA-AIBI 2	jaayeon	${31.38}_{(13)}$	${0.9417}_{(10)}$	$?$	Tesla V100	PyTorch	flip/rotate ( $\times 8$ )	$L_{1}$
NCIA-Lab	Han-Soo-Choi	${31.37}_{(14)}$	${0.9269}_{(15)}$	$2.92$	TITAN RTX	PyTorch	None	MS-SSIM/ $L 1$
Couger AI 1	sabarinathan	${31.34}_{(15)}$	${0.9296}_{(13)}$	$0.23$	GTX 1080	Keras/Tensorflow	None	MSE/SSIM
Visionaries	rajatguptakgp	${19.97}_{(16)}$	${0.6791}_{(16)}$	?	GTX 1050 Ti	PyTorch	None	MSE

Table 2: Results and rankings of methods submitted to the sRGB denoising track.

Figure 3: Combined PSNR and SSIM values of all methods from both rawRGB (in blue) and sRGB (in red) tracks. Note the different axes and scales for each track.

Main ideas

All of the proposed methods are based on deep learning. Specifically, all methods employ convolutional neural networks (CNNs) based on various architectures. Most of adapted architectures are based on widely-used networks, such as U-Net

[ronneberger2015unet], ResNet [he2016deep], and DenseNet [huang2017densely]

. The main ideas included re-structuring existing networks, introducing skip connections, introducing residual connections, and using densely connected components. Other strategies have been used such as feature attention for image denoising

[anwar2019real], atrous spatial pyramid pooling (ASPP) [chen2018encoder], and neural architectural search (NAS) [elsken2018neural].

Most teams used $L_{1}$ loss as the optimization function while some teams used $L_{2}$ loss or adopted a mixed loss between $L_{1}$ , $L_{2}$ , multi-scale structural similarity (MS-SSIM) [wang2003multiscale], and/or Laplace gradients.

Top results

The top methods achieved very close performances, in terms of PSNR and SSIM. In the rawRGB track, the top two methods are $0.01$ dB apart in terms of PSNR, whereas in the sRGB track, the top three methods have $\sim 0.1$ dB difference in terms of PSNR, as shown in Figures 1 and 2. The differences in SSIM values were similarly close. In terms of PSNR, the main performance metric used in the challenge, the best two methods for rawRGB denoising are proposed by teams Baidu Research Vision and HITVPC&HUAWEI, and achieved 57.44 and 57.43 dB PSNR, respectively, while the best method for sRGB denoising is proposed by team Eraser and achieved 33.22 dB PSNR. In terms of SSIM, as a complementary performance metric, the best method for rawRGB denoising is proposed by the team Samsung_SLSI_MSL and achieved a SSIM index of 0.9979, while the best SSIM index for sRGB denoising is achieved by the Eraser team.

Ensembles

To boost performance, most of the methods applied different flavors of ensemble techniques. Specifically, most teams used a self-ensemble [timofte2016seven] technique where the results from eight flipped/rotated versions of the same image are averaged together. Some teams applied additional model-ensemble techniques.

Conclusion

From the analysis of the presented results, we can conclude that the proposed methods achieve state-of-the-art performance in real image denoising on the SIDD+ benchmark. The top methods proposed by the top ranking teams (i.e., HITVPC&HUAWEI, Baidu Research Vision, Eraser, and Alpha) achieve consistent performance across both color spaces—that is, rawRGB and sRGB (see Figure 3).

4 Methods and Teams

4.1 Hitvpc&huawei

Distillating Knowledge from Original Network and Siamese Network for Real Image Denoising

The team used distillating knowledge and NAS (Neural Architecture Search technology) to improve the denoising performance. The proposed network is based on both MWCNN [liu2018multi] and ResNet [he2016deep] to propose the mwresnet (multi-level wavelet resnet). The team used the proposed network to design the Siamese Network by use of NAS technology. Both networks can complement each other to improve denoising performance in distillating knowledge stage. Only the Siamese Network is used in the final denoising stage. The network architecture proposed by the team is shown in Figure 4.

Figure 4: The network architecture proposed by the HITVPC&HUAWEI team.

4.2 Baidu Research Vision

Neural Architecture Search (NAS) based Dense Residual Network for Image Denoising

The Baidu Research Vision team first proposed a dense residual network that includes multiple types of skip connections to learn features at different resolutions. A new NAS based scheme is further implemented in PaddlePaddle [paddlepaddle] to search for the number of dense residual blocks, the block size and the number of features, respectively. The proposed network achieves good denoising performance in the sRGB track, and the added NAS scheme achieves impressive performance in the rawRGB track. The architectures of the neural network and the distributed SA-NAS proposed by the team are illustrated in Figure 5.

Figure 5: The architectures of the neural network and the distributed SA-NAS scheme proposed by the Baidu Research Vision team.

4.3 Eraser

Iterative U-in-U network for image denoising (UinUNet)

The team modified the down-up module and connections in the DIDN [yu2019deep]. Down-sampling and up-sampling layers are inserted between two modules to construct more hierarchical block connections. Several three-level down-up units (DUU) are included in a two-level down-up module (UUB). The UinUNet architecture proposed by the team is shown in Figure 6.

Figure 6: The UinUNet network architecture proposed by the Eraser team.

Kernel Attention CNN for Image Denoising (KADN)

is inspired by Selective Kernel Networks (SKNet) [li2019selective], DeepLab V3 [chen2018encoder], and Densely Connected Hierarchical Network for Image Denoising (DHDN) [park2019densely]. The DCR blocks of DHDN are replaced with Kernel Attention (KA) blocks. KA blocks apply the concept of atrous spatial pyramid pooling (ASPP) of DeepLab V3 to apply the idea of SKNet that dynamically selects features from different convolution kernels. The KADN architecture proposed by the team is shown in Figure 7.

Figure 7: The KADN network architecture proposed by the Eraser team.

4.4 Alpha

Enhanced Asymmetric Convolution Block (EACB) for image restoration tasks [Liu2020MMDM]

Based on ACB[ding2019acnet]

, the team added two additional diagonal convolutions to further strengthen the kernel skeleton. For image restoration tasks, they removed the batch normalization layers and the bias parameters for better performance, and used the cosine annealing learning rate scheduler

[LoshchilovSGDR] to prevent gradient explosions. They used a simple version of RCAN [zhang2018image] as the backbone. The specific modifications are: (1) Remove all channel attention (CA) modules. Too many CAs lead to an increase in training and testing time, and bring little performance improvement. (2) Remove the upsampling module to keep all the features of the same size. (3) Add a global residual to enhance the stability of the network and make the network reach higher performance in the early stages of training.

4.5 Samsung_SLSI_MSL

Real Image Denoising based on Multi-scale Residual Dense Block and Cascaded U-Net with Block-connection [Bao_2020_CVPR_Workshops]

The team used three networks: residual dense network (RDN) [zhang2018residual], multi-scale residual dense network (MRDN), and Cascaded U-Net [ronneberger2015unet] with residual dense block (RDB) connections (CU-Net). Inspired by Atrous Spatial Pyramid Pooling (ASPP) [chen2018encoder] and RDB, the team designed multi-scale RDB (MRDB) to utilize the multi-scale features within component blocks and built MRDN. Instead of skip-connection, the team designed U-Net with block-connection (U-Net-B) to utilize an additional neural module (i.e., RDB) to connect the encoder and the decoder. They also proposed and used noise permutation for data augmentation to avoid model overfitting. The network architecture of MRDN proposed by the team is shown in Figure 8, and CU-Net is detailed in [Bao_2020_CVPR_Workshops].

Figure 8: The MRDN architecture proposed by the Samsung_SLSI_MSL team.

4.6 ADDBlock

PolyU-Net (PUNet) for Real Image Denoising

The team utilized the idea of Composite Backbone Network (CBNet) architecture [liu2019cbnet] used for object detection. They used a U-net architecture [park2019densely] as the backbone of their PolyU-Net (PUNet). They constructed recurrent connections between backbones with only addition and without upsampling operation contrast to CBNet to prevent distortion of the original information of backbones. Additionally, contrary to CBNet, a slight performance gain was obtained by sharing weights in the backbone networks. The network architecture proposed by the team is shown in Figure 9.

Figure 9: The PUNet architecture proposed by the ADDBlock team.

4.7 Tyan

Parallel U-net for Real Image Denoising

The team proposed parallel U-net for considering global and pixel-wise denoising at the same time. Two kinds of U-net were combined in a parallel way: one traditional U-net for global denoising due to its great receptive field, and another U-net with dilated convolutions replacing the pooling operations; to preserve the feature map size. Both U-nets take the same input noisy image separately and their outputs are concatenated and followed by a 1x1 convolution to produce the final clean image. The network architecture proposed by the team is shown in Figure 10.

Figure 10: The Parallel U-net architecture proposed by the Tyan team.

4.8 Uiuc Ifp

Using U-Nets as ResNet blocks for Real Image Denoising

The team concatenated multiple of the U-Net models proposed in [liu2019learning, zhou2020image]. Each U-Net model is treated as a residual block. The team used eight residual blocks in their model. Model ensemble was used to improve the performance. Specifically, the team trained ten separate models and deployed the top three models that achieves the best results on the validation set. In the testing phase, the team first applied rotating and flipping operations to augment each testing image. Then, a fusion operation is applied to the results obtained from the three high-performance models.

4.9 Nju-Iitj

Learning RAW Image Denoising with Color Correction

The team adapted scaling the Bayer pattern channels based on each channel’s maximum. They used Bayer unification [liu2019learning] for data augmentation and selected Deep iterative down-up CNN network (DIDN) [yu2019deep] as their base model for denoising.

4.10 Panda

Pyramid Real Image Denoising Network

The team proposed a pyramid real image denoising network (PRIDNet), which contains three stages: (1) noise estimation stage that uses channel attention mechanism to recalibrate the channel importance of input noise; (2) at the multi-scale denoising stage, pyramid pooling is utilized to extract multi-scale features; and (3) the feature fusion stage adopts a kernel selecting operation to adaptively fuse multi-scale features. The PRIDNet architecture proposed by the team is shown in Figure 11.

Figure 11: The PRIDNet architecture proposed by the Panda team.

4.11 Rainbow

Densely Self-Guided Wavelet Network for Image Denoising [liu2020densely]

The team proposed a top-down self-guidance architecture for exploiting image multi-scale information. The low-resolution information is extracted and gradually propagated into the higher resolution sub-networks to guide the feature extraction processes. Instead of pixel-shuffling/unshuffling, the team used the discrete wavelet transform (DWT) and the inverse discrete wavelet transform (IDWT) for upsampling and downsampleing, respectively. The used loss was a combination between the L1 and the Laplace gradient losses. The network architecture proposed by the team is shown in Figure

12.

Figure 12: The network architecture proposed by the Rainbow team.

4.12 TCL Research Europe

Neural Architecture Search for image denoising [mozejko_superkernel_2020]

The team proposed an ensemble model consisting of 3 - 5 sub-networks. Two types of sub-networks are proposed: (1) the Superkernel-based Multi Attentional Residual U-Net and (2) the Superkernel SkipInit Residual U-Net. The superkernel method used by the team is based on [stamoulis2019single].

4.13 Boe-Iot-Aibd

Raw Image Denoising with Unified Bayer Pattern and Multiscale Strategies

The team utilized a pyramid denoising network [zhao2019pyramid] and Bayer pattern unification techniques [liu2019learning] where all input noisy rawRGB images are unified to RGGB bayer pattern according to the metadata information. Then inputs are passed into Squeeze-and-Excitation blocks [hu2018squeeze] to extract features and assign weights to different channels. Multiscale densoising blocks and selective kernel blocks [li2019selective] were applied. The network architecture proposed by the team is shown in Figure 13.

Figure 13: The network architecture proposed by the BOE-IOT-AIBD team.

4.14 LDResNet

Mixed Dilated Residual Network for Image Denoising

The team designed a deep and wide network by piling up the dilated and residual (DR) blocks equipped with multiple dilated convolutions and skip connections. In addition to the given noisy-clean image pairs, the team utilized extra undesired-clean image pairs as a way to add some noise on the ground truth images of the training data of the SIDD dataset. The network architecture proposed by the team is shown in Figure 14.

Figure 14: The network architecture proposed by the LDResNet team.

4.15 Ewha-Aibi

Denoising with wavelet domain loss

The team used an enhanced deep residual network EDSR [lim2017enhanced] architecture with global residual skip and input that is decomposed with stationary wavelet transform and used loss in wavelet transform domain. To accelerate the performance of networks, channel attention [woo2018cbam] is added every fourth res block.

4.16 Stair

Down-Up Scaling Second-Order Attention Network for Real Image Denoising

The team proposes a Down-Up scaling RNAN (RNAN-DU) method to deal with real noise that may not be statistically independent. Accordingly, the team used the residual non-local attention network (RNAN) [zhang2019residual]

as a backbone of the proposed RNAN-DU method. The down-up sampling blocks are used to suppress the noise, while non-local attention modules are focused on dealing with more severe, non-uniformly distributed real noise. The network architecture proposed by the team is shown in Figure

15.

Figure 15: The RNAN Down-Up scaling network (RNAN-DU) architecture proposed by the STAIR team.

4.17 Couger AI

Lightweight Residual Dense net for Image Denoising

The team proposed a U-net like model with stacked residual dense blocks along with simple convolution/convolution transpose. The input image is first processed by a coordinate convolutional layer [liu2018intriguing] aiming to improve the learning of spatial features in the input image. Moreover, the team used the modified dense block to learn the global hierarchical features and then fused these features to the output of decoder in a more holistic way.

Lightweight Deep Convolutional Model for Image Denoising

The team proposed also to train the network without the coordinate convolutional layer [liu2018intriguing]. This modification achieves better results in the testing set compared to the original architecture.

4.18 Zju231

Deep Prior Fusion Network (DPFNet) for Real Image Denoising

The team presented DPFNet based on U-net [ronneberger2015unet]. They utilize the DPF block and the Residual block, which are both modified versions of the standard residual block in ResNet [he2016deep], for feature extraction and image reconstruction. Compared with the Residual block, the DPF block introduces an extract 1 $\times$ 1 convolutional layer to enhance the cross-channel exchange of feature maps during the feature extraction stage.

4.19 NoahDn

Learnable Nonlocal Image Denoising

The team proposed a method that explicitly use the nonlocal image prior within a fully differentiable framework. In particular, the image is processed in a block-wise fashion, and after a shallow feature extraction, self-similar blocks are extracted within a search window and then jointly denoised by exploiting their nonlocal redundancy. The final image estimate is obtained by returning each block in its original position and adaptively aggregating the overcomplete block estimates within the overlapping regions.

4.20 NCIA-Lab

SAID: Symmetric Architecture for Image Denoising

The team proposed a two-branch bi-directional correction model. The first branch was designed to estimate positive values in the final residual layer, while the second branch was used to estimate negative values in the final residual layer. In particular, the team built their model on top of the DHDN architecture [park2019densely] by adapting two models of the DHDN architecture.

4.21 Dahua_isp

Dense Residual Attention Network for Image Denoising

The team optimized the RIDNet [anwar2019real] with several modifications: using long dense connection to avoid gradients vanishing; and adding depth-wise separable convolution [howard2017mobilenets] as the transition.

4.22 Visionaries

Image Denoising through Stacked AutoEncoders

The team used a stacked autoencoder to attempt denoising the images. While training, under each epoch, image pairs (original and noisy) were randomly shuffled and a Gaussian noise with mean and standard deviation of the difference between the original and noisy image for all 160 image pairs was added.

Acknowledgements

We thank the NTIRE 2020 sponsors: Huawei, Oppo, Voyage81, MediaTek, DisneyResearch $|$ Studios, and Computer Vision Lab (CVL) ETH Zurich.

Appendix A Teams and Affiliations

NTIRE 2020 Team

Title: NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results

Members:
Abdelrahman Abdelhamed¹ (kamel@eecs.yorku.ca),
Mahmoud Afifi¹ (mafifi@eecs.yorku.ca),
Radu Timofte² (radu.timofte@vision.ee.ethz.ch),
Michael S. Brown¹ (mbrown@eecs.yorku.ca)

Affiliations:
¹ York University, Canada
² ETH Zurich, Switzerland

Hitvpc&huawei

Title: Distillating Knowledge from Original Network and Siamese Network for Real Image Denoising

Members: Yue Cao¹ (hitvpc_huawei@163.com), Zhilu Zhang¹, Wangmeng Zuo¹, Xiaoling Zhang², Jiye Liu², Wendong Chen², Changyuan Wen², Meng Liu², Shuailin Lv², Yunchao Zhang²

Affiliations: ¹ Harbin Institute of Technology, China ² Huawei, China

Baidu Research Vision

Title: Neural Architecture Search (NAS) based Dense Residual Network for Image Denoising

Members: Zhihong Pan¹ (zhihongpan@baidu.com), Baopu Li¹, Teng Xi², Yanwen Fan², Xiyu Yu², Gang Zhang², Jingtuo Liu², Junyu Han², Errui Ding²

Affiliations: ¹ Baidu Research (USA), ² Department of Computer Vision Technology (VIS), Baidu Incorporation

Eraser

Title: Iterative U-in-U network for image denoising, Kernel Attention CNN for Image Denoising

Members: Songhyun Yu (3069song@naver.com), Bumjun Park, Jechang Jeong

Affiliations: Hanyang University, Seoul, Korea

Alpha

Title: Enhanced Asymmetric Convolution Block (EACB) for image restoration tasks

Members: Shuai Liu ¹ (18601200232@163.com), Ziyao Zong¹, Nan Nan¹, Chenghua Li²

Affiliations: ¹ North China University of Technology,
² Institute of Automation, Chinese Academy of Sciences

Samsung_SLSI_MSL

Title: Real Image Denoising based on Multi-scale Residual Dense Block and Cascaded U-Net with Block-connection

Members: Zengli Yang (zengli.y@samsung.com), Long Bao, Shuangquan Wang, Dongwoon Bai, Jungwon Lee

Affiliations: Samsung Semiconductor, Inc.

ADDBlock

Title: PolyU-Net (PUNet) for Real Image Denoising

Members: Youngjung Kim (read12300@add.re.kr), Kyeongha Rho, Changyeop Shin, Sungho Kim

Affiliations: Agency for Defense Development

Tyan

Title: Parallel U-Net for Real Image Denoising

Members: Pengliang Tang (tpl21200@outlook.com), Yiyun Zhao

Affiliations: Beijing University of Posts and Telecommunications

Uiuc Ifp

Title: Using U-Nets as ResNet blocks for Real Image Denoising

Members: Yuqian Zhou (zhouyuqian133@gmail.com), Yuchen Fan, Thomas Huang

Affiliations: University of Illinois at Urbana Champaign

Nju-Iitj

Title: Learning RAW Image Denoising with Color Correction

Members: Zhihao Li¹ (lizhihao6@outlook.com), Nisarg A. Shah²

Affiliations: ¹ Nanjing University, Nanjing, China, ² Indian Instititute of Technology, Jodhpur, Rajasthan, India

Panda

Title: Pyramid Real Image Denoising Network

Members: Yiyun Zhao (yiyunzhao@bupt.edu.cn), Pengliang Tang

Affiliations: Beijing University of Posts and Telecommunications

Rainbow

Title: Densely Self-guided Wavelet Network for Image Denoising

Members: Wei Liu (liujikun@hit.edu.cn), Qiong Yan, Yuzhi Zhao

Affiliations: SenseTime Research; Harbin Institute of Technology

TCL Research Europe

Title: Neural Architecture Search for image denoising

Members: Marcin Możejko (marcin.mozejko@tcl.com), Tomasz Latkowski, Lukasz Treszczotko, Michał Szafraniuk, Krzysztof Trojanowski

Affiliations: TCL Research Europe

Boe-Iot-Aibd

Title: Raw Image Denoising with Unified Bayer Pattern and Multiscale Strategies

Members: Yanhong Wu (wuyanhong@boe.com.cn),

Pablo Navarrete Michelini, Fengshuo Hu, Yunhua Lu

Affiliations:Artificial Intelligence and Big Data Research Institute, BOE

LDResNet

Title: Mixed Dilated Residual Network for Image Denoising

Members: Sujin Kim (sujin.kim@snu.ac.kr)

Affiliations: Seoul National University, South Korea

ST Unitas AI Research (STAIR)

Title: Down-Up Scaling Second-Order Attention Network for Real Image Denoising

Members: Magauiya Zhussip (magauiya@stunitas.com), Azamat Khassenov, Jong Hyun Kim, Hwechul Cho

Affiliations: ST Unitas

Ewha-Aibi

Title: Denoising with wavelet domain loss

Members: Wonjin Kim (onejean81@gmail.com), Jaayeon Lee, Jang-Hwan Choi

Affiliations: Ewha Womans University

Couger AI

Title: Lightweight Residual Dense net for Image Denoising, Lightweight Deep Convolutional Model for Image Denoising

Members: Priya Kansal (priya@couger.co.jp), Sabari Nathan (sabari@couger.co.jp)

Affiliations: Couger Inc.

Zju231

Title: Deep Prior Fusion Network for Real Image Denoising

Members: Zhangyu Ye¹ (qiushizai@zju.edu.cn), Xiwen Lu², Yaqi Wu³, Jiangxin Yang¹, Yanlong Cao¹, Siliang Tang¹, Yanpeng Cao¹

Affiliations: ¹ Zhejiang University, Hangzhou, China ² Nanjing University of Aeronautics and Astronautics, Nanjing, China ³ Harbin Institute of Technology Shenzhen, Shenzhen, China

NoahDn

Title: Learnable Nonlocal Image Denoising

Members: Matteo Maggioni
(matteo.maggioni@huawei.com), Ioannis Marras, Thomas Tanay, Gregory Slabaugh, Youliang Yan

Affiliations: Huawei Technologies Research and Development (UK) Ltd, Noah’s Ark Lab London

NCIA-Lab

Title: SAID: Symmetric Architecture for Image Denoising Members: Myungjoo Kang (mkang@snu.ac.kr), Han-Soo Choi, Sujin Kim, Kyungmin Song

Affiliations: Seoul National University

Dahua_isp

Title: Dense Residual Attention Network for Image Denoising

Members: Shusong Xu (13821177832@163.com), Xiaomu Lu, Tingniao Wang, Chunxia Lei, Bin Liu

Affiliations: Dahua Technology

Visionaries

Title: Image Denoising through Stacked AutoEncoders

Members: Rajat Gupta
(rajatgba2021@email.iimcal.ac.in), Vineet Kumar

Affiliations: Indian Institute of Technology Kharagpur