A Practical Solution for SAR Despeckling with Only Single Speckled Images

12/13/2019
by   Jianguo Sun, et al.
Microsoft
0

In this letter, we aim to address synthetic aperture radar (SAR) despeckling problem with the necessity of neither clean (speckle-free) SAR images nor independent speckled image pairs from the same scene, a practical solution for SAR despeckling (PSD) is proposed. Firstly, to generate speckled-to-speckled (S2S) image pairs from the same scene in the situation of only single speckled SAR images are available, an adversarial learning framework is designed. Then, the S2S SAR image pairs are employed to train a modified despeckling Nested-UNet model using the Noise2Noise (N2N) strategy. Moreover, an iterative version of the PSD method (PSDi) is also proposed. The performance of the proposed methods is demonstrated by both synthetic speckled and real SAR data. SAR block-matching 3-D algorithm (SAR-BM3D) and SAR dilated residual network (SAR-DRN) are used in the visual and quantitative comparison. Experimental results show that the proposed methods can reach a good tradeoff between speckle suppression and edge preservation.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 3

page 4

07/21/2020

Balance Scene Learning Mechanism for Offshore and Inshore Ship Detection in SAR Images

This letter proposes a novel Balance Scene Learning Mechanism (BSLM) for...
04/02/2017

SAR image despeckling through convolutional neural networks

In this paper we investigate the use of discriminative model learning th...
09/09/2017

Learning a Dilated Residual Network for SAR Image Despeckling

In this letter, to break the limit of the traditional linear models for ...
06/28/2020

SAR Image Despeckling by Deep Neural Networks: from a pre-trained model to an end-to-end training strategy

Speckle reduction is a longstanding topic in synthetic aperture radar (S...
01/11/2018

Quadrature compressive sampling SAR imaging

This paper presents a quadrature compressive sampling (QuadCS) and assoc...
01/22/2022

Learning Efficient Representations for Enhanced Object Detection on Large-scene SAR Images

It is a challenging problem to detect and recognize targets on complex l...
07/28/2020

Change Detection Using Synthetic Aperture Radar Videos

Many researches have been carried out for change detection using tempora...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Synthetic aperture radar (SAR) images are usually corrupted with speckle noise, which leads to the degradation of image quality and affects the performance in various applications of remote sensing, e.g., classification [12] and change detection [16]. Several methods have been proposed to mitigate the speckle in SAR images, including filtering methods [7, 6], wavelet shrinkage [3, 9], SAR block-matching 3-D algorithm (SAR-BM3D) [13].

Recently, convolutional neural network (CNN) based supervised learning has been employed for SAR image despeckling, which can reduce speckle noise by learning relationships between speckled and clean ground truth images with CNN models, e.g., SAR convolutional neural network (SAR-CNN)

[4], SAR image despeckling network (SAR-IDN) [15], and SAR dilated residual network (SAR-DRN) [20]. However, there are few clean SAR images in practical applications. Hence, speckle-free optical images with the single channel are usually employed as the clean ground truth for SAR image despeckling, and synthetic speckled images can be obtained by adding speckle noise to original images. Then, such speckled-to-clean image pairs can be used as training data in CNN despeckling models. However, because of the differences in imaging mechanism and image features of SAR and optical images, i.e., gray-level distribution and spatial correlation [5], it is not the optimal solution to achieve SAR image despeckling by directly employing the aforementioned CNN models, which are trained on optical images using supervised learning.

More recently, the Noise2Noise (N2N) [8] strategy has shown its ability for image denoising without using any clean ground truth images. By employing the N2N strategy, CNN model can still achieve high denoising performance (e.g., zero-mean distribution noise) with mean square error (MSE) loss, as long as two independent noisy images from the same underlying clean image are available. This provides the possibility for SAR image despeckling without using clean ground truth images. However, to acquire two independent speckled SAR images from same scene is quite difficult. Hence, it is still impossible to solve the SAR image despeckling problem by merely employing the N2N strategy, as only single speckled SAR images are available.

In this letter, we introduce a more practical despeckling solution for SAR image in the situation of only single speckled images are available, namely, a practical solution for SAR despeckling (PSD). Our main contributions are summarized as follows:

  1. []

  2. To generate speckled-to-speckled (S2S) SAR image pairs, an adversarial learning framework that consists of two generators and a discriminator is presented, which are trained by an alternating optimization strategy.

  3. By using the obtained S2S SAR image pairs, an advanced Nested-UNet model [21] is trained to achieve despeckling with the N2N strategy. In addition, an iterative version of the PSD method (PSDi) is proposed, which can further improve the despeckling performance.

  4. Visual and quantitative experiments conducted on synthetic speckled and real SAR data show that the proposed methods notably suppress speckle noise with better preserving edge, which outperform SAR-BM3D and SAR-DRN.

Ii Methodology

Fig. 1: The overall flowchart of the proposed PSD method. (left) Adversarial learning for S2S SAR image pairs generation: two generators , , and a discriminator are trained by using the adversarial loss , the backward cycle consistency loss , and the total variation (TV) loss . (right) N2N despeckling without clean ground truth images: the despeckling model is trained by using the MSE loss .

In this section, firstly, the SAR speckle noise model is introduced. Then, we present implement details of the proposed PSD method as shown in Fig. 1

, which consists of two parts: 1) an adversarial learning framework for S2S SAR image pairs generation and its loss functions for optimization are presented; 2) the N2N despeckling strategy without using any clean ground truth images is described, and the despeckling network architecture of the modified Nested-UNet is given.

Ii-a SAR speckle noise model

Let be the observed (speckled) SAR image with the size of , be the underlying (despeckled) clean image, and be the speckle noise. Then Y can be obtained by following model [5]:

(1)

where denotes the Hadamard product (i.e., entry-wise product) of two matrices. It is well-known that, for a SAR image, N

often follows a Gamma distribution with unit mean and variance

, where is the number of looks in SAR imaging process, where a smaller

indicates stronger speckle. The aim of despeckling is to estimate

X from the observed image Y. Based on the established speckle noise model in (1), our proposed approach in Section II-B can be used to generate S2S SAR image pairs from same scenario.

Ii-B Adversarial learning for S2S SAR image pairs generation

In this section, to generate S2S SAR image pairs, we introduce an adversarial learning framework, which consists of two generators , , and a discriminator , as shown in Fig. 1 (left). , and denote the parameters (i.e., weights and biases) of , and , respectively.

The generator is used to generate the “fake” speckled SAR images , expressed as follows

(2)

and the network architecture of is the same as the despeckling model which will be described in Section II-C. And the discriminator determines the distance between the distribution of the “fake” speckled SAR images and the distribution of the input speckled images, and its network architecture is designed as suggested in [2]. and are trained by the adversarial loss of Wasserstein Generative Adversarial Networks (WGAN) [1] , which can be formulated as

(3)

where denotes the expectation operator.

Here, to make change speckle distribution while preserving the basic edge features of the original SAR image, is used to bring the output of back to the original SAR image, namely, . We design the network architecture of following the denoising CNN (DnCNN) [19]. and are trained by the backward cycle consistency loss following [2] and [22], can be written as

(4)

In addition, to smooth the output of , the generator is also trained by the total variation (TV) loss , which is defined as

(5)

where is the pixel value in . can reduce the difference of adjacent pixel values in the despeckled images .

With the defined loss functions, an alternating optimization strategy is applied to optimize , and , which can be described as an adversarial min-max problem, expressed as follows

(6)

where , , and are the predefined weights for , and , respectively. To prevent from being over-smooth, the predefined weight for should be far less than and [15]. After reaching a steady state through adversarial learning, i.e., until the discriminator cannot distinguish the “fake” speckled SAR images from the input speckled images Y, then we can obtain the S2S SAR image pairs from the same scene in the situation when only single speckled SAR images are available, which can be expressed as follows

(7)

where and are two independent speckle matrices with the same number of looks .

Ii-C N2N despeckling without clean ground truth images

After obtaining the S2S SAR image pairs , as shown in Fig. 1 (right), we can employ the N2N strategy [8] to train the despeckling model without using any clean ground truth images. Here, MSE loss is used to optimize , which is formulated as

(8)

where denotes the parameters (i.e., weights and biases) of .

Fig. 2: The network architecture of Nested-UNet. Different from the original UNet, Nested-UNet has convolution layers on skip pathways and dense skip connections on skip pathways.

Due to the characteristic of speckle noise, as expressed in (1), the expected value of speckled image is the same as that of underlying clean (despeckled) image, hence, the despeckled images can be generated by using without any clean ground truth images. This is particularly true when the despeckling model is trained on a large dataset. A modified Nested-UNet [21] is adopted as the despeckling model as shown in Fig. 2

. Nested-UNet is chosen because of its advantage in improving the gradients flow throughout the network by its convolution layers on skip pathways and dense skip connections on skip pathways. The original Nested-UNet is polished to be more effective for SAR image despeckling by removing batch normalization layers of each convolutional block, which has been proven in other image processing tasks, e.g., image super-resolution

[10].

Except the above PSD method, we also propose an iterative version of PSD method, named PSDi. For PSD method, the training data (the input and ground truth images) of , i.e., S2S SAR image pairs , are generated by using . However, since no ground truth (clean or speckled) images are available in the training process of , the despeckling ability of is not decent. Compared with , can provide more powerful despeckling ability since ground truth (speckled) images are used in its training process. Therefore, to make the distribution of generative S2S SAR image pairs closer to that of real SAR images, we apply instead of in the proposed PSDi. Then, the new S2S image pairs can be obtained as follows

(9)

With the new S2S image pairs, a more effective despeckling model can be obtained by using N2N strategy again. Thereby, we can complete the training process of the despeckling model with only single speckled SAR images.

Iii Experiments and results

The number of looks 1 4 16
Index PSNR SSIM PSNR SSIM PSNR SSIM
Speckled Input 11.08 0.0519 15.17 0.1261 20.73 0.3052
SAR-BM3D [13] 16.99 0.1799 19.88 0.2525 22.24 0.3604
SAR-DRN [20] 25.25 0.5831 28.39 0.7143 29.77 0.8061
PSD (Proposed) 25.47 0.7023 28.45 0.7571 30.04 0.8200
PSDi (Proposed) 25.67 0.7196 28.67 0.7763 29.92 0.8284
TABLE I: Performance comparison in terms of average PSNR/SSIM on synthetic speckled data.
(a) (b) (c) (d) (e) (f)
Fig. 3: Illustration of despeckling results on synthetic speckled image corrupted by 1-look speckle. Full results (top row), Magnified results (bottom row). (a) Speckle-free image, (b) Speckled image, (c) SAR-BM3D [13], (d) SAR-DRN [20], (e) PSD (proposed), (f) PSDi (proposed).
(a) (b) (c) (d) (e)
Fig. 4: Despeckling performance comparison on (top) ALSO-2 image corrupted by 2-look speckle and (bottom) Sandia image corrupted by 1-look speckle, where several homogeneous and edge areas are identified by the red and green boxes, respectively. (a) Original SAR image, (b) SAR-BM3D [13], (c) SAR-DRN [20], (d) PSD (proposed), (e) PSDi (proposed).
Data Original
SAR-
BM3D [13]
SAR-
DRN [20]
PSD
(proposed)
PSDi
(proposed)
ALSO-2 Area 1 5.4501 13.7571 13.2693 14.8630 15.2900
ALSO-2 Area 2 5.8544 23.4989 24.8332 26.7084 27.2935
ALSO-2 Area 3 6.2657 23.8957 19.1693 25.6111 25.9474
Sandia Area 1 7.7393 21.6378 18.8353 23.0581 23.5764
Sandia Area 2 11.4771 112.2791 54.4402 114.6707 116.5001
TABLE II: Performance comparison in terms of ENL on real SAR data.

Iii-a Experimental setup

In this work, 1-look SAR image patches (cropped to 9696 pixels) obtained from the Sentinel-1 are used to train the proposed networks. In the training process of the adversarial learning (Section II-B), to keep the noise level in the original 1-look SAR images Y consistent with that in the generated speckled images , the number of looks for each generated speckled image is set to be 1. In the training process of the N2N despeckling (Section II-C), to make the network available to different speckle levels, the number of looks for each training image pair is randomly set to be 1, 2, 4, 8 and 16, respectively. The predefined weights , , and are set to be 1, 1 and 0.1, respectively. The networks , , , and are trained using the Adam optimizer by setting , , and , where the learning rates are all initialized as

and reduced by half with each 8 epochs. The network

is trained using the RMSProp optimizer with an initial learning rate of

, and its clipping value is set as 0.02, and the critic iteration is 5. The training process is set to have 32 epochs with mini-batch size 16 in the adversarial learning (Section II-B) and the N2N despeckling (Section II-C).

To verify despeckling effectiveness of the proposed methods PSD () and PSDi (), experiments are conducted on both synthetic speckled data and real SAR data, as compared with the state-of-the-art methods, i.e., SAR-BM3D [13] and SAR-DRN [20]. For SAR-BM3D, it needs to know the number of looks without any additional training, whereas SAR-DRN needs clean ground truth images for training, but does not need to know . Hence, in the training process of SAR-DRN, around

speckle-free optical images from ImageNet

[14] are used as the ground truth images, and the input speckled images can be obtained by adding multiplicative speckle noise to original images. Other parameters in SAR-DRN are set as suggested in [20].

Iii-B Performance comparison on synthetic speckled data

In this experiment, 100 optical remote images (selected from the AID dataset [18]) are used for analyzing. To verify the despeckling effectiveness with the known noise level, the number of looks

of these images is set as 1, 4 and 16, respectively. Since the clean ground truth images are available, the indices of peak signal to noise ratio (PSNR) and structural similarity index (SSIM) 

[17] are used for evaluation, the comparison results are given in Table I. As can be seen from Table I, our proposed PSD and PSDi methods outperform SAR-BM3D and SAR-DRN in all noise levels. Moreover, the PSDi method can further improve the despeckling performance compared with the PSD method. Fig. 3 shows a despeckling sample on an AID image corrupted by 1-look speckle. As can be seen, only our proposed PSD and PSDi methods can remove speckle effectively. Note that SAR-DRN needs clean ground truth images for training, whereas our proposed methods can achieve better performance without using clean ground truth images.

Iii-C Performance comparison on real SAR data

Data
SAR-
BM3D [13]
SAR-
DRN [20]
PSD
(proposed)
PSDi
(proposed)
ALSO-2 Area 4 0.8802 0.8838 0.8836 0.8838
ALSO-2 Area 5 0.8709 0.8728 0.8729 0.8729
Sandia Area 3 0.9103 0.9140 0.9102 0.9104
Sandia Area 4 0.9075 0.9103 0.9076 0.9077
TABLE III: Performance comparison in terms of EPD-ROA on real SAR data.
Fig. 5: The tradeoff between the average EPD-ROA and the average ENL.

To validate the despeckling effectiveness of proposed methods in practical applications, we also conduct a despeckling experiment on real SAR data. Two real SAR images are employed for test. One is an agriculture area image nearby Brazil which is obtained by the ALSO-2 sensor (L-band) and corrupted by 2-look speckle. The other is an aircraft display image corrupted by 1-look speckle and courtesy of Sandia National Laboratories, Radar ISR (Ku-band). In this experiment, the images are cropped to 512512 and 10241024 pixels, respectively. The despeckling results are given in Fig. 4. To evaluate the degree of speckle reduction, the equivalent number of looks (ENL) [11] is used as the performance metric. Table II gives the comparison results in terms of ENL value, which are calculated from several homogeneous areas identified by the red boxes in Fig. 4 (a). To evaluate the edge preservation, the edge-preservation degree based on the ratio of average (EPD-ROA) [11] is used as the measurement. Table III gives the comparison results in terms of EPD-ROA value, which are calculated from several edge areas identified by the green boxes in Fig. 4 (a).

From Table II and Table III, we can see though SAR-BM3D performs well in speckle reduction, the results are over-smooth in despeckling performance. SAR-DRN does not show high speckle suppression ability on real SAR images like that on synthetic speckled images, this may because of the differences between the training data (synthetic speckled images) and the testing data (real SAR images). In summary, as shown in Fig. 5, the proposed PSD and PSDi methods can achieve overwhelming despeckling while preserving great edge features, as compared with SAR-BM3D and SAR-DRN methods. Without clean ground truth images that are used to train the despeckling network, our proposed methods still can achieve better performance while only single speckled SAR images are available. This means our proposed methods are more useful in practical applications.

Iv Conclusion

In this letter, we presented practical deep learning-based methods for SAR image despeckling. Different from other deep learning-based despeckling methods, our proposed methods can achieve despeckling with only single speckled SAR images. In addition, a modified despeckling Nested-UNet is used as the despeckling model. Experiments conducted on both synthetic speckled data and real SAR data demonstrated the superiority of our proposed methods, as compared with state-of-the-art methods in conventional and deep learning-based, respectively. In our future study, we will apply our proposed methods for the polarimetric SAR image despeckling.

References

  • [1] M. Arjovsky, S. Chintala, and L. Bottou (2017) Wasserstein generative adversarial networks. In Proc. Int. Conf. Mach. Learn., pp. 214–223. Cited by: §II-B.
  • [2] S. Cha, T. Park, and T. Moon (2019) GAN2GAN: Generative noise learning for blind image denoising with single noisy images. arXiv preprint arXiv:1905.10488. Cited by: §II-B, §II-B.
  • [3] S.G. Chang, B. Yu, and M. Vetterli (2000) Adaptive wavelet thresholding for image denoising and compression. IEEE Trans. Image Process. 9 (9), pp. 1532–1546. Cited by: §I.
  • [4] G. Chierchia, D. Cozzolino, G. Poggi, and L. Verdoliva (2017) SAR image despeckling through convolutional neural networks. In Proc. IEEE Int. Geosci. Remote Sens. Symp., pp. 5438–5441. Cited by: §I.
  • [5] G. Di Martino, M. Poderico, G. Poggi, et al. (2013) Benchmarking framework for SAR despeckling. IEEE Trans. Geosci. Remote Sens. 52 (3), pp. 1596–1615. Cited by: §I, §II-A.
  • [6] D.T. Kuan, A.A. Sawchuk, T.C. Strand, and P. Chavel (1985) Adaptive noise smoothing filter for images with signal-dependent noise. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-7 (2), pp. 165–177. Cited by: §I.
  • [7] J.S. Lee (1980) Digital image enhancement and noise filtering by use of local statistics. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2 (2), pp. 165–168. Cited by: §I.
  • [8] J. Lehtinen, J. Munkberg, J. Hasselgren, et al. (2018) Noise2Noise: Learning image restoration without clean data. In Proc. Int. Conf. Mach. Learn., pp. 2965–2974. Cited by: §I, §II-C.
  • [9] H.C. Li, W. Hong, Y.R. Wu, and P.Z. Fan (2013) Bayesian wavelet shrinkage with heterogeneity-adaptive threshold for SAR image despeckling based on generalized gamma distribution. IEEE Trans. Geosci. Remote Sens. 51 (4), pp. 2388–2402. Cited by: §I.
  • [10] B. Lim, S. Son, H. Kim, et al. (2017) Enhanced deep residual networks for single image super-resolution. In

    Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. Workshop

    ,
    pp. 1132–1140. Cited by: §II-C.
  • [11] X. Ma, P. Wu, Y. Wu, and H. Shen (2017) A review on recent developments in fully polarimetric SAR image despeckling. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 11 (3), pp. 743–758. Cited by: §III-C.
  • [12] M. Masoud, S. Bahram, and M. Fariba (2017) The effect of PolSAR image despeckling on wetland classification: introducing a new adaptive method. Can. J. Remote Sens. 43 (5), pp. 485–503. Cited by: §I.
  • [13] S. Parrilli, M. Poderico, C.V. Angelino, and L. Verdoliva (2012) A nonlocal SAR image denoising algorithm based on LLMMSE wavelet shrinkage. IEEE Trans. Geosci. Remote Sens. 50 (2), pp. 606–616. Cited by: §I, Fig. 3, Fig. 4, §III-A, TABLE I, TABLE II, TABLE III.
  • [14] O. Russakovsky, J. Deng, H. Su, et al. (2015) ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115 (3), pp. 211–252. Cited by: §III-A.
  • [15] P. Wang, H. Zhang, and V. Patel (2017) SAR image despeckling using a convolutional neural network. IEEE Signal Process. Lett. 24 (12), pp. 1763–1767. Cited by: §I, §II-B.
  • [16] R. Wang, J. Chen, L. Jiao, and M. Wang (2019) How can despeckling and structural features benefit to change detection on bitemporal SAR images?. Remote Sens. 11 (4), pp. 421. Cited by: §I.
  • [17] Z. Wang, A. Bovik, H. Sheikh, et al. (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Process. 13 (4), pp. 600–612. Cited by: §III-B.
  • [18] G.S. Xia, J. Hu, F. Hu, et al. (2017)

    AID: A benchmark data set for performance evaluation of aerial scene classification

    .
    IEEE Trans. Geosci. Remote Sens. 55 (7), pp. 3965–3981. Cited by: §III-B.
  • [19] K. Zhang, W. Zuo, Y. Chen, et al. (2017) Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. on Image Process. 26 (7), pp. 3142–3155. Cited by: §II-B.
  • [20] Q. Zhang, Q. Yuan, J. Li, et al. (2018) Learning a dilated residual network for SAR image despeckling. Remote Sens. 10 (2), pp. 196. Cited by: §I, Fig. 3, Fig. 4, §III-A, TABLE I, TABLE II, TABLE III.
  • [21] Z. Zhou, M.M.R. Siddiquee, N. Tajbakhsh, and J. Liang (2018) Unet++: A nested u-net architecture for medical image segmentation. In Proc. Int. Workshop Deep Learn. Med. Image Anal. Int. Workshop Multimodal Learn. Clin. Decis. Support, pp. 3–11. Cited by: item 2, §II-C.
  • [22] J.Y. Zhu, T. Park, P. Isola, and A. Efros (2017)

    Unpaired image-to-image translation using cycle-consistent adversarial networks

    .
    In Proc. IEEE Int. Conf. Comput. Vis., pp. 2223–2232. Cited by: §II-B.