I Introduction
SAR image change detection technology is widely used in earth observation tasks such as environmental protection, urban research, and forest resource monitoring [1]. Compared with optical images, multitemporal SAR images are usually contaminated by inherent speckle noise, which impacts much negative effects on change detection algorithms [2][5].
Due to the extreme shortage of multitemporal SAR images with ground truth, supervised methods are limited in SAR image change detection. At present, unsupervised methods are widely studied and applied in this field. The main steps of unsupervised methods usually include: 1) Preprocessing; 2) difference image (DI) generation; 3) classification. A common method of generating DI is to calculate the logratio of the two SAR images pixel by pixel, the obvious disadvantage of which is the poor noise robustness. In classification, unsupervised clustering algorithms are applied at early stages. For example, means clustering [6], multiple kernel means clustering [7], and fuzzy means clustering [8] have been extensively studied to classify pixels or produce pseudolabel training samples to subsequent classifiers from a DI. However, these existing clustering algorithms have significant disadvantages. On one hand, these distancebased clustering algorithms are extremely sensitive to speckle noise, which often leads to treat changes caused by speckles as real terrain objects changes. On the other hand, most of the existing clustering algorithms assume that the changed and unchanged classes are balanced. In many cases, the pixels of changed class are far less than the those of unchanged class, which is typical imbalanced. Traditional clustering methods will cause excessive false alarms when faced with imbalanced data.
In recent years, deep learning exhibits excellent performance in pattern recognition. Many researchers have introduced deep learning into SAR image change detection and achieved superior performance. Gao
et al. use clusterbased PCANet [9] and CWNN [10] to exploit the key information, respectively. In [3], Geng et al. proposed a saliencyguided deep neural network for SAR image change detection. Li
et al. proposed a SAR image change detection method based on convolutional neural network (CNN) [11]. Recently, researchers have noticed the imbalance in SAR image change detection. Wang
et al. proposed an imbalanced learning method by morphologically supervised PCANet [12]. To our best knowledge, there are still few studies on the imbalanced SAR images change detection, it is a fact that deep learning methods have extremely high requirements on the quality of training samples. However, the problem is that, with strong speckle noise and imbalanced data, it is hard to obtain high reliable pseudolabel training samples for deep learning models. Apparently, high quality DI and effective clustering methods are benefit for this issue.In this research, a novel approach based on deep learning is proposed for imbalanced SAR image change detection. Especially, a novel DI generation method to get deep difference image (DDI) and a parallel FCM clustering (PFCMC) method are developed in the proposed approach. Our approach integrates DDI, PFCMC and PCANet to implement imbalance multitemporal SAR image with strong speckle noise change detection, which is abbreviated as DP_PCANet. Our main contributions are as follows:
1) Based on the ideas of convolution and pooling in deep learning, we developed weighted pooling and cumulative weighted pooling. This method of generating DDI can effectively suppress various speckle noise and enhance terrain changes than traditional method.
2) PFCMC method is developed for imbalanced SAR image based on combining nonlinear mapping, Gabor wavelets and basic FCM, it can provide more reliable pseudolabels training samples in the case of significantly imbalanced SAR images.
3) In PCANet classification, oversampling and undersampling are employed to reduce the negative impact of imbalanced data.
The remainder of this letter is organized as follows. Section II describes the proposed approach. Experimental results, comparison and analysis are in Section III. Finally, the conclusion is given in Section IV.
Ii Methodology
In this letter, the proposed approach mainly includes three parts: 1) generating a DDI; 2) parallel FCM clustering; 3) a PCANet with SVM are designed to classify pixels based on pseudolabel training samples. The flowchart of the proposed approach is illustrated in Fig. 1.
Iia Deep Difference Image Gerneration
In deep CNN, averagepooling and maximumpooling are important processes to extract deep features. Although averagepooling is effective in suppressing noise, it can also easily result in excessive loss of details information. As for maximumpooling, it can effectively enhance image features, however, it is more sensitive to noise. Motivated by the above ideas, we have proposed a weightedpooling method for generating a DDI based on the distance measure of the local window.
If the local window is size of , where
is odd , the weightedpooling kernel
is a matrix with the same size as shown in Equation (1). Each element value is determined by the distance between the current position and the matrix center, which is calculated by Equation (2).(1) 
(2) 
As a special case, the value of the element at the center of the matrix is defined as . After weightedpooling convolution, each pixel in the weighedpooling image can be represented by Equation (3), where is pixel in the original image
(3) 
Given two multitemporal SAR images and , where , convolution of each SAR image using the same weighted pooling kernel to obtain two pooled images and , where denote 2D convolution operation. Then, the logratio image is calculated as:
(4) 
The logratio operator has weak antinoise performance, there is still a certain degree of speckle noise in
. Based on the sparse distribution of noise outliers and the aggregated distribution of real change in logratio image, we apply weightedpooling operation with different local window size to
for times, obtaining pooled images. Then, all pooled images is accumulated to generate the DDI denoted by :(5) 
where and denotes a pooled image of after weightedpooling with parameter .The process of generating the DDI is exhibited in Fig. 2.
IiB Parallel FCM Clustering
In SAR image change detection, the number of changed pixels is often much less than that of unchanged pixels, The traditional clustering methods have poor clustering effect in case of imbalanced data.We develop parallel FCM clustering based on
mapping method to generate high quality pseudo labels. In order to enhance the difference of pixel categories and effectively reduce the impact of sample imbalance characteristics, the PFCM method first employs two sigmoid functions to map DDI to obtain two mapped images. The Sigmoid mapping function is with two parameters
and , and a variable is mapped by (6).(6) 
The Gabor wavelet transform is used to extract features from the two mapped images to obtain two sets of pixellevel feature vectors and
respectively, details of Gabor feature extraction can be referred in [9]. At last, FCM and simple coding method are used to get the final clustering results. The detailed descriptions of the parallel FCM algorithm are as follows:
1) Input: Deep difference image .
2) Step1: Obtain by normalizing and centralizing , apply two mapping to with two different parameter sets pixelbypixel to get and .
3) Step2: Perform Gabor feature extraction to and respectively. Two Gabor feature vector sets are obtained, which are denoted as and , where represents a Gabor feature vector.
4) Step3: FCM is utilized to perform two class clustering on and to get two label sets and respectively, where represents a label corresponding to a Gabor feature vector. The value of is 0 or 1. The simple averaging operation is used to encode the two label sets, obtaining the final label set , where .
5) Step4: If , assign the corresponding pixel to the changed class , if , assign the corresponding pixel to the unchanged class , the others are assigned to the intermediate class .
6) Output: The clustering result can be denoted by an image with labels {, , }.
The pixels belonging to
have the high probability to be changed, while the pixels belonging to
have the high probability to be unchanged. These two kinds of pixels can be chosen as samples for training PCANet model to classify those pixels belonging to .IiC PCANet Classification Model
PCANet is a kind of deep learning model with strong robustness to noise [13]. In this letter, a PCANet with the similar two stages structure in [9] and linear SVM are adopted as the classification model.
Patches of size are generated pixel by pixel on two weightedpooled SAR images and . The corresponding patches are connected to form a new patch set denoted as , where . Oversampling the pixels of and downsampling the pixels of to maintain a certain balance between the sample numbers. Randomly select imagepatch samples and vectorize them to obtain feature vectors, then subtract vector mean from each a vector and combine them to a new feature matrix . The expression for calculating the PCA filter is as:
(7) 
where is a function that maps to a matrix , and means the th
principle eigenvector of
. Then the th filter output of the first stage in PCANet is:(8) 
where denotes 3D convolution.
The first layer of PCANet calculation is completed to obtain . The process of the second stage is similar to the first stage, the same method is applied to each to get the second stage output where .
(9) 
We binarize outputs to obtain
, where is a Heaviside step function, whose value is one for positive and zero otherwise. Around each pixel of , the vector of binary bits can be converted into an integer value, the conversion formula is as follows:(10) 
The single integervalue image is obtained, and each pixel is an integer in the range . We further transform into a histogram denoted by . Then the PCANet feature of the imagepatch can be defined as
(11) 
Every imagepatch is processed by the above method to get PCANet feature, and the extracted features and clustering labels are fed into linear SVM to train model. Use the model to further classify the pixels belonging to into and . Reconstruct the labels by position to get the final changed map.
Iii Experimental Results and Analyses
Iiia Experimental Setup
In order to evaluate the performance of the proposed approach, we apply three real SAR datasets. The first is called the A dataset, which presents a section of two SAR images acquired by ERS2 SAR sensor over the city of San Francisco. Each of these images is size of . The second dataset and the third dataset are named the B and C dataset respectively, which presents two SAR images captured by the COSMOSkymed SAR sensor with a size of . The SAR images in B and C contain relatively strong speckle noise, and the areas of changed and unchanged in these SAR images are extremely imbalanced.
The following indicators are employed to evaluate the proposed approach: false positives (FP), false negatives (FN), percentage correct classification (PCC) and Kappa coefficient (KC). The FP denotes the number of unchanged pixels that are false detected, while FN refers the number of real changed pixels are detected as unchanged. True negatives (TN) indicates the unchanged pixels that are correctly detected, and true positives (TP) is the correct detection of changed pixels. Let Nc and Nu refer the number of truly changed and unchanged pixels of the ground truth respectively. We employ to measure the imbalance between the changed class and unchanged class. Overall errors (OE) is utilized to indicate the number of pixels for all detected errors. The PCC refers the ratio of correctly classified pixels to all pixels, which is expressed as . The KC is considered as an essential criterion, which is for consistency check. It is calculated as:
(12) 
(13) 
For comparison, four stateoftheart approaches are utilized on the above three datasets. These approaches are PCA_kmeans [14] , NR_ELM [15], Gabor_PCANet [9] and CWNN[10]. Moreover, the parameters of these methods are set with reference to the corresponding papers.
In the proposed DP_PCANet approach, we set and
, respectively. In order to reduce the number of hyperparameters, we define the relationship and constraint between the two sets of the center biases
in the function:.As mentioned in [9], the parameter of PCANet include imagepatch size , the filter numbers and , and the training samples . We set and the . At the stage of generating DDI and parallel clustering, we set for A; for B; for C.
IiiB Analysis of Result
Comparing (d) and (e) in Fig. 3, it can be found that The proposed method of generating DDI can effectively suppress speckle noise and enhance the characteristics of the real changed area for all three datasets. This is mainly because the weighted pooling has noise reduction on the original image, in addition, the image features are effectively retained. Then, the cumulated weightedpooling can further weaken the sparse highvalue noise on the logratio image, and further strengthen real changes. The DDI is significantly better than traditional DI visually.
The imbalanced ratio was calculated: , and . As can be seen from (f) in Fig. 3, the PFCMC method can effectively adapt to images of different imbalanced ratio, and clustering results image is satisfied for all three datasets.
In Table I, we exhibit the values of the abovementioned indicators to evaluate these change detection methods. The PCC and KC of DP_PCANet are 99.08% and 0.9296, respectively, which are slightly higher than 98.90% and 0.9164 of Gabor_PCANet, 98.94% and 0.9231 of CWNN. It demonstrates that the change detection result of DP_PCANet is almost identical to the Ground truth. For the dataset B with strong speckle noise, the PCC of DP_PCANet reaches 99.52%, the KC reaches 0.7177. For C dataset, the PCC of DP_PCANet reaches 98.65% and the KC is 0.6081. The indicators of our proposed algorithm are higher than other methods on three datasets. Comparing the change detection results of these five approaches in Fig. 3, we draw the conclusion that proposed approach is superior to other methods. Above results illustrates that our algorithm’s robustness and generalization are significantly better than other methods on the three sets of data, and have extremely strong detection performance.
The degree of imbalance in the three datasets is quite different, the imbalance of dataset B is very serious, especially. Various imbalance in these datasets has a strong negative effect on the clustering accuracy and the performance of the deep learning classification model. Due to imbalanced characteristic, the other four methods identify large number of unchanged pixels as changed pixels, leading to great false alarms. Compared with other methods, the FP of our proposed approach are 221, 266, and 585 on the three sets of data. DP_PCANet has the lowest FP and maintains the lowest OE, while keeping the highest PCC and KC. Experimental results demonstrate that DP_PCANet has strong adaptive ability and generalization performance on imbalanced multitemporal SAR image change detection tasks.
Model  Results on the A dataset  

FP  FN  OE  PCC  KC  
PCA_kmeans  13496  218  13714  0.7907  0.3170 
NR_ELM  10137  10  10147  0.8452  0.4161 
Gabor_PCANet  311  409  720  0.9890  0.9164 
CWNN  545  150  695  0.9894  0.9231 
DP_PCANet  221  382  603  0.9908  0.9296 
Model  Results on the B dataset  
FP  FN  OE  PCC  KC  
PCA_kmeans  60119  241  60360  0.6228  0.0220 
NR_ELM  20721  236  20957  0.8690  0.0912 
Gabor_PCANet  23761  131  23892  0.8507  0.0862 
CWNN  7194  417  7611  0.9524  0.2077 
DP_PCANet  266  503  769  0.9952  0.7177 
Model  Results on the C dataset  
FP  FN  OE  PCC  KC  
PCA_kmeans  62095  180  62275  0.6108  0.0567 
NR_ELM  15714  1121  16835  0.8948  0.1885 
Gabor_PCANet  20860  695  21555  0.8653  0.1733 
CWNN  7854  898  8752  0.9453  0.3487 
DP_PCANet  585  1663  2248  0.9860  0.6092 
IiiC Analysis of Parameter b and T
We choose a set of to explore the relationship between the cumulative times and the performance of DP_PCANet, is adopted as the evaluation indicator. The same way is also used to analyze the center bias . The results are exhibited in Fig. 4.
From Fig. 4, it can be found that the parameter and have little effect on the dataset A, our algorithm can always maintain excellent performance for data A, the highest is 0.9296 when and . For dataset B and C with strong speckle noise, the result becomes better as increases. When reaches 9 and more, the tends to be stable and maintain a highelevel. The above experimental results prove that more cumulative times has a stronger effect on reducing speckle noise and retaining the true change. In theory, the benchmark of center bias is 0. The clustering algorithm has no preference of any kind when . Experimental results show the and are 98.76%, 0.5542 for data B and 98.60%, 0.5549 for data C when , it explains the benchmark parameters allow DP_PCANet to show good performance in most data. there is a most suitable center biases for SAR data with different characteristics, It is worth further research to automatically select appropriate .
Iv Conclusion
In this letter, we have proposed a robust imbalanced multitemporal SAR change detection approach. The proposed DDI generation method can effectively reduce speckle noise and enhance features. A parallel FCM clustering method was developed to increase the gap between the change class and the unchanged class, which can obtain excellent clustering performance under imbalanced data, thereby providing highly reliable pseudolabel training samples. Oversampling and undersampling were employed to mitigate the imbalance effects on PCANet. Experiment results confirmed the effectiveness, generalization and robustness of the proposed approach.
References
 [1] M. N. Sumaiya and R. Shantha Selva Kumari, “Logarithmic meanbased thresholding for SAR image change detection,” IEEE Geosci. Remote Sens. Lett., vol. 13, no. 11, pp. 1726–1728, Nov. 2016.
 [2] M. Gong, Z. Zhou, and J. Ma, “Change detection in synthetic aperture radar images based on image fusion and fuzzy clustering,” IEEE Trans. Image Process., vol. 21, no. 4, pp. 2141–2151, Apr. 2012.
 [3] J. Geng, X. Ma, X. Zhou and H. Wang, “Saliencyguided deep neural networks for SAR image change detection,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 10, pp. 7365–7377, Oct. 2019.
 [4] M. Gong, J. Zhao, J. Liu and Q. Miao, “Change detection in synthetic aperture radar images based on deep neural networks,” IEEE Trans. Neural Netw., vol. 27, no. 1, pp. 125–138, Jan. 2016.
 [5] L. Jia, M. Li, Y. Wu, P. Zhang, H. Chen, and L. An, “Semisupervised SAR image change detection using a clusterneighborhood kernel,” IEEE Geosci. Remote Sens. Lett., vol. 11, no.8 pp. 1443–1447, Aug. 2014.

[6]
Y. Zheng, X. Zhang, B. Hou, and G. Liu, “Using combined difference image and kmeans clustering for SAR image change detection,”
IEEE Geosci. Remote Sens. Lett., vol. 11, no. 3, pp. 691–695, Mar. 2014.  [7] L. Jia, M. Li, P. Zhang, Y. Wu, and H. Zhu, “SAR image change detection based on multiple kernel kmeans clustering with localneighborhood information,” IEEE Geosci. Remote Sens. Lett., vol. 13, no. 6, pp. 856862, Jun. 2012.
 [8] H. Li, T. Celik, and W. J. Emery, “Gabor feature based unsupervised change detection of multitemporal SAR images based on twolevel clustering,” IEEE Geosci. Remote Sens. Lett., vol. 12, no. 12, pp. 2458–2462, Dec. 2015.
 [9] F. Gao, J. Dong, B. Li, and Q. Xu, “Automatic change detection in synthetic aperture radar images based on PCANet,” IEEE Geosci. Remote Sens. Lett., vol. 13, no. 12, pp. 1792–1796, Dec. 2016.
 [10] F. Gao, X. Wang, Y. Gao, J. Dong , and S. Wang, “Sea ice change detection in SAR images based on convolutionalwavelet neural networks” IEEE Geosci. Remote Sens. Lett., vol. 16, no. 8, pp. 1240–1244, Aug. 2019.
 [11] Y. Li, C. Peng, Y. Chen, and L. Jiao, “A deep learning method for change detection in synthetic aperture radar images,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 8, pp. 5751–5763, Aug. 2019.
 [12] R. Wang, J. Zhang, J. Chen, and L. Jiao, “Imbalanced learningbased automatic SAR images change detection by morphologically supervised PCANet,” IEEE Geosci. Remote Sens. Lett., vol. 16, no. 4, pp. 554–558, Apr. 2019.
 [13] T. H. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, and Y. Ma, “PCANet: A simple deep learning baseline for image classification?,” IEEE Trans. Image Process., vol. 24, no. 12, pp. 5.17–5.32, Dec. 2015.

[14]
T. Celik, “Unsupervised change detection in satellite images using principal component analysis and kmeans clustering,”
IEEE Geosci. Remote Sens. Lett., vol. 6, no. 4, pp. 772–776, Oct. 2009.  [15] F. Gao, J. Dong, B. Li, Q. Xu, and C. Xie, “Change detection from synthetic aperture radar images based on neighborhoodbased ratio and extreme learning machine,” J. Appl. Remote Sens., vol. 10, no. 4, Dec 2016, Art. no. 084684.
Comments
There are no comments yet.