Magnetic resonance imaging (MRI), as an attractive non-invasive medical imaging technique, plays an important role on the diagnosis of pathological and physiological alterations in living tissues and organs of human body, as it can provide high resolution three dimensional (3D) images with anatomical and functional contrast due to different MR measurable tissue parameters. The quality of MR images can, however, easily be degraded by random noise generated during image acquisition. Increased noise level can dramatically affect the accuracy of diagnoses and also the reliability of quantitative image processing including segmentation, registration and classification. To avoid these problems, it is therefore essential to remove the noises of MR images (image denoising) before implementing further image processing.
In the past years a number of image denoising methods have been reported. Martin-Fernandez and Villullas Martin-Fernandez2015
applied a new method with shrinkage of wavelet coefficients based on the conditioned probability of being noise. The involved the parameters are calculated by means of the expectation maximization method. In addition, Change et al.Chang2015 took full use of the block representation advantage of nonlocal means 3D method (NLM3D) to restore the noisy slice from different neighboring slices and employed a post processing step to remove noise further while preserving the structural information of 3D MRI. Zhang et al. Zhang2015
investigated the application and improvement of higher-order singular value decomposition (HOSVD) to denoise MR volume data and achieved comparable performance to that of block-matching 4D method (BM4D). Manjón further et alManjon2015 proposed a novel method for MRI denoising that exploits both the sparseness and self-similarity properties of MR images with the state-of-art performance. Baselice et al.Baselice2017 exploited Markov random fields to achieve the combination of details preservation and noise reduction without any supervision. Bhujle and Chaudhuri Bhujle2013 carried out nonlocal means denoising on squared magnitude images and thus compensated the introduced bias. Chang and Change Chang2015a applied an artificial neural network associated with image texture feature analysis to establish a predictable parameter model which is able to automate the denoising procedure. Golshan and Hasanzadeh Golshan2015 proposed a new filtering method based on the linear minimum mean square error (LMMSE) estimation, which employs the self-similarity property of MR data to restore the noise-less signal. Varadarajan and Haldar Varadarajan2015 described a novel majorize-minimize framework that allows penalized maximum likelihood estimates obtained by solving a series of much simpler regularized least-squares surrogate problems. While robust denoising performances are usually able to be achieved, most of these methods, however, include complex optimization processes for images Zhang2017a and therefore are time-consuming. Furthermore, these methods generally involve the optimization of a non-convex cost function and requires a manual parameter selection, meaning that the above methods usually need a procedure to estimate the noise level parameter.
To avoid optimization in test phase and directly apply discriminant learning, in this study we extended feed-forward denoising conventional neural networks (DnCNNs) which was originally proposed by by Zhang et al.Zhang2017a to restore the noise-free MR images from the noisy ones. To adapt to 3D volume, we applied a multi-channel approach for learning that we hypothesize will a faster and stabler training as well as a more robust denoising performance. To validate these, four other state of art denoising methods were also employed and compared with our proposed method. To our best knoweldge, this paper shows the first deep learning based method to denoise MR images with Rician noise.
The DnCNNs proposed by Zhang et al. Zhang2017a is developed from a VGG network to make it suitable for image denoising. To apply this model in 3D data, we used the multi-channel version of DnCNNs (MCDnCNN). The goal of this network is to compute the noise-free image. He et al. He
pointed out that the network must preserve all input details since the image is discarded and the output is generated from the learned features alone. With many weight layers, this becomes an end-to-end relation requiring very long-term memory. For this reason, the vanishing/exploding gradients problem can be critical. It was shown thatHe the residual networks are easier to be optimized, and can gain accuracy from considerably increased depth. Therefore rather than directly outputing the noise-free image, the network is designed to predict the noise at each pixel of the original image . Therefore, the proposed DnCNN implicitly removes the latent clean image with the operations in the hidden layers. The noise free intensity can be computed as .
This network, as illustrated in Fig 1
, consists of one input layer of convolution with rectified linear unit (ReLU), eight layers of convolution with batch normalization and RelU and one output layer of convolution. The output layer generates an image with the same size as the input. This network does not include any max-pooling layer as the output should have the same size as the input. For the first layer, we use 64 kernels of size 3x3x. For grey level 2D image, the can be set to 1. In our application, we are dealing with MR volume. One option is to process the volume slice by slice and is 1 accordingly. In this study, to make use of context information across neighboring slices of a particular slice , we set the size of the input to be by by 5. and are the size of each 2D slice and the number of neighboring slices including slice is 5. The kernels of following layers including the last layer are also set to have a size of 3x3x64. Two key featuresZhang2017a of this network, residual learning formulation and batch normalization, are incorporated to speed up training as well as to boost the denoising performance.
To optimize parameters of this proposed residual network, the averaged mean square error between the desired residual image and the estimated ones from the noisy input is calculated as the loss function of the networks:
After computing the noise for every slice across the volume we can restore the original noise free volume.
So far, most existing state-of-the-art denoising methods depends on a fixed noise level. When applied to a MRI volume with unknown level of Rician noise, the common way is to first estimate the noise level parameter, and then denoise using the parameter. This makes the denoising results affected by the accuracy of noise estimation. To demonstrate the capability of DnCNN in general image denoising, we conducted two different training approaches. We first trained the DnCNN model on a specific noise level (MCDnCNNs) to evaluate the superior performance of our proposed deep learning approach compared with the-state-of-the-art Rician noise denoising approaches. Second, we extended our DnCNN model to a general model (MCDnCNNg) for Rician denosing with unknown noise level.
2.2 Comparison with other methods
To evaluate the denoising performance of our method on 3D MR images, four well-established denoising methods have also been tested and compared in this study.
2.2.1 optimized block-wise non-local means (NLM) denoising filter (Coupe)
This method was developed by Coupe P et al. Coupe2008 , based on the classic NLM denoising filter. Four new features, including a fully-automated tuning of the smoothing parameter, a selection of the most relevant voxels, a blockwise implementation and a parallelized computation have been combined into this filter, resulting in a better denoising performance with a significant decrease of computation time.
2.2.2 3D wavelet subbands mixing method (WSM)
This method was an extension of the optimized block-wise NLM filter mentioned above. Coupe P et al Coupe2008a proposed a mixing of a 3D sub-bands wavelet on this filter to further improve the performance of image noise removal, while keeping the computation time comparable.
2.2.3 3D oracle-based discrete cosine transform (DCT) filter (ODCT3D)
The local DCT denoising filter was adaptedManjon2012a to deal with Rician noise by using a pseudo-oracle principle. Compared with the two filters mentioned above, improved denoising results have been shown by using this filter with much shorter computation time (81s, 110s vs 10s).
2.2.4 3D prefiltered rotationally invariant nonlocal means filtering (PRI_NLM3D)
3D PRINLM filter is a new rotationally invariant version of the original NLM filter. It was developed Manjon2012a with the usage of the prefiltered image obtained by the above-mentioned DCT denoising. Compared to the other three filters, PRINLM filter was reported to have the best denoising performance with less than 1 minute processing time.
2.3 MR image acquisition and evaluation
We have applied our training and test scheme on three subsets from two public datasets: IXI dataest and Brainweb datasets. One critical problem of deep learning approach is the weak general applicability. Networks trained on one dataset from a specific manufacture or setting may not perform well in a different dataset. To investigate the general applicability of our network, we collected different datasets in which one was used for training on one dataset while the other datasets were tested.
2.3.1 IXI-Hammersmith dataset
Hammersmith dataset is a subset of IXI dataset(http://brain-development.org/ixi-dataset/). The data was acquired in Hammersmith Hospital using a Philips 3T system (T1 parameters; Repetition Time = 9.60; Echo Time = 4.60; Number of Phase Encoding Steps = 208; Echo Train Length = 208; Reconstruction Diameter = 240.0; Acquisition Matrix = 208 x 208; Flip Angle = 8.0).
In total 30 sets of T1 weighted MR brain images from healthy subjects were randomly selected from IXI Database . 20 out of 30 images were used for training the neuralnetwork with the left 10 images for evaluating the described methods.
For Rician denoising model training with either known or unknown noise level, we set the patch size as 60 x 60, and use sliding window strategy to obtain 150,000 patches to train the corresponding model. We use Adam for stochastic optimization and a mini-batch size of 64. We train 50 epochs for our two different models. The learning rate was decayed exponentially from 1e−1 to 1e−4 for the 50 epochs. MatConvNet package was used to train the proposed models.
2.3.2 IXI-Guys dataset
The second dataset is Guys dataset with 10 images acquired at Guy’s Hospital using a Philips 1.5T system(T1; Repetition time = 9.813; Echo time = 4.603; Number of Phase Encoding Steps = 192; Echo Train Length = 0; Reconstruction Diameter = 240; Flip Angle = 8).
2.3.3 Brainweb dataest
The second dataset is Simulated Brain Database (http://brainweb.bic.mni.mcgill.ca/brainweb/). The SBD contains a set of realistic MRI data volumes produced by an MRI simulator，which were widely used to evalulate the performance of donoising approachesFeng2016 ; Zhang2014 ; AksamIftikhar2014 ; Coupe2008b .
To evaluate the performance of the denoising from our proposed methods and other methods, we applied these methods on the data with different noise level from 1% to 15% with an increase of 2%. We use two quantitative measuresZhang2015 . One is the peak signal to noise ratio (PSNR) which is defined as:
where is the root mean square error between denoised data and the noise-free data. The second measure is the global of structure similarity index (SSIM) which is the average of structure similarity index () Manjon2012 defined as :
where and are the mean of the data and and and are constants. and
are the variances andis the covariance of and . has been computed with a 3x3x3 voxel kernel.
3.1 Results from IXI-Hammersmith dataset
Tables 1 and 2 and Figure 2 summarize the PSNR and SSIM measures in IXI-Hammersmith dataset with different methods . The noise-specific model MCDnCNN (MCDnCNNs) significantly outperformed all other methods in terms of both and . Second only to MCDnCNNs, our proposed model MCDnCNN for general noise is also better than other compared methods in terms of from noise level 1% to 15%. and from noise level 5% to 15%. .
Figure 3 shows an example of denoising result from different methods on data with 15% noise. Visually, dedicated 15% noise-specific model MCDnCNNs gives the most clean result. All other methods did not remove noise in the background sufficiently. In the brain foreground, both MCDnCNNs and MCDnCNNg achieve visually good result. All other methods removed noise but also lost details of the image.
3.2 Results from IXI-Guys dataset
Figure 4 summarizes the PSNR and SSIM with different methods from Guys dataset. Although we did not train on this dataset, the results shows that our two models still achieved a good performance. The noise-specific model MCDnCNNs shows most robust performance compared to all other methods in terms of and . Next to MCDnCNNs, our proposed model MCDnCNNg for general noise is also significantly better than other compared methods in terms of from noise levels 3% to 13%. However in terms of , from noise levels 3% to 15%, ODCT3D and PRI_NLM3D opposed our general model. It should be noted that both MCDnCNNg and MCDnCNNs are not retrained on this dataset.
Figure 5 shows an example of denoising result from different methods on data with 15% noise from Guys dataset. We can see that the effect is similar to that from the IXI-Hammersmith dataset.
3.3 Results from Brainweb dataset
Figure 6 summarizes the PSNR and SSIM in Brainweb dataset from different methods. In analogous to the finding shown above, the noise-specific model MCDnCNN (MCDnCNNs) significantly outperforms all other method in terms of and . Second only to MCDnCNNs, our proposed model MCDnCNN for general noise (MCDnCNNg) is also significantly better than other compared methods in terms of and from noise levels 5% to 15%.
Figure 7 shows an example of denoising result from different methods on data with 15% noise from Brainweb dataset. We can see that the effect is similar to that from the IXI-Hammersmith dataset.
4 Discussion and Conclusion
In this paper, we present a novel approach to denoise robustly MR images based on the deep convolutional feed forward neural Zhang2017a . This net is multi-channel-based combined with residual learning. Our proposed noise-specific feed forward denoising model outperforms all other methods in term of the peak signal to noise ratio and the global of structure similarity index in all our three test datasets. Moreover, our general noise-applicable model is also better than the other compared methods in two datasets. The reason why in one dataset the general model achieves not qualified performance is probably because the model needs to be retrained. In summary, testing with datasets including IXI-Hammersmith, IXI-guys and Brainweb data, our general model shows good general applicability and compare favorably to other conventional methods.
The advantage of our proposed method is that the new training scheme is within an unified framework. Our general denosing model is trained with the noisy images from a wide range of noise levels. To test a given image, this model can be applied without assuming or estimating its noise level in advance. Although the performance of our general noise-level model is slightly worse than our noise-level specific model (each model was trained with images with a specific noise level). The general model however can be applied for MR images directly out-of-box in real practice. It should be noted that all other compared methods need parameter-tuning for a specific noise level. Before applying the most robust denosing methods, the noise level on a specific image needs to be estimated and this estimation may affect denosing effect. To improve the performance, the general model can be further trained and tuned in the corresponding dataset.
In this paper, we demonstrated that our denoising model trained from one MR dataset with 3T can be directly applied on other two datasets with a different magnetic field strength and the performance of our noise-specific deep-learning model outperforms that from other methods in the other two datasets. It should be noted that the Brainweb dataset used here is simulated noise-free data and results from this dataset is most convincing. This validates that the general applicability of the deep-learning based denoise model.
In this paper, a deep convolutional neural network was proposed for image denoising. The max-pooling is not used to keep output the same size to that of input image. In the future, we are going to explore other possibilities such as deep convolutional generative adversarial networks(DCGAN)Denton2015 ; Goodfellow2014 ; Halbritter2017 and u-net. In u-net, although max-pooling is used, at the same network one upsampling scheme is applied to restore the size of the output image.
In our future work, instead of making use of a multi-channel network, we would investigate the effect of feeding the whole 3D volume to a complete 3D convolutional neural network. We would also like to explore the possibilities of extending this proposed method to remove the image noise in lung, cardiac or abdominal MRI studies.
- (1) M. Martin-Fernandez and S. Villullas, “The em method in a probabilistic wavelet-based mri denoising,” Computational and mathematical methods in medicine 2015 (2015).
- (2) L. Chang, G. ChaoBang, and Y. Xi, “A mri denoising method based on 3d nonlocal means and multidimensional pca,” Computational and mathematical methods in medicine 2015 (2015).
- (3) X. Zhang, Z. Xu, N. Jia, W. Yang, Q. Feng, W. Chen, and Y. Feng, “Denoising of 3d magnetic resonance images by using higher-order singular value decomposition.,” Medical image analysis 19, 75–86 (2015).
- (4) J. V. Manjón, P. Coupé, and A. Buades, “Mri noise estimation and denoising using non-local pca,” Medical image analysis 22(1), 35–47 (2015).
- (5) F. Baselice, G. Ferraioli, and V. Pascazio, “A 3d mri denoising algorithm based on bayesian theory.,” Biomedical engineering online 16, 25 (2017).
- (6) H. V. Bhujle and S. Chaudhuri, “Laplacian based non-local means denoising of mr images with rician noise.,” Magnetic resonance imaging 31, 1599–1610 (2013).
- (7) Y.-N. Chang and H.-H. Chang, “Automatic brain mr image denoising based on texture feature-based artificial neural networks.,” Bio-medical materials and engineering 26 Suppl 1, S1275–S1282 (2015).
- (8) H. M. Golshan and R. P. R. Hasanzadeh, “An optimized lmmse based method for 3d mri denoising.,” IEEE/ACM transactions on computational biology and bioinformatics 12, 861–870 (2015).
- (9) D. Varadarajan and J. P. Haldar, “A majorize-minimize framework for rician and non-central chi mr images.,” IEEE transactions on medical imaging 34, 2191–2202 (2015).
- (10) K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising.,” IEEE transactions on image processing : a publication of the IEEE Signal Processing Society 26, 3142–3155 (2017).
- (11) K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition. arxiv 2015,” arXiv preprint arXiv:1512.03385 .
- (12) P. Coupé, P. Yger, S. Prima, P. Hellier, C. Kervrann, and C. Barillot, “An optimized blockwise nonlocal means denoising filter for 3-d magnetic resonance images,” IEEE transactions on medical imaging 27(4), 425–441 (2008).
- (13) P. Coupé, P. Hellier, S. Prima, C. Kervrann, and C. Barillot, “3d wavelet subbands mixing for image denoising,” Journal of Biomedical Imaging 2008, 1 (2008).
- (14) J. V. Manjón, P. Coupé, A. Buades, D. L. Collins, and M. Robles, “New methods for mri denoising based on sparseness and self-similarity,” Medical image analysis 16(1), 18–27 (2012).
- (15) A. Feng, J. Peng, and X. Wu, “Global denoising for 3d mri,” in Eighth International Conference on Digital Image Processing (ICDIP 2016), 100331Q–100331Q, International Society for Optics and Photonics (2016).
- (16) X. Zhang, G. Hou, J. Ma, W. Yang, B. Lin, Y. Xu, W. Chen, and Y. Feng, “Denoising mr images using non-local means filter with combined patch and pixel similarity.,” PloS one 9, e100240 (2014).
- (17) M. Aksam Iftikhar, A. Jalil, S. Rathore, and M. Hussain, “Robust brain mri denoising and segmentation using enhanced non-local means algorithm,” International Journal of Imaging Systems and Technology 24(1), 52–66 (2014).
- (18) P. Coupé, P. Yger, S. Prima, P. Hellier, C. Kervrann, and C. Barillot, “An optimized blockwise nonlocal means denoising filter for 3-d magnetic resonance images,” IEEE transactions on medical imaging 27(4), 425–441 (2008).
- (19) J. V. Manjón, P. Coupé, A. Buades, D. Louis Collins, and M. Robles, “New methods for mri denoising based on sparseness and self-similarity.,” Medical image analysis 16, 18–27 (2012).
- (20) E. L. Denton, S. Chintala, R. Fergus, et al., “Deep generative image models using a￼ laplacian pyramid of adversarial networks,” in Advances in neural information processing systems, 1486–1494 (2015).
- (21) I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, 2672–2680 (2014).
- (22) S. Halbritter, “Generative adversarial networks,” (2017).