Research Article | Open Access
An Improved Image Denoising Model Based on Nonlocal Means Filter
The nonlocal means filter plays an important role in image denoising. We propose in this paper an image denoising model which is a suitable improvement of the nonlocal means filter. We compare this model with the nonlocal means filter, both theoretically and experimentally. Experiment results show that this new model provides good results for image denoising. Particularly, it is better than the nonlocal means filter when we consider the denoising for natural images with high textures.
Let be a noisy image that we want to deal with. The aim of image denoising is to recover the original image from a noisy image , where is the noise. This is an inverse problem, and in general there is no solution. Nevertheless, there are many useful methods to deal with this problem, for example, the median filter, the Wiener filter, the bilateral filter, variational methods [1–5], sparse representation based methods , wavelet-based methods , the convolutional neural network based methods [8–10], nonlocal self-similarity based methods [11–20], and collaborative filtering methods [12–15, 21–23].
Among all, nonlocal self-similarity models received recently much attention and play an important role in image processing. K. Dabov et al.  proposed the sparse 3D transform-domain collaborative filtering (BM3D) method, in which one exploits the inherent nonlocal self-similarity property of natural images, groups similar 2D image fragments into 3D groups, and then deals with these 3D groups by the developed collaborative filtering procedure. The BM3D and its variants proposed in [15, 21–26] have been demonstrated to achieve state-of-the-art performance. In [21, 22], the BM3D is applied for coherent noise reduction in digital holography. In , an iterative algorithm combining BM3D technique was proposed for the salt and paper (S&P) noise removal. In , V. Katkovnik et al. introduced the CD-BM3D and iterative CD-BM3D algorithms based on BM3D for interferometric phase image estimation. The nonlocal means filter (NLM) proposed by Buades et al. , known for its simplicity and excellent ability to efficiently remove additive white Gaussian noise (AWGN) while preserving edges, has triggered an important and continuous development in image processing. Not only lots of the NLM’s modified versions, the mixed schemes integrating NLM idea, and NLM-based iterative approaches are applied to remove AWGN, but also other kinds of noise, such as signal-dependent nonadditive speckle noise , additive mixed noise , and S&P noise . Existing NLM-based iterative approaches have also been shown to provide better results than their predecessors [17–19]. In , MRI deniosing was performed by a union method (the Wiener enhanced NLM), in which the noisy MR image was deniosed successively by the traditional median filter, the Wiener filter, and the NLM filter. The experiments demonstrated that the union method obtained better results than the NLM for denoising MRI corrupted by AWGN . H.Y. Liu et al.  proposed a two-steps iterative regularized denoising approach based on the NLM and the total variation. First the prefiltered image was obtained by using the NLM with the improved similarity weights. Then, they presented a modified TV model based on the prefiltered image to achieve final denoised results. X.T. Wang et al.  proposed an iterative nonlocal means filter for S&P noise removal (S&P-INLM). S&P-INLM includes three stage operations. First, a spatial hard threshold method is used to mark the positions of noisy pixels and noise-free pixels of an image corrupted by S&P noise, and then the noisy pixels in the image are prefiltered by a median filter while noise-free pixels are left unchanged. The prefiltering is used to make the noise distribution in the prefiltered image close to the Gaussian distribution. Finally, the noisy pixels in the prefiltered image are filtered by the proposed NLM-based iterative algorithm , while noise-free pixels are not filtered. The scheme proposed in  is based on the detection results of noisy pixels’ position and has achieved good results in removing S&P noise. However it is not suitable for the white Gaussian noise removal since the positions of the noisy pixels corrupted by white Gaussian noise are hard to detect.
As mentioned above, in image processing, nonlocal self-similarity based methods play an important role in image denoising. Recently, the BM3D and its modifications have been shown to outperform the NLM in many cases. Nevertheless, the NLM is still popular due to its advantages that the algorithm is simple and has good denoising performance while preserving edges. In this paper, we want to present an improved NLM-based approach for removing AWGN from noisy natural images, which is different from previous NLM-based iterative schemes and NLM modifications. Our proposed model has better denoising performance than the NLM for natural images with high textures, while its computation is almost as simple as the NLM.
Motivated by the Yaroslavsky filter , Buades, Coll and Morel proposed the original NLM . This model is given by where , , and is defined byHere is a Gaussian kernel with standard deviation , and in (1) is defined byWe defineas a weight that measures the similarity of between a neighborhood of and a neighborhood of . The parameter in (4) controls the decay of the weight. In order to ensure that only pixels with a similar neighborhood have a large weight, usually corresponds to the standard deviation of the image noise. Formula (1) is equivalent to where is defined by is the denoised image for . In many applications one gets a very good denoised image , though this model is very simple. This indicates that the weight captures the feature of images very well. There are other choices of the weight . Here, we just use this weight.
Numerically the nonlocal means filter could be written asif we denote a digital image by a function .
Variational methods also work quite well for image denoising. In variational methods one considers the minimizer of a suitable functional, which usually satisfies a partial differential equation. Then we use the minimizer as the denoised image. This is one way to study the inverse problem mentioned at the beginning of this paper. The most popular functional used in image processing are the Dirichlet functionaland the TV functionalAccording to these functionals there are models and TV models for image denoising. Particularly the TV model proposed by Rudin, Osher, and Fatemi  is very useful in image processing.
Motivated by the work of Kindermann, Osher, and Jones , Gilboa and Osher [20, 29] wanted to combine the TV model and the nonlocal means filter into a nonlocal TV model, so that the corresponding model is just a regularization of the nonlocal means filter. They realized this idea successfully, by introducing suitable nonlocal operators, which was motivated by the work of Zhou-schölkopf [30, 31]. Having nonlocal operators, it is natural to propose a nonlocal version of the ROF model, the NLTV model , and other some nonlocal models. The nonlocal models they proposed are very useful in image processing. Their models have been used and extended in various problems; see, for example, [32–40].
Motivated by the work of Gilboa-Osher and the work in geometric analysis (see, for example, ), Jin-Jost-Wang proposed new nonlocal operators . These new nonlocal operators match the geometric structure of images well by using the weight . With these new nonlocal operators, they proposed a new nonlocal model . The new nonlocal model provides good denoising results. By experiments we found that with only a few iterations in this nonlocal model one can already obtain very good results for image denoising. This motivates us to propose in this paper an improved nonlocal means filter model (INLM)Here is defined by (5) and denotes the twice use of the nonlocal means filter .
The computation of INLM is almost as simple as the NLM. Our experiments show that the INLM model is much better than and also better than the nonlocal means filter (NLM) for many images with higher textures.
2. A New Nonlocal Model and Its Discrete Version
In this section we first briefly recall this new nonlocal variational setting and then present its discrete version, especially the discrete version of the new nonlocal model. Motivated by this model we propose at the end of this section our model as an improved nonlocal means filter.
2.1. A New Nonlocal Model
Let be a noisy image that we want to consider and the weight , computed by (4), which measures the similarity of images. Let be defined by (6). Motivated by differential geometry, in  the authors used ω and to define the L2-norms for functions and vector field . First one defines a scalar product for functions with respect to byHence the -norm for functions is defined by For vector fields one defines a scalar product with respect to byfor any pair of vector fields . The difference vector field of a function (an image) is defined bywhich is different from the nonlocal gradient vector field given in [20, 29, 30]. Here one views as a metric. A map is viewed as a vector field. Using and one can define the divergence operator byThen one can check that for any and From this variational setting it is natural to define a (nonlocal) Dirichlet functional and a (nonlocal) Laplacian of functions which can also be defined directly from the difference operator and the divergence operator div. This functional (18) is a regularization of the nonlocal means filter, which is the same as the functional obtained by Gilboa and Osher. However, the viewpoint of Jin-Jost-Wang is different from that of Gilboa and Osher. This difference leads Jin-Jost-Wang to introduce a different nonlocal TV model and model in . Here we only consider the model , The difference to the nonlocal setting of Gilboa-Osher appears only in the fidelity term, where a different definition of -norm of a function is used. This difference makes (though slightly) different between Gilboa-Osher’s nonlocal model and Jin-Jost-Wang’s model. It was showed that Jin-Jost-Wang’s nonlocal model is more closer the nonlocal means filter than Gilboa-Osher’s nonlocal model. The Euler-Lagrange equation of (20) isor equivalently
2.2. Its Discrete Version
In this subsection, we present the discrete version of the new nonlocal model, which has its own interest. Now we represent a digital image by a (discrete) function , where is a set . Hence is a digital image with pixels . For simplicity, we denote by for . The weight is denoted by , an symmetric matrix with nonnegative entries. Then is given bywhich is a discrete version of . For a digital image we define its weight -norm by This is the discrete version of (13). For a pair of we define a scalar product byThen the difference vector field defined above is given bywhich is a discrete version of (15). As above a map from is considered as a vector field. For a pair of vector fields we define a scalar productFrom the scalar product we have a weight -norm of , which is given byNow for a vector field , its divergence is defined byOne can then check as above: for any and This means that div is the adjoint operator of . Now with the gradient operator and the divergence operator we can define the (normalized) Laplacian of function as above by (19)or equivalently,where is the nonlocal average of A (discrete) function satisfying , or equivalent , is called a harmonic function. It is easy to check that a harmonic function is a critical point of the following nonlocal Dirichlet energy: This functional is the discrete version of functional (18), which is viewed as a regularization of the nonlocal means filter. Now the new nonlocal model has the following discrete version:One can easily check that the minimizer (critical) point of satisfieswhich is the discrete version of (22). To find a minimizer , the following heat equation is considered in which is the (negative) gradient flow of in the space of functions with the -norm with respect to . The discrete version of (37) iswhere . The initial condition isIf , then the coefficients in (38) are nonnegative. Therefore we have the following.
Proposition 1. This algorithm is stable, provided that
2.3. An Improved Nonlocal Means Filter Model
In  Jin-Jost-Wang presented experiments to show that their new nonlocal model is very good in image denoising. It was observed in  that with a few iterations one can already get good results. In this paper we choose and and iterate (38) twice. For the first iteration of (38) we haveThen iterating formula (38) again we obtainThis is the model we want to propose in this paper. Let us rewrite it in the following form:Recall that is the twice use of nonlocal means filter (NLM2). Our experiments show that our model (43) is much better than and also better than the nonlocal means filter for many images as shown in our experiments.
For experiments, the first task is how to compute the weight . We compute it by following the method given in . For a given image one computes the weight by using the difference of patches around each point (pixel). The patch of size around is given bywhere is an odd integer. Let be the Gaussian weight Euclidean distance, i.e.,where is the discrete version of Now the weight we will use is computed by
In the computation of the weight we use , i.e., patch of size . The search window as mentioned above is of size .
Give a noisy image , where is the ordinary true image and is a random noise (white Gaussian noise) with mean zero and standard deviation . In order to compare different models, we use the peak signal-to-noise ratio (PSNR) to measure images and compare noisy images and denoised images. The peak signal-to-noise ratio (PSNR) is defined bywhere is the maximum possible pixel value of the image . MSE is the mean squared error defined byHere denotes the denoised image from the noisy image .
3.1. On Test Images
The images we use are taken from USC-SIPI image database with the same name and three standard test images (Cameraman, Barbara, and Lena). The three standard test images and USC-SIPI images database are widely used in the image processing literature, for example, [2, 11, 20, 36, 40]. All images in this experiment are of size .
The parameter in (47) usually corresponds to the standard derivation of the noise. If is small, noise remains in the denoised image, while larger will oversmooth the image. So if NLM is applied twice, should be large for the first time and much smaller for the second. In our experiments, for the first use of NLM we searched the best (i.e., the best ), denoted by , so that PSNR of denoised images has the largest value. Then we applied NLM method again to denoised images to get denoised images and used to calculate the weight (47) and the weighted average value (7), here using is better than using . For the second use of NLM we also searched the best , denoted by , to make PSNR of be the highest. The experiment results are presented in Table 1; the corresponding denoised images are showed in Figures 1 and 2.
The experiment shows that PSNR of denoised image by INLM is much better than NLM2. It is also better than NLM for most images. INLM improves the NLM, while NLM2 gets worse results than the NLM.
In Figures 3, 4, and 5 we present local zooming images taken from the Barbara image and the 1.1.13 image. From these three images one can see that the denoised images obtained by the INLM model are closer to the original images than the denoised images by the NLM model. The INLM model can preserve more image edges and details while deniosing.
We also compared our INLM with the Wiener enhanced NLM proposed in . The experiments in  demonstrated that the Wiener enhanced NLM  has much better performance than the NLM for removing AWGN from MR images. We wonder how it performs for removing AWGN from noisy natural images. Based on this consideration, we applied the Wiener enhanced NLM  to the natural images as shown in Table 1: Cameraman, Barbara, Lena, Images 1.1.10, 1.1.11, 1.1.12, 1.1.13, 1.3.01, 1.3.02, 1.4.02, and 1.5.03, corrupted by AWGN with the standard derivation , respectively. The experiment results showed that the PSNRs of the denoised images by the Wiener enhanced NLM  are lower than NLM, NLM2, and our model INLM. It means that, for denoising natural images corrupted by AWGN, the Wiener enhanced NLM  does not perform as well as when denoising MRI corrupted by AWGN.
3.2. On Real Images
In order to further illustrate the performance and the robustness of our model, we also apply our INLM model to real natural images and compare the results with those obtained by NLM and NLM2. Three real images used in the experiments are of size and shown in Figures 6, 7, and 8. We use the blind metric BRISQUE proposed by A. Mittal et al.  to assess denoised images. BRISQUE is one of the state-of-the-art blind no-reference image quality metrics when noise level in the image is unknown. The smaller the BRISQUE value is, the better the performance of image restoration is . We calculate BRISQUE values by directly using the software of BRISQUE which is available online at http://live.ece.utexas.edu/research/quality/.
In this experiment, let the decay parameter and . The experiment results in Table 2 show that BRISQUE values of denoised image by INLM are the smallest, which means that INLM is better than other two models. The corresponding denoised images are showed in Figures 6, 7, and 8.
In this paper we first present a discrete version of Jin-Jost-Wang’s nonlocal model which was proposed in . Then we propose an improved nonlocal means filter (INLM) (43). This INLM is as simple as the NLM and is simpler than Jin-Jost-Wang’s nonlocal model proposed in . Our experiments show that this INLM restores the original image from noisy image very well while preserving the image edges and details. Particularly this model is much better than the method by using the NLM model twice. It is also better than the NLM model when we consider the denoising for images with high textures.
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This work was supported by Natural Science Foundation of Zhejiang Province [no. LY17F010015].
- L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1-4, pp. 259–268, 1992.
- G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing, Partial Differential Equations and the Calculus of Variations, vol. 147 of Applied Mathematical Sciences, Springer Science + Business Media, LLC, New York, NY, USA, 2nd edition, 2006.
- T. F. Chan and J. Shen, Image Processing and Analysis: Variational, Pde, Wavelet, and Stochastic Methods, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2005.
- J. Xu, A. Feng, Y. Hao, X. Zhang, and Y. Han, “Image deblurring and denoising by an improved variational model,” International Journal of Electronics and Communications, vol. 70, no. 9, pp. 1128–1133, 2016.
- L. Jiang, J. Huang, X.-G. Lv, and J. Liu, “Alternating direction method for the high-order total variation-based Poisson noise removal problem,” Numerical Algorithms, vol. 69, no. 3, pp. 495–516, 2015.
- M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Transactions on Image Processing, vol. 15, no. 12, pp. 3736–3745, 2006.
- N. Mustafa, J. Ping, S. Ahmed, and M. Giess, “Medical image de-noising schemes using wavelet transform with fixed form thresholding,” International Journal of Advanced Computer Science and Applications, vol. 6, no. 10, pp. 173–178, 2015.
- W. Jifara, F. Jiang, S. Rho, M. Cheng, and S. Liu, “Medical image denoising using convolutional neural network: a residual learning approach,” The Journal of Supercomputing, pp. 1–15, 2017.
- F. Zhang, N. Cai, J. Wu, G. Cen, H. Wang, and X. Chen, “Image denoising method based on a deep convolution neural network,” IET Image Processing, vol. 12, no. 4, pp. 485–493, 2018.
- K. Isogawa, T. Ida, T. Shiodera, and T. Takeguchi, “Deep shrinkage convolutional neural network for adaptive noise reduction,” IEEE Signal Processing Letters, vol. 25, no. 2, pp. 224–228, 2018.
- A. Buades, B. Coll, and J. M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 490–530, 2005.
- S. Lefkimmiatis, “Non-local Color Image Denoising with Convolutional Neural Networks,” in Proceedings of the 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR '17), pp. 5882–5891, Honolulu, HI, USA, July 2016.
- M. Diwakar and M. Kumar, “CT image denoising using NLM and correlation-based wavelet packet thresholding,” IET Image Processing, vol. 12, no. 5, pp. 708–715, 2018.
- Y. Zhou, M. Lin, S. Xu et al., “An image denoising algorithm for mixed noise combining nonlocal means filter and sparse representation technique,” Journal of Visual Communication and Image Representation, vol. 41, pp. 74–86, 2016.
- K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007.
- A. Uzan and A. Stern, “Speckle denoising in digital holography by nonlocal means filtering,” Applied Optics, vol. 52, no. 1, pp. A195–A200, 2013.
- X. Wang, S. Shen, G. Shi, Y. Xu, and P. Zhang, “Iterative non-local means filter for salt and pepper noise removal,” Journal of Visual Communication and Image Representation, vol. 38, pp. 440–450, 2016.
- N. Joshi, S. Jain, and A. Agarwal, “An improved approach for denoising MRI using non local means filter,” in Proceedings of the 2nd IEEE International Conference on Next Generation Computing Technologies (NGCT '16), pp. 650–653, Dehradun, India, October 2016.
- H. Liu, Z. Zhang, L. Xiao, and Z. Wei, “A structure-preserved nonlocal iterative regularization model for image denoising,” in Proceedings of the 6th International Congress on Image and Signal Processing (CISP '13), pp. 336–340, Hangzhou, China, December 2013.
- G. Gilboa and S. Osher, “Nonlocal operators with applications to image processing,” Multiscale Modeling & Simulation, vol. 7, no. 3, pp. 1005–1028, 2008.
- V. Bianco, P. Memmolo, M. Paturzo, and P. Ferraro, “On-speckle suppression in IR digital holography,” Optics Expresss, vol. 41, no. 22, pp. 5226–5229, 2016.
- V. Bianco, P. Memmolo, M. Paturzo, A. Finizio, B. Javidi, and P. Ferraro, “Quasi noise-free digital holography,” Light: Science & Applications, vol. 5, no. 9, 2016.
- V. Katkovnik and K. Egiazarian, “Sparse phase imaging based on complex domain nonlocal BM3D techniques,” Digital Signal Processing, vol. 63, pp. 72–85, 2017.
- I. Djurović, “BM3D filter in salt-and-pepper noise removal,” Eurasip Journal on Image and Video Processing, vol. 2016, no. 13, 2016.
- H. Zhong, K. Ma, and Y. Zhou, “Modified BM3D algorithm for image denoising using nonlocal centralization prior,” Signal Processing, vol. 106, pp. 342–347, 2015.
- Z. Liu, S. Wang, and M. Zhang, “Improved sparse 3D transform-domain collaborative filter for screen content Image denoising,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 32, no. 3, 2018.
- L. P. Yaroslavsky, Digital Picture Processing, An introduction, vol. 9 of Springer Series in Information Sciences, Springer-Verlag, 1985.
- S. Kindermann, S. Osher, and P. W. Jones, “Deblurring and denoising of images by nonlocal functionals,” Multiscale Modeling & Simulation, vol. 4, no. 4, pp. 1091–1115, 2005.
- G. Gilboa and S. Osher, “Nonlocal linear image regularization and supervised segmentation,” Multiscale Modeling & Simulation, vol. 6, no. 2, pp. 595–630, 2007.
- D. Zhou and B. Schölkopf, “A regularization framework for learning from graph data,” in Relational Learning and Its Connections to Other Fields, pp. 132–137, 2004.
- D. Zhou and B. Schölkopf, “Regularization on discrete spaces,” in Pattern Recognition, Proceedings of the 27th DAGM Symposium, pp. 361–368, Berlin, Germany, 2005.
- H. Chang, X. Zhang, X.-C. Tai, and D. Yang, “Domain decomposition methods for nonlocal total variation image restoration,” Journal of Scientific Computing, vol. 60, no. 1, pp. 79–100, 2014.
- S. Tang, W. Gong, W. Li, and W. Wang, “Non-blind image deblurring method by local and nonlocal total variation models,” Signal Processing, vol. 94, no. 1, pp. 339–349, 2014.
- X. Liu and L. Huang, “A new nonlocal total variation regularization algorithm for image denoising,” Mathematics and Computers in Simulation, vol. 97, pp. 224–233, 2014.
- C.-A. Deledalle, N. Papadakis, J. Salmon, and S. Vaiter, “Clear: covariant least-square refitting with applications to image restoration,” SIAM Journal on Imaging Sciences, vol. 10, no. 1, pp. 243–284, 2017.
- Y. Lou, X. Zhang, S. Osher, and A. Bertozzi, “Image recovery via nonlocal operators,” Journal of Scientific Computing, vol. 42, no. 2, pp. 185–197, 2010.
- C. Deng, S. Wang, W. Tian, Z. Wu, and S. Hu, “Approximate sparsity and nonlocal total variation based compressive MR image reconstruction,” Mathematical Problems in Engineering, vol. 2014, Article ID 137616, 13 pages, 2014.
- X. Zhang, M. Burger, X. Bresson, and S. Osher, “Bregmanized nonlocal regularization for deconvolution and sparse reconstruction,” SIAM Journal on Imaging Sciences, vol. 3, no. 3, pp. 253–276, 2010.
- X. Zhang and T. F. Chan, “Wavelet inpainting by nonlocal toral variation,” Inverse Problems and Imaging, vol. 4, no. 1, pp. 191–210, 2010.
- Y. Jin, J. Jost, and G. Wang, “A nonlocal version of the Osher-Solé-Vese model,” Journal of Mathematical Imaging and Vision, vol. 44, no. 2, pp. 99–113, 2012.
- J. Jost, “Equilibrium maps between metric spaces,” Calculus of Variations and Partial Differential Equations, vol. 2, no. 2, pp. 173–204, 1994.
- Y. Jin, J. Jost, and G. Wang, “A new nonlocal variational setting for image processing,” Inverse Problems and Imaging, vol. 9, no. 2, pp. 415–430, 2015.
- Y. Jin, J. Jost, and G. Wang, “A new nonlocal H1 model for image processing,” Journal of Mathematical Imaging and Vision, vol. 48, no. 1, pp. 93–105, 2014.
- A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Transactions on Image Processing, vol. 21, no. 12, pp. 4695–4708, 2012.
Copyright © 2018 Yan Jin et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.