Abstract
The nonlocal means algorithm is widely used in image denoising, but this algorithm does not work well for high-intensity noise. To overcome this shortcoming, we establish a coupled iterative nonlocal means model in this paper. Considering the computation complexity of the new model, we realize it by using multiscale wavelet transform and propose an asymptotic nonlocal filtering algorithm which can reduce the influence of noise on similarity estimation and computation complexity. Moreover, we build a new nonlocal weight function based on the structure similarity index. Simulation results indicate that the proposed approach cannot only remove the noise but also preserve the structure of image and has good visual effects, especially for highly degenerated images.
1. Introduction
Noise in images introduced by formation and transmission is unavoidable, which makes postprocessing difficult. Denoising is always an important research in image processing and computer vision. Numerous denoising techniques have been proposed in the image processing literature including variation regularization [1–6], Partial Differential Equation [7, 8], wavelet shrinkage [9], and neighborhood filtering [10, 11]. In particular, the nonlocal means (NLM) algorithm, which is first presented in [12] by Buades et al., has drawn much research attention lately due to its excellent performance.
How to measure the similarity of original clean patches through noisy patches is the heart of the NLM algorithm. The Gaussian weighted distance is used to measure the similarity on different image patches in NLM, which is easily influenced by noise and does not use the structure information in the patch. In order to improve the accuracy of similarity estimation, multiple weight functions have been developed [13–16]. Tasdizen [13] proposed a principle neighborhood dictionary that measures the patch similarity in the domain of principle component analysis. Other transform domain based NLM algorithms are also proposed such as [14, 15]. Kervrann et al. [16] proposed a Bayesian NLM framework that measures the patch similarities based on the statistical distribution of noise. Rehman and Wang [17] directly used the structure similarity (SSIM) index to estimate the nonlocal weights (SSIM-based nonlocal means, SSIM-NLM). All these weights functions can generate good denoising results for low-intensity noise, while not working well for high-intensity noise.
In order to reduce the influence of noise, iterative NLM algorithm has been proposed in [18], and it has been extended in [19, 20]. They update the filtered image in iterative process, while estimating the nonlocal weights over original noisy image which will reintroduce noise. Another iterative NLM algorithm has been proposed in [21] from variation formulation. Though the similarity of patches may be measured more accurately from the denoised image rather than the noisy image, the residual noise is still in the results because the weighted averaging is implemented on noisy image. From statistics, Deledalle et al. proposed a similar iterative approach in [22]. The above two kinds of iterative NLM algorithms have better denoising effect; however, these approaches only update one variable during iteration. This noncorresponding could bring deviation. In order to avoid it, [23] established a new regularization functional by using the maximum entropy to estimate the nonlocal weights. This model can obtain better results because the nonlocal weights and filtered image both are updated in iterative process. However, its computational complexity is high.
In fact, the iterative NLM algorithm switches strong filtering into multiple relatively weak filtering. As we know, multiscale transform is an effective tool to realize asymptotic approximation of the signal. Based on this, multiscale NLM algorithms have been proposed in [24, 25]. Compared to [25] (multiscale NLM, MSNLM), our algorithm proposed in [24] can better preserve the details because subimage has orientation information with wavelet transform. Thus, in this paper, we use multiscale wavelet transform to realize the new coupled iterative NLM model and call it asymptotic nonlocal filtering (ANLF). In ANLF algorithm, the noisy image is transformed into a series of spatial subimages; then the nonlocal weights are estimated by relatively clean low-frequency part and weighted averaging is applied at the same layer to remove the noise. ANLF has the same way as [24] but out of different considerations. Moreover, we give a new nonlocal weight function based on SSIM index [26].
To summarize, in this paper, we propose a novel asymptotic nonlocal filtering for image denoising. Extensive experimental results show our method is valid. The contributions of this paper are threefold.(i)We establish a new NL regularization functional and deduce the corresponding coupled iterative NLM model.(ii)Using multiscale wavelet transform to realize the coupled iterative NLM model, the new method reduces the computational complexity.(iii)A new nonlocal weight function is given based on SSIM index which can provide more accurate similarity estimation.
2. Coupled Iterative Nonlocal Model
In this section, we review the NLM algorithm and previous related iterative NLM algorithms. Then, we establish a coupled iterative nonlocal model which makes a good tradeoff between noise suppression and edge preservation.
2.1. NLM Algorithm
We assume that the input noisy image is where : () is composed of the original clean image and the independent additive white Gaussian noise with zero mean and variance . NLM algorithm estimates the gray value of each pixel as the weighted average of all pixels whose Gaussian neighborhood looks like the neighborhood at pixel : where , is Gaussian function with standard deviation , is a normalizing factor, and is the decay parameter of weight function.
The similarity function represents the affinity between different pixels of the image, and it is defined as a similarity measure based on the -norm between intensity values. The discrete version of the NLM algorithm can be represented as where and is a small neighborhood centered at the pixel , is the weighted -norm and is the standard deviation of the Gaussian kernel, and is a normalizing factor.
2.2. Previous Iterative NLM Algorithm
In 2005, Kindermann et al. [27] first proposed nonlocal regularization functional based on the calculus of variation and diffusion processes. And the following weighted nonlocal functional is proposed by Gilboa and Osher [18]: where , , , .
By using fixed point method, the corresponding iterative scheme is obtained:
The weights in (5) depend on original noisy image and are not involved in the iterative process. This will lead to reintroduced noise.
Different from the method shown in (4), Brox et al. [21] proposed another variation formulation based on a trivial variation principle, which can be written as
And they obtained the corresponding iterative scheme by using fixed point method:
Deledalle et al. [22] presented further iterative weighted maximum likelihood denoising with probabilistic patch-based weights (iPPB), which updates the weights in a data-driven way using both the noisy patch and iteratively filtered results. Same as (7), iPPB can give more accurate similarity weights, but its weighted averaging is always implemented over original noisy image .
2.3. Coupled Iterative Nonlocal Model
The averaging procedure of the above iterative algorithms does not correspond to the weight function because there is only variable ( or ) in updating. This strategy is not reasonable from the perspective of game theory. Inspired by the nonlocal regularization for inverse problems [23], we directly use the maximum entropy approach to estimate and consider the following joint optimization: where is the regularization parameter.
Using the alternating minimum and fixed point method for (8), we obtain
It is obvious that (9) is a neighborhood filtering (Gaussian filtering), and the regularization parameter of (8) is just the filtering parameter which controls smoothing degree. This coupled system avoids the disadvantages of the methods mentioned in Section 2.2 through the mutual influence between and .
Assume that the total number of image pixels is , that the patch size is , and that the searching window is of size . The complexity of NLM is . The complexity of iterative NLM algorithm is , where is the number of iterations. It will not achieve the intention edge-preserving if is too small. Such as, literature [22] used 25 iterations to ensure iPPB algorithm to reach convergence. That is to say, the complexity of proposed coupled iterative NLM is high. Moreover, the number of iterations and the selection criterion of smoothing parameter are hard to be determined. In order to solve these problems, we propose an asymptotic nonlocal filtering algorithm in the next section.
3. Asymptotic Nonlocal Filtering
For high-intensity noise, the coupled iterative NLM algorithms perform better than NLM in edge-preserving because they switch a strong filtering into multiple weaker filtering. It is well known that multiscale transform is the effective tool to realize asymptotic approximation of the signal and when the signal is decomposed into different subbands, the noise is decomposed and its variance is reduced layer by layer. According to this, we realize the coupled iterative NLM system by using multiscale wavelet transform.
3.1. The Framework of ANLF
Wavelet transform is used to realize image multiscale decomposition because it is simple and can maintain the structure of image better than others (such as LP [25]).
Let the noisy image belong to zero layer. Using the Mallat algorithm [28], is decomposed into layers. At layer , the low-frequency part and the high-frequency part are, respectively, reconstructed from scale coefficient and wavelet coefficients , , , and then , ().
After wavelet transform, the noise of obeys or approximately obeys Gaussian distribution, so NLM can be applied to directly and we can obtain the denoised image by and the weights are computed by : where is the decay parameter of the weights at layer , depending on the noise variance of .
Then, we replace by and repeat the process above at the layer . This process will not stop until the layer . We call this algorithm ANLF or -ANLF and show whole process in Algorithm 1.
| ||||||||||||||||||
In the framework of ANLF, the observed image is transformed into a relatively clean spatial subimage sequence by using wavelet transform and the coupled iterative NLM filtering is implemented asymptotically. As with analysis in Section 2.3, the complexity of ANLF is . It is lower than the complexity of coupled iterative NLM algorithm because the desired clean subimage can be provided with three to five layers’ wavelet decomposition for most images. This suggests that ANLF improves computational efficiency and reduces the influence of noise, especially for highly degenerated images. Furthermore, the corresponding low-frequency part is used to calculate the similarity weights instead of . That is, (12) shares the basic idea of the NLM with weighted -norm in similarity measuring. Compared with the Gaussian filtering, wavelet transform is used here to preserve the structure features and an asymptotic approximation framework can be obtained naturally. In addition, human visual system is more sensitive to image structural changes [29]. Thus, ANLF can provide more accuracy similarity weights, and the restored image with this algorithm is more suitable for human visual perception.
In the later discussion, we use the following discrete form:
3.2. Improved Weight Function
In -ANLF algorithm, the similarity of noisy image patches is estimated by corresponding low-frequency patches. But (13) only uses the gray information of patches while ignoring their structure. In addition, the -norm is sensitive to noise. In edge regions, the small weight may be assigned to the structurally similar patches because their -norm is large, and then the noise cannot be removed effectively by the weighted averaging.
The SSIM index [26] could remedy the weaknesses of (13) because it combines the brightness, contrast, and structure information of the patch. So we use SSIM index to improve the accuracy of similarity estimation and give the following nonlocal weight function: Here the SSIM index between the image patches and is defined as where is constant, , , and are the mean, standard deviation, and cross correlation between the two patches, respectively. The positive functions , are not constant like [17]. They, depending on noise level , are used to balance the tradeoff between noise suppression and edge preservation.
Equation (16) suggests that the better the similarity to reference patch is, the bigger the SSIM is, so this patch has higher ratio in weighted averaging. The values of , control the smoothing degree and ratio of reflects that more attention should be paid on mean or variance: if , (16) stresses on the mean; namely, the filtering of (15) focuses on smoothing; if , (16) stresses on the variance; namely, the filtering of (15) focuses on edge-preserving. Therefore one can get a balance between smoothing and edge-preserving by choosing appropriate functions . We select the and according to the following principles:(1)if the noise level is low, smoothing degree should be small and the emphasis of processing should be laid on structure preservation, so , both are small and satisfy ;(2)if the noise level is high, smoothing degree should be large and the emphasis of processing should be laid on noise suppression, so , both are large and satisfy .
As analyzed in Section 3.1, the noise levels of low-frequency sequences are always low in framework of ANLF. So we take and at layer as where is constant. Of course, one can construct other forms ,.
Using (15) to compute the similarity weights, the -ANLF can be modified to SSIM-ANLF and we show it in Algorithm 2.
| ||||||||||||||||||
4. Experiments and Analysis
In this section, experimental results are reported to validate the efficiency of the proposed algorithm. The corresponding algorithms are implemented in Matlab 7.0 environment on an AMD 1.90-Ghz dual-core PC. The results obtained by our methods are compared with that obtained from the NLM, iPPB, SSIM-NLM, MSNLM, and BM3D [30] algorithm.
In order to quantitatively measure the denoising capability of different methods, the reference originals, also called clean images, are known and the images used in our experiments are only corrupted by additive Gaussian white noise of a given variance. Thus, the MSSIM [26] and PSNR (dB) indexes can be used to measure the denoising performance. The MSSIM and PSNR are, respectively, defined by where and are the reference and the restored image, respectively; is the image contents at the th local window; and are the sizes of the image.
4.1. The Choice of Parameters
Thirty-six experiments are done to test the performance of our algorithms on six images shown in Figure 1. For all experiments, we select the parameters to achieve the best MSSIM values. A search window of size and a similarity patch of size are used. The maximum decomposition layer is 3. and is taken as empirical value (, ). Of course, it can be chosen according to the method which presented in [31]. The parameters have been set to , .
For iPPB and BM3D, we used the codes provided by the authors with their optimal parameter in all these algorithms and the iterative number of iPPB is 25.
4.2. Testing the Robustness of ANLF
In this section, the experiments are done to test the robustness of the -ANLF to wavelet transform. In Table 1, we list the results of seven wavelet transforms on three images. We can find that the standard variances of PSNR and MSSIM are less than 0.1 and 0.005, respectively. These results indicate that the -ANLF algorithm is not sensitive to the chosen wavelet transform. Therefore in following experiments we only use the Db8 wavelet to show the results of our method.
4.3. Simulation Results and Analysis
In this section, the part of denoising results is given to illustrate the validity of method which is proposed in this paper.
In Figure 2, we show the clips of denoising results to demonstrate the detail-preserving capability of the proposed method. We can see that the capacity of detail-preserving of proposed methods is better than the NLM, iPPB, SSIM-NLM, and MSNLM and similar to BM3D’s. It is easy to find the result of BM3D appeared fake textures in flat area, and SSIM-ANLF almost did not.
(a) Noisy image
(b) NLM
(c) iPPB
(d) SSIM-NLM
(e) MSNLM
(f) BM3D
(g) -ANLF
(h) SSIM-ANLF
In Figures 3 and 4, we show two denoising results to demonstrate the edges- preserving capability of the proposed methods. Comparing the results shown in Figure 3, some artificial edges are obtained from the NLM, BM3D, and -ANLF algorithm, and obscured image with some highlight edges is obtained from the iPPB algorithm, while SSIM-ANLF estimates the edges well and the restored image is more faithful to the original clean image. The denoising results of image with weak edges are shown in Figure 4, where we can easily find that the SSIM-ANLF algorithm is efficient on edge preservation; it especially does not produce false edges in the flat area.
(a) Noisy image
(b) NLM
(c) iPPB
(d) SSIM-NLM
(e) MSNLM
(f) BM3D
(g) -ANLF
(h) SSIM-ANLF
(a) Noisy image
(b) NLM
(c) iPPB
(d) SSIM-NLM
(e) MSNLM
(f) BM3D
(g) -ANLF
(h) SSIM-ANLF
The results shown in Figure 5 demonstrate that SSIM-ANLF algorithm can preserve the edges while removing the noise for highly degenerated images. The related values of PSNR and MSSIM which are listed in Table 2 indicate that our methods are valid.
(a) Noisy image
(b) NLM
(c) iPPB
(d) SSIM-NLM
(e) MSNLM
(f) BM3D
(g) -ANLF
(h) SSIM-ANLF
5. Conclusion
In this paper, we have proposed a novel asymptotic nonlocal filtering algorithm to realize the coupled iterative NLM by using multiscale wavelet transform. The proposed approach reduces the computation complexity of iterative NLM and the influence of noise on similarity estimation. Moreover, we build a new nonlocal weight function which obviously improves the accuracy of similarity estimation. Especially for highly degenerated images, SSIM-ANLF can preserve the edges and fine details information and possess good vision effect.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (nos. 61271294, 61362029, and 61271452) and the Key Project of Chinese Ministry of Education (no. 2011155).