Abstract
We propose a sparsity based approach to remove additive white Gaussian noise from a given image. To achieve this goal, we combine the local prior and global prior together to recover the noise-free values of pixels. The local prior depends on the neighborhood relationships of a search window to help maintain edges and smoothness. The global prior is generated from a hierarchical sparse representation to help eliminate the redundant information and preserve the global consistency. In addition, to make the correlations between pixels more meaningful, we adopt Principle Component Analysis to measure the similarities, which can be both propitious to reduce the computational complexity and improve the accuracies. Experiments on the benchmark image set show that the proposed approach can achieve superior performance to the state-of-the-art approaches both in accuracy and perception in removing the zero-mean additive white Gaussian noise.
1. Introduction
Image denoising, aiming at generating clean images by removing the noises, plays an important role in various tasks of image processing and computer vision, such as feature extraction, object detection, and pattern recognition [1–3]. Since noise-free image can significantly improve the performance of these tasks, image denoising has drawn a growing number of attentions in recent years. Under dissimilar circumstances, images can be corrupted by different types of noises, for example, Gaussian noise, salt-and-pepper noise, and quantization noise, in the process of acquisition and transmission. Among these noises, additive white Gaussian noise, which is usually caused by high temperature or lacking of illumination, is the most conventional and has been widely studied in the past decades. In this paper, we mainly focused on removing this classic type of noise.
Given an image corrupted by additive white Gaussian noise , we can formulate the relationships among , , and noise-free image as follows:where the mean and standard deviation of are 0 and , respectively. To recover from in (1), a number of approaches [4, 5] have been proposed and achieve satisfying results.
Recently, Nonlocal Means (NLM) filtering has been rapidly developed and widely used in image processing tasks. NLM, originally introduced by Buades et al. in [6], tries to utilize all the other pixels in the image to recover the noise-free value of target pixel. NLM has achieved a superior performance [7–9] over previous approaches while preserving the integrity of relevant image information and has been theoretically proven to be effective in dealing with zero-mean Gaussian white noise. In order to globally generate effective correlations among pixels in the target image, the traditional NLM algorithm needs to calculate every pairwise similarity in a large search window. It causes high computing complexity and limits the application of NLM in practice. Therefore, there is an urgent requirement to develop an algorithm to reduce the search range and efficiently construct global correlations among pixels.
Recent studies about the Compressed Sensing (CS) have shown that the information of a given image is always redundant, and a specific pixel can be sparsely represented by other pixels in the image. This inspires us to reduce the search range of NLM by finding the sparse representation of the target pixel. Sparse representation has been used in many computer vision and image processing tasks, such as face recognition, motion segmentation, and image restoration. Specially, sparsity based approaches have been well studied, because of its superior performance on various tasks. Xu et al. [10] have utilized sparsity to solve the image smoothing and image deblurring problem. Wang et al. [11] adopt sparsity in image segmentation. Nie et al. [12] propose a hierarchical sparsity based approach to tackle the intrinsic image decomposition problem and achieve superior performance to the previous approaches. These successful applications of sparsity inspire us to explore the potentiality of it to resolve image denoising problem.
In this paper, we propose a sparsity based approach to remove the additive white Gaussian noise from a given image. To recover the noise-free value of a pixel, we combine the local and global information, together, to achieve more accurate result. The local correlation is adopted by exploring the relationships of pixels in a small search window ( pixels in our experiments) to preserve edges and ensure local smoothness. The global correlation is generated by finding the sparse representation based on sparsity to reduce the search range. Moreover, we also adopt Principal Component Analysis (PCA) to measure the similarities between features of pixels, which can both reduce the amount of calculation and improve the accuracies. Experiments on benchmark images corrupted by different level of zero-mean Gaussian white noise show that the proposed approach gains superior performance to the state-of-the-art image denoising approaches.
The rest of this paper is organised as follows: Section 2 will briefly review the NLM algorithm and sparsity. Section 3 will specify the proposed based nonlocal denoising algorithm in detail. In Section 4, we show the experimental results that demonstrate the effectiveness of our algorithm. Section 5 will summarise and conclude this paper.
2. Related Work
In this section, we will briefly review the previous studies about NLM and sparsity, the two most correlated algorithms to the proposed image denoising approach.
Nonlocal Means. NLM is a popular technique developed by Buades et al. [13], which achieves unprecedented performance in image denoising task. However, the superior performance is achieved at the cost of high computational complexity. For a target pixel , the standard NLM computes the relevances based on image patches between and all the other pixels in the image. Then, the linear combination result of similar and homologous pixel values is assigned as the noise-free value of . The high computational complexity makes standard NLM less practical in image denoising task. Salmon [14] conducted experiments to study two factors in NLM, search window size and weight of the central patch, and concluded that the performance increased when the window is larger and the way of the central weight was a crucial issue. Some approaches have been proposed to accelerate the NLM or find a more appropriate definition of weight. References [15, 16] terminated the distortion computation between patches based on probabilistic early termination. But the reduction in the amount of similarity measurement resulted in the loss of image information. Darbon et al. [17] separated the pixels in neighbourhood to offer a parallel implementation in contemporary memory sharing of computer architectures, which had special requirement for hardware. In terms of weight, Zhong et al. [18] estimated the similarities between noise-free patches instead of noisy observations. The denoising scheme did secondary filtering based on the original NLM, which resulted in more calculation amount. References [19, 20] built a dictionary to project the image content to the subspace and then defined a metric between a pixel and the neighbouring ones. According to region characteristics, region-based NLM [21, 22] adaptively changed the similarity patch size to preserve edges. These methods did provide a more accurate metric of distance and gained a better performance than the standard NLM. However, because the search area is a region centered on the target pixel, the previous NLM based approaches are seminonlocal in essence.
Sparsity. sparsity based approaches have been widely used in many vision and graphic tasks and acquired superior performance. Hyder and Mahata [23] proposed an iteratively approximate -norm based on fixed point to reconstruct sparse signal and achieved an obvious improvement in noisy environment. In [10, 24], Xu et al. developed gradient minimization to control the nonzero gradients among neighborhoods, which indicated that the prominent image structures needed to be preserved in image smoothing. Later, they extended sparsity to image deblurring and achieved notable improvement both in convergence speed and result quality. Mancera and Portilla [25] selected coefficients by minimizing the nonzeros with -norm in in-painting problem and gained a remarkable performance compared to practical -norm based methods. A Gaussian function was provided by Mohimani et al. [26] to obtain a continuous smooth estimation of -norm and found the sparsest solution for linear equations of an underdetermined system. The process speed was improved by two orders of magnitude faster than while retaining the same accuracy. Lopez et al. [27] adapted -norm to make Least-Squares Support Vector Machines (LS-SVM) sparsity in classification and regression problems. As a result, the amount of support vectors was reduced significantly than standard LS-SVM while keeping a comparable accuracy. However, this procedure was computationally expensive. Wang et al. [11, 28] built a constructed graph based on representation of features, which gave a better description of superpixels, to perform image segmentation. Experimental results showed that their approach achieves competitive results compared with state-of-the-art methods. Nie et al. [12] constructed a sparse representation for pixels by solving the minimization problem, then formulated a sparse prior to preserve the global consistency during the decomposition of intrinsic image, and achieved superior performance. The success of sparsity in various tasks promotes us to utilize it to deal with image denoising problem.
3. Our Method
3.1. The Proposed Formulation
Let , , and represent noise-free image, additive Gaussian noise, and observed image, respectively. According to (1), the value of pixel in can be defined as the addition of th pixel values in and ; that is, . We formulate the proposed denoising model as follows: where is the restored value of pixel , and represent the local prior and global prior at pixel , and is a normalization factor.
Local prior explores the correlations among pixels in a local window. It can preserve edges and encourage smoothness. We define the local prior at pixel as where represents the pixel set in a local search window centred at and represents the local similarity between pixel and pixel . In our experiments, the size of local search window is set as . We will illustrate the detailed definition of in the next section.
Global prior builds the relationships between pixels in the whole image. It can promote the global consistency and improve the denoising results. The definition of is given as follows: where represents the set of pixels globally correlated with and represents the nonlocal similarity. The generation of and calculation of will be specified in Section 3.3.
The normalization factor is defined as
3.2. Local Prior Acquisition
According to (3), the local prior is defined as the summation of weighted pixel values in a local search window. In this section, we will specify the definition of similarity .
For a given pixel , we represent its feature by concatenating the gray intensity of nearby pixels within a local square patch centred at pixel , and the size of the patch is set as in our experiments. We use to denote the feature set of all pixels in image , where is the feature vector of pixel and its dimension is . For pixel , the similarity between pixels and is defined as follows:where represents the Euclidean distance and is the parameter which controls the decay of weights related to the level of noise. For a specific image, is a constant. The similarity definition given in (6) can help maintain the simplicity of algorithm; however, the computation process is time-consuming. In addition, the original feature vector always includes redundant and noisy information. To reduce the feature dimension and refine the feature representation, we project the original feature vector onto a subspace generated by Principle Component Analysis (PCA).
To achieve this goal, we first randomly select pixels in image . We use to denote the set of feature vectors of selected pixels. Let be the number of principle components and let be the sorted eigenvectors in descending order according to their respective eigenvalues, which are generated from the set . The number is determined bywhere is a threshold which means the utilization rate of information, usually set as 0.95 in our experiments. Then, we project the original feature vector on the -dimensional PCA subspace aswhere represents the distance of th original feature vector’s projection on the th basis vector. Since the basis vectors are orthonormal, we can calculate the distance between pixel and pixel on the PCA subspace as We use to replace the traditional Euclidean distance. Finally, the local similarity between pixels and is defined as
3.3. Global Prior Acquisition
Based on the mechanism of sparse coding representation, a specific signal can be sparsely represented by the dictionary with the coefficient vector , that is, , subject to , and is a positive integer. Accordingly, for a given pixel , we still use to represent its feature vector. To generate the global correlations of pixel , we solve the following minimization problem based on sparsity:where represents the sparse dictionary of pixel and is the corresponding coefficient vector. The construction of dictionary for each pixel can significantly affect the accuracy and speed of the solving process. Inspired by the hierarchical sparsity proposed in [12], which has shown superior performance both in effectiveness and efficiency, we also adopt a hierarchical framework in this paper.
For image , given the downsampling ratio and layer number , we first build an image pyramid . represents the image in th layer, and corresponds to the original image . For the top layer , we build the sparse dictionary for pixel by features of all the other pixels in , that is, , where represents the feature of th pixel in and is the number of pixels in . By solving the coefficient vector , we can find pixels which are globally correlated with pixel through the nonzero coefficients in . For layer (), we build sparse dictionary for th pixel by the derivative relationships from . Specifically, we first find pixel ’s corresponding pixel in by downsampling. By , we can find the pixel set correlated with . For each pixel , we can get the corresponding image patch in by upsampling. Then, we use feature vectors of pixels in to construct the sparse dictionary for th pixel in layer , where represents the union of multiple sets.
Finally, we can find the set of pixels which are globally correlated to pixel in by nonzero coefficients in . For pixel , the global similarity between pixels and is defined by the Normalized-Residual aswhere , if , and , otherwise.
4. Experimental Result
We test our approach on twelve benchmark images: Lena, Boat, Barbara, Man, Cameraman, House, Peppers, Couple, Hill, Fingerprint, Mandrill, and Monarch. Images are corrupted by three levels of additive zero-mean white Gaussian noise, with standard variance , respectively. Parameters to achieve local prior are set as illustration in Section 3.2. For parameters in global prior acquisition, we set the sparsity , the downsampling ratio , and the layer number . The parameter settings are generated by fine-tuning on the training set, including Montage, Airplane, Bridge, and Truck, to the achieve highest average accuracy. We quantitatively evaluate the denoising quality with two conventional measurements: Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). PSNR represents noise removal effect, defined as follows: withwhere and represent the denoising image and noise-free image, respectively, and is the number of pixels in image (). stands for the noise variance of despeckled image. SSIM is used to describe the noise removal quality and it is much closer to human vision, given bywhere and are the mean and variance of noise-free image , respectively, and the similar definition of and to denoising image . is the covariance of and . and are constants, which are used to avoid instability when and are very close to zero.
Figures 1 and 2 show the image denoising results on different levels of noise by the proposed approach. From Figures 1 and 2, we can observe that the denoising images on different noise levels achieve good visual sense. Our approach can maintain detailed structure well and has remarkable noise reduction at the flat region. Moreover, our approach can also preserve the global consistency.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
We also compare the proposed approach with five NLM based image denoising approaches: NLM [6], NL-TV [7], NLMPCA [20], NLSAP [22], and R-NL [29]. The proposed approach is denoted as -NLM. The comparison results are shown in Table 1. From Table 1, we can observe that our approach can always achieve the highest PSNR and SSIM score in different noise levels. And our approach achieves average 0.7 PSRN improvement and 0.05 SSIM improvement to original NLM algorithm, which specifies the efficiency of proposed approach. Moreover, our approach offers superior performance on 7 test images released by state-of-the-art approach R-NL, average 0.16 PSNR improvement and 0.02 SSIM improvement, especially on the image Man and Barbara.
To further demonstrate the advantage of our approach, Figures 3 and 4 show the comparison of detailed structures of different approaches. Figure 3(a) is a detailed show of original image Lena and Figure 3(b) is the corresponding noisy image with . Figures 3(c)–3(g) show the detail of denoising results of NLM, NL-TV, NLMPCA, NLSAP, R-NL, and proposed model -NLM. From Figure 3(g), we can observe that the boundary between white part and black part in the eye of image Lena is obvious; besides, the intensities of both parts in two eyes are consistent. That specifies that our model is more effective in finding the global correlations among pixels. But the results generated from other approaches have some small drawbacks; for example, the boundary is fuzzy, or the similar parts in two eyes are inconsistent. In addition, the forehead is much smoother on -NLM than other models. This can also demonstrate that our approach can construct appropriately nonlocal correlations and encourage global consistency. Figure 4(a) is the detail of hand of noise-free image Man and Figure 4(b) is the noisy image with . Figures 4(c)–4(g) are the denoising results of different algorithms. From Figure 4(g), we can see that the bracelet in the hand of man is much clear by -NLM. The result also illustrates the superior performance of our approach.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
5. Conclusion
This paper presents an image denoising approach based on sparsity. We combine local prior and global prior together to effectively generate image denoising results. The local prior can preserve edges and encourage local smoothness. To effectively and efficiently calculate the local similarity, we adopt Principle Component Analysis to reduce the feature dimension and refine the feature representation, while the global prior encourages global consistency. The major contribution of this paper is that we utilize hierarchical sparsity to construct global correlations, which is more effective and efficient to reduce the redundant information in the whole image. This differs from traditional nonlocal approaches which trivially search similar pixels in a search window. Experiments on the benchmark images show the superior performance of the proposed approach to state-of-the-art approaches in different noise levels.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
The authors would like to extend their sincere gratitude to Professor Dang Jianwu, for his supports and efforts on this work. The research is supported in part by the National Natural Science Foundation (Surface Project no. 61175016 and Surface Project no. 61304250).