Abstract
Images are often corrupted by impulse noise. In this paper, an alternating direction minimization with continuation algorithm is modified and iteratively used to remove the impulse noise in images by exploring its selfsimilarity. A patchbased nonlocal operator and sparse representation are married in the  optimization model to be solved. Simulation results demonstrate that the proposed algorithm outperforms typical denoising methods in terms of preserving edges and textures for both saltandpepper noise and randomvalued impulse noise. It can be also applied to suppress impulse noiselike artifacts in real mural images.
1. Introduction
Image denoising is a fundamental problem in image processing. Besides the commonly processed Gaussian noise, impulse noise (Figure 1(a)) is another typical type of noise generated due to noisy sensors or channel transmission errors [1]. In mural images, the impulse noiselike artifacts are also observed because of the bacterial plaque (Figure 1(b)) or missing objects (Figure 1(c)). A classical impulse denoising method is median filtering, which replaces the central pixel with a medianvalue within a local window. Median filtering is with low computation but often loses image structures when noise is heavy. The denoising performance is expected to be greatly improved if the image structures are extracted and preserved in denoising.
(a)
(b)
(c)
In recent years, sparse representation method has attracted great attention to provide effective representation of image structures and found promising applications in image denoising [2–5], image decomposition [6, 7], image fusion [8, 9], pattern recognition [10–13], and so forth. In impulse noise removal, sparse representation is remarkably effective at smoothing away noise [14–16], but it may lose image structures which are not sparsely represented by its predefined basis such as wavelet [3, 14]. Although the adaptive dictionary training [15, 16] improves the denoising performance, its computation cost of training dictionaries while removing noise is relatively high. Besides, the commonly used total variation [17–20] is highly related to enforcing the sparsity of finite difference of images. However, total variation assumes that images are piecewise smooth; thus, textures would be easily lost after denoising [15, 16, 21, 22]. Therefore, exploring the sparse representation for impulse noise removal is still challenging.
Similarity is often rooted in repeated patterns of images [23–25] and greatly improves image quality in deblurring [26] and medical image reconstruction [27]. Similarity information was previously investigated in block matching 3D frames [26]. Patchbased nonlocal operator (PANO) [27] is a linear operator to model the sparse representation of similar patches, which can be viewed as an alternative form of the block matching 3D frames. Unlike the traditional sparsifying methods, for example, conventional wavelets or finite difference, PANO provides an adaptive sparse representation of image with a very fast training phase. Besides, the linearity of PANO allows setting up explicitly a sparsitybased denoising model.
The contributions of this paper are summarized as follows.(1)The impulse noise is removed by making use of similaritymotivated sparse representation of images. Mathematically, we model the denoising problem as an  minimization problem with PANO which provides the sparse representation of similar image patches.(2)To solve the optimization problem, a fast alternating direction minimization with continuation algorithm is developed by taking the patchbased representation property into account.(3)Besides removing saltandpepper noise and randomvalue impulse noise in images, the proposed method is also applied in recovering mural images from corrupted data. Experimental results on simulated data and real mural images verify that the proposed method outperforms other methods in removing impulse noise or impulse noiselike artifacts.
2. Model
In this section, the impulse noise model and typical denoising methods are briefly summarized. Then, the proposed denoising model is presented.
2.1. Impulse Noise
Image contaminated with impulse noise is usually modeled as where is the noiseless image, is the observation, denotes an elementwise product, is a sample drawn from an i.i.d. multivariate Bernoulli distribution with success probability , represents a vector with all elements being equal to one, is either saltandpepper noise or randomvalued impulse noise.
For the saltandpepper noise, the element of satisfies where . For the randomvalued impulse noise, is drawn from uniform distribution in . In this paper, , , and are set.
2.2. Proposed Denoising Model
With a maximum a posteriori estimator on the impulse noise and prior knowledge of image, the noiseless image is estimated by minimizing where can be chosen as total variation assuming that the image is piecewise constant [19, 28] or norm of wavelet coefficients assuming that the image is sparse in the wavelet domain [14]. However, total variation is observed to easily lose textures [16] and wavelet may lose image structures that are not sparsely represented by its predefined basis. Thus, how to preserve the edges and textures remains open for the impulse denoising problem.
In this paper, image selfsimilarity is employed to provide an optimized sparse representation of images. Assuming that the similarity information of image is known with PANO, an  regularization model is proposed to remove the impulse noise, where promotes the sparsity using PANO, removes the outliers in images, and balances between the sparsity and outliers removing.
PANO is a linear operator to model the sparse representation of similar patches [27, 29]. For a given image , PANO is defined as where denotes the patch extraction, means grouping some of the patches into a 3D cube, and is 3D Haar transform. Figure 2 illustrates the process of grouping patches into a cube. First, an image is extracted into patches. Then, some similar patches are grouped together and stacked into 3D cubes. Finally, a sparse representation will be achieved by performing a 3D Haar transform on each cube, where is the sparse vector for the th cube [27, 29]. The adjoint operator of is , and , where is a diagonal matrix whose th diagonal entry is a counter indicating the repetitions that the th pixel is grouped into the 3D cubes. Therefore, by taking this patchbased representation property into account, the traditional alternating direction algorithm can be derived to achieve fast computation.
(a)
(b)
(c)
3. Algorithm
To numerically solve the optimization problem in (4), an alternating direction minimization with continuation (ADMC) algorithm [30, 31] is modified to remove the impulse noise. ADMC algorithm is also used for image deblurring [32, 33]. Here, by introducing auxiliary variables and to (6), one has
As , the solution of (7) approaches that of (6). In practice, we use the previous solution as a “warm start” for the next alternating optimization as gradually increased. does not need to be very large for image denoising. Simulation results show that increasing it will not improve the quality of denoised image but need more computation time. Thus is suggested. When is fixed, (7) can be solved in an alternating fashion between , , and . Consider the following.(1)Fix and ; solve , whose solution is where denotes the soft thresholding operation on with threshold [30, 31].(2)Fix and ; solve , whose solution is (3)Fix and ; solve , whose solution is
To construct PANO, similarity needs to be learnt so that some similar patches are grouped together and stacked into 3D cubes. However, the similarity is corrupted for a noisy image. To obtain a reasonable similarity, we need to have an initial denoised image as a guide image. The similarity is first learnt from a denoised image using conventional methods such as median filtering and then learnt twice from the denoised image using PANO. The process of the modified algorithm is summarized as Algorithm 1.

4. Results
In simulations, the proposed PANObased denoising method is compared with median filtering [1], waveletbased sparse representation denoising, and total variation denoising. The dualtree complex wavelet, which was used in impulse denoising [14], is adopted as the wavelet transform. PANO is applied with the typical parameter settings: the number of similar patches is 16, the patch size is 8 × 8, and the search region is 39 × 39.
Objective image quality assessments, peak signaltonoise ratio (PSNR), and mean measure of structural similarity (MSSIM) [34] are evaluated. The PSNR is defined as where is the number of pixels in an image and and are the original and denoised images, respectively. MSSIM is defined as where and are the image content at the th local window and SSIM evaluates local reconstruction errors by measuring the similarity between two images in a local window [34]. A higher MSSIM value means that structural information is better preserved and more suitable for human visual system.
The experiments run on 4 Cores 2.6 GHz CPU laptop computer with 12 GB RAM. The computational time of the proposed method is approximately 25 (or 70) seconds for image with size 256 × 256 (or 512 × 512). For the simulated noisy images, the window size of the median filtering and the regularization parameters of the rest methods are optimized to maximize MSSIMs while removing most of the impulse noise.
4.1. Choose Guide Images
The similarity is assumed to be known in the proposed denoising model. However, the similarity is also corrupted for a noisy image. To obtain a reasonable similarity, we need to have an initial denoised image as a guide image. In this section, we will discuss how to choose a guide image.
Since conventional denoising methods are available, one may consider taking the denoising result by median filtering, total variation, or wavelet denoising as the guide image to learn the similarity. When a better denoised image is obtained using the proposed approach, one may further utilize the result as an updated guide image.
As shown in Figure 3, different initial guide images lead to comparable denoising results with 30% noise. So median filtering is chosen to produce the initial guide image since its computation is the fastest. After the similarity is learnt for 3 times, both the PSNR and MSSIM of the proposed method become stable, and more similarity learning will cost more computation time. Hence, learning the similarity 3 times is sufficient.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
4.2. Remove Simulated Impulse Noise
Images (Figures 3(a), 4(a), and 4(d)), which are usually used in image processing, are treated as ground truth images; saltandpepper noise or randomvalued impulse noise is manually added in simulations.
(a)
(b)
(c)
(d)
(e)
(f)
4.2.1. SaltandPepper Noise Removal
The performances of the four denoising methods for saltandpepper noise are compared. As shown in Figure 5, repeated image patterns, for example, textures in Barbara, are obviously lost using median filtering and total variation. Although wavelet recovers the textures much better than the above two methods, it suffers from losing edges. The proposed method preserves these features best, with the clearest textures in Barbara and edges in both House and Boat.
As shown in Figure 6, the proposed method achieves the highest PSNRs when the noise level is below 40% and the highest SSIMs for all noise levels. When the noise becomes heavier (e.g., 50%), the performance of the proposed method gradually dropped. As Figure 7 shows, the qualities of the initial guide images using median filtering, total variation, and wavelet denoising methods are all unsatisfactory; thus, similarity cannot be learnt well so that the denoising results of the proposed method are also not satisfactory. Table 1 shows the performance of the proposed method using different initial guide images under heavy noise. Suppose we have the original image as the initial guide image so that similarity information can be exactly leant; the proposed method can remove the noise very well and preserve the image structures. In summary, the proposed method leads to comparable denoising results when ground truth image is not available.
(a)
(b)
(c)
(d)
(e)
(f)
4.2.2. RandomValued Impulse Noise Removal
The performances of the four denoising methods for randomvalued impulse noise are also compared. As shown in Figure 8, edges and textures are best preserved by the proposed method. Under different noise levels, proposed method achieves the highest MSSIMs (Figure 9). These observations imply that the proposed method preserves the best image structures during removing the randomvalued impulse noise.
(a)
(b)
(c)
(d)
(e)
(f)
4.3. Remove Impulse NoiseLike Artifacts in Mural Images
In the section, the impulse noiselike artifacts in realistic mural images are suppressed using these denoising methods. As shown in Figure 10, both median filtering and total variation will easily blur the images. On the contrary, the proposed method preserves the image details much better and suppresses most the impulse noiselike artifacts as marked. We can observe that the lost image structures are minimal among these methods (Figure 11). Therefore, the proposed method can also be used to remove these impulse noiselike artifacts of real dataset.
4.4. The Effect of Noise Detectors
Noise detectors are very useful to improve the denoising performance of saltandpepper noise. Chan et al. [18] proposed to use a noise detector first and apply a weighted regularization model with less penalization on the noisefree pixels. Following this idea, we use a noise detector to distinguish the noisy pixels from the rest of the pixels in the preliminary results. The locations of detected noise candidates are stored in a set , whose complementary set stores locations of the rest of the pixels. We tried a weighted  regularization model as where is a diagonal matrix, whose th diagonal entry is . A small weight is assigned for noise candidates and a large weight is set for the rest of pixels. is chosen to be total variation or PANO.
Simulation results listed in Table 2 show that noise detectors can significantly improve the denoising performance. When both methods are equipped with noise detectors, the proposed method still outperforms total variation with higher MSSIMs and comparable PSNRs. This implies that the proposed method can preserve image structures better, as shown in Figure 12.
(a)
(b)
(c)
(d)
(e)
An efficient method to suppress the noise via combining PANO with noise detectors can be found in [35]. However, the noise detectors proposed in [18] only work for saltandpepper noise. How to detect the noise candidates in randomvalued noisy image is still unknown. Besides, how to solve the proposed weighted regularization model efficiently and setting the weight optimally need comprehensive investigation. We leave these as the future work.
5. Conclusions
Impulse noise removal with similaritymotivated sparse representation is proposed. With a patchbased nonlocal operator to model sparse representation of similar image patches, the  optimization problem is solved by a modified iterative alternating direction minimization algorithm.
Experiments on simulated data demonstrate that the proposed method significantly improve the recovery of textures and edges while removing both saltandpepper noise and randomvalued impulse noise. It can be also used to remove the impulse noiselike artifacts in mural images well. Denoising performance of the proposed method may be further improved if one can learn a proper similarity from serious noisy image.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (61302174, 61201045), Scientific Research Foundation for the Introduction of Talent at Xiamen University of Technology (YKJ12021R), Open Fund from Key Lab of Digital Signal and Image Processing of Guangdong Province (2013GDDSIPL07, 54600321), Fundamental Research Funds for the Central Universities (2013SH002), and Natural Science Foundation of Shaanxi Province of China (2013JM8015).