Abstract
Image denoising is an important problem in many fields of image processing. Boosting algorithm attracts extensive attention in recent years, which provides a general framework by strengthening the original noisy image. In such framework, many classical existing denoising algorithms can improve the denoising performance. However, the boosting step is fixed or nonadaptive; i.e., the noise level in iteration steps is set to be a constant. In this work, we propose a noise level estimation algorithm by combining the overestimation and underestimation results. Based on this, we further propose an adaptive boosting algorithm that excludes intricate parameter configuration. Moreover, we prove the convergence of the proposed algorithm. Experimental results that are obtained in this paper demonstrate the effectiveness of the proposed adaptive boosting algorithm. In addition, compared with the classical boosting algorithm, the proposed algorithm can get better performance in terms of visual quality and peak signal-to-noise ratio (PSNR).
1. Introduction
Image denoising is a fundamental problem in image processing, computer vision, pattern recognition, and so on. Consider a noisy image modeled aswhere is a clean image and is added white Gaussian noise with zero-mean and standard deviation . The goal of denoising is to resolve the clean image in (1). To this end, many technique and methods, such as spatial adaptive filters, transform-domain methods, sparse representation, and processing of local patches have been explored to study this problem, which leads to state-of-the-art denoising algorithms (denoisers), including the NLM [1], K-SVD [2], EPLL [3], BM3D [4], BM3D-SAPCA [5], and LSSC [6].
Although the algorithms mentioned above are effective in denoising application, the performance can be improved by employing some specific techniques, such as “Twicing”, Bregman iteration, and SAIF. In this paper, we focus on boosting skills originally used in machine learning, which improve denoising performance by multiple reusing [7–10] of weak denoisers to achieve strong ones. A general boosting algorithm [11] for image denoising can be expressed aswhere is a denoiser and is the denoised image of the th iteration. Essentially, the boosting algorithm (2) is to repeatedly denoise the residuals, , and add them back to the denoised image. However, Talebi et al. [10] pointed out that the number of iterations must be tuned carefully since the sequence obtained by (2) is not always convergent to the best restoration.
To conquer this problem, Romano et al. [7] proposed another boosting algorithm, called SOS, which can be expressed asSOS strengthens the noisy image by adding portion of the denoised image to the noisy image before operating the next iteration denoiser and subtracting the same portion of the outcome. SOS has excellent performance in denoising application. In addition, it is guaranteed to converge to an optimal solution, which enables us to obtain stopping criteria easily.
We note that the denoisers in (2) and (3) are invariant operators. In other words, the parameter in the denoisers is fixed regardless of the noise levels in the different iteration steps. The authors in [7] predefine a constant to estimate the initial noise level of and then set with to estimate the noise level of . In the following weaker denoising steps, the constant parameter is used in denoisers . Obviously, such scheme is not precise since s are the progressively restoring images which lead to a decreasing noise level estimation of with increasing. To solve this problem, in this paper we propose an adaptive boosting algorithm for image denoising application. Adaptive denoisers are used in boosting rather than an invariant operator , where the parameter is a noise level estimator of .
The estimation of the noise level plays a key role in our adaptive boosting algorithm. In the last decades, many methods on noise level estimation have been proposed [12–20]. Among these methods, the blind evaluation of noise level in textured images was widely studied [15, 16], which leads to many noise level estimation methods that were wildly used in processing the highly textured images. Different from the methods that were reported in [15, 16], in this paper, we focus on patch-based methods which have received a lot of attention due to the sound theoretical basis and promise performance, such as PCA [17], WT [18], and others [19, 20]. Chen et al. [19] pointed out that PCA and WT methods could lead to an underestimation of the noise level by using the Blom’s theorem [21]. Jiang et al. [20] proposed a noise level estimation method (called JZ in the following) based on the eigenvalues of covariance matrix of the flat patches. We in this paper prove that JZ method is an overestimation of the noise level. Combining WT underestimation and JZ overestimation, we propose a new estimator to obtain higher accurate estimation. The main contributions of this paper are as follows:(i)We propose an adaptive boosting algorithm for image denoising application, in which an adaptive weaker denoiser is introduced rather than an invariant operator often used in the traditional boosting.(ii)We prove that JZ method is an overestimation of the noise level by using Blom’s theorem. In addition, combining WT underestimation and JZ overestimation, we propose a new estimator to obtain higher accurate estimation.(iii)We prove the convergence of the proposed adaptive boosting algorithm. And some experiments are conducted to validate the proposed algorithm.
This paper is organized as follows: in Section 2, we present some related works concerning noise level estimation and image quality assessment. In Section 3, a noise level estimation algorithm and an adaptive boosting algorithm for image denoising application are proposed. In Section 4, the experiment procedure and the trial comparison between the proposed method and the initial denoising algorithm are described in detail. We summarize, conclude, and discuss the directions of future research in Section 5.
2. Previous Related Work
2.1. Eigenvalues and Noise Level
Liu et al. [18] gave a noise level estimation algorithm based on weak textured patches. According to their algorithm, the noise level of the image is estimated as follows:where is the ith eigenvalues of , in which is a set of weak textured patches that are selected by Algorithm 1 that was reported in [18], and is the covariance matrix of . By selecting flat patches in noisy image, Jiang et al. [20] gave another noise level estimation as follows:where denotes the ith eigenvalues of and is the covariance matrix of . In. (4) and (5), is the element number of each patch. The basis of the above two methods is to extract a set of patches with Gaussian distribution. Chen et al. [19] proved the following theorems about the distribution of the eigenvalues.
Lemma 1 (see [19]). Given a set of random variables , with each element following Gaussian distribution independently, the distribution of the noise estimation converges to the distribution when becomes sufficiently large.
Lemma 2 (see [19]). Let denote the cumulative distribution function of a standard Gaussian distribution. Given that independent random variables generated from the normal distribution with order , then the expected value of can be approximated by with .
2.2. Natural Image Quality Evaluator (NIQE)
The denoising algorithm is usually an iterative process in which the number of iterations needs to be selected such that the denoised image can achieve the best visual effect. For this problem, no-reference/blind image quality assessment models [22–26] are introduced. Recently, Mittal et al. [25] proposed a blind image quality assessment model called NIQE. In this method, the quality of a given image is expressed as follows:where , and , are the mean vectors and covariance matrices of the natural MVG model and the distorted image’s MVG model. Compared with other methods, NIQE does not have to train on large databases of human opinions of distorted images and has low computation complexity. Thus, it is suited for determining the optimal numbers of iterations. More details about NIQE can be found in [25].
3. The Proposed Method
In this section, we discuss the theoretical foundation of the noise level estimation algorithm and give an estimation that linearly combines the overestimated and underestimated results to evaluate the noise levels. Then an adaptive boosting algorithm for image denoising is proposed. The details of these steps are presented in the following subsections.
3.1. Noise Level Estimation
Using Lemmas 1 and 2 we can establish the following result.
Corollary 3. The estimation of in (4) satisfies . Particularly, if , then , where
Proof. If , then . By adding to both sides, we have ; that is, . Then, we have . Thus , with and . In (4), the last eigenvalue is selected as the noise level; it follows that . From Lemma 2, the following relationship holds:Thus .
Note that . Then ; that is, . Because is a monotonic function, we can obtain . Then, . Finally, we multiply both sides of the above equation by and simplify it to obtain .
Theorem 4. If , then noise level can be estimated bywhere and .
Proof. From (7) and (8), we obtain the following:Let , . Then, (10) becomesTherefore, can be solved by (11), that is,The collection of weak textured patches and the flat patches may not be the same. We extract the flat patches from the weak textured patches and denote it by . Then, both (7) and (8) hold on the collection , and the proposed algorithm for noise level estimation can be described in Algorithm 1. In experiment, we set to ensure the efficiency of Algorithm 1 and obtain more accurate noise level. The results of the corresponding proof experiments can be seen in Section 4.1. In the following section, Algorithm 1 will be plugged in a new adaptive boosting algorithm.
| ||||||||||||||||||||||||
3.2. Adaptive Boosting Algorithm
Based on the analysis for denoiser in (3), the noise level is the main parameter in the iteration. We show that the denoiser can be improved by the following boosting procedure:
Strengthen the signal by accumulating the previous denoised image to the noisy input image.
Estimate the noise level of the strengthen image.
Project the strengthen image to the range less than 1; i.e., divide the strengthen image by its infinite norm (maximum).
Operate the denoiser on the project image and back-project the range of the outcome to match the clean image .
The main equations that describe the above procedure can be written aswhere the infinite norm represents the maximum value of all elements, is the noise level of , the parameter controls the strength of the denoiser, and controls the signal emphasis. Our full image denoising algorithm is given in Algorithm 2.
| ||||||||||||||||||||
3.3. Convergence Analysis of Algorithm 2
In this section, bounded denoiser and linear denoiser are introduced and some proposition are analysed, then the convergence of Algorithm 2 is proved.
Definition 5 ((bounded denoiser) [27]). A bounded denoiser with a parameter is a function such that for any input ,for some universal constant independent of and .
Definition 6 (linear denoiser). For the given constants , , , the denoiser is a linear operator, if
Milanfar [8] pointed out that many denoisers satisfy Definitions 5 and 6. Based on this, Romano et al. [7] proved the convergence of SOS algorithm by neglecting the nonlinear term. According to these definitions and the following properties, we prove the convergence of Algorithm 2.
Proposition 7. For the given and , .
Proof. Since is a bounded denoiser, we have .
In iterative equation (13), is the th iterative approximation of . Denote the error between and by ; that is, . Then it follows from the iterative equation thatSubstituting and into (16), we can getDenote . And let . Using the central limit theorem, is a normal random variable with . Furthermore, we denote the standard deviation of by . This implies that in (17) can be considered as clean portion and a zero-mean Gaussian white noise with standard deviation . Thus, the noise level of estimated by Algorithm 1 is bounded. Then, we have the following proposition.
Proposition 8. There exists a constant satisfying .
Proposition 9. For all , and .
Proof. Since , we have , for . Then .
Assume . Then . Furthermore, noting that , we can getTherefore, the mathematical inductive method enables us to get .
Since , we have .
Proposition 10. For any positive integer , the following inequalities hold:
Proof. For any given , , it holds that . Therefore, for all noise level , we haveNoting thatand employing (20) and Proposition 9, we haveUsing the iteration , we have . ThenSubstituting (22) into (23), we getSince , then we haveThis completes the proof.
Next, we further consider the convergence of Algorithm 2.
Theorem 11. If is bounded and linear denoiser, then the main iterations of Algorithm 2 are convergent; that is, as .
Proof. It is easy to obtainFurther applying Definition 5 and substituting (22) into (26), we haveSimilarly,Applying Definition 5 and substituting (25) into (28), we obtainOn the other hand, the iteration leads toSubstituting (27) and (29) into (30), we haveFinally, Applying Proposition 7 and substituting (31) into (32), we obtainFrom Proposition 8 both and are all bounded. And since and are finite, we can get as , by taking the limit on both sides of (33).
3.4. Parameters Configuration
In Algorithm 2, we select the state-of-the-art algorithms: BM3D and BM3D-SAPCA. The source code of these algorithms can be obtained from the original authors and we use the default parameters. When the denoiser is given, the parameters in Algorithm 2 are and . In the remainder of this section, we mainly discuss the influence of and on the final results.
Considering and and discreting by using a step-size , we get the parameters set . Then, we introduce these parameters into the proposed Algorithm 2 and apply the proposed algorithm to noisy image. Figure 1(a) shows the relationship between and PSNR defined as , where MSE is the mean squared error between the original image and its denoised version. The relationship between and optimizing truncation iteration number is shown in Figure 1(b). It can be seen from Figure 1, when the parameters and are small, we can get the smallest PSNR and the largest iteration number. In this case, the efficiency of the denoising is the lowest. When both and achieve the highest value, the number of the iteration is the minimum, but the PSNR is not the maximum value. It is obvious that the PSNR has some volatility with . The iteration number of the area corresponding to the optimal PSNR value maintains consistency. In order to get the best denoising performance and reduce the computation complexity, suitable parameters and must be determined. According to the results that are present in Figure 1, the PSNR is highest and the number of iterations is acceptable when and . Therefore, we select (0.635, 0.472) in the following experiment.
(a) Relationship among , , and PSNR
(b) Relationship among , , and iterations
4. Results
4.1. Noise Level Estimation Results
Usually, the value of Bias, Std, and RMSE are used to evaluate the noise estimation performance. We use well-known criteria: accuracy, reliability, and overall performance which have been considered in most literatures. In detail, and
We test our method on Tampere image datasets (TID2008) [28] which contains 25 images of size . All images in this datasets are disturbed by zero-mean Gaussian noise with different standard deviations = 5, 10, 15, 20, 25, 30, 40, 50, 60, 70, 80, 90, and 100. According to Figure 3(a), the estimated results are almost as same as the true noise level. As shown in Figure 3(b), the value of Bias, Std, and RMSE are all close to zero, which illustrates the accuracy of the Algorithm 1.
4.2. Denoising Results
4.2.1. Objective Measurement Results
In this section, we provide detail results of the proposed methods. Based on the completely blind image quality assessment method NIQE, we always consider the following stopping rule: , and . The final outcome of Algorithm 2 is . We evaluate the competing methods on standard image processing test dataset (SIPTD) including Forman, Lena, House, Fingerprint, and Peppers, whose scenes are shown in Figure 2. These images are extensively tested in image processing. Thus it is convenient to compare our method with other methods under the same condition. All the test images are corrupted by an additive zero-mean Gaussian noise with a variance . The denoising performance is evaluated by using the PSNR, structural similarity (SSIM) [29], and dissimilar index (DSI) [30]. All results obtained by competing denoising method are shown in Tables 1 and 2. The results that appear in the column are obtained by applying the BM3D or BM3D-SAPCA on using the accurate noise standard deviation. The results that appear in the SOS and “OURS” columns are obtained by applying SOS that was reported [7] and the proposed Algorithm 2, respectively. The best results for each image and noise level are highlighted. In contrast, Algorithm 2 achieves higher PSNR and SSIM than those of the other schemes. Compared with BM3D, PSNR obtained by Algorithm 2 achieves about 0.45dB improvement on the image Peppers with the noise level 50. The results presented in Table 3 indicate that the proposed approach achieves the highest average improvement of the PSNR and SSIM among all compared methods. Hence, for SIPTD dataset, the proposed algorithm is effective.
(a) Foreman
(b) Lena
(c) House
(d) Fingerprint
(e) Peppers
(a)
(b)
Moreover, we perform Algorithm 2 on all the grass images from the MeasTex texture dataset [31] which contain rich texture features. The comparison results are presented in Table 4. Compared with SOS methods, the average improvements of PSNR and SSIM demonstrate significant comparative advantages for almost all noise levels. It is clear that the proposed algorithm offers better restoration of texture images.
Finally, we use DSI (the Matlab realization of the DSI metric are available at http://ponomarenko.info/flt.htm.) which obtains the largest spearman rank order correlation coefficient values to mean opinion scores to evaluate the visual quality of the denoised images. According to the results present in Tables 3 and 4, the proposed Algorithm 2 obtained lower average DSI value for each denoiser on the considering datasets in most case.
4.2.2. Detail Contrast
In this section, we compare the performance of considered denoising algorithms in preserving image details. Figure 5 shows the fragments of noisy () Baboon image and the corresponding images denoised by different denoisers. The first row is the original image and its noisy version. The second row shows the denoised result corresponding to different parameters: exact , SOS, and by using the denoiser BM3D. The third row is the denoised result obtained by the denoiser BM3D-SAPCA. The enlarge fragments in each subfigure are helpful for demonstrating the good quality of the denoised images of faithful detail preservation. Figures 5(f) and 5(i) show more details close to the original image.
Figure 6 shows the fragments of noisy () fingerprint image and the corresponding images denoised by different denoisers. According to Figure 6, our approach outperforms BM3D, SOS, and BM3D-SAPCA, numerically and visually.
Fragments of noisy () and denoised Foreman are shown in Figure 7. For this relatively high level of noise, the proposed Algorithm 2 attains a good preservation of sharp details, such as the lines on the wall in Foreman image. Meanwhile the smooth regions, such as the hat in Foreman image, are also of a good preservation. All the denoised images obtained by using the proposed Algorithm 2 have the fewest disturbing artifacts.
4.2.3. Computational Time
We discuss the efficiency of the proposed algorithm in this section. SOS and Algorithm 2 in this paper need several iterations during the denoising application because these two algorithms may cost more time than the initial denoiser. Table 5 shows the results of computational time between SOS and Algorithm 2. Compared with SOS algorithm, Algorithm 2 has a certain advantage in processing time for higher noise level while BM3D is selected as the denoiser. According to the results presented in Table 5, the processing time of Algorithm 2 in this paper is lower than SOS for each noise level when the denoiser BM3D-SAPCA is used. Furthermore, the average processing time spent per image of Algorithm 2 is less than that of SOS.
5. Conclusion and Discussion
In this paper, we proposed a new adaptive boosting denoising algorithm by plugging an accurate noise level estimation. Experience shows that the algorithm can improve performance of denoising algorithm which depended on noise level and can also keep the image’s edges and detail information well. Though it is a convenient tool for improving various denoising algorithm, there are still serval directions that we are interested in future works. Firstly, in order to get optimal output image, we use NIQE algorithm to assess the quality of output image . When are no longer decreasing, the final result of Algorithm 2 is given by . Figure 4(a) shows the result; the highest PSNR is obtained when first increase. By using Theorem 11, a straightforward stopping criterion ( is sufficiently small) can be obtained. Although this straightforward stopping criterion avoid the output of Algorithm 2 back to , the PSNR does not always increase when decrease. Figure 4(b) shows the relationship between PSNR and with ; the smallest does not give the optimal PSNR. Compared with Figure 4(a) and 4(b), we can find that the stopping rule in Algorithm 2 is helpful for determining the optimal numbers of iterations. Secondly, there are two main parameters and in the boosting method. How to select these parameters for optimal denoising results needs further research. Finally, we hope that other image restoration problems, such as image deblurring/inpainting, can use similar embedding method.
(a)
(b)
(a) Original image
(b) Clean patch
(c) Noise patch
(d) BM3D, , 25.43
(e) BM3D, SOS, 25.40
(f) BM3D, ours, 25.50
(g) BM3D-SAPCA, , 25.68
(h) BM3D-SAPCA, SOS, 25.66
(i) BM3D-SAPCA, ours, 25.73
(a) Original image
(b) Clean patch
(c) Noise patch
(d) BM3D, , 28.81
(e) BM3D, SOS, 28.81
(f) BM3D, ours, 28.82
(g) BM3D-SAPCA, , 28.94
(h) BM3D-SAPCA, SOS, 28.96
(i) BM3D-SAPCA, ours, 29.00
(a) Original image
(b) Clean patch
(c) Noise patch
(d) BM3D, , 30.04
(e) BM3D, SOS, 30.13
(f) BM3D, ours, 30.18
(g) BM3D-SAPCA, , 30.32
(h) BM3D-SAPCA, sos, 30.50
(i) BM3D-SAPCA, ours, 30.71
Data Availability
It should be noted that a software release of the proposed algorithm in our manuscript is available online: https://ww2.mathworks.cn/matlabcentral/fileexchange/67924-abd.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 11671307, 61561019, 61763009, and 11761030 and by Doctoral Scientific Fund Project of Hubei University for Nationalities under Grant No. MY2015B001.