Abstract
This paper addresses the image restoration problem which remains a significant field of image processing. The fields of experts- (FoE-) based image restoration has been discussed and some open issues including noise estimation and parameter selection have been approached. The stochastic method FoE performs fairly well; meanwhile it might also produce unsatisfactory outcome especially when the noise is grave. To improve the final performance, we introduce the integration with deterministic method K-SVD. The FoE-treated image has been used to obtain the dictionary, and with the help of sparse and redundant representation over trained dictionary, the K-SVD algorithm can dramatically solve the problem, even though the pretreated result is of poor quality under severe noise condition. The experimental results via our proposed method are demonstrated and compared in detail. Meanwhile the test results from both qualitative and quantitative aspects are given, which present the better performance over current state-of-art related restoration algorithms.
1. Introduction
Image restoration is a field that focuses on removing noise without sacrificing important image structures including edges and details. It is often assumed that the true image has been corrupted by independent identically distributed (i.i.d.) additive Gaussian noise with zero mean and standard deviation . The corrupted image is Also it could be seen as the original image has been degraded to the noisy image through specific transform . Thus we try to find out the inverse transform to restore the original image.
Mathematically this inverse problem might be highly ill-posed since the solution may not be unique or even inexistent. Generally, there are two kinds of opinion to solve this inverse restoration problem, namely, deterministic and stochastic. Algebraic viewpoint solves the problem by adding penalty function to the cost function as proposed by Tikhonov. The minimization of functional in the regularization can be written as While the data fidelity term maintains image characteristics and the regularization term ensures the stability of the final result, as to the stochastic aspect, image prior knowledge is of great importance to the whole restoration. The maximum likelihood (ML) and maximum a posteriori (MAP) principles provided theoretical foundation; meanwhile the statistics-based estimation methods offer feasible solutions. The ML estimation has been used to maximize the conditional probability density . Under the Gaussian noise assumption, the inference can be made . It can be revealed that the solution is similar to the deterministic method as least squares expression. However, for the lack of prior information, unsatisfactory performance is often led. The more widely used MAP method added image prior knowledge to pursue better results. Through Bayes theorem, the estimation can be made as The added image prior knowledge can make the problem properly posed. Generally speaking, the need for prior models of image occurs in many vision problems, and whenever the “uncertainty” exists, the prior model comes into play.
Fields of experts (FoE) proposed by Roth and Black has developed a new method for learning priors aiming at low-level vision problems [1]. The key is to formulate priors as a high-order Markov random field (MRF) defined over large neighborhood systems. And the advantage is that the prior probability of the image can be defined in terms of a random field with overlapping cliques, while the potential functions are modeled as a product of experts (POE). Having been successfully used in many vision problems including image denoising and inpainting, the FoE has provided a principled way to practical tasks. Nevertheless, FoE is not that perfect and some issues can be further researched and discussed. Firstly, in FoE it is assumed that the noise is known, while it is not realistic in most cases. Also the relation between weight parameter and noise standard deviation and the iteration time are worthwhile to probe. In addition, a more important fact is that the FoE may not have fine performance under severe noise condition.
Focusing on these topics, we proposed some improvements to the original FoE method in this paper. And the paper is organized as follows. Noise estimation method and FoE concerning parameter relation issues will be discussed in Section 2. In Section 3 the K-SVD method will be employed to improve the denoising performance and experimental results will be presented in Section 4. We also give conclusion and further discussion at last in Section 5.
2. Fields of Experts in Image Restoration
2.1. Fields of Experts
The training and inference of traditional Markov random field (MRF) model are NP-hard. In order to manipulate MRF model to describe abundant structure information sufficiently, Roth and Black combine the MRF with sparse coding to build the MRF with homogeneous potential functions, which can be also called fields of experts. Unlike other prior models, FoE use richer prior models that combined more filters over larger neighborhoods orderly [1].
Let the pixels in an image be presented as in which is node and is edge. A square neighborhood system can also be defined. The center of each neighborhood is located on pixel (node) and each neighborhood system defines a maximal clique in the graph. According to Hammersley-Clifford theorem, the probability of this density model belongs to Gibbs distribution . And is the potential function for clique , while is the partition function [2, 3]. Also it can be written as . And Over the whole range of image, in which and is the expert function. Hence
According to the posterior probability and the i.i.d. Gaussian noise assumption, the likelihood can be written as In which ranges the whole image. Thus the gradient log-likelihood is . The log-prior knowledge is , where is the convolution between filter and image and . After introducing the iteration time , update rate , and weight , the gradient ascent restoration algorithm can be Unlike other prior models, FoE use richer prior models that combined more filters over larger neighborhoods orderly.
2.2. Noise Estimation
It is assumed that the noise is given in FoE algorithm, while it might be unrealistic. Therefore we present a fast and simple method for estimating the variance of additive zero mean Gaussian noise. Without loss of generality, suppose that the image has size of width and height (namely, a image) and each pixel has the grayscale range 0, 1,…, 255. Image structure like edges have strong second-order differential components and the noise estimator should be insensitive to the Laplacian of the image. Considering these issues, a mask has been made to estimate the noise: It is assumed that the estimated noise standard deviation is at each pixel. Thus the mask has zero mean and variance . We make the convolution between the noisy image and the mask, denoted as . The mask will be applied to the whole range of the noisy image, which will bring to each pixel. We can calculate estimated noise variance through the average over the convolution results. And the standard deviation is the square root: As shown in Table 1. The estimated noise deviation is close to the ground-truth noise level , and the average relative error is 2.2%.
Unlike the other noise estimation methods in [4–7], the proposed one is neither time consuming nor theoretically complex; nevertheless the performance is favorable. Above all, the blind restoration can be realized through the noise estimation.
2.3. Parameter Relation and Selection
As can be seen in (8), the final result is affected by several parameters, the relationship between them needs our special attention, and the selection also calls for notice.
In order to figure out appropriate lambda value for noise standard deviation sigma , the sigma-lambda pairs here are determined experimentally to give good denoising performance with the model. According to the cubic spline interpolation and the hand-crafted empirical data, the relation between noise standard variance and lambda has been tested and the calculated result curve is drawn as shown in Figure 1. We test the sigma from 1 to 100 at the step size 1 and it can be figured out that if the noise standard variance is beyond 30, the lambda maintains nearby 6.
Also the relation between iteration times and peak signal-to-noise ratio (PSNR) of the final outcome image has been verified as in Figure 2. In the test 10000 times, iteration has been adopted to the Lena image under three kinds of noise level (, 30, and 50, resp.). It can be figured out that for more serious noise condition the FoE method is more difficult to converge, while the default iteration is often set to 3000 times.
To be honest, such iteration is rather time consuming especially when the time is more than 5000; for the case of 10000 times, the output is even worse, while it also remains a problem that such troublesome iteration may not lead to convergence as can be seen from the example shown in Figure 2.
In our application, we choose an FoE prior knowledge with 8 filters of pixels, update rate of 0.1 and perform 3000 iterations. And with the estimated noise standard deviation, the blind image restoration can be realized.
3. The Integration of FoE and K-SVD
Apart from the above-mentioned convergence problem, the more critical issue that makes us anxious is the poor performance under grave noise condition. Actually it is not a special individual case through our observation. Once the noise standard deviation exceeds 30, the final outcome will not be that desirable, neither visual nor numerical. In some extreme examples (basically when ), the FoE that seems powerless to master the situation in such cases can be seen from the cases shown in Figure 3. An original clean Lena has been corrupted by the Gaussian noise with and 100, respectively, and FoE restoration has been applied to both of them; however, the outcome is nearly horrible. Frankly speaking, they are almost the same as the noisy images without any treatment.
(a)
(b)
As further probe of the above two examples, after the noise estimation test via our proposed method in Section 2, it can be computed that the remaining noise standard deviation is still 51.31 and 74.42, respectively. Thus these FoE-treated denoise results can be also regarded as “noisy images” but “known” noise. In order to settle the inefficient problem when noise is severe, the dictionary based method caught our attention.
K-SVD is an effective dictionary training method as an extension of K-mean [8, 9]. The objective is to solve the minimization problem in (11). Assume that is the dictionary, is the sparse representation, and is the patch of image :
Let , , and stand for dictionary, training signal, and sparse parameter vector of dictionary, respectively. is the set of training signals and is the solution vector set from to . The learning process can be expressed as In which is the upper limit of nonzero components of sparse coefficient, namely, the maxdiff of .
The construction of dictionary can be divided into two steps: sparse coding and update. Fix the dictionary at first and use orthonormal matching pursuit [10] to get the sparse representation of over dictionary . Then fix the and pursue better . Initial dictionary will not be the optimum; thus iteration will be employed and we need to solve the equation
The closed-form solution is While K-SVD can easily handle the severe noise cases with satisfaction, this leads us to the combination. As to the demarcation of the “sever noise,” it might not be arbitrary that we judge the threshold as considering the relation curve in Figure 1.
In our work, the dictionary has been trained using patches from the denoise result but actually nearly “corrupted image" treated via FoE. From this point of view, the FoE seems more like pretreatment, fairly effective but still significant. K-SVD looks like deterministic method considering the minimization routine, while FoE resembles stochastic from the view of MRF. Therefore, the combination can be regarded as the collaboration of stochastic method and deterministic method. The concise procedures are shown as in Algorithm 1.
| ||||||||||
4. Experimental Results
In this section we demonstrate the results achieved by applying the proposed method on several test images. Four bitmap (bmp) formatted grayscale images Cameraman, Lena, House, and Dollar, all with 256 256 pixel size, have been chosen as the test images as shown in Figure 4. Among these images, Cameraman and House have more smooth areas and sharp edges, while Lena and Dollar have much more details. The subtle difference can influence the final restoration results and leads to diverse subjective visual feelings.
(a)
(b)
(c)
(d)
To test the method, we performed a chain of experiments, and various state-of-art methods have been chosen to make the comparison. For different noise levels we denoise the images using simple Wiener filtering (via MATLAB wiener2), Split Bregman method for total variation (for the cases of anisotropic and isotropic, ATV and ITV has been named, resp.) in [11–13], nonlocal mean (NL-mean) in [14–16], and the FoE-based proposed method.
Figures 5 and 6 display the entire restoration results of the five above-mentioned methods, while Figures 7 and 8 place emphasise on the detail parts. Generally the proposed method has produced the most vivid effect compared with the others. Even within noisy circumstance, the detail can be better reserved, the edge can be restored, and noise speckles can be removed to some extent.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
The Wiener filter has the lowest complexity and the result is sometimes acceptable; ATV and ITV are a little complicated, while the final performances are not that stable compared with our expectation. NL-mean has developed rapidly since its birth, while the outcome is also impressive. The proposed method, although relatively sophisticated to some extent, outperformed these state-of-art methods and detail comparison will be displayed soon.
Peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM) in [17] are employed to objectively characterize the quality. PSNR are commonly used to judge the denoise results in which higher value stands for better quality. In our experiments it has been defined as while and are true image and restored image, respectively. And PSNR is given in decibels (dB). Unlike PSNR, SSIM measure two images considering human visual characteristic which extract structure information; therefore the reservation degree of image structure information is quantized by the equation where is the mean value, is the variance, and is the covariance. Parameters and control the stability. SSIM ranges from 0 to 1, while 1 stands for the perfect restoration.
As to the noise standard deviation, we test from 10 to 100 at the step of 10, and Figure 9 shows the PSNR comparison of different restoration methods. The proposed method almost stays at the top of all curves for different deviation condition. In some low noise cases (Lena and House at noise standard deviation 10) it has been surpassed by the NL-mean.
But it should also be pointed out that the noise standard deviation is assumed to be known in NL-mean method, while we only employ the estimation values.
Figure 10 depicts the SSIM comparison of different restoration methods. The proposed method outperforms the others especially under severe noise condition. Wiener has fair results overall, while ATV and ITV fall down rapidly. The NL-mean has competitive results particularly in low noise environment, but striped noise still occurs for higher noise case.
5. Summary and Future Work
In this paper we have discussed the FoE-based image restoration method. The parameter detail including noise estimation has been discussed. Also FoE and K-SVD models have been combined in the process and the advantages of both have been utilized.
To summarize, the emphasis of this paper includes the following three aspects. First, a noise estimation method has been proposed to realize blind image restoration; secondly, original FoE has some pending parameter issues and we have made the research and discussion; finally as the further improvement, to promote the restoration performance under severe noise condition, we combine the FoE method with K-SVD method which can be seen as the cooperation of deterministic and stochastic methods. From the experiment results, the proposed method has been examined and the comparison with state-of-art methods demonstrated the validity.
On the other hand, the proposed method is still not perfect and some shortcomings can be optimized. Future work can be extended along the following directions. The oversmooth phenomenon still exists in the restoration and needs to be improved. Also the model selection, prior knowledge of images can be studied more deeply. The i.i.d. additive Gaussian noise has been focused in this paper, while we may need further approach for the statistically dependent noise like the fractal noise case mentioned in [18]. For different noise situation, the properties of images will be altered and the proposed method also needs to be further optimized.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by National Natural Science Foundation of China (no. 61301095), Heilongjiang Province Natural Science Foundation for the Youth (no. QC2012C070), Heilongjiang Province Natural Science Foundation (F201345), and the Fundamental Research Funds for the Central Universities of China (no. HEUCF140807).