Abstract

A novel semiblind defocused image deconvolution technique is proposed, which is based on multivariate local polynomial regression (MLPR) and iterative Wiener filtering (IWF). In this technique, firstly a multivariate local polynomial regression model is trained in wavelet domain to estimate defocus parameter. After obtaining the point spread function (PSF) parameter, iterative wiener filter is adopted to complete the restoration. We experimentally illustrate its performance on simulated data and real blurred image. Results show that the proposed PSF parameter estimation technique and the image restoration method are effective.

1. Introduction

Shooting a real world image with a camera through an optical device gives a 2D image where at least some parts are affected by a blur and noise. Images can be blurred by atmospheric turbulence, relative motion between sensors and objects, longer exposures, and so on, but the exact cause of blurring may be unknown. Restoration of blurred noisy images [13] is one of the main topics in many processing. The literatures [46] have given good methods to improve image qualities. The purpose of image restoration is to reconstruct an unobservable true image from a degraded observation. An observed image can be written, ignoring additive noise, as the two-dimensional (2D) convolution of the true image with a linear space-invariant () blur, known as the . Restoration in the case of known blur, assuming the linear degradation model, is called linear image restoration and it has been presented extensively in the last three decades giving rise to a variety of solutions [710]. In many practical situations, however, the blur is unknown. Hence, both blur identification and image restoration must be performed from the degraded image. Restoration in the case of unknown blur is called blind image restoration [1113]. Existing blind restoration methods can be categorized into two main groups: (i) those which estimate the a priori independent of the true image so as to use it later with one of the linear image restoration methods, such as zero sheet separation, generalized cross validation, and maximum likelihood and expectation maximization based on the image model [1416], and (ii) those which estimate the and the true image simultaneously, such as nonnegative sand support constraints recursive inverse filtering, maximum likelihood and conjugate gradient minimization, and simulated annealing [17, 18]. Algorithms belonging to the first class are computationally simple, but they are limited to situations in which the has a special form, and the true image has certain features. Algorithms belonging to the second class, which are computationally more complex, must be used for more general situations. In this paper, a kind of semi-blind image restoration algorithm is proposed in case of known the blur type (defocused blurring).

In general, discrete model for a linear degradation caused by blurring can be given by the following equation: where * indicates two-dimensional convolution, represents in original image, is the degraded image, represents the two-dimensional , and is the additive noise. In this paper, we deal only with additive Gaussian noise, as it effectively models the noise in many different imaging scenarios. The difficulty in solving the restoration problem with a spatially varying blur commonly motivates the use of a stationary model for the blur. This leads to the following expression for the degradation system:

The use of linear techniques for solving the restoration problem is facilitated by using space-invariant model. Models that utilize space-variant degradations are also common, but lead to more complex solutions. As for defocus blur, is modeled as a uniform intensity distribution within a circular disk, where disk radius is the only unknown parameter for this type of blur.

Many existing image restoration algorithms assume that the is known, but in practical it is not always the case. The restoration without knowing of the is called blind image restoration. Fourier methods can be used to estimate the defocus parameter through calculating a ratio of power of high frequencies portion to that of low frequencies portion. However, a main drawback of the method is its bad noise immunity. To solve this problem, [19] proposed a novel algorithm to overcome this shortcoming, the RBF neural network is applied to fit . This scheme has good fitting, but bad prediction. To avoid the weak generalization ability, an more efficient method for estimating parameter is proposed in this paper. The prediction ability of these two methods is compared with the trained five images. The steps of the presented algorithm in this paper is as follows: Firstly we construct feature vectors of several blurred images with known defocus radius in wavelet domain, then a multivariate local polynomial estimation model [2022] is trained using the vectors as inputs and defocus parameters as outputs. After the model is trained, the new defocused images are applied to the trained model for predicting the parameter . For a semi-blind defocused image, can be estimated through calculating the feature vectors and using it as input of the trained model. With known radius , many traditional algorithm could be applied to restore the degraded image. In this paper, iterative Wiener filtering [23] is adopted to image restoration.

2. Relationship between Wavelet Coefficients and

The wavelet transform provides a powerful and versatile framework for image processing. It is widely used in the fields of image denoising, compression, fusion, image restoration [2426], and so forth.

The two-dimensional discrete wavelet transform () [27, 28] hierarchically decompose an input image into a series of successively lower resolution images and their associated detail images. is implemented by a set of filters, which are convolved with the image rows and columns. An image is convolved with low-pass and high-pass filters and the odd samples of the filtered outputs are discarded resulting in downsampling the image by a factor of 2. The level wavelet decomposition of an image results in an approximation image and three detail images , and in horizontal, vertical, and diagonal directions respectively. Decomposition into levels of an original image results in a down sampled image of resolution with respect to the image as well as detail images.

When an image is defocused, edged in it are smoothed and widened. The amount of high frequency band decreased, and that corresponding to low frequency band increases.

In order to denote the relationship between wavelet coefficients and defocused radius , we define five variables named , and as follows: where represents the summation of all coefficients' absolute value, is total number of coefficients.

An original image is blurred artificially by a uniform defocus with whose value rang from 1 to 20. The relationship between and is shown in Figure 1, where the curves are normalized in interval. When increases, and decrease monotonously.

In order to estimate defocus parameter , only known the roughly similar relationship is not enough. As shown in Figure 2, every image has monotonous curve between and , but they are not superposition. For a degraded unknown image, cannot be calculated because the curve of the given image is not known. For example, if of image “rice” has been calculated, and then we estimate according curve if “ic” in Figure 2, wrong results are obtained obviously. To solve this problem, one of the methods is to choose neural networks. Computational artificial neural networks are known to have the capability for performing complex mappings between input and output data, but neural network method has bad generalization ability. Here we propose a multivariate local polynomial regression model to estimate . The variables are chosen to train the multivariate local polynomial estimation model. Prediction Comparison is made to verify the advantages of multivariate local polynomial fitting.

3. Training Multivariate Local Polynomial Estimation Model

Multivariate local polynomial fitting is an attractive method both from theoretical and practical point of view. Multivariate local polynomial method has a small mean squared error compared with the Nadaraya-Watson estimator which leads to an undesirable form of the bias and the Gasser-Muller estimator which has to pay a price in variance when dealing with a random design model. Multivariate local polynomial fitting also has other advantages. The method adapts to various types of designs such as random and fixed designs, highly clustered and nearly uniform designs. Furthermore, there is an absence of boundary effects: the bias at the boundary stays automatically of the same order as the interior, without use of specific boundary kernels. The local polynomial approximation approach is appealing on general scientific grounds: the least squares principle to be applied opens the way to a wealth of statistical knowledge and thus easy generalizations. In this Section, we briefly outline and review the idea of the extension of multivariate local polynomial fitting [2022] to the parameter of defocused .

3.1. Multivariate Kernel Function

To localize data in the -dimension, we need a multikernel function. Generally speaking, a multivariate kernel function refers to an -variate function satisfying

Here and hereafter, we use to indicate multivariate integration over the -dimensional Euclidean space.

There are two common methods for constructing multivariate kernel functions. For a univariate kernel , the product kernel is given by and the spherically symmetric kernel is defined as where is a normalization constant and . Popular choices of include the standard -variate normal density and the spherical kernel The latter is the optimal kernel, according to Fan and Gijbels [21] and Su [22].

The localization in multivariate nonparametric regression is frequently carried out by the kernel weighting. Let be a symmetric positive-definite matrix called a bandwidth matrix. The localization scheme at a point assigns the weight where is the determinant of the matrix . The bandwidth matrix is introduced to accommodate the dependent structure in the independent variables. For practical problems, the bandwidth matrix is taken to be a diagonal matrix. The different independent variables will be accommodated into different scales. For simplification, the bandwidth matrix is designed into ( denoting the identity matrix of order ).

3.2. Multivariate Predictor with Local Polynomial Fitting

Suppose that the input vector is , the model is fitted by the function

Our purpose is to obtain the estimation of function , this paper, we use the th order multivariate local polynomial to predict the defocused parameter value based on the point of the test image. The polynomial function can be described as where

In the multivariate prediction method, denotes the trained image feature vectors. Using pairs of , for which the values are already known, the coefficients of is determined by minimizing

For the weighted least squared problem, a matrix form can be described by where and is the

We then have the least squared solution with multivariate local polynomial fitting as follows: or, when is inverse, the estimation can be written by then we can get the estimation where .

Computing the will suffer from large computational cost. we can use the recursive least squared method to reduce the computation complexity, and it is very powerful especially in the real time prediction problems. There are several important issues about the bandwidth, the order of multivariate local polynomial function and the kernel function which have to be discussed. The three problems will be presented in Section 3.3.

3.3. Parameters Selections

For the multivariate local polynomial predictor, there are three important problems which have significant influence to the prediction accuracy and computational complexity. First of all, there is the choice of the bandwidth matrix, which plays a rather crucial role. The bandwidth matrix is taken to be a diagonal matrix. For simplification, the bandwidth matrix is designed into . So the most important thing is to find the bandwidth . A too big bandwidth underparameterizes the regression function, causing a large modeling bias, while a too small bandwidth overparameterizes the unknown function and results in noisy estimates. In theory, there exists an optimal bandwidth in the meaning of mean squared error, such that

But the optimal bandwidth cannot be solved directly. So we discuss how to get the asymptotically optimal bandwidth. There are quite a few important techniques for selecting the bandwidth. such as cross-validation and plug-in bandwidth selectors. a conceptually simple technique, with theoretical justification and good empirical performance, is the plug-in technique.

Another issue in multivariate local polynomial fitting is the choice of the order of the polynomial. Since the modeling bias is primarily controlled by the bandwidth, this issue is less crucial however. For a given bandwidth , a large value of would expectedly reduce the modeling bias, but would cause a large variance and a considerable computational cost. Since the bandwidth is used to control the modeling complexity, and due to the sparsity of local data in multidimensional space, a higher-order polynomial is rarely used. We use the local quadratic regression to indicate the flavor of the multivariate local polynomial fitting, that is to say, .

The third issue is the selection of the kernel function. In this paper, of course, we choose the optimal spherical kernel function, which minimizes the asymptotic of the resulting multivariate local polynomial estimators, as our kernel function.

3.4. Estimating the Defocused Parameter

Twenty original images are chosen to train the model. The images are defocused artificially with whose value ranging from 2 to 7. So the total number of training samples are 120. Then feature vectors are constructed using variables of each image:

The defocused parameter is the model output.

When training samples are given, obtaining weights matrix , according to the relationship between the and , then the defocused parameter can be calculated using the trained model.

4. Iterative Wiener Filter

Wiener filtering (minimizing mean square error) is commonly used to restore linearly-degraded images. To obtain optimal results, there must be accurate knowledge of the covariance of the ideal image. In this section, the so-called iterative Wiener filter [19, 23] is used to restore the original image.

The imaging system is assumed to be linear shift invariant with additive, independent, white noise processes of known variance. the model for the observed image is given in matrix notation by where is the ideal image. The optimal linear minimum mean-squared error, or Wiener restoration filter given by where , requires accurate knowledge of , the autocorrelation of ideal image . However, in practical situations is usually not available and only a single copy of the blurred image to be restored, , is provided. In the absence of a more accurate knowledge of the ideal image , the blurred image is often used in its place simply because there is no other information about readily available. The signal is subsequently used to compute an estimate of and this estimate is used in place of in (4.2).

The following summarizes the iterative Wiener filtering procedure.

Step 1 (Initialization). Use to compute an initial estimate of by

Step 2 (Filter Construction). Use , the th estimate of to construct the restoration filter given by

Step 3 (Restoration). Restore by the filter to obtain , the estimate of

Step 4 (Update). Use to compute an improved estimate of , given by

Step 5 (Iteration). Increment and repeat Steps 2, 3, 4, and 5.

5. Experimental Results and Analysis

The experiments are carried out by using the Matlab image processing toolbox. The performance of the proposed image restoration algorithm has been evaluated using the classical gray-scale image, image, image, and image in Matlab toolbox. To verify the good ability of restoration of the proposed algorithm, one real blurred image is used for the deconvolution procedure. The results show our method is very successful for this kind of blurred image.

In image restoration studies, the degradation modelled by blurring and additive noise is referred to in terms of the metric blurred signal-to-noise ratio (). This metric for a zero-mean image is given by where is the noise-free blurred image and is the additive noise variance.

For the purpose of objectively testing the performance of linear image restoration algorithms, the improvement in signal-to-noise ratio is often used. is defined as where and are the original and degraded image pixel intensity values and is the restored true image pixel intensity value. cannot be used when the true image is unknown, but it can be used to compare different methods in simulations when the true image is known.

In order to find the good performance of the proposed multivariate local polynomial Regression method (MLPR) compared with the RBF neural network algorithm (RBFNN) which is investigated in the literature [19], the same defocused blurred images are used for the experiments. Mean squared prediction errors are shown in Table 1. From Table 1, we can conclude that the prediction results of predictor are significantly better than the RBF neural network method in the same simulated data.

Figures 3, 4, 5 and 6, in which the true images, blurred images and estimated true images are depicted in the left, middle and right column, respectively, illustrate how the method behaves in , , and images. It is clear from Figures 36 that performance of the new method is effective in different images. Figure 7 also shows that the presented algorithm is good for real blurred image.

6. Conclusions

A new method that is based on multivariate local polynomial regression model and iterative Wiener filtering for semi-blind restoration of blurred noisy images was proposed in this paper. Defocused parameter was estimated by a multivariate local polynomial regression model trained in wavelet domain. The main advantages of the proposed technique are that it is not only robust to noise because wavelet transform has an excellent denoising ability, but also effective to artificially and practically defocused blurred image. Restoration is successfully realized by the iterative Wiener filter, resulting in improved the image quality. The algorithm was justified via simulation and real image. Defocused image parameter can be successfully estimated by using trained model. Experimental results show the proposed algorithm is reliable and robust for defocused blurred image restoration.

Acknowledgments

This work was supported by Chongqing CSTC foundation of China (CSTC2009BB2420), Chongqing CMEC foundation of China (KJ100810, KJ100818), and CQUT foundation of China (2007ZD16).