Abstract
A semiblind image deconvolution algorithm with spatially adaptive total variation (SATV) regularization is introduced. The spatial information in different image regions is incorporated into regularization by using the edge indicator called difference eigenvalue to distinguish flat areas from edges. Meanwhile, the split Bregman method is used to optimize the proposed SATV model. The proposed algorithm integrates the spatial constraint and parametric blur-kernel and thus effectively reduces the noise in flat regions and preserves the edge information. Comparative results on simulated images and real passive millimeter-wave (PMMW) images are reported.
1. Introduction
Image deconvolution is a classical inverse problem that has been widely investigated in astronomical, medical, and remote sensing imaging. Over the years, many deconvolution algorithms have been developed (for reviews see [1, 2]). In this paper, we are concerned with the semiblind deconvolution, in which the point spread function (PSF) is described as a parametric Gaussian form. The degraded model can be described as where denotes the convolution operator with a Gaussian kernel parameterized by its width , that is, , denotes the system noise, and and are the observed image and the true image, respectively.
Deconvolution is an ill-posed problem and should be regularized. The TV regularization was first proposed by Rudin et al. [3] in image denoising. Because of its advantages in preserving edge and detailed information, it has attracted strong research interest [4–6]. Nevertheless, TV model has certain shortcomings. First, TV regularization favors a piecewise constant solution, and, as a result, the processing results in the flat region are poor and some “staircase effects” are produced (Figure 1(b)). Second, TV-based method is sensitive to the regularization parameter. For instance, a large regularization parameter can help reduce the noise in flat regions but blur the edge and detailed information at the same time (Figure 1(c)).
(a)
(b)
(c)
In many practical situations, the PSF can be modeled by the physics/optics of the imaging set-up and described as parametric function. Then, during the deconvolution, only the PSF parameters need to be estimated. It is often called semiblind deconvolution. Some works are available in [2, 6, 7]. In [7], Bar et al. presented an integrated variational framework for simultaneous semiblind restoration of Gaussian blur and image segmentation. The objective functional is where denotes the edge map and is the support of the image. They have demonstrated successful semiblind image restoration that is more robust than the total variation blind image restoration [4] where the kernel form is not restricted. However, the framework of [7] is very sensitive to noise. Extracting the edge map will fail in the presence of noise. Therefore, it is limited in practical application. Further illustration can be found in [8].
In this paper, a semiblind image deconvolution model with spatially adaptive TV regularization (SATV) [9] is proposed. An effective spatial information extractor called difference eigenvalue, recently proposed by Tian et al. [10], is used to extract the spatial information in the image. For each pixel, a weighted parameter determined by spatial information is added to constrain the TV regularization strength. For flat area pixels, a large weight parameter is set to suppress noise. On the contrary, for nonsmooth area pixel, a small weighed parameter is set to preserve edge and detail information. To make the SATV regularization easy to optimize, the split Bregman method is used to solve the nonsmooth energy model. Our experimental results show that the proposed SATV method can automatically balance the regularized strength between different spatial property regions in the image, and thus it can effectively reduce the noise in flat regions as well as preserving the edge and detailed information.
The remainder of this paper is as follows. In Section 2, the proposed model and efficient solution are presented. In Section 3, experimental results are given. Finally, conclusions are drawn in Section 4.
2. SATV Model and Optimization Solution
In this section, our spatially adaptive TV semiblind deconvolution model is presented, and then the split Bregman method is used to solve the model.
2.1. Formulation of Our Model
In this paper, a spatially weighted TV regularization model considering the spatial dependent property of the TV regularization is introduced. The key problem is to select a good spatial information indicator, which can distinguish edges from flat areas in the image. Several alternative edge indicators have been proposed, such as gradient operator, Gaussian curvature, difference curvature [11], and difference eigenvalue [10]. As shown in Figure 2, the difference eigenvalue is more robust to indicate the edge under significant noise. So we use difference eigenvalue to indicate the spatial information.
(a)
(b)
(c)
(d)
The difference eigenvalue [10] is based on the Hessian matrix of the image. We use a Gaussian filtered version of the Hessian matrix to improve the robustness to noise: where denotes the Gaussian kernel with the parameter (the size is 5 × 5 and in this paper). The two eigenvalues of , denoted by and , are given by Let denote the larger eigenvalue and let denote the other eigenvalue. Here and correspond to the maximum and minimum local variation at a pixel, respectively. The difference eigenvalue edge indicator is defined as where is a weighting factor which is used to achieve balance between detail enhancement and noise suppression. Its value is estimated from the gray-level variance and defined by where and are minimum and maximum gray-level variances of , respectively. For a given pixel with coordinates , the gray-level variance is calculated from its 3 × 3 neighborhood:
With the difference eigenvalue edge indicator , the SATV model proposed in this paper is defined as Here is the spatial weight and is the contrast factor. To summarize, several good features of SATV can be drawn: (1) for flat area pixels, because is close to zero, is close to 1, which means that a large TV regularization strength is enforced, and then the noise will be well suppressed; (2) for edge and detailed pixels, because is very large, is small and almost close to zero, and this weakens the TV regularization strength, so the edge and texture will be well preserved.
Substituting in (2) with SATV presented in (9), we introduce a new total bounded variation-based image restoration model as
Note that the quadratic regularization term about provides the probability to discriminate the structure of stability results from that of the nonquadratic bounded variation term [12].
2.2. Numerical Algorithm
The split Bregman iteration method [5, 13] is used to solve the nonsmooth model (10). The discrete version of (10) is expressed as The discrete spatially weighted TV model is defined by where and denote the horizontal and vertical first-order differences at pixel , respectively. By letting , this yields a constraint optimization problem:
Rather than considering (13), we will consider an unconstraint problem: Then the corresponding split Bregman algorithm is given as shown in Algorithm 1. Here, is a small positive constant.
| ||||||||||||||||||
Note that can be computed by direct formula and to get we solve a linear system of equation: where is the adjoint operator of . Minimizing with respect to the scalar parameter , one can obtain where The bisection method was used to solve (17) for . The discrete support of the Gaussian was limited at about . Equation (15) is computed in the Fourier domain, which helps accelerate the convolution operation [14].
3. Experiments’ Results
In this section, numerical results are presented to test the proposed method. For comparison, the semiblind restored method proposed by Bar et al. [7] is also tested. We call it Bar-SBD method later.
Peak-signal-to-noise ratio (PSNR) and the structural similarity (SSIM) [12] are used to measure the restored quality. For the simulation, the satellite, Lena, and cameraman images were blurred by Gaussian PSFs with standard deviation 2.1 of sizes 11 × 11 and 15 × 15, respectively. Then zero-mean Gaussian noise with standard deviations 0.001, 0.005, and 0.01 is added by the MATLAB command imnoise. The initial value was 0.5 and convergence tolerance was 10−3.
Example 1. Figure 3 shows the restoration results for the PSF size of 11 × 11 and noise level of 0.005. From the three images, it can be seen that our method has restored much sharper results with less residual noise. Moreover, our method showed good convergence property. The estimated kernel width converged nearly to the accurate value after about 15 iterations, as illustrated in the bottom line of Figure 3.
(a)
(b)
(c)
(d)
Example 2. We restored the images degraded by Gaussian PSF size of 15 × 15 and the three noise levels. The PSNR and the calculating times of the two methods are listed in Table 1. is the estimated kernel width. The implementations were in MATLAB on a 2.93 GHz PC. The advantage of our method appears obvious. In every case, our method has achieved the higher PSNR and the lower computational time. Also, in every case, our method obtains more accurate estimates of the actual blur parameter. These results show that our method outperforms the Bar-SBD method at a wide range of noise levels.
Example 3. A restoration example of real passive millimeter-wave (PMMW) image is shown in Figure 4. The PMMW image was taken with a 94 GHz passive millimeter-wave imager at London Heathrow airport.
(a)
(b)
Because of the long wavelength of millimeter-wave, the PMMW image often suffers poor spatial resolution.
Image deconvolution has been extensively researched to improve the resolution [15]. From the results, it can be seen that sharper details are restored by our method, for instance, the number “2” and the car at right corner. Benefiting from the spatially adaptive regularization and split Bregman solution, the proposed method converges well for the practical degraded image.
Example 4. Another restoration example of real PMMW image is shown in Figure 5. It can be observed from Figure 5(b) that the recovered image by Bar-SBD amplifies the noise obviously. As shown in Figure 5(c), the restored image by proposed method looks natural and the noise is suppressed well. The convergence process of the proposed method is illustrated in Figure 5(d).
(a)
(b)
(c)
(d)
We remark that there are several parameters to be tuned up in our proposed methods, and we have to choose optimal parameters in order to make the relatively fair comparisons. We have tried our best to determine the optimal parameters. In (9), the parameter in the spatial weight controls the ability to distinguish the edge and texture area from the flat and ramp area. To show the effect of the parameter on the restored result, Table 2 lists the PSNR and SSIM value when is selected to be 0.0005, 0.01, 0.05, 0.1, 0.5, 1, and 5, respectively. It can be seen that the PSNR and SSIM values show little change with the change of the parameter and achieve little higher values when ranges from 0.1 to 1. In this paper, the parameter was set to be 0.5.
The parameters , , , and were selected manually to provide the best PSNR results. We first fix the parameters , , , and and adjust such that it gives the best restoration results measured by PSNR; in our implementation, we have found that is a relatively fair value in most cases for different images. The remaining parameters are tuned up one by one. This procedure is repeated several times for each parameter until they become stable. In this paper, we set parameter . The parameters and were chosen by trial and error tests. In all our experiments, the parameters used were and . Although the better results could be obtained with “optimal” tuning of these parameters, it was found that these heuristic values achieved good results in most cases. In fact, the same parameter only needs to be tuned up slightly for different images by the same blur-kernel and corrupted by the same noise level; this significantly reduces the computational time of looking for good parameters.
4. Conclusion
In this paper, we proposed a semiblind deconvolution method with spatially adaptive TV regularization. The proposed method can automatically balance the regularization strength between different spatial property regions in the image. Also, the split Bregman method is used to optimize the nonsmooth model. Comparative results on simulated and real degraded images show that the proposed method can effectively reduce the noise in flat region as well as preserving the edge and detailed information.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research work is supported by Science and Technology Department of Jiangsu Province of China (no. BE2011747, no. BE2011822) and National Natural Science Foundation of China (no. 61105015). The authors would like to thank Dr. Peter R Coward of QinetiQ Malvern, Worcs., UK, for providing the real experiment data of PMMW image.