Abstract
It has been proved that total generalized variation (TGV) can better preserve edges while suppressing staircase effect. In this paper, we propose an effective hybrid regularization model based on second-order TGV and wavelet frame. The proposed model inherits the advantages of TGV regularization and wavelet frame regularization, can eliminate staircase effect while protecting the sharp edge, and simultaneously has good capability of sparsely estimating the piecewise smooth functions. The alternative direction method of multiplier (ADMM) is employed to solve the new model. Numerical results show that our proposed model can preserve more details and get higher image visual quality than some current state-of-the-art methods.
1. Introduction
Image restoration refers to the problem of recovering image that satisfies people’s needs from an observed image that degraded by different blur and noise. The problem is mainly applied to remote sensing, medical image, video cameras, and other fields [1–5]. There are many factors that cause image degradation, such as atmospheric turbulence, camera shake, and relative motion between the camera and the object [6]. To obtain higher quality images, many methods have been proposed to solve the degenerate model, for instance, variational analysis [7]. In this paper, the degenerate model can be expressed as the following form:where , are degenerate and real images, respectively, is a blur operator, we use the “fspecial” function to describe the blur operator “” in this paper, and is the additive white Gaussian noise with variance and zero-mean.
Recovering from is an inverse problem. In order to deal with this problem, scholars have done a lot of researches on this problem. One of the most effective ways to deal with this problem is adding some regularized terms to objective function. This leads to the following restoration model:where denotes the Euclidean norm, is a regularized term which is used to regularize the solution, and is a positive regularization parameter which controls the two terms.
The traditional regularization terms include the Tikhonov-like regularization [8] and the total variation (TV) regularization [9]. Tikhonov-like regularization: , where is finite different operator. Due to the fact that Tikhonov-like regularization tends to make images overly smooth in the process of image processing, it fails to preserve sharp edges. On the contrary, TV regularization does better in protecting the sharp edge of image [10]. TV regularization: or . In (2), is isotropic regularization term if the norm is 2-norm, and is anisotropic regularization term if the norm is 1-norm. TV regularization was first introduced by Rudin et al. in 1992 and widely used to solve problem (1). Although the TV regularization is better than the Tikhonov-like regularization in preserving sharp edges, it often produces staircase effects. The reason of causing the staircase effects is that TV regularization tends to transform the smooth regions of the solution into piecewise constant regions during solving the minimization problem [11, 12].
The TV regularization effectively reduces the noise and preserves the sharp edges. But there still exist some undesired staircase effects in smooth regions of the restored images. To suppress the staircase effects, many improved models based on TV regularization term are proposed, such as high-order TV model [12–14], hybrid TV model [15–19], fractional order TV model [20–28], and total generalized variation model [29–34]. The TGV regularization is a useful tool to remove the staircase effect as well as preserve the sharp edge. In this paper, we focus on the total generalized variation regularization which can be seen as a generalization of total variation. The concept of TGV regularization was first proposed by Bredies et al. [29] as penalty function for image processing. It is worth noticing that TGV involves and balances higher-order derivatives of . This results in the fact that the reconstruction by using TGV regularization can preserve edges while suppressing staircase effect. In order to solve the TGV model efficiently, many optimization algorithms have been proposed, such as Newton’s method, split Bregman method, alternating direction method of multipliers, and gradient descent method [35–43]. Experiments show that TGV has the superior performance to TV based regularization models in image reconstruction. In other words, TGV regularization model has some advantages in restraining the staircase effects caused in TV based regularization models.
In recent years, sparse representation aroused people’s attention [44]. In most cases, images are usually sparse in some domain such as Fourier, cosine, wavelet, and wavelet frame [45, 46]. In fact, images can be sparsely approximated by proper wavelet frame [47–49]. So, the sparse representation based on the wavelet frame has become a hot topic in the process of image restoration. Correspondingly, the effective model is the -norm of the wavelet frame coefficients, because of its sparsity and convexity. Numerical experiments show that the models based on wavelet frame and variational methods can significantly improve the quality of images [50, 51]. However, the only fly in the ointment is that the Gibbs-like oscillations emerge frequently around the image discontinuities.
Therefore, to better reconstruct the degraded image and simultaneously preserve image features, a new edgepreserving regularization scheme is reported in this work. Inspired by the above-mentioned advantages of wavelet frame based methods and TGV, and avoiding their main shortcomings, we concentrate on a novel hybrid regularizers model for image restoration by combining TGV and wavelet frame. Owing to the proposed model making good use of the advantages of wavelet frame and total generalized variation regularization, the new proposed model can not only protect the sharp edges of the images, but also make good use of the sparse prior information. In addition, the staircase effects are also effectively suppressed.
Experimentally, we employ the ADMM technique to develop a restoration method to solve the proposed model. First, three auxiliary variables are introduced to transform the proposed model into a new constrained problem. Next, we utilize the variable splitting technique to transform the constrained problem into the unconstrained problem. Finally, the obtained unconstrained problem can be solved by the alternative direction method of multiplier. Numerical experiments show that the new proposed model is very effective compared to several state-of-the-art methods in eliminating staircase effects and recovering some details.
The rest of the paper is organized as follows. In Section 2, we briefly introduce the concept of TGV and wavelet frame. In Section 3, we present the proposed model and use the alternative direction method of multiplier (ADMM) to solve it. In Section 4, we show several numerical experiments to demonstrate the effectiveness of the proposed model. Finally, we summarize this article in Section 5.
2. Review of Total Generalized Variation and the Wavelet Frame
In this section, we will briefly introduce the concepts of the TGV and wavelet frame.
2.1. Total Generalized Variation
The concept of the TGV was first proposed by Bredies et al. [29], which is considered as a generalization of TV. Later, many scholars applied TGV to the field of image processing to suppress the staircase effects. The experimental results have certain advantages over the existing methods; see [30, 33]. The TGV model can be defined as where , it is the space of symmetric k-tensors on ; is image domain, is the space of compactly supported symmetric tensor field; is the norm, and is fixed positive parameter; is the generalization of the divergence operator of order to the tensor field. From the definition of , we can see that TGV is a generalization of TV. When , , can be seen as the classical TV; for more details, see [29, 33, 34].
In this paper, we focus on the second-order TGV, which can be written as where is the space of all symmetric matrices and is the space of continuously differentiable symmetric matrix field with the compact support in . For , the divergence of is defined as ; , and the infinity norm of and is given by In the following, we focus on the dimension and denote the spaces , and . As noted in [29, 32], the discrete of can be rewritten as the following equivalent form:where denotes symmetrized derivative, represents the original image, is the ordinary gradient operator, and represent two first-order forward finite difference operators in directions and . Furthermore, we denote , which is the ith row of and , and is the gradient of . Let and Then, we can discretize the symmetric operator asand the ith component of is denoted as
2.2. The Wavelet Frame
In this subsection, we briefly introduce some notations of wavelet frames. In the discrete setting, let stand for the fast tensor product framelet decomposition and be the fast reconstruction; then we obtain for any images based on the unitary extension principle (UEP) [52] since , where is the identity matrix. The construction of framelets can be obtained according to the unitary extension principle (UEP) as well. In experiments, we will use the piecewise linear B-spline framelets constructed by [52] considering the balance of the time and quality. We denote an -level framelet decomposition of as where denotes the index set of all framelet bands and is the wavelet frame coefficients of in bands at level . We will also use to denote the frame coefficients; i.e., , where For more details on discrete algorithms of framelet transforms, see [53, 54].
To use the sparseness of the wavelet frame coefficients, there are several different wavelet frame based models proposed in the papers, i,e., the synthesis based approach [55, 56], the analysis based approach [57, 58], and the balanced approach [51, 53]. Although these approaches are different, they can guarantee a clear recovery from the unknown clean image as long as certain conditions are satisfied.
3. The Proposed Model and Algorithm
In this section, we will introduce a new hybrid regularization model based on TGV and wavelet frame. The new model can suppress the staircase effect while protecting the edge of the image. Alternating direction method of multipliers is used to solve the proposed minimization problem.
3.1. The New Model
Combining wavelet frame and TGV regularizations, we propose the following hybrid regularization model:where is the total generalized variation of ; its definition has been given in Section 2.1. Equation (11) can be rewritten aswhere are positive regularization parameters that balance the four terms for minimization. is the discrete wavelet frame transform; more details have been described in Section 2.2. Note that we name the proposed hybrid regularization model (12) as the TGVframe restoration model.
3.2. The Proposed Algorithm for the New Model
In this subsection, the alternating direction method of multipliers is employed to solve TGVframe restoration model (12). First, by introducing three auxiliary variables , , , we can transform model (12) into the following constrained problem:
For the above constrained problem (13), using the classical quadratic penalty method, we can transform it into the following unconstrained problem:where are positive penalty parameters. For the above unconstrained optimization problem (14), its augmented Lagrange function is given bywhere , , are Lagrange multipliers, which control positive penalty parameters going to infinity. According to the classical ADMM, we should solve the following iterative scheme:
For the -subproblem, it can be rewritten asBased on the optimality condition [59], the minimization problem (17) can be solved by the following linear equation:where refers to the first-order difference operator and is conjugate operator of . Under the periodic boundary condition for , and are block circulant matrices [59]. So and can be diagonalized by the Fourier transform. The Fourier transform of is defined as . is the inverse Fourier transform of . By using the Fourier transform, we can gain the solution of as follows:where
Solving the -subproblem is similar to -subproblem, and the solution of -subproblem can be given byBecause , we only need to get and .
Then, for , we havefor ,The solutions of problems (22) and (23) are similar to that of problem (18), so we can solve problems (22) and (23) with several FFTs and IFFTs [59] under the periodic boundary condition.
For the minimization of in (16), the subproblem is given byProblem (24) can be solved by the the following four-dimensional shrinkage operator:
For -subproblem, it can be given byThe solution of problem (26) is given by the one-dimensional shrinkage operation:
For -subproblem, it can be written asThe solution of problem (28) can be given by the following two-dimensional shrinkage operation:
Finally, we update Lagrange multiplier through the following formula, respectively,
In this paper, our algorithm is simply denoted as TGVframe. The TGVframe algorithm of the deblurring and denoising is summarized as shown in Algorithm 1.
4. Numerical Experiments
In this section, in order to show the effectiveness of the proposed model in suppressing staircasing effect, protecting the sharp edges, and removing Gaussian noise in image restoration, we compare the proposed models with FTVd version 4.1 [10], TVframe [49], and TGV [29]. All the experiments were performed under Windows 10 and MATLAB 2012a running on a desktop with an core i5 Duo central processing unit at 2.50 GHz and 4 GB memory to ensure the fairness and effectiveness of the test.
In the experiments, we select “Cameraman” (), “Lena” (), “House” (), and “Barbara” () as the tested images; see Figure 1. For all experiments, the tested images are degraded by different blur kernel and additional Gaussian white noise with different standard deviation. We use signal-to-noise ratio (SNR) and structural similarity (SSIM) to measure the quality of the restored images. The larger the SNR is, the better the image quality is. The SNR and SSIM are defined as

(a) Cameraman

(b) Lena

(c) House

(d) Barbara
Here , are the ideal image and the restored image, respectively. is the mean intensity value of , and and are the mean value of the and , respectively. and are the variance of and , respectively, and is covariance of and . and are positive constants which can be seen as stabilizing constants for nearzero denominator values. The SSIM is an index used to measure the similarity between the restored image and the ideal image. For more details about SSIM, see [60].
In order to have fair comparisons in all the experiments, we terminate all the algorithms by the following stopping condition:
4.1. Gaussian Blur Experiment
In this subsection, Gaussian blur experiment is considered. It is known that the size of blur kernel determines the quality of the recovered images to a certain extent. The larger the blur kernel is, the lower the recovered image quality is. In order to test the effectiveness of our method for different blur kernels, we do a test for the images with different size of blurring kernels. The values of SNR and SSIM by four different methods under different Gaussian blur kernels are shown in Table 1. In this test, we select the well-known image “Cameraman” which degraded by being blurred with a Gaussian kernel and contaminated by Gaussian noise with standard variance as tested image displayed in Figure 2(b), and the recovered results are shown in Figure 2. In our method, we select parameters , , and ; = 300 and = 10 in FTVd; , in TVframe; and , in TGV. Figure 3 shows the enlarged results of a part of Figure 2. From Figures 2 and 3 and Table 1, it is not difficult to see that our method still has good restoration when the size of blur kernel reached , which shows that our method is robust to blur kernel. Meanwhile, our method can also better suppress the staircase effect. In order to show our proposed method is convergent, we also plot the changing curve of SNRs versus iteration numbers in Figure 4.

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

(a) , Gaussian(11, 11)

(b) , Gaussian(9, 9)
4.2. Average Blur Experiment
In this experiment, we test the well-known image “house” which has many texture details that help to show the effectiveness of our method. The “house” image is degraded by blurring kernel with average kernel and noise derivation ; see Figure 5(b). The effect diagrams that are restored with FTVd, TVframe, TGV, and our method are shown in Figures 5(c)–5(f). In this test, the parameters , , and are used in TGVframe; in FTVd, and ; the parameters and are selected in TVframe; the parameters , are used in TGV. The restoration results are shown in Figure 5. We have enlarged some details of the four restored images for a better visualization and the effects of the enlarged part are shown in Figure 6. As it is seen in the zoomed parts, our method outperforms the other three methods. The values of SNR and SSIM of the four images are listed in Table 2. From Figures 5 and 6 and Table 2, we can find that the restored images by FTVd, TGV, and TVframe have some piecewise constant regions, while our method does better and has better qualities with more details, textures, and less stair-casing effect.

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)
4.3. Motion Blur Experiment
In this experiment, the image “Barbara” is degraded by blurring kernel with motion kernel and noise derivation . In TGVframe, we select , , and ; the parameters = 400 and = 15 are selected in FTVd; the parameters = 2e-4 and = 1e-4 are selected in TVframe; the parameters , are used in TGV. The restoration results by FTVd, TVframe, TGV, and TGVframe are shown in Figures 7(c)–7(f). It is not difficult to find that the visual effect of the restored image by our method is quite competitive with those restored by the other three methods. Several zoomed regions of Barbara image are shown in Figure 8. We can easily find that our method performs better than the other methods in detail texture. The values of SNR and SSIM are listed in Table 3. The values of SNR and SSIM of the recovered image by our method are superior to those by the other methods: FTVd, TVframe, and TGV. To further illustrate the effectiveness of our method, we also verify the effect of our method for image restoration with different noise under the same blurring kernel. The experimental results are shown in Table 4. We can see that our method is also applicable to different noises.

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)
4.4. Out-of-Focus Blur Experiment
In this experiment, the image “Lena” is degraded by out-of-focus blur kernel with radius 9. In this subsection, we use the MATLAB function “fspecial“ to characterize the out-of-focus blur. Besides, we add the Gaussian white noise with to the image “Lena.” The parameters in the out-of-focus blur test are set as follows: we select , , in our method; = 300, = 15 in FTVd; , in TVframe; , in TGV. In Table 5, we summarize the image Lena restoration results of our method, FTVd, TGV, and TVframe; and the recovered images are shown in Figures 9(c)–9(f). We can see that our method can get higher SNRs and SSIMs and higher image visual quality. In addition, the zoomed parts of “Lena” are listed in Figure 10. As it is seen in Figures 10(c)–10(f), our method can restore more details than the other methods and can overcome the staircase effects.

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)
4.5. The Experiment Compared with BM3D
In this subsection, we do an experiment compared with BM3D [61] which is a better deblurring and denoising method. In this test, we select the image “Cameraman” which degraded by pixel Gaussian blur with two different standard deviations and contaminated by Gaussian noise with standard variance . The results are shown in Figure 11 and the enlarged parts are shown in Figure 12. It is easy to see that our method achieves a better recovery effect than BM3D.

(a) Degenerated image with standard deviation 1.6

(b) SNR = 23.06, SSIM = 0.5937

(c) SNR = 23.51, SSIM= 0.6152

(d) Degenerated image with standard deviation 2.5

(e) SNR = 22.23, SSIM = 0.5586

(f) SNR = 22.85, SSIM = 0.5845

(a)

(b)

(c)

(d)

(e)

(f)
5. Conclusion
In this paper, we propose an effective image restoration model based on total generalized variation and wavelet frame. In addition, we employ the alternating direction method of multipliers to solve it. Experimental results demonstrate that our method not only has better visual resolution, but also can suppress the staircase effect compared to some current state-of-the-art methods.
Data Availability
The test images in the article can be freely downloaded from this site http://sipi.usc.edu/database/.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the National Key Research and Development Program of China under Grant 2017YFC1405600, by the Training Program of the Major Research Plan of National Science Foundation of China under Grant 91746104, by National Science Foundation of China under Grant 61101208 and Grant 11326186, by Qindao Postdoctoral Science Foudation, China (2016114), by a Project of Shandong Province Higher Educational Science and Technology Program, China (J17KA166), and by Joint Innovative Center for Safe and Effective Mining Technology and Equipment of Coal Resources, Shandong Province of China and SDUST Research Fund (2014TDJH102).