Research Article | Open Access
Yi Xu, Ting-Zhu Huang, Jun Liu, Xiao-Guang Lv, "Split Bregman Iteration Algorithm for Image Deblurring Using Fourth-Order Total Bounded Variation Regularization Model", Journal of Applied Mathematics, vol. 2013, Article ID 238561, 11 pages, 2013. https://doi.org/10.1155/2013/238561
Split Bregman Iteration Algorithm for Image Deblurring Using Fourth-Order Total Bounded Variation Regularization Model
We propose a fourth-order total bounded variation regularization model which could reduce undesirable effects effectively. Based on this model, we introduce an improved split Bregman iteration algorithm to obtain the optimum solution. The convergence property of our algorithm is provided. Numerical experiments show the more excellent visual quality of the proposed model compared with the second-order total bounded variation model which is proposed by Liu and Huang (2010).
Image restoration problem is one of the earliest and most classic linear inverse problems [1–3]. In this class of problems, a noisy indirect observation of an original image is modeled as where is a bounded linear operator representing the convolution and denotes the additive noise.
Equation (1) is a typically ill-posed inverse problem; that is, a small change in will lead to huge deviation in the solution . Hence, to keep the numerical stability, the regularization method known as adding a regularization term to the energy function has been developed. The original scheme introduced by Tikhonov and Arsenin  is given by where the nonnegative function which regularizes the solution by enforcing certain prior constrains on original image is known as regularizationpenalty and the which measures the violation of the relation between and its observation is known as fidelity. The scale is called the regularization parameter; it compromises fidelity with penalty. By this Tikhonov regularization method, we can compute stable approximations to the original solution. However, this smoothness penalty model does not preserve the edge, sparsity pattern, and texture well because of its isotropic smoothing properties. To overcome this shortcoming, Rudin et al.  proposed the total variation (TV) based regularization scheme (the ROF model) where denotes a bounded subset with Lipschitz boundary, , and represents the distributional derivative of . Many computational methods [5–11] for solving (3) sprang up in recent years. For the moment we may think of the nonsmooth penalty term as the seminorm. More precisely, As a result, the bounded variation (BV) seminorm is endowed with . Then the Banach space (Ω) is essentially an extension of . This model has been extremely successful in a wide variety of image restoration problems, such as image denoising [5, 12], signal processing [13, 14], image deblurring , image decomposition, and texture extraction . However, this model also usually produces staircase effects and new edges that do not exist in the true image.
In [17, 18], the authors concentrate especially on the full norm as the regularization term, compared with the TV regularization, which is preferable due to its ability to preserve edges in the original image during the reconstruction process. Then specify the original problem to the following variation model : where is a closed, convex subset of . The space endowed with the norm is a Banach space. A quadratic regularization term is utilized in model (5) comparing with model (3), it serves two advantages. One advantage is that, for , it provides a coercive term for the subspace of constant function which is in the kernel of the -operator (in this model represents the gradient operator). The other advantage of the quadratic regularization term is that it provides the probability to discriminate the structure of stability results from that of the nonquadratic term.
As mentioned in [19–21], this technique preserves edges well, but the obtained images for this model are often piecewise constant. To prevent the staircase effect, we penalize jumps more; this can be achieved by taking the second derivation into account. So in this paper we present an improved model; that is, the second-order diffusive term is replaced by fourth-order diffusive term in the model (5); new model substantially reduces the staircase effect, while preserving sharp jump discontinuities (edges). Here we rewrite the proposed model: where the Frobenius norm of the Hessian matrix is (see ) and the is defined as follows .
Definition 1. Let be an open subset with Lipschitz boundary. is a subspace of functions such that the following equation is satisfied: where stands for the set of functions in with compact support in .
The organization of the rest of paper is as follows. In Section 2, we give a detailed description of the Bregman iteration method. In Section 3, we elaborate on the analysis of the extended split Bregman iteration method for the proposed model. In Section 4, the convergence analysis is displayed. Numerical experiments intended for demonstrating the effectiveness of our model are provided in Section 5. Finally, concluding remarks are given in Section 6.
2. Bregman-Related Algorithms
Bregman iteration is a concept that originated in function analysis for finding extrema of convex function , which was initially introduced and studied by Osher et al. for image processing . Now we will show the general formulation of Bregman iteration technique.
2.1. Bregman Iteration
In , Goldstein and Osher considered the generalized constrained minimizations of the form where and defined in are convex functions. The associated unconstrained problem is where is the positive parameter which should be chosen extremely large.
Definition 2. The Bregman distance of function between and is where stands for duality product and is in the subdifferential of at with We assume that is differentiable then problem (11) can be iteratively solved by  The convergence analysis of this Bregman iterative scheme was provide in . The computational performance of Bregman iteration relays on how fast the subproblem can be solved. Let , where is a linear operator. As show in [25, 27], iteration (14) can be reformulated as a simplified form This Bregman iteration which was proposed by Osher et al. for TV based image denoising  has two advantages ; the first one is that this method converges very quickly, especially for problem where contains 1-regularization term. The second advantage is that the parameter in (14) remains constant; so for the purpose of fast convergence, we can choose which minimizes the condition number of the subproblem. Due to the highefficiency and robustness of Bregman iteration method, it has been widely used for image reconstruction [27–29].
2.2. Split Bregman
Goldstein and Osher  proposed the split Bregman iteration based on the split formulation provided in  to attack the general 1-regularized optimization problem (11). In a recent paper , this method is used to solve general variational models for image restoration. The crucial point of the split Bregman method is that it could separate the 1 and 2 portions in (11). They converted (11) into the constrained optimization problem where and stand for the convex functions. Then transform it into an unconstrained problem This problem is similar to (11); so they enforced the simplified Bregman iterative algorithm to problem (18) This is called two-phase split Bregman iterative algorithm.
Then pay attention to the subproblem Due to the “split” of the 1 and 2 components of this function, the minimization problem was performed by iteratively minimizing with respect to and separately Step 1: , Step 2: .The speed of the split Bregman method is relayed on how quickly the two steps can be solved. A wide variety optimization techniques can be used to solve Step 1, for instance, the Fourier transform method and conjugate gradient method. Step 2 could be attacked with the extremely fast shrinkage formula, namely, where
3. The Proposed Algorithm
Due to , estimation procedures are more appropriate for image restoration, and TV norm is essentially norm of derivatives. However, the fourth-order total variation term in model (6) is continuous expression; by utilizing the similar method of discretizing total variation in , we can deduce the discrete fourth-order total variation (let represent the operator ):
So the proposed model (6) can be rewritten as follows:
Then we perform the split Bregman iteration to solve problem (24); this yields a constrained problem Obviously, problem (25) is equivalent to the following unconstraint problem: Concretely, the extended split Bregman iterative for solving (26) is depicted as with the update formula for
For more precisely, we derive our algorithm as follows.
Algorithm 3. We have the following steps.(1) Set input value . Set .(2) Update from (3) Update from (4) Update from (5) If stopping criterion holds, output the . Otherwise, set and go to Step 2.
4. Convergence Analysis
In this section, we concentrate on the rigorous convergence proof of our iterative algorithm. Our analysis below is similar to that in [33, 34], where the authors presented the analysis of the unconstrained split Bregman iteration in detail.
We note that the subproblems (27) are convex and differentiable; so the first-order optimality conditions for and are obtained as follows: where . The condition (32) will be used for analyzing the convergence property of our algorithm.
Proof. Reference  has shown that problem (26) exists a unique so the first order optimality condition holds
where and ; let ; then (32) is equivalent to
This illustrates that and are a fixed points of (27).
Let and denote the sequence generated by algorithm (27), and and represent the errors, respectively. Then subtracting every equation in (32) from the corresponding equations in (35), we give the result where ; we take the inner product of the first and second equations in (36) with respect to and separately; then followed by the same manipulations applied to the third equation of (36), we obtain Obviously, the third equation of (37) can be rewritten as Combining the first two equations of (37) with (38), we have By summing the equation bilaterally from to , we get Because , and is convex, then we have The fact that all terms involved in (40) are nonnegative leads to the following expression: With the assumption and letting , inequality (42) suggests the following conclusions: And hence The Bregman distance satisfies Associating this equation with (47), we get Again, together with (46) and the continuous property of , the following expression hold: With the similar skills, we have Combining (50)-(51), we obtain Finally, from (44) and (52), the main results are shown as follows: This proves Theorem 4.
5. Numerical Experiments
In this section, a number of experiments are performed to demonstrate the effectiveness and efficiency of our proposed split Bregman iteration (PSBI) algorithm for the fourth-order diffusive model (6), which will be compared with the split Bregman iteration (SBI) for second-order diffusive model provided in . All experiments are generated in MATLAB 7.10 environment on a desktop with Windows 7 operating system, 3.00 GHz Intel Pentium(R) D CPU, and 1.00 GB memory.
The performance of all algorithms is measured by the improved signal-to-noise ratio (ISNR) and mean squared error (MSE) defined as where , and denote the degraded, original, and recovered images, and and are the sizes of image, respectively. The higher ISNR or the lower MSE, the better quality of the deblurred images. Moreover, the stopping criterion of all algorithms measured by the difference between the consecutive iterations of the deblurred image satisfies the following inequality:
Three classical grayscale images with resolution of pixels in Figure 1 are used for synthetic degradations in our experiments. The blur kernels used for blurring are Gaussian blur (size of pixels, variance of 3), linear motion blur (length of 10 pixels and direction of 30 degrees), out-of-focus blur (size of 10 10 pixels, defocus radius of 4) and uniform blur (size of pixels). Periodic boundary conditions are considered in the following experiments.
In the first experiment, the original image “Cameraman” with complex background is blurred by Gaussian blur, out-of-focus blur, linear motion blur, and uniform blur, respectively. The blurred images are showed in Figures 2(a)–2(d), Figures 2(e)–2(h) show the deblurred results corresponding to the split Bregman iteration method for all the blur cases, and the selected parameter values corresponding to the SBI method are the same with those the authors given out in . In order to give explicit comparison, we show the restored images obtained by our PSBI method in Figures 2(i)–2(l), and the selected parameter values are , respectively. Through comparing the “sky” in those deblurred images we can see that the PSBI method recovers more details (removes the stair effect) than SBI method. However, due to the fourth-order diffusive term, our proposed method costs a little more time than the SBI method. Table 1 gives the CPU times, ISNR values, and MSE values of the SBI method and PSBI method for all the blur kernels mentioned above, which shows that our algorithm works well for images with complex background.
In the next test, we use Gaussian blur and out-of-focus blur to degrade the “Boat” image and then run the two algorithms many times to obtain the best results. In PSBI method, the selected parameters and iterations are for Gaussian blur and for out-of-focus blur, respectively. Figure 3 shows the recovered results of the two methods. And for the purpose of better illustrations, we provide a close-up of image region in Figure 4. We can see from Figure 4 that the restored images estimated by PSBI are better when compared to the ones deblurred by SBI; for example, see the “letter” in the stern of “Boat.” Meanwhile, Table 2 illustrates the ISNR values obtained by PSBI are higher than those obtained by SBI while the MSE values are lower. So, it is clear that our method can effectively reduce the “staircase” which always appeals in the two-order diffusive model.
Figures 5 and 6 show the comparison between PSBI and SBI algorithm for image “Lena.” Firstly we blur the “Lena” by linear motion blur and uniform blur and then select the deblurred result that looks best after carefully tuning the parameters; for linear motion blur, the parameter values and iterations are , and for uniform blur, they are . It can be seen from Figures 5 and 6 that many staircase effects in the smooth regions can be detected obviously in the images restored by the SBI model while seldom appear in our results, and the upper-left corners in image “Lena” deblurred by PSBI are more visual comparing with that obtained by SBI model. Table 3 shows that the restored images obtained by our model have higher ISNR values and lower MSE values than the second-order model.
In this paper, we propose the fourth-order total bounded variation regularization based image deblurring model and exploit the split Bregman iteration method to solve this new model. The convergence analysis of our algorithm is provided. Numerical experiments show that our algorithm works well for images with complex background or simple background. In our synthetic experiments, the fourth-order diffusive model yields better results than the second-order diffusive model. It is believed that the proposed model can be extended to further applications in image processing and computer vision.
The authors would like to thank the referees very much for their valuable comments and suggestions.
- C. R. Vogel, Computational Methods for Inverse Problems, vol. 23 of Frontiers in Applied Mathematics, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Pa, USA, 2002.
- M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging, Institute of Physics Publishing, Bristol, UK, 1998.
- H. Andrews and B. Hunt, Digital Image Restoration, Prentice Hall, Englewood Cliffs, NJ, USA, 1977.
- A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems, V. H. Winston & Sons, Washington, DC, USA, 1977.
- L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D, vol. 60, no. 1–4, pp. 259–268, 1992.
- Y. Li and F. Santosa, “An affine scaling algorithm for minimizing total variation in image enhancement,” Tech. Rep., Cornell University, Ithaca, NY, USA, 1994.
- R. Acar and C. R. Vogel, “Analysis of bounded variation penalty methods for ill-posed problems,” Inverse Problems, vol. 10, no. 6, pp. 1217–1229, 1994.
- A. Chambolle and P.-L. Lions, “Image recovery via total variation minimization and related problems,” Numerische Mathematik, vol. 76, no. 2, pp. 167–188, 1997.
- T. F. Chan and P. Mulet, “On the convergence of the lagged diffusivity fixed point method in total variation image restoration,” SIAM Journal on Numerical Analysis, vol. 36, no. 2, pp. 354–367, 1999.
- C. R. Vogel and M. E. Oman, “Iterative methods for total variation denoising,” SIAM Journal on Scientific Computing, vol. 17, no. 1, pp. 227–238, 1996.
- J. Zhang, K. Chen, and B. Yu, “An iterative Lagrange multiplier method for constrained total-variation-based image denoising,” SIAM Journal on Numerical Analysis, vol. 50, no. 3, pp. 983–1003, 2012.
- C. R. Vogel and M. E. Oman, “Iterative methods for total variation denoising,” SIAM Journal on Scientific Computing, vol. 17, no. 1, pp. 227–238, 1996.
- E. J. Candes and T. Tao, “Decoding by linear programming,” IEEE Transactions on Information Theory, vol. 51, no. 12, pp. 4203–4215, 2005.
- D. L. Donoho, “Compressed sensing,” IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, 2006.
- T. Chan and J. Shen, Theory and Computation of Variational Image Deblurring, IMS Lecture Notes, 2007.
- T. F. Chan, S. Esedo\=glu, and M. Nikolova, “Algorithms for finding global minimizers of image segmentation and denoising models,” SIAM Journal on Applied Mathematics, vol. 66, no. 5, pp. 1632–1648, 2006.
- X. Liu and L. Huang, “Split Bregman iteration algorithm for total bounded variation regularization based image deblurring,” Journal of Mathematical Analysis and Applications, vol. 372, no. 2, pp. 486–495, 2010.
- G. Chavent and K. Kunisch, “Regularization of linear least squares problems by total bounded variation,” ESAIM Control, Optimisation and Calculus of Variations, vol. 2, pp. 359–376, 1997.
- T. Chan, A. Marquina, and P. Mulet, “High-order total variation-based image restoration,” SIAM Journal on Scientific Computing, vol. 22, no. 2, pp. 503–516, 2000.
- T. F. Chan, S. Esedoglu, and F. E. Park, “A fourth order dual method for staircase reduction in texture extraction and image restoration problems,” CAM Report, UCLA, 2005.
- Y.-L. You and M. Kaveh, “Fourth-order partial differential equations for noise removal,” IEEE Transactions on Image Processing, vol. 9, no. 10, pp. 1723–1730, 2000.
- H.-z. Chen, J.-p. Song, and X.-C. Tai, “A dual algorithm for minimization of the LLT model,” Advances in Computational Mathematics, vol. 31, no. 1–3, pp. 115–130, 2009.
- Z.-F. Pang and Y.-F. Yang, “Semismooth Newton method for minimization of the LLT model,” Inverse Problems and Imaging, vol. 3, no. 4, pp. 677–691, 2009.
- L. M. Brègman, “A relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming,” USSR Computational Mathematics and Mathematical Physics, vol. 7, no. 3, pp. 200–217, 1967.
- S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, “An iterative regularization method for total variation-based image restoration,” Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 460–489, 2005.
- T. Goldstein and S. Osher, “The split Bregman method for 11 regularized problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 2, pp. 323–343, 2009.
- W. Yin, S. Osher, D. Goldfarb, and J. Darbon, “Bregman iterative algorithms for 11-minimization with applications to compressed sensing,” SIAM Journal on Imaging Sciences, vol. 1, no. 1, pp. 143–168, 2008.
- J. Darbon and S. Osher, Fast Discrete Optimization for Sparse Approximations and Deconvolutions, UCLA, 2007.
- Y. Wang, J. Yang, W. Yin, and Y. Zhang, “A new alternating minimization algorithm for total variation image reconstruction,” SIAM Journal on Imaging Sciences, vol. 1, no. 3, pp. 248–272, 2008.
- Y. Wang, W. Yin, and Y. Zhang, “A fast algorithm for image deblurring with total variation regularization,” CAAM Technical Report TR07-10, Rice University, Houston, Tex, USA, 2007.
- J.-F. Cai, B. Dong, S. Osher, and Z. Shen, “Image restoration: total variation, wavelet frames, and beyond,” Journal of the American Mathematical Society, vol. 25, no. 4, pp. 1033–1089, 2012.
- L. Moisan, “How to discretize the Total Variation of an image,” Proceedings in Applied Mathematics and Mechanics, vol. 7, no. 1, pp. 1041907–1041908, 2007.
- J.-F. Cai, S. Osher, and Z. Shen, “Split Bregman methods and frame based image restoration,” Multiscale Modeling & Simulation, vol. 8, no. 2, pp. 337–369, 2009.
- S. Setzer, “Split Bregman algorithm, Douglas-Rachford splitting and frame shrinkage,” Computer Science, vol. 5567, pp. 464–476, 2009.
Copyright © 2013 Yi Xu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.