Mathematical Problems in Engineering

Volume 2013, Article ID 571052, 8 pages

http://dx.doi.org/10.1155/2013/571052

## A Bidirectional Flow Joint Sobolev Gradient for Image Interpolation

^{1}College of Computer Science, Chongqing University, Chongqing 400030, China^{2}College of Mathematics and Statistics, Chongqing Technology and Business University, Chongqing 400067, China^{3}College of Mathematics and Science, Chongqing University, Chongqing 400044, China^{4}Department of Mathematics & KLDAIP, Chongqing University of Arts and Sciences, Yongchuan, Chongqing 402160, China

Received 9 February 2013; Accepted 9 May 2013

Academic Editor: Ezzat G. Bakhoum

Copyright © 2013 Yi Zhan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

An energy functional with bidirectional flow is presented to sharpen image by reducing its edge width, which performs a forward diffusion in brighter lateral on edge ramp and backward diffusion that proceeds in darker lateral. We first consider the diffusion equations as gradient flows on integral functionals and then modify the inner product from to a Sobolev inner product. The experimental results demonstrate that our model efficiently reconstructs the real image, leading to a natural interpolation with reduced blurring, staircase artifacts and preserving better the texture features of image.

#### 1. Introduction

Digital image interpolation is an inverse process of imaging process which samples higher resolution scene to lower resolution lattices. It is an important technology in digital image processing, artificial intelligence, pattern recognition, and modern industries. Producing visually pleasing resulting image is the key task in image interpolation. Many methods do their service for these problems, usually known as edge directed, level-set based, or isophote oriented, varying considerably.

The methods based on partial differential equations (PDEs) or energy variational usually have better effects in deblurring. Morse and Schwartzwald [1, 2] reduced the zigzagging artifacts by restricting curvature of the interpolated isophotes (equi-intensity contours). Minimum curvature is required on isophotes of the interpolated images. Malgouyres and Guichard [3] considered the preservation of one-dimensional structure to lead to the resulting images without blurring effects. Aly and Dubois [4] proposed a model-based TV regularization image up-sampling methods. Image acquisition process is modelled after a lowpass filtering followed by sampling. However, TV minimization is based on the assumption that the desirable image is almost piecewise constant, which yields a result with oversmoothed homogeneous regions. Belahmidi and Guichard [5] have improved the TV-based interpolation by developing a nonlinear anisotropic PDE, hereafter referred to as BG interpolation method. Similar to [6], the nonlinear diffusion coefficient is locally adjusted according to image features such as edges, textures, and moments, which performs forward diffusion near homogeneous regions while backward diffusion near edges with strength and orientation adapted to image structures. To sharpen edges further, bidirectional diffusion (flow) was proposed [7–9]. In [7], a bidirectional flow is presented to sharpen image by reducing its edge width, which performs a fuzzy backward (inverse) diffusion along the gradient direction to the isophote line (edge), while it does a certain forward diffusion along the tangent direction on the contrary. A fractional-order bidirectional flow was also present in [9].

Generally, the solution of the energy functional is obtained by a steepest descent iteration with the discretized gradient. Richardson [10] demonstrated substantial benefits of Sobolev gradient method to a variety of PDEs that arise in image processing in computational efficiency. Sundaramoorthi et al. [11] presented Sobolev active contours with a Sobolev gradient in a space of smooth curves. Kazemi and Danaila [12] used Sobolev gradient method in conjunction with the steady-state solution of the Navier-Stokes equation in order to fill in missing pieces in a digital image. Yuan and He [13] proposed variational level set methods for image segmentation based on both and Sobolev gradients.

In this paper we focus on a new method for image interpolation based on variational regularization and Sobolev gradient. The variational regularization is a bidirectional flow in which forward diffusion and backward diffusion occur on both sides of image edge *instead of* along the gradient direction to the isophote line (edge) and the tangent direction as in [7]. We consider the diffusion equations as gradient flows on integral functionals and then modify the inner product from to a Sobolev inner product. The rest of the paper is organized as follows. In Section 2, we review the method based on variational method, PDEs method, and bidirectional diffusion method. The proposed variational regularization and Sobolev gradient for image interpolation are presented in Section 3. In Section 4 we demonstrate the experimental results on natural images which suggest that this method outperforms previous work, and Section 5 is for conclusion.

#### 2. Background

The imaging processing of a digital lower resolution image from a high-resolution image can be formulated as where is a sparse matrix that combines both filtering and downsampling processes. The image interpolation is to solve unknown image from the ill-posed inverse problem (1). It is an efficient approach to reduce it to a minimization problem in a regularization-based framework as follows [4]: where is a regularization parameter that controls the tradeoff between and . The data fidelity function is generally formulated in the classical least-squares sense as . The variational regularization has been widely used in many areas of image processing [4, 7, 14]. The main advantage of the variational regularization is that it allows incorporating various prior knowledge, such as shape and intensity distribution, smoothing image edges while preserving important features of image. All of these formulations are different forms of a unified functional form of the variational regularizer given by Using Euler’s equation, the minimizer of problem (3) is the steady-state solution of the nonlinear PDE given by A direct calculation verifies The two second-order directional derivatives are calculated as follows:

A simple and effective function is , which corresponds to TV regularization. This derived the image interpolation based on the TV regularization under the level set method [4]

An analogue regularization of (5) was proposed by Belahmidi and Guichard based on the classical heat diffusion model [5], formulated as follows: where the function is typically defined as with a constant to be tuned for a particular application. The role of the diffusion coefficient is to control the smoothing adaptively.

In order to sharpen image by reducing its edge width, a fuzzy bidirectional flow framework based on generalized fuzzy set has been presented for image denoising by [7] as follows: where , , , , are positive constant numbers, is a hyperbolic tangent function, and is a Gaussian function with standard deviation .

In all these models, is the second-order directional derivative in the direction of the gradient and in the direction orthogonal to the gradient . From the viewpoint of geometry, the evolution process in the artificial time given by these models is seen as energy dissipation process in two orthogonal directions and [4]. The diffusion process of along with will preserve the location and the intensity transitions of the contours, while smoothing along them maintaining their crispness. This diffusion term is used to maintain edges with smooth isophotes in [4, 5]. But this process cannot decrease the width of edges caused by the upsampling process , and it will introduce some blur. In (8), the forward diffusion process of the grey values along conserves the edges by the stop function . Unfortunately, the ability of the forward diffusion term in (8) is limited since it is a unilateral diffusion. The feature-dependent fuzzy bidirectional flow (10) has better effects in decreasing the width of edges since it is a bilateral diffusion across the image counters. However, the second-order directional derivative in the direction of gradient locate inaccurately the center of the “slop” of image edges, which reduces the ability of decreasing the width of edges. In the next section, we chose an appropriate function and improved its Sobolev gradient to decrease the width of edges.

#### 3. The Variational Regularization and Sobolev Gradient

If we consider the gray-level value at a pixel to be analogous to the amount of particles, each having one unit of “mass,” stacked at the pixel, then in order to decrease the width of edges, we would like to move particles from the lower part of a “slope” upwards and from the higher part downwards. This process enhances the ratio of gray level on both sides of image edges. It can be achieved by enlarging the small and middle gradients in interpolation process. Since the arc tangent function rapidly increases near zero while very slowly on larger arguments, we take the arc tangent function for in (3); that is, . Equation (3) is rewritten as

Thus, our energy functional for image interpolation is defined as The problem now is to minimize or at least find critical points of the energy functional for which the Euler-Lagrange equation is a nonlinear PDE. There are three approaches to minimize : Newton’s method, evolving the parabolic PDE to steady state, and steepest descent iteration [15]. As pointed in [15], these methods all have some drawbacks. With the Sobolev gradient, however, steepest descent is often very effective [16]. Next we derive the Sobolev gradient of similar to [15, 17].

Define a differential operator by and let From (12) the Fréchet derivative of is For , is a bounded linear functional on , and hence, by the Riesz representation theorem, there exists , termed the Sobolev gradient, such that for all , where denotes the adjoint of . For , is also bounded on and is represented by an gradient for all . The two gradients are thus related by where Since is a dense subspace of , (18) can be extended to by continuity. In order to obtain an expression for , we integrate by parts in (15): with on the boundary of . Hence We improve the variant coefficients of the divergence and Laplace terms in (21) to obtain terse formula and better interpolation effects as follows: where the parameter , controls the tradeoff between the diffusion along the direction orthogonal to the gradient and the bidirectional diffusion.

Now we illustrate the action of the regularization terms in (22). As mentioned at the outset of this section, in order to decrease the width of edges, we would like to move particles from the lower part of a “slope” upwards and from the higher part downwards. It can be achieved by an energy dissipation process in two opposite directions, forward diffusion in brighter lateral on edge ramp and backward in darker lateral. In brighter lateral on edge ramp, and the second term in (22) is negative, and the dissipation process given by (22) diffuses backward. While in darker lateral, and the second term in (22) is positive, the dissipation process given by (22) diffuses forward.

#### 4. Numerical Algorithm and Experimental Results

In this section, we develop a fully discrete numerical method to approximate problem (22). We recall first the notations in the finite differences scheme used in our paper. Let and be the space and time steps, respectively, and let be the grid points. Let be an approximation of the function , with . Then for each (given ),(1)discretize by finite differences and compute from (22),(2)introduce the notation (will correspond to ),(3)solve with in using the semi-implicit scheme as in [16]: start with (or the previous ), iterate for until reaching the steady-state solution : (4)update . The parameters are fixed by hand at some fixed values. In general, parameter varies largely according to image features while in a certain range ([2,10], [0.5,5], [20,50]) are better.

We evaluate the performance of the proposed interpolation method in comparison with some representative work in the literatures: new edge-directed interpolation (NEDI) [18], the BG [5], the fuzzy bidirectional flow method (FBF) [7], and the local self-similarity algorithm (LSS) [19]. Experiments results are shown in Figures 1–4. The choice of the parameters is based on subjective quality of the results assessed informally by our personal preference as human viewers in terms of edge sharpness, contour crispness, no ringing in smooth regions, and no ringing near edges. The first experiment directly interpolates a koala image and a Barbara image (portion) by a factor of and , respectively, shown in Figures 1 and 2. In the second experiment, the images were first lowpass filtered and subsampled by a factor of or ; then the subsampled image was interpolated to the original image size, shown in Figures 3, 4, and 5.

From these figures, we can see that the LSS method sharpens the edges of image but smoothes unduly the image features, which produces the cartoon effects in result images. For example, the blocks or lumps of irregular shape appear in the fur of koala in face and forelimb (Figure 1), the strip (Figure 2), in face of old man and on the edge of hat (Figure 3), and in the petal (the detail texture disappeared in Figures 4 and 5). It is remarkable especially for larger expanded factor as shown in Figures 2 and 3, which results from using several small filters for larger factor. For example, to achieve a magnification of 5 it uses 5 : 4, 4 : 3, 4 : 3, 3 : 2, and 3 : 2. However, the effects appear even if only one filter is adopted, as shown in Figure 1. The FBF sharpens the image edges but cannot obtain smooth edges, unduly smooth image homogeneous areas, as shown in Figures 1 and 2. The BG produces better results than the LSS and FBF; the blur visual effect is still remarkable. Our approach not only sharpens the edges obtaining crisp and clear image edges not only, but also preserves the image detail feature resulting in the best visual effects. It results from the evolution process in the artificial time given by (22). The diffusion process of along the orthogonal direction to the gradient smooth image contours along them while the bidirectional flow decreases the width of edges maintaining their crispness. For example, the fur of koala, the fuzz on the hat of the old man, the stamen, and the stria in petals are seen clearly.

We use the classic PSNR to characterize the difference between the reference image and the output of a method. We use several test images including parrot, old man, and two flower images. To show the true power of the interpolation algorithms, we first downsampled each image by a factor of or and then interpolated the result back to its original size. The PSNR is shown in Table 1. From the table, the proposed method yields improved PSNR in all the experiments. This improvement may be attributed to the fact that our approach works better than other methods.

#### 5. Conclusion

In this paper, a new image interpolation model based on energy variational is proposed. It diffuses forward and backward simultaneously on both sides of the image’s edge. A Sobolev gradient method is adopted to minimize the functional. We have shown improvement over our method on a subjective scale and in many cases with an improvement in PSNR.

#### Acknowledgments

This work was supported by Natural Science Foundation Project of CQ CSTC (Grant no. cstcjjA40012), Education Committee Project Research Foundation of Chongqing (no. KJ120709, no. KJ131209), the National Natural Science Foundation of China under Grant nos. 61202349 and 11226268, and the Research Foundation of Chongqing University of Arts and Sciences (no. R2012SC20).

#### References

- B. S. Morse and D. Schwartzwald, “Image magnification using level-set reconstruction,” in
*Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition*, vol. 1, pp. I-333–I-340, December 2001. View at Scopus - B. S. Morse and D. Schwartzwald, “Level-set image reconstruction,” in
*Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition*, pp. 333–340, December 2001. View at Scopus - F. Malgouyres and F. Guichard, “Edge direction preserving image zooming: a mathematical and numerical analysis,”
*SIAM Journal on Numerical Analysis*, vol. 39, no. 1, pp. 1–37, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - H. A. Aly and E. Dubois, “Image up-sampling using total-variation regularization with a new observation model,”
*IEEE Transactions on Image Processing*, vol. 14, no. 10, pp. 1647–1659, 2005. View at Google Scholar · View at MathSciNet - A. Belahmidi and F. Guichard, “A partial differential equation approach to image zoom,” in
*Proceedings of the International Conference on Image Processing (ICIP '04)*, pp. 649–652, October 2004. View at Scopus - G. Gilboa, N. Sochen, and Y. Y. Zeevi, “Forward-and-backward diffusion processes for adaptive image enhancement and denoising,”
*IEEE Transactions on Image Processing*, vol. 5, no. 7, pp. 689–703, 2002. View at Google Scholar · View at Scopus - S. Fu, Q. Ruan, W. Wang, F. Gao, and H. D. Cheng, “A feature-dependent fuzzy bidirectional flow for adaptive image sharpening,”
*Neurocomputing*, vol. 70, no. 4, pp. 883–895, 2007. View at Publisher · View at Google Scholar · View at Scopus - Y. Zhan, M. H. Wang, Q. Wan, and M. Li, “Modified algorithm with bidirectional diffusion for TV image interpolation,”
*Journal of Software*, vol. 20, no. 6, pp. 1694–1702, 2009. View at Publisher · View at Google Scholar · View at Scopus - Z. Ren, C. He, and M. Li, “Fractional-order bidirectional diffusion for image up-sampling,”
*Journal of Electronic Imaging*, vol. 21, no. 2, Article ID 023006, 2012. View at Publisher · View at Google Scholar - W. B. Richardson, Jr., “Sobolev gradient preconditioning for image-processing PDEs,”
*Communications in Numerical Methods in Engineering*, vol. 24, no. 6, pp. 493–504, 2008. View at Publisher · View at Google Scholar · View at MathSciNet - G. Sundaramoorthi, A. Yezzi, and A. C. Mennucci, “Sobolev active contours,”
*International Journal of Computer Vision*, vol. 73, no. 3, pp. 345–366, 2007. View at Publisher · View at Google Scholar · View at Scopus - P. Kazemi and I. Danaila, “Sobolev gradients and image interpolation,”
*SIAM Journal on Imaging Sciences*, vol. 5, no. 2, pp. 601–624, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - Y. Yuan and C. He, “Variational level set methods for image segmentation based on both ${L}_{2}$ and Sobolev gradients,”
*Nonlinear Analysis: Real World Applications*, vol. 13, no. 2, pp. 959–966, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - L. Alvarez, P.-L. Lions, and J.-M. Morel, “Image selective smoothing and edge detection by nonlinear diffusion,”
*SIAM Journal on Numerical Analysis*, vol. 29, no. 3, pp. 845–866, 1992. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - R. J. Renka, “Image segmentation with a Sobolev gradient method,”
*Nonlinear Analysis: Theory, Methods and Applications A*, vol. 71, no. 12, pp. e774–e780, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - J. W. Neuberger,
*Sobolev Gradients and Differential Equations*, Lecture Notes in Mathematics, Springer, Berlin, Germany, 1997. View at MathSciNet - M. Jung, G. Chung, G. Sundaramoorthi, L. A. Vese, and A. L. Yuille, “Sobolev gradients and joint variational image segmentation, denoising and deblurring,” in
*Electronic Imaging*, vol. 7246 of*Proceedings of SPIE*, p. 72460I, January 2009. View at Publisher · View at Google Scholar · View at Scopus - X. Li and M. T. Orchard, “New edge-directed interpolation,”
*IEEE Transactions on Image Processing*, vol. 10, no. 10, pp. 1521–1527, 2001. View at Publisher · View at Google Scholar · View at Scopus - G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,”
*ACM Transactions on Graphics Journal*, vol. 30, no. 2, article 12, 2011. View at Publisher · View at Google Scholar · View at Scopus