Research Article | Open Access
Superresolution of Images Using Area Preserving Geometric Evolution Laws
PDE-based approaches are used to obtain superresolution of images. Using level set method for area constraint geometric evolution laws allows to remove pixelation in images without removing small-scale features. We compare mean curvature flow, surface diffusion, and Willmore flow for this purpose.
The goal of superresolution is to construct out of a given low-resolution image an image with higher resolution. Superresolution is a common problem in image processing and numerous algorithms have been proposed . Most of these algorithms are based on interpolation and lead to an improvement of the image but also to imperfect reconstructions with false high-frequency components. For interpolation methods based on PDEs, which are mostly adapted from inpainting problems, we refer to [2–6]. These approaches try to reconstruct the geometric properties of the images by evolving their level curves. In order to maintain fidelity to the original low-resolution image, different approaches are used. A standard level set method to modify pixel intensity which is based on evolution of mean curvature flow , where is the intensity and the mean curvature, as used for example, in [7, 8] for edge-preserving smoothing, noise removal, and other image enhancement, smoothes away feature objects, as all level curves are shortening and thus will not lead to appropriate superresolution images. In , this is circumvented by imposing constraints that preserve accuracy to the original image using unchanging anchor pixels defining explicit topology constraints. In , the problem is attacked from a different point of view, by constraining the mean curvature flow problem in order to preserve the area enclosed by each level curve. We will follow this approach and extend it to other geometric evolution problems of higher order. Higher-order PDEs have been applied to various problems in image processing, for example, [9–11]. They are, however, not yet been used for superresolution. Only in  a higher-order model based on a Cahn-Hilliard equation is applied to superresolution of binary images, especially text.
The paper is organized as follows: in Section 2, we describe a level set formulation for area conserving mean curvature flow, surface diffusion, and Willmore flow. In Section 3, we discuss an appropriate discretization which conserves all level curves simultaneously. In Section 4, the algorithms are applied to test problems. Thereby we start from a given image, lower the resolution, and use bicubic interpolation to create an image that is again . Our three area preserving evolution laws are applied to this image for a short time and the results are compared with the original image. We also apply the algorithms to text images which are only known at a low resolution. In Section 5, we draw conclusions.
2. Level Set Method
For a given initial curve , a geometric evolution law defines a family of curves , with . If is described implicitly as a specific level curve of a function , we can evolve by evolving by solving , with a given normal velocity. If is determined through a geometric evolution law, we can implicitly evolve the curve by solving this level set equation . To simultaneously evolve all level curves of a level set function , we can define a global energy using the coarea formula. Thereby, we consider with a given energy density on the curve and an appropriate gradient flow with respect to a specific metric Following , we interpret as an element of the manifold of level set ensembles and identify a tangent vector on with a motion velocity of the corresponding level curve via the classical level set equation . This allows to define the corresponding metric on which reads for the metric and for the metric For both cases, we are now able to rewrite the simultaneous gradient flow of all level curves in terms of the level set function which is equivalent to for all functions . From (7) the weak form can be obtained for various geometric evolution laws in level set form by choosing appropriate energy density , see Table 1 (e.g., [14, 15]).
To define area preserving versions of these geometric evolutions laws, we introduce a Lagrange multiplier and define the Lagrangien with the area of the interior domain inside of . We thus obtain with uniquely determined by the constraint : The corresponding evolution laws read as follows:(i)area preserving mean curvature flow (ii)surface diffusion (which is already area preserving) with the surface Laplacian;(iii) area preserving Willmore flow thereby denotes the shape operator on and the Frobenius norm.
To construct a level set formulation to simultaneously evolve all level curves under the constraint, that their enclosed area is preserved, we follow the same lines as for the unconstraint problem and define to obtain or equivalently for all functions and . For area preserving mean curvature flow ( and ), we thus obtain The same formulation can be obtained by rewriting the formulation in  in weak form. For surface diffusion ( and ), we obtain with the projection onto the tangent space defined for every on . For area preserving Willmore flow ( and ), we obtain with a weighted curvature.
3. Semi-Implicit Finite Element Discretization
We discretize the constraint evolution problems first in space using piecewise linear finite elements and then in time based on a semi-implicit backward Euler scheme. We thereby follow the discretizations of the unconstrained problems [11, 14] and treat the Lagrange multiplier explicit by evaluating (with ) for different discrete values of and then interpolating to obtain , see . The algorithms are implemented in the adaptive finite element toolbox AMDiS .
We first demonstrate the smoothing and area preserving properties of the algorithms. We, therefore, evolve a perturbed circle. Thereby the radius of is initially defined as with angle . Figure 1 shows the initial curve and the evolution of its averaged curvature over time towards the constant value . We here only use the constraint for and let all other level lines evolve accordingly, thus only is computed and . For all cases, the area remains constant; however, the smoothing differes from which a different behaviour if applied to images is expected.
4. Numerical Results
We start with a simple application, namely, the superresolution of text images. Therefor, we use the logo of our university and apply area preserving Willmore flow, mean curvature flow, and surface diffusion to smooth the jagged parts of the text given at a low resolution. Figure 2 shows the results and the improvement if compared with bicubic interpolation. The differences obtained with the three evolution laws are minor.
The described algorithms can be extended to color images by applying them separately to the individual color planes. As already mentioned in , the possible separation of such an approach, due to the independent evolution, is not of significant for the obtained results.
We consider two images, the “Lena’’ image and the “Monarch’’ image. For both images, originally given with a resolution , the resolution is reduced to and bicubic interpolations is used to produce again a image. We apply our three level set algorithms to the resulting images. Figure 3 shows results on the “Lena’’ image.
Applying the same conserved evolution laws to the “Monarch’’ image results in Figure 4.
In both cases, all area conserved evolution laws result in smoother contours, which is directly visible at the shoulder and the hat in the “Lena’’ image and along the white stripes in the “Monarch’’ image. The jagged parts remaining after bicubic interpolations are removed. Without the area constraint, the evolution would oversmooth the images and significant detail would be lost. The applied constraints help to maintain these details. The difference in the three evolution laws is most significant for mean curvature flow, which reduces the sharpness of the edges. The results for Willmore flow and surface diffusion are comparable.
We consider PDE bases approaches to reconstruct geometric properties of images by evolving their level curves. Applying these level set reconstructions to obtain superresolution images had already been shown to lead to improved results, as jagged parts of the image can be smoothed. However, unconstrained evolutions will lead to oversmoothing. The goal thus is to constrain the evolution laws such that visually significant properties remain. A simple attempt in this direction, which does not require image-specific treatments is to constrain the area of each level curve. We have shown that such constrained geometric evolution laws can improve results of other methods for image magnification. The resulting PDEs turn to become nonlocal which requires special treatment. For all cases of area conserved mean curvature flow, surface diffusion and area conserved Willmore flow we use a semi-implicit discretization in which the Lagrange multiplier is treated explicitly which drastically reduced the complexity of the discretization scheme of the higher-order PDEs. A comparison of the three evolution laws shows an improvement if fourth-order PDEs are used. Small-scale details seem to better remain than for the second-order PDE. The difference between area conserved Willmore flow and surface diffusion is only marginally in the considered examples. As surface diffusion is already area conserving and thus computationally less expensive, it should be the method of choice.
The authors would like to thank P. Smereka for fruitfull discussions.
- L. Alvarez, P. L. Lions, and J. M. Morel, “Image selective smoothing and edge detection by nonlinear diffusion,” SIAM Journal on Numerical Analysis, vol. 29, no. 3, pp. 845–866, 1992.
- H. A. Aly and E. Dubois, “Image up-sampling using total-variation regularization with a new observation model,” IEEE Transactions on Image Processing, vol. 14, no. 10, pp. 1647–1659, 2005.
- M. Bertalmio, A. Bertozzi, and G. Sapiro, “Navier-stokes, fluid dynamics, and image and video inpainting,” in Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 355–362, 2001.
- A. L. Bertozzi, S. Esedog̃lu, and A. Gillette, “Inpainting of binary images using the Cahn-Hilliard equation,” IEEE Transactions on Image Processing, vol. 16, no. 1, pp. 285–291, 2007.
- M. Burger, C. Stöcker, and A. Voigt, “Finite element-based level set methods for higher order flows,” Journal of Scientific Computing, vol. 35, no. 2-3, pp. 77–98, 2008.
- V. Caselles, J. M. Morel, and C. Sbert, “An axiomatic approach to image interpolation,” IEEE Transactions on Image Processing, vol. 7, no. 3, pp. 376–386, 1998.
- T. F. Chan, S. H. Kang, and J. Shen, “Euler's elastica and curvature-based inpainting,” SIAM Journal on Applied Mathematics, vol. 63, no. 2, pp. 564–592, 2002.
- M. Droske and M. Rumpf, “A level set formulation for Willmore flow,” Interfaces and Free Boundaries, vol. 6, no. 3, pp. 361–378, 2004.
- S. Esedoglu and P. Smereka, “A variational formulation for a level set representation of multiphase flow and area preserving curvature flow,” Communications in Mathematical Sciences, vol. 6, no. 1, pp. 125–148, 2008.
- F. Malgouyres and F. Guichard, “Edge direction preserving image zooming: a mathematical and numerical analysis,” SIAM Journal on Numerical Analysis, vol. 39, no. 1, pp. 1–37, 2001.
- R. Malladi and J. A. Sethian, “A unified approach to noise removal, image enhancement, and shape recovery,” IEEE Transactions on Image Processing, vol. 5, no. 11, pp. 1554–1568, 1996.
- E. Meijering, “A chronology of interpolation: from ancient astronomy to modern signal and image processing,” Proceedings of the IEEE, vol. 90, no. 3, pp. 319–342, 2002.
- B. S. Morse and D. Schwartzwald, “Image magnification using level-set reconstruction,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '10), pp. 333–340, December 2001.
- S. Osher and J. A. Sethian, “Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations,” Journal of Computational Physics, vol. 79, no. 1, pp. 12–49, 1988.
- C. Stöckert and A. Voigt, “Geodesic evolution laws—a level set approach,” SIAM Journal on Imaging Sciences, vol. 1, no. 4, pp. 379–399, 2008.
- S. Vey and A. Voigt, “AMDiS—adaptive multidimensional simulations,” Computing and Visualization in Science, vol. 10, no. 1, pp. 57–66, 2007.
Copyright © 2012 Caroline Jäger et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.