- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Journal of Applied Mathematics
Volume 2013 (2013), Article ID 107120, 14 pages
Multiscale Image Representation and Texture Extraction Using Hierarchical Variational Decomposition
1College of Mathematics and Statistics, Chongqing University, Chongqing 401331, China
2College of Mathematics and Physics, Chongqing University of Science and Technology, Chongqing 401331, China
Received 19 April 2013; Revised 1 July 2013; Accepted 30 July 2013
Academic Editor: Ke Chen
Copyright © 2013 Liming Tang and Chuanjiang He. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In order to achieve a mutiscale representation and texture extraction for textured image, a hierarchical decomposition model is proposed in this paper. We firstly introduce the proposed model which is obtained by replacing the fixed scale parameter of the original decomposition with a varying sequence. And then, the existence and convergence of the hierarchical decomposition are proved. Furthermore, we show the nontrivial property of this hierarchical decomposition. Finally, we introduce a simple numerical method for the hierarchical decomposition, which utilizes gradient decent for energy minimization and finite difference for the associated gradient flow equations. Numerical results show that the proposed hierarchical decomposition is very appropriate for multiscale representation and texture extraction of textured image.
A grayscale image can be represented by a function with , where is an open, bounded, and connected subset of , typically a rectangle or a square [1, 2]. We are interested in the decomposition of into two components, [3–5], or three components, [6–9], where represents piecewise-smooth (cartoon or structure) component of and represents the oscillatory component of , that is, texture, and represents the residual (noise). Image decomposition is an important image processing task, which is widely used in image denoising [4, 10, 11], deblurring [12, 13], image representation [5, 13], texture extraction or discrimination [6, 14], and so on. It has seen much recent progress, much of which has particularly been made through the use of variational framework to model oscillatory component that represents texture; see, for example, [2–6, 8–14]. We give here some classical examples of image decomposition models by variational approaches that are most related to our present work.
A celebrated decomposition easier to implement is the total variation (TV) minimization model by Rudin, Osher, and Fatemi (ROF)  for image denoising, in which an image is split into and : which yields so-called decomposition. This model is convex and easy to solve in practice. The function allows for discontinuities along curves; therefore, edges and contours are preserved in the restored image .
However, as Meyer pointed out in , the function space is not the most suitable one to model oscillatory components, since the oscillatory functions do not have small -norms. He suggested using , the dual space of , instead of for the oscillatory components. However, there is no known integral representation of continuous linear functional on . To address this problem, Meyer used another slightly larger space to approximate . Using to characterize oscillatory components yields the decomposition by solving the following variational problem: The decomposition model can better extract texture; however, it cannot be directly solved in practice due to the nature of the -norm [4, 6, 14], for which there is no standard calculation of the associated Euler-Lagrange equation. Vese and Osher [6, 14] first overcame this difficulty by replacing the space with (). Then, the decomposition model (2) is approximated by the following minimization problem:
In , Vese and Osher did not solve (3) directly but adapted the model by adding a fidelity term into the energy functional to guarantee . In detail, their variational formulation is defined as In this decomposition, the image is discomposed into three components, with , , and .
The previous models are examples of a larger class of the fixed scale decompositions (the scale parameters in these models are fixed). It has been argued that a human visualizes a scene in multiple scales [16, 17]. Then, multiscale approaches are appropriate for image representation because a single scale may not be a perfect simulation of the human visual perception. In order to achieve reliable image information in different scales, both the large-scale and small-scale behaviors should be investigated and incorporated appropriately. Thus, a natural way to address this problem is the multiscale analysis.
Tadmor et al. [5, 13] presented a hierarchical decomposition based on the ROF model (1) to achieve multiscale image representation, in which the scale parameter is not fixed, but a varying sequence: starting with an initial scale , and then, successive application of the following dyadic refinement step produces, after such steps, the hierarchical decomposition of :
In this study, we focus on multiscale representation and texture extraction for textured image. As discussed previously, the decomposition is not the best one for textured image, so using hierarchical decomposition (7) introduced by Tadmor et al. to implement multiscale representation and texture extraction for textured image is obviously not the best choice. We thus in this paper propose the hierarchical decomposition using the model (4), which enables us to capture an intermediate regularity between and and oscillation between and . We here adopt decomposition because is a very suitable function space to model oscillatory patterns [6, 14]; in addition, the -norm is easier to solve in practice. In the proposed hierarchical decomposition, the scale parameter is not fixed but varies over a sequence of dyadic scales. Consequently, the decomposition of a textured image is not predetermined but is resolved in terms of layers of intermediate scales. So, we can achieve multiscale image representation. Compared to Tadmor et al.’s 2-tuple hierarchical decomposition, the proposed 3-tuple hierarchical decomposition can precisely extract texture in different scales.
So far, there have been a lot of efficient variational decomposition models for textured image, much of which follows Meyer’s work. The decomposition introduced by Vese and Osher is the first one to practically solve the Meyer’s model presented in (2), in which cartoon component is measured in and texture component in , instead of . We here recall the definition and some known results of , and , which are much related to our present study.
Definition 1. Let be an open subset with Lipschitz boundary. Then, is the subspace of such that the following quantity is finite. Further, is called the -norm.
Remark 2. with the norm of is a Banach space, but one does not use this norm since it possesses no good compactness property. Classically, in one works with the -weak* topology, which is defined as convergence to in -weak* topology if and only if converges to strongly in and converge to for all in .
Theorems 3 and 4 show the compactness and lower semicontinuity of .
Theorem 3 (see ). If is a uniformly bounded sequence in , then there exist a subsequence and in such that converge to in the -weak* topology.
Definition 5. consists of distributions which can be written as endowed with the norm
Definition 6. consists of distributions which can be written as
endowed with the norm
For every , the space above can be identified with the space , the dual space to the Sobolev space , where . In fact, the norm is a dual norm to the Sobolev norm . And the space which is the dual to the space . Moreover, if , the spaces approximate the space . By the Sobolev imbedding theorems, we obtain that , where is a constant which is independent of but . So, for any , these are larger spaces than and allow for different choices of weaker norms for the oscillatory component .
For instance, consider the sequence of one-dimensional functions defined on . Then, , where . It is easy to check that(1);(2) as ;(3) as .
This simple example demonstrates that an oscillatory function has a small -norm as well as -norm which both approach to zero as the frequency of oscillations increases, but importantly, not with a so small -norm. So, -norm and -norm are more suitable than -norm to measure textures in image decomposition. In addition, -norm is weaker than -norm. So using -norm to measure oscillatory functions, we also can exactly capture the texture in the energy minimization process.
For the space , we have the following results which will be used in what follows.
Proposition 7 (see ). If , then there exists with and on , such that .
Proposition 8. If , then . Indeed, .
Replacing with (), Vese and Osher introduce the following convex minimization problem; that is, decomposition: where are tuning parameters. The first term insures that , the second gives us , while the third term is a penalty on the norm in of . Clearly, if and , this model is formally an approximation of the model (2) originally proposed by Meyer in .
In what follows, to simplify the notations, we always write , , and instead of , , and , respectively.
3. The Proposed Hierarchical Decomposition
3.1. Description of Hierarchical Decomposition
We firstly modify the original decomposition presented in (4) to a single parameter pattern with a constraint condition . The new decomposition is defined as Here, the constraint condition ensures that the sum of texture and residual (noise) has zero mean. In this study, the parameter in (14) is viewed as a scale factor which can be used to measure the scale of the extracted cartoon, especially texture. If the value is too small, then only the small scale feature (coarser texture) is allocated in , while most of the large scale feature (smoother texture) is swept into the residual component . If is too large, however, all the textures are extracted indiscriminately, regardless of their distinct scales.
To achieve multiscale description of a textured image, we here propose a hierarchical decomposition based on (14), which enables us to effectively extract textures in different scales.
For a given scale , the minimizer of is interpreted as a decomposition, , such that captures textures in the scale , while the textures above remain unresolved in . The residual still consists of significant textures when viewed under a larger scale than , say : with where captures textures in the scale , while the textures above remain unresolved in . The process of (15) can be continued to capture the missing large scale textures.
The proposed hierarchical decomposition can be stated as follows:
starting with an initial scale , where Proceeding with successive applications of the dyadic refinement step (15), we have where From (19), we obtain, after such steps, the hierarchical decomposition of as follows: The partial sum, , provides a multiscale representation of , in which lies in the intermediate scale spaces between and , and lies in the intermediate scale spaces between and . Another application of this hierarchical decomposition is multiscale texture extraction. Indeed, represents the textures in the scales ranging from to .
3.2. Existence of Hierarchical Decomposition
The existence of our hierarchical decomposition is directly derived from the following result, actually, which can be used for original decomposition by replacing with , but Vese and Osher did not give proof for it in their papers.
Theorem 9. For (), the following minimization problem has a solution such that and .
Proof. Since for all and , . We can find a minimizing sequence such that and for all . Then, we have uniformly
Here, the constant may be changed from line to line. By the Sobolev-Poincare inequality, we have
where is the volume of . We thus obtain by (23), which implies that is uniformly bounded in since for all . Because is bounded, is also uniformly bounded in . By (23), we thus have
By Theorem 3, there exists and a subsequence (still denoted by ), such that converge to in -weak* topology and weakly in . In particular, by lower semicontinuity for the -weak* topology (Theorem 4), we can obtain
Since is uniformly bounded in , by (23) we have that is uniformly bounded in . Therefore, there exists such that (up to a subsequence) converges to weakly in . By weak lower semicontinuity of -norm, we deduce the following property:
For , by Proposition 7, there exists such that ( is the distribution space) and , which implies () due to . Therefore, there exist , such that, up to a subsequence, converges to weak* in .
We next prove that . Let ( is the test function space); then, Taking (using weak topology and weak* topology), we obtain This implies . And since , a.e. Therefore, . By weak* lower semicontinuity, it follows that By (26)–(30), we have which implies that is a solution for (22). The proof is completed.
3.3. Nontrivial Property of Hierarchical Decomposition
In this study, if the solution of (22) satisfies or , then the decomposition is called the nontrivial decomposition. If () is nontrivial for any , then the hierarchical decomposition (21) is called the nontrivial hierarchical decomposition. Conversely, if the minimization problem (22) has only zero solution, that is, , then the decomposition of is trivial, which makes no sense for image decomposition. In what follows, we discuss the existence of the nontrivial hierarchical decomposition in (21).
Firstly, similar to (but slightly different from) Definition 5.3 of , we here define a new quantity to measure the -function, which will play a key role in our following study.
Definition 10. Let . Then, for any and , one defines
where denote inner product.
By the definition of , we have the following results.
Proposition 11. Let . If , then .
Proof. For any , , and , replacing with and noting that , we have
By , we can deduce that
By the definition of , we have
By Theorem 9, the minimization problem (22) must have solutions. Next, simulating hierarchical decomposition proposed by Tadmor et al. , we show some properties for these solutions, which will be used to demonstrate the nontrivial property for our hierarchical decomposition.
Lemma 12. Let . If the minimization problem (22) has a zero solution, then .
Proof. Since (22) has a zero solution, then for any and , we have and that is, This inequality can be rewritten as Substituting by and by in (38) and taking and , respectively, we obtain By the definition of , we have .
Lemma 13. Let . If , then the solution of (22) is nonzero; that is, or . Furthermore, and satisfy
Proof. The first assertion is proved directly by Lemma 12.
Because is the solution of (22), for any , , and , we have By the triangle inequality, we obtain So, the inequality (41) is changed into Expanding the second term on left side of the last inequality, we can obtain Dividing both sides of the last inequality by and taking , we obtain Dividing both sides of (44) by and taking , we also obtain The inequalities (45) and (46) imply that By definition of , we have
Let . Replacing with in the inequality (41), we have So, Dividing both sides of the last inequality by and then taking and , respectively, we obtain the equality (40): So, which, due to or , implies By definition of and (48), we have .
Theorem 14. Let with , and is the solution of (22). Then, for any initial scale , the decomposition is nontrivial for any . In other words, any hierarchical decomposition of given in (21) is nontrivial.
Proof. Since , we have by Proposition 11. By Lemma 13, the decomposition is nontrivial, and . Because , again by Lemma 13, the decomposition is nontrivial, and which means such nontrivial decomposition can continue.
For the th decomposition, by Lemma 13, we have which means the th decomposition is nontrivial. In conclusion, any hierarchical decomposition of given in (21) is nontrivial when .
Remark 15. By Theorems 9 and 14, we can deduce that for any -function with , there must be a nontrivial hierarchical decomposition. This result is much significant for image hierarchical decomposition. In general, a digital image is a nonnegative -function with , so any hierarchical decomposition of must be nontrivial.
3.4. Convergence of Hierarchical Decomposition
For the hierarchical decomposition given in (21), we have the following convergence result (Theorem 17) in the topology, which is similar to the convergence result of hierarchical decomposition proposed by Tadmor, Nezzar, and Vese (see Theorem 2.2 in  for details). To prove Theorem 17, we need the following lemma.
Lemma 16. If , then there are , and so that , where is a constant independent of .
Theorem 17. Let . Then, the hierarchical decomposition given in (21) satisfies In addition, the following “energy” estimate holds:
Proof. By Lemma 16, there exist and , such that , where does not depend on . Since is a solution of (22), we have Thus, which, by , implies By , we have Therefore, as . The proof of the first assertion is completed.
Equation (57) can be seen as the -energy decomposition of in our hierarchical decomposition. In addition, the multiscale nature of our hierarchical extraction can be quantified in terms of this energy decomposition.
4. Numerical Implementation
In this section, we present the details of numerical implementation for our hierarchical decomposition:
Taking , we obtain the following equivalent formulation of (67) in terms of , , and : where .
Minimizing the energy in (68) with respect to , and yields the following Euler-Lagrange equations: If the exterior normal to the boundary is denoted by , then the associated boundary conditions for , , and are
Equation (69) with boundary condition (72) implies that holds. Indeed, by taking the integral for each side of (69) and using the Gaussian formula, we obtain Since , by Proposition 8, we have . Therefore, .
The details are as follows:(i) fixed , find the solution of with the initial condition ,(ii) fixed , find the solution of with the initial conditions , , respectively.
We use a simple explicit finite difference scheme to solve (77)-(78). The image domain is discretized by the space steps and . Then, the grid is defined as We denote the time step by , and (). Let be the value of at the grid . In order to compute the right hand side of (77)-(78), we denote Then, (77)-(78) can be approximated by the following discretizations (to remove the singularity when and , we introduce a regularity parameter ): with the initial condition where is the curvature of the level set of at the grid , defined by with the initial condition where .
5. Numerical Results
We present four numerical examples in this section to demonstrate the efficiency of multiscale texture extraction and image representation using the proposed hierarchical decomposition for textured images. Test images, shown in Figure 1, are two synthetic images and two real images. In all experiments, we take the time step , the space step , the initial scale , and the regular parameter .
For the choice of , by the theoretical analysis in Section 2, we have that -norms are weaker than -norm for any . So, any choice of with is suitable. Here, similar to what was done by Vese and Osher in [6, 14], we tested the model (68) with different values of ; our observation is that results are very similar, while the case of yields faster calculations per iteration. Thus, we set in the following. We note in passing that some different approaches based on duality principle have been proposed, such as [21, 22], to solve (67) with . We here adopt the method introduced by Vese and Osher because this study is following their work in [6, 14].
(i) Image hierarchical decomposition: Figure 2 shows the hierarchical decomposition results for a synthetic textured image for 7 steps. The first column shows the cartoon components of the initial image in different scales. We can see that these cartoon components are very little different visually. This phenomenon is compatible with the theory of causality of scale space. The second column shows the “textures+100” (plus a constant for illustration purposes) of the image in different scales. It is clear that the textures can be gently extracted by increasing the value of scale parameter , because this image involves the textures of different scales: coarser textures correspond to the smaller scales, while smoother textures correspond to the larger scales. The third column shows “residuals+100,” from which we can clearly see that some textures and edges are swept into these residual components when the value of scale parameter is smaller, and then, they are gradually swept out and absorbed by and by increasing the value of the scale parameter . Figure 3 shows the plots of the -energy of , -energy of , and -energy of , respectively.
(ii) Multiscale texture extraction: Figure 4 shows the results of multiscale texture extraction using hierarchical decomposition for another synthetic textured image for 9 steps. The first two images show the initial and final cartoon components which have little visual difference; this phenomenon is identical with the results of the first experiment. The next nine images show the texture components in different scales, which can be used as the results of multiscale texture extraction for this synthetic textured image. We remark that the larger scale textures are gradually resolved from the residual in terms of the increasing scale.
(iii) Multiscale texture extraction and image representation: Figure 5 shows the results of multiscale texture extraction and image representation using hierarchical decomposition of a fingerprint for 6 steps. The second column of this figure shows the extracted texture in different scales. The s are shown in the last column of this figure, which can be used as a multiscale representation of the original image. We can clearly see that, from top to bottom of this column, an additional amount of blurred texture is resolved in terms of the refined scaling for edges.
(iv) Multiscale image representation for noisy textured image: Figure 6 shows the hierarchical decomposition results of a noisy Barbana for 6 steps. The last column of this figure shows s which can be seen as restored images in different scales. Clearly, when the value of is smaller, such as , there are a few textures and noises in the restored images, much of which is swept into residual components. When , some textures of the image are recovered on the headscarf of Barbana while removing the smaller scale noises from the entire image. If we continue the decomposition into smaller scales, then noise will reappear in the components, since the refined scales reach the same scales of the noise itself. From the last column of this figure, we can obtain restored image from noisy Barbana in different scales according to our requirements.
In this paper, in order to achieve multiscale image representation and texture extraction for textured image, we presented a hierarchical decomposition model which combines the idea of hierarchical decomposition introduced by Tadmor et al. with the decomposition proposed by Vese et al. In addition, we proved the existence and the convergence of the hierarchical decomposition, and the nontrivial property of this decomposition is also discussed. But the uniqueness of this hierarchical decomposition has not been proved in this paper. The authors will be concerned about this problem in the successive research.
- G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing, Springer, New York, NY, USA, 2001.
- T. Chan, J. Shen, and L. Vese, “Variational PDE models in image processing,” Notices of the American Mathematics Society, vol. 50, no. 1, pp. 14–26, 2003.
- L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D, vol. 60, no. 1–4, pp. 259–268, 1992.
- S. Osher, A. Sole, and L. Vese, “Image decomposition and restoration using total variation minimization and the H1,” Multiscale Modeling and Simulation, vol. 1, no. 3, pp. 349–370, 2003.
- E. Tadmor, S. Nezzar, and L. Vese, “A multiscale image representation using hierarchical (BV; L2) decompositions,” Multiscale Modeling and Simulation, vol. 2, no. 4, pp. 554–579, 2003.
- L. Vese and S. Osher, “Modeling textures with total variation minimization and oscillating patterns in image processing,” Journal of Scientific Computing, vol. 19, no. 1–3, pp. 553–572, 2003.
- J. Shen, “Piecewise images and the Mumford-Shah-Sobolev model for segmented image decomposition,” Applied Mathematics Research Express, vol. 4, pp. 143–167, 2005.
- T. M. Le and L. Vese, “Image decomposition using total variation and div(BMO),” Multiscale Modeling and Simulation, vol. 4, no. 2, pp. 390–423, 2005.
- C. W. Lu and G. X. Song, “Image decomposition using adaptive regularization and div (BMO),” Journal of Systems Engineering and Electronics, vol. 22, no. 2, pp. 358–364, 2011.
- A. Chambolle, R. DeVore, N. Lee, and B. Lucier, “Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage,” IEEE Transactions on Image Processing, vol. 7, no. 3, pp. 319–335, 1998.
- Z. Jin and X. Yang, “Analysis of a new variational model for multiplicative noise removal,” Journal of Mathematical Analysis and Applications, vol. 362, no. 2, pp. 415–426, 2010.
- I. Daubechies and G. Teschke, “Variational image restoration by means of wavelets: simultaneous decomposition, deblurring, and denoising,” Applied and Computational Harmonic Analysis, vol. 19, no. 1, pp. 1–16, 2005.
- E. Tadmor, S. Nezzar, and L. Vese, “Multiscale hierarchical decomposition of images with applications to deblurring, denoising and segmentation,” Communications in Mathematical Sciences, vol. 6, no. 2, pp. 281–307, 2008.
- L. Vese and S. Osher, “Image denoising and decomposition with total variation minimization and oscillatory functions,” Journal of Mathematical Imaging and Vision, vol. 20, no. 1-2, pp. 7–18, 2004.
- Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, American Mathematical Society, 2001.
- K. H. Liang and T. Tjahjadi, “Adaptive scale fixing for multiscale texture segmentation,” IEEE Transactions on Image Processing, vol. 15, no. 1, pp. 249–256, 2006.
- Y. F. Pu and J. L. Zhou, “A novel approach for multi-scale texture segmentation based on fractional differential,” International Journal of Computer Mathematics, vol. 88, no. 1, pp. 58–78, 2011.
- L. Evans and R. Gariepy, Measure Theory and Fine Properties of Functions,, CRC Press, Boca Raton, Fla, USA, 1992.
- A. Chambolle and P. L. Lions, “Image recovery via total variation minimization and related problems,” Numerische Mathematik, vol. 76, no. 2, pp. 167–188, 1997.
- G. Aubert and J. Aujol, “Modeling very oscillating signals. Application to image processing,” Applied Mathematics and Optimization, vol. 51, no. 2, pp. 163–182, 2005.
- J. F. Aujol, G. Aubert, L. Blanc-Féraud, and A. Chambolle, “Image decomposition into a bounded variation component and an oscillating component,” Journal of Mathematical Imaging and Vision, vol. 22, no. 1, pp. 71–88, 2005.
- A. Chambolle and T. Pock, “A first-order primal-dual algorithm for convex problems with applications to imaging,” Journal of Mathematical Imaging and Vision, vol. 40, no. 1, pp. 120–145, 2011.