Abstract
When denoising with the method of Weickert’s anisotropic diffusion equation, the textures and details will be compromised. A fidelity term is added to Weickert’s equation, and the coefficient of fidelity term will vary adaptively with the instant image, which makes the diffusion term and the fidelity term come to a better compromise. Otherwise, when deciding the edge directions, because of the strong smoothness of linear Gaussian function, a few other edge directions hiding in the main direction will be lost. To preserve these detailed edge directions, Gaussian kernel is substituted for nonlinear wavelet threshold. In addition, in order to preserve the textures and details as much as possible, a nonlocal diffusion tensor was introduced, and the two eigenvalues are reset by combining the two methods: edge-enhancing diffusion and coherence-enhancing diffusion. Experiments show that the new model has an obvious effect on preserving textures and details.
1. Introduction
Partial differential equation has been widely used in image processing. PDE-based image processing can retrospect to Cabor and, Jain, but it is really founded by Koenderink and Wikin [1]; they proposed the concept of scale space and their contributions made up of the foundation of PDE-based image processing. The scale space of image can be obtained through evolving image by classical heat diffusion equation, but it is not the only way to get the scale space of image. In the late 1980s, Hummel proposed the criterion of constructing scale space. Based on the criterion, Perona and Malik proposed their anisotropic diffusion equation.
However, Perona-Malik model only takes the position information into account, and the diffusion coefficient of the position is decided by the magnitude of gradient modulus. Weickert introduced the concept of structure tensor and proposed his anisotropic diffusion equation. Weickert’s diffusion equation takes local variations of the gradient orientation into account, in some sense, it is really anisotropic. Two mean models of this type are edge-enhancing model and coherence-enhancing model. But the two models have limitations.
In order to overcome the defects, a fidelity term is added to Weickert’s diffusion equation, and the coefficient of fidelity term can adaptively vary with the instant image. In addition, to preserve more detailed edge directions hiding in the main direction, we substitute the nonlinear wavelet threshold operator for the Gaussian kernel. In order to preserve the textures and details as much as possible, a nonlocal diffusion tensor was introduced, and the two-eigenvalues are reset by combining the two methods: edge-enhancing diffusion and coherence-enhancing diffusion. Experiments show that the new method has an obvious improvement in vision effect.
2. Weickert’s Anisotropic Diffusion Equation
In PM diffusion equation, the diffusion degree is estimated by the magnitude of gradient modulus. In order to take the local variations of the gradient orientation into account, Weickert defined a more general descriptor, and structure tensor is proposed. The anisotropic diffusion becomes as follows: where , is a Gaussian kernel, which avoids the influence of noise in estimating the gradient; , it is called a structure tensor [2, 3]; the local information is averaged by convolving component wise with a Gaussian kernel , which avoids the fact that neighbouring gradients with same orientation but opposite sign offset one another. is called diffusion tensor; it has the same orthonormal basis of eigenvectors as , and the choice of corresponding eigenvalues of depends on the demand. Suppose that the two orthonormal bases and corresponding eigenvalues are , and , , indicates the orientation maximizing the gray-value fluctuations, while is perpendicular to . The eigenvalues and convey the shape information. Isotropic structures are characterized by , that is, the smooth areas, linelike edges by , and corners by .
In edge-enhancing diffusion model [4], . Supposing that the two orthonormal bases and corresponding eigenvalues are and , the eigenvalues of diffusion tensor are where is the eigenvalue that corresponds to and is the eigenvalue that corresponds to . We can notice that is large and decreases with the increase of ; this process can preserve edges, while, cannot get right edge direction. If is too small, the noises cannot be fully avoided, and the decided direction of edge will be fluctuated greatly; if is too large, the neighbouring gradients with the same orientation but opposite sign will offset one another, and we cannot get right edge directions. So, edge-enhancing diffusion is only applicable for images only with large structures.
Different from edge-enhancing diffusion model, in coherence-enhancing model [5], . Supposing that the two orthonormal bases and corresponding eigenvalues are and , the eigenvalues of diffusion tensor are where , which keeps the diffusion tensor uniformly positive definite. Because of the act of large-scale Gaussian kernel, a few edge directions hiding in the main direction will be lost. In addition, because of the small , even if in the smooth area, is much larger than , and many artificial edges are formed in the smooth area. So, coherence-enhancing model is only applicable for images with only linelike edges such as fingerprint image.
3. The Decision of Edge Directions Based on Nonlinear Wavelet Threshold
Based on the previous analysis, we know that some detailed edge directions will be lost because of the strong smoothness of Gaussian kernel [6]. So, we substitute the Gaussian kernel for the nonlinear wavelet threshold. The new structure tensor based on nonlinear wavelet threshold is as follows: where is the nonlinear wavelet threshold operator. The comparison of edge directions of a remote image, based on and Gaussian operator , is illustrated in Figure 1. The edge direction is depicted by grey values: horizontal direction is depicted in black and vertical direction in white. We choose as the soft wavelet threshold operator.
(a)
(b)
(c)
Figure 1(a) is the result of a noise remote sensing image, and Figure 1(b) is the result of edge directions found by . We can see that some detailed directions have been lost. Figure 1(c) is the result of edge directions found by . We can even see that more detailed edge directions have been detected, and it get more accurate edge directions.
4. The New Model Based on Nonlocal Diffusion Tensor
Suppose that the image function , is a bounded open region, and the notion of derivatives was extended to a nonlocal framework by the following definition [7]: where , , is a positive measure defined between the points and . To keep up with standard notations related to graphs [8], we define the weights as follows: Thus, . We assume, that the weights are symmetric; that is, .
The nonlocal derivative can be written as
The nonlocal gradient is defined as the vector of all partial derivatives:
We denote vectors as . The standard inner product is used for functions For vectors, we define a dot product and an inner product The magnitude of a vector is
With the above inner products, for vector , the nonlocal divergence is defined as
The Laplacian can now be defined by
Note that in order to get the standard Laplacian definition which relates to the graph Laplacian, we need a factor of .
We can also formulate nonlocal curvatures as follows: where
Based on the analysis in Section 3 and the definition of nonlocal operators mentioned previously, we extend the local tensor diffusion to nonlocal one; that is, where is the nonlocal gradient and is the nonlocal divergence. is a symmetric positive definite nonlocal diffusion tensor. Starting from image , series of images are obtained, the noises in which will be gradually reduced. The final denoised image should preserve textures and details of the original image as much as possible. Next, how to determine the weights and the nonlocal diffusion tensor will be discussed.
4.1. The Definition of Weights
The calculation of weights was divided into two steps. First, a distance measure of two points should be given, and then the weights can be defined according to the distance measure.
Taking into account the similarity and Euclidean distance, the distance measure of and is given [9] as follows: where is a Gauss function with standard deviation , the first integral term measures the similarity of and , and is the Euclidean distance between and . Only both the gray value and the Euclidean distance of the two pixels are similar; the two pixels are the similar pixels.
Now, the weights can be defined according to the distance measure . For model (17), in the diffusion process, we hope the similar pixels interact with each other, so the weights of similar pixels should be larger; that is, the weights should be a decreasing function of distance . For each point we define the following set of area (a parameter) within a search neighborhood around (where , ). where is the area of region . In the discrete case, expresses the number of pixels in region and is an area parameter. The significance of (19) is that in a larger neighborhood of , pixels whose gray values are the most similar to are searched; the pixels constitute a set recorded as . The weights of pixel to are defined as
4.2. The Design of Diffusion Tensor
In the area of isotropic structures, strong diffusion is needed in two directions to denoise efficiently, and the two eigenvalues should be large; in linelike edges, strong diffusion is needed in edge direction to enhance the edges, little diffusion is needed in the direction perpendicular to edges to preserve edges, the eigenvalue in edge direction should be large, and the one perpendicular to edges should be small; in the corner, little diffusion is needed to preserve the corner and the two eigenvalues should be small.
Based on the above analysis, the two eigenvalues are reset by combining edge-enhancing diffusion model and coherence-enhancing model. Letting , the new eigenvalues are as follows: where and are as follows: where is a weight function, and it decreases with and satisfies . In this paper, we choose . can be other operators such as exponent form.
Now, we analyze the eigenvalues in different areas. In isotropic-structure areas, , so ; , the two eigenvalues are large, which assure the strong diffusion in two directions and the effective de-noising in smooth areas.. In linelike edges, . Now is smaller, and is larger, which assures the preservation and enhancement of edges. In the corner, , and are both smaller, which assures the little diffusion in two directions and the preservation of corners.
Similar to local structure tensor proposed above, the nonlocal structure tensor of each pixel is defined as follows: where , is the nonlinear wavelet threshold operator [10].
Supposing that the two orthonormal bases and the corresponding eigenvalues of are , and , , the two orthonormal bases of diffusion tensor are same as , and the two corresponding eigenvalues of diffusion tensor are reset as above.
After determining the weights and the nonlocal diffusion tensor , the nonlocal tensor diffusion model (17) is obtained.
5. Anisotropic Diffusion Equation with Adaptive Fidelity Term
5.1. Anisotropic Diffusion Equation with Scalar-Parameter Fidelity Term
New eigenvalues are obtained by the method of Section 4, and the new diffusion tensor, expressed as , is determined. In order to preserve the detailed structures as much as possible and make the final denoised image faithful to initial noise image, a fidelity term with parameter is added to the equation, and the new diffusion equation is as follows:
Suppose that the noise can be approximated by Gaussian white noise, and the variance is [11], when the equation reaches its stable condition, and does not vary with the time . So the equation is
After denoising, we hope that the residual part contains all noises; that is,
Multiplying (25) by and integrating over , we get
Substituting (26) into (27), we get
After each step of iteration, substitute the instant image into (28), and we get the new parameter of fidelity term, which is proper for the instant time scale. Next iteration is processed by substituting the new into (24). Experiments show that diffusion equation with fidelity term is better in preserving edges and details than the one without fidelity term.
5.2. Anisotropic Diffusion Equation with Adaptive Fidelity Term
Let the initial noise image be ; our model consists of three components: , is the Cartoon approximation, and is the remainder noncartoon part; it typically consists of textures, small-scale details, thin lines, and so forth. The only assumption we make is that it has zero mean. They should be preserved during the course of denoising. is an additive noise. In the paper, the residue of an image is .
Now, we define the power of image as its variance [10]:
The mark , as mentioned above, denotes the different parts of an image. The power of noise is defined as its variance, which is known or can be estimated and denoted by .
Let us define first a measure which we refer to as the local power [11]: where is a normalized () and radially symmetric smoothing window and is the expected value. From the definition of the local power it follows that
Different contents need different extent of denoising. In the position of large structures, denoising efficiently is needed. After denoising, we hope that the power of the residual part is equal to the power of noise. While in the position of detailed structures, such as textures, in order to preserve the details, as a compromise, parts of the noise should be left in the denoised image; we hope that the power of the residual part is less than the power of noise. Now, suppose that , and it is prior information; the definition of this prior information will be given in next chapter.
The scalar was replaced by an adaptive function , and the diffusion equation is as follows: whose stable condition is We multiply (33) by and integrate over as follows: In order to compute , we denote as the convolution of function with window as follows: Substituting (35) into (34), after a change in the order of integrals in the we get where . A sufficient condition is Substituting (37) into (35), we can get the value of .
Now, the definition of will be discussed. In (37), in order to get the value of , the problem reduces to finding . should adaptively vary with the content of position . When lies in the large-structure area, should be large and fidelity becomes small, and then the diffusion equation can denoise efficiently in this position. When lies in the area which contains details, should be small and fidelity term becomes large, and then the diffusion equation can reject the denoising and preserving details.
For images with only large structures (so-called cartoon-type image), it requires full denoising, and the power of the residue should approximate the power of noise; that is . In fact, many images are noncartoon types; they typically contain textures, small-scale details, and so forth. If it is still fully denoised, these details will be lost or blurred. To overcome the defect that the denoised image will distort because of the loose of details, denoising and detail preserving should come to a better compromise. So the non-cartoon images require less denoising, and some noises still need to be in the denoised image. Now, the power of the residue should be less than the power of noise; that is .
After the previous analysis, we use the scalar-parameter method mentioned in Section 5.1 to choose the first metastable scale. In our paper, we choose (this condition is actually implicit as there is no stable scale with residue power below the noise level). Thus, after the process of scalar parameter, cartoon part of the image and the noise and texture parts are differentiated. A majority of details and textures are left in the residual part.
Substituting for into (28), we can get the residue by iterating (24). Substituting into (27), we can get the local power , we assign
Now, let us analyse (38). When lies in the large-structure area, the local power of the residue is almost constant (), and hence , and this scheme degenerates to the scalar-parameter one mentioned in Section 5.1. We get a high quality denoising process. When lies in the area which contains details, the local power of the residue contains two parts: , so and , and now, becomes small and becomes large. We get a low quality denoising process to preserve detailed structures.
Using the scalar-parameter method mentioned in Section 5.1 to choose the first metastable scale, after each step of iteration, substitute the instant image into (35). We get the new parameter of fidelity term, which is proper for the instant time scale. Next iteration is processed by substituting the new into (32). Experiments show that the diffusion equation with adaptive fidelity term is better in preserving edges and details than that with scalar-parameter fidelity term.
6. Experiment Results and Analysis
Because “barb,” “tiger”, and “remote sensing image” have more detailed structures, we choose these three images as the experiment images. To highlight the treatment of the details, we intercept texture parts: the legs of BARB.
We compare the processing results of six models: edge-enhancing model, coherence-enhancing model, local-tensor model without fidelity term, local tensor model with adaptive fidelity term, nonlocal tensor model without fidelity term, and nonlocal-tensor model with adaptive fidelity term. In our experiments, , three-layer wavelet decomposition is made, and time interval .
Figures 2, 3, and 4 are the denoising results of these six methods. (a) is the original image, (b) is the noise image, and (c) is the results of edge-enhancing model. Because it cannot get the right edge direction, the textures of legs in “woman” have disappeared and the details and fine edges in “Tiger” and remote sensing image have been blurred. (d) is the results of coherence-enhancing model. Although the textures in the leg can be vaguely seen, the details in “Tiger” and remote sensing image are even enhanced; many artificial edges appear in smooth areas. (e) is the results of local-tensor model without fidelity term, (f) is the results of local-tensor model with fidelity term, (g) is the result of nonlocal-tensor model without fidelity term, and (h) is the result of nonlocal-tensor model with fidelity term. We can notice that the models based on nonlocal tensor are better than the ones based on local tensor, and the models with fidelity term are better than the ones without fidelity term.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Tables 1, 2, and 3 are the performance parameters of these three images, respectively. In order to emphasize the preserving of detailed structures, we choose both parts with more textures and details to process; thus, the two contradictions, preserving details and denoising, should come to a better compromise, so PSNR increases a little, but the vision effect has improved much.
7. Conclusions
A fidelity term, adaptively varying with the time scale, is added to Weickert’s anisotropic diffusion equation. The conflict, preserving details and denoising, will come to a better compromise. More detailed structures will be preserved during the course of denoising. Otherwise, to preserve a few other edge directions hiding in the main edge direction, Gaussian kernel is substituted for nonlinear wavelet threshold operator. In addition, in order to preserve the textures and details as much as possible, a nonlocal diffusion tensor was introduced and the two eigenvalues are reset by combining the two methods: edge-enhancing diffusion and coherence enhancing diffusion. Experimental results show that the nonlocal adaptive image denoising model has an obvious effect on preserving textures and details.
Acknowledgments
This work is supported by the National Natural Science Funds (Grant nos. 61070087, 61001183, and 11101292) and Science and Technology Research Foundation of Shenzhen (Grant no. JC201005280508A).