Abstract
In this paper, we introduce two novel total variation models to deal with speckle noise in ultrasound image in order to retain the fine details more effectively and to improve the speed of energy diffusion during the process. Firstly, two new convex functions are introduced as regularization term in the adaptive total variation model, and then, the diffusion performances of Hypersurface Total Variation (HYPTV) model and Logarithmic Total Variation (LOGTV) model are analyzed mathematically through the physical characteristics of local coordinates. We have shown that the larger positive parameter in the model is set, the greater energy diffusion speed appears to be, but it will cause the image to be too smooth that required adequate attention. Numerical experimental results show that our proposed LOGTV model for speckle noise removal is superior to traditional models, not only in visual effect but also in quantitative measures.
1. Introduction
With the development of digital image technology, a large number of digital images are transmitted and compressed through various channels. However, image corruption usually is unavoidable during transmission and storage, and the resultant noise quite often seriously affects the visual effect of the image. Clearly, high-quality images are desirable in many areas, such as in medical imaging and pattern recognition. Thus, image denoising is a critical step in image processing and computer version, which plays an important role in various applied areas, especially in medical imaging, video processing, and remote sensing. On the other hand, it is also an important preprocessing process for other image processing that relies on subsequent processing.
Today, image denoising becomes research of focus, and many image denoising methods have been proposed such as Lee filter [1], Kuan filter [2], locally adaptive statistic filters [3–5], PDE-based and curvature-based methods [6, 7], wavelet transform based thresholding methods [8], and total variational [9–11]. In addition, the method based on machine learning [12–15] has received wide attention in recent years, such as deep learning [12, 13], linear regression [14], and Bayesian learning [13, 15].
In 1992, Rudin et al. [10] proposed a denoising model based on the total variation: where represents the TV regularization term, is an original image that is without noise, is noisy image, and represents the Gaussian random noise with mean zero and standard deviation . represents the regularization parameter which can be used to balance the fidelity terms and regularized terms in TV model, and represents the norm of the image gradient.
It is well known that medical ultrasonic images may have many noises of speckle, which will bring a significant problem in terms of the quality of ultrasonic images and cover up the lesions of certain important tissues. Further, it brings great difficulties to the actuate diagnosis and identification of certain specific diseases and can create the potential risk of missed diagnosis and misdiagnosis. Thus, it is very desirable to eliminate the speckle noise in ultrasound image and simultaneously retain the important features in practices. As mentioned in article [3], the speckle noise in medical ultrasonic images can be formulated in the following form: where is a noisy image, and represents the Gaussian random noise with zeros mean and standard deviation .
In this paper, we focus on the image denoising form by using variation method, where the noisy image is ultrasound speckle noise. Based on the model (2) and the characteristics of the Gaussian distribution, Krissian et al. in the article [16] derived a date fidelity term:
Within the variational framework, the data fidelity function is derived from the degradation model (2). One of the technical approaches to solve the variational model is the regularization technique, which minimizes the cost function to obtain stable and accurate solutions. In general, the image denoising variation method is to consider: where is a regularization term that represents a prior information about the object to be restored, and is a fidelity term to ensure that the restoration is not far from the original observation . represents the regularization parameter which can balance the fidelity term and the regularization term.
In [11], motivated by the classical ROF model [10], the authors proposed a convex variational model for removing the speckle noise in ultrasound image. The convex variational model involves the TV regularization term and convex fidelity term (see Equation (5)): where represents the TV regularization term, and represents the directional gradient of . The existence and uniqueness of the solution of model (5) is proved in [11]. In this paper, we call the model in (5) the “JIN’s model.” In [17], the authors proposed a well-balanced speckle noise reduction (WBSN) model that can detect edges.
Although TV regularization is effective for image denoising, it also leads to some staircase effects that is undesirable. In order to solve this problem, many methods based on improved TV regularization are proposed, such as high-order TV regularization [18–20], several hybrid TV regularization [21], the improved infimal convolution [22, 23], nonlocal TV model [24, 25], fractional order TV model [26], and anisotropic TV model [27, 28]. Fractional theory [29, 30], wavelet [31], and statistical information [32–34] are also employed to deal with intensity inhomogeneity or noise. Although these denoising methods can reduce the staircase effects in the restoration of additive noise images, however, there are more staircase effects that appeared in the restoration of speckle noise images. In this paper, we introduce HYPTV model and LOGTV model to reduce speckle noise and staircase effects in ultrasound images in an effective way.
In numerical algorithms, most of the energy function minimization problems can be transformed to an Euler-Lagrange equation and then be solved by using the finite difference method. However, the choice of adequate regularization terms is critical in terms of solution accuracy. Moreover, when solving the Euler-Lagrange equation, the energy diffusion form of the noise image in different regions is required to handle differently based on the physical characteristics of local coordinates, since the diffusion velocity of different parameters to the energy function is different in the two models. The new proposed HYPTV model and the LOGTV model, as shown in this paper, not only can preserve the edges of the restored images well when restoring the ultrasonic image with speckle noise but also better reduce the staircase effect generated during the recovery process.
The rest of this paper is as follows: in Section 2, we review some background knowledge. In Section 3, we propose two new models based on variation; meanwhile, we not only analyze diffusion performance of the proposed models but also give the corresponding numerical algorithms. Section 4 shows five different experiments and results. The paper ends with concluding remarks in Section 5.
2. Preliminary
2.1. Some Theoretical Background
The Euler-Lagrange equation was developed in the 1750s by Euler and Lagrange in connection with their studies of the tautochrone problem.
Theorem 1. A multidimensional generalization comes from considering a function on variables. If is open, bounded Lipschitz domain in , then is extremized only if satisfies the partial differential equation: For model (5), the corresponding Euler-Lagrange equation is as follow: where and , respectively, represent gradient operators and divergence operators. Using gradient descent method, we can get the model as follows: where is the unit out normal vector of .
Without losing generality, in the following, we consider the grayscale images as matrices.
Definition 2. Let , the gradient operators on the space and the divergence operators on the space are defined as: where are the first-order forward and backward discrete derivation operators in the -direction and -direction, respectively, which are defined as: Applying the above gradient operators and divergence operators to model (9), we can obtain the equivalent minimization problem.
Definition 3. Let be a convex subset of , A function is called convex if
The following facts are easily checked:
Theorem 4. If functions and are convex and have the same domain definition, then is also convex.
Proof. Let functions and are convex. According to the Definition 3, for any and , we have Hence is convex.
Theorem 5. If a differentiable function satisfied then is convex.
2.2. The Condition of TV Regularization Term
Although TV regularization is very effective in image restoration, it usually generates some staircase effects. Thus, it is suggested in literature to use general variational methods, i.e., to consider: where represents a potential function, and the case leads to the total variation regularization term. In the literature [35], the author Costanzino chooses that leads to the well-known harmonic model.
In practice, we prefer good smoothing in some domain where the intensity of variations is relatively weak. This can be achieved by requiring a function to satisfy the following conditions:
Near the edge of the image, the intensity of variations is strong. If we would like to preserve the edge, then the function should satisfy the following conditions:
With the conditions (16) and (17), the function is convex and nondecreasing function, such as:
There functions are convex and nondecreasing on , as shown in Figure 1.
In this paper, we will use two new functions and , which appear to be quite effective for image processing, in particular, for ultrasound image denoising. Obviously, the functions and are convex and nondecreasing.
3. The Proposed Restoration Model
In this section, we propose adaptive total variation model for image restoration. We use the finite difference method to solve the Euler-Lagrange equation directly, and then find the minimum value of the energy function.
3.1. The Adaptive Total Variation Model
Apply the selected function to model (15), we propose adaptive total variation model, where is a regularization term, and ; is a fidelity term; represents the regularization parameter which can balance fidelity term and regularization term.
Firstly, the energy functional is convex, which guarantees the existence of the minimal solution of the model (19).
Theorem 6. The energy functional is convex. That is to say, for any , and , we have: where .
Proof. Firstly, the function is convex; according to Definition 3, for any , and , we have
Meanwhile, we have (The proof step of the inequality is in the appendix.)
Therefore,
This proof is established.
Secondly, the uniqueness of the minimum solution of the model (19) can also be proved.
Theorem 7. If and are two minimize solutions of model (19), then we have .
Proof. According to Theorem 6, we have If , then the above assumption gives a contradiction that is not a minimize solution.
3.2. Diffusion Performance
In this subsection, we mainly analyze the diffusion performance and diffusion speed of the energy function of HYPTV model and LOGTV model.
3.2.1. Diffusion Performance of HYPTV Model
Firstly, we use the finite difference method to solve the HYPTV model, associated with the potential function .
From Definition 2, we can obtain the corresponding Euler-Lagrange equation HYPTV model that as follows:
Using gradient descent method, Equation (25) can be transformed to: where is the unit out normal vector of .
In order to analyze the diffusion performance, local image coordinate system is established. As shown in Figure 2, the -axis represents the direction parallel to the image gradient at the pixel level, and the -axis is the corresponding vertical direction.
According to Figure 2, we can know:
So, Equation (26) can be rewritten as: where
The and are control functions of the diffusion along the -direction and -direction, respectively. Now, we consider the diffusion of image restoration. Some test images are shown in Figure 3.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(1) Smooth area. When , . This shows that the diffusion form of the energy Equation (19) is isotropic. In other words, the energy diffusion rate along direction and direction is very close in the process of image restoration in the smooth region. And the rate of energy diffusion is obviously positively correlated with the parameter .
(2) Sharp area. When , we obtain . This shows that the diffusion form of the energy Equation (19) is anisotropic. In other words, the energy diffusion rate in -direction in Equation (28) is much larger than that in the -direction in the sharp region. But the gradient does not exceed 255, so . One can see that the larger the parameter is set, the smaller the limit becomes. And the rate of energy diffusion is obviously positively correlated with the parameter .
3.2.2. Diffusion Performance of LOGTV Model
Secondly, we use the finite difference method to solve the LOGTV model, which the potential function is .
From the Definition 2, we can obtain the corresponding Euler-Lagrange equation LOGTV model that as follows:
Using gradient descent method, Equation (30) can be transformed to: where is the unit out normal vector of .
Hence, Equation (31) can be rewritten as: where
and are control functions of the diffusion along the -direction and -direction, respectively. Now, we consider the diffusion of image restoration.
(1) Smooth area. When , and , we can obtain and . This shows that the diffusion form of the energy Equation (19) is isotropic. In other words, the energy diffusion rate along direction and direction is very close in the process of image restoration in the smooth region. When , and , we can obtain and . This shows that the diffusion form of the energy Equation (19) is anisotropic. However, the gradient of noise image is relatively large, so in the smooth region, whatever the value of , it has little effect on the model.
(2) Sharp area. When , we can obtain . This shows that the diffusion form of the energy Equation (19) is anisotropic. In other words, the energy diffusion rate in -direction in Equation (28) is much larger than that in the -direction in the sharp region. But the gradient does not exceed 255, so . One can see that the larger the parameter is set, the smaller the limit becomes. And the rate of energy diffusion is clearly positively correlated with the parameter .
3.3. Numerical Implementation
We will describe the corresponding numerical algorithm in this section. Firstly, the HYPTV model can be solved by discretization as follows: where , and represents time step. Furthermore, the iterative formula can approximate as: for ; , and represent the size of the image. Here: where , and is a positive parameter that is close to zero. With boundary conditions:
Now, note Equation (25), the two sides are multiplied by , and then, the integral on the domain can be obtained:
Because the Gaussian noise have mean 0 and variance , we can obtain:
In the process of iteration, we always use the previous solution to calculate the next solution. The optimization algorithm for HYPTV model is given in the following (Algorithm 1).
Secondly, the LOGTV model can be solve by discretization as follows: where . Furthermore, the iterative formula can approximate as:
Similar to the HYPTV model, the optimization algorithm for LOGTV model is given in the following (Algorithm 2).
4. Experimental Results
In this section, we present numerical results to demonstrate the effectiveness of the HYPTV and LOGTV model in image restoration. Firstly, to evaluate the quality of restored images, we use the peak signal-to-noise ratio (PSNR) value and the structure similarity (SSIM) index, which are defined as follows: where is the clean image, and is the restored image. is the average of , is the standard deviation of , and and are some constants for stability. Secondly, we calculated the noise deviation reduction (NDR) at each iteration as a convergence condition; where represents the results of the iterations, respectively. Finally, the stopping condition (NDR) for the HYPTV and LOGTV models is as follows:
In the numerical experiment, we will use the noise image as the initial value, that is, . Moreover, the gray values of all original images are in range [0, 255].
4.1. Denoising Effect of Different Parameters of HYPTV Model and LOGTV Model
In this example, we use different parameter values in the algorithm to test the effect of HYPTV and LOGTV models. The test images is shown in Figure 3(a), and the noise levels are and . Figure 4 shows the different PSNR values and iteration numbers when different values are used in the HYPTV model algorithm. Figure 5 shows the different PSNR values and iteration numbers when different values are used in the LOGTV model algorithm. Firstly, we can also see that PSNR is the largest in . Secondly, with the increase of parameters, the speed of image restoration is faster. Based on the above analysis, when , the denoising performance of HPYTV and LOGTV models is close to the best. Therefore, in the following experiment, we choose in HPYTV and LOGTV models.
4.2. Denoising Effect of the HYPTV and LOGTV Model
In this subsection, we use the algorithm to test the effect of the HYPTV and LOGTV model on image denoising. In Figures 6 and 7, we show the original images, the noise images, and the restored images by HYPTV and LOGTV models. Figures 6(a) and 7(a) are original images, Figures 6(b) and 7(b) are noisy images, which the noise levels are and , respectively. Figures 6(c) and 7(c) are corresponding restored images by HYPTV model. Figures 6(d) and 7(d) are corresponding restored images by LOGTV model. Table 1 shows that the PSNR values, SSIM values, and the numbers of iteration for the different test images can be got by using the HYPTV model and the LOGTV model. “Noise image PSNR” is the peak signal-to-noise ratio of noisy images and original images. “Denoising image PSNR” is the peak signal-to-noise ratio of restored images and original images. “Iter” is the number of iterations of the algorithm. From the results, it is obvious that the HYPTV model and the LOGTV model are fairly effective in reducing the speckle noise and edge-preserving.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
4.3. Reduction of Staircase
In this subsection, we test the reduction of the staircase effect in HYPTV model and LOGTV model by image in Figure 8. Figure 8(a) shows original images (“house”). Figure 8(b) shows detailed images of Figure 8(a), respectively. Figure 8(c) shows noise images which noise standard deviation . Figure 8(d) shows detailed images of Figure 8(c). The original images contain a lot of details information, such as textures edges and in homogeneous regions.
(a)
(b)
(c)
(d)
Figure 9 displays the restoration results of the noisy “house” image, respectively. Figures 9(a)–9(e) was restored ROF model [10], ATV model [36], JIN’s model [11], HYPTV model, and LOGTV model, respectively. Figures 9(f)–9(j) is corresponding details images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
According to the results, we can see that JIN’s model, HYPTV model, and LOGTV model had a good effect in removing image noise and preserving image edges. In addition, we can clearly see staircase effect in the detailed images obtained by ROF model, ATV model, and JIN’s model. At the same time, the staircase effect of HYPTV model and LOGTV model restored images is reduced. Although both the HYPTV model and the LOGTV model can reduce the ladder effect, the LOGTV model is better than the HYPTV model.
4.4. Comparison with ROF Model, ATV Model, and JIN’s Model
In this subsection, we compare the effect of the ROF model, ATV model, and JIN’s model with HYPTV and LOGTV models for some images. The test original images are shown in Figure 3(“house,” “peppers,” “boat,” “pirate,” and “bird”), with two sizes of and three sizes of . Figure 10 shows some noise images with different standard deviation.
(a)
(b)
(c)
(d)
(e)
Figures 11–15 display the restoration results for images (“peppers,” “boat,” “house,” “pirate,” and “bird”) through ROF model, ATV model, JIN’s model, HYPTV model, and LOGTV model. The noise versions of “peppers”, “boat” and “house”, and “pirate” and “bird” are obtained by model (2) with standard deviations 2, 3, and 4, respectively. In addition, the detailed images of the restored images are also displayed. Table 2 shows the PSNR and SSIM values for different test images by using the ROF model, ATV model, JIN’s model, HYPTV model, and LOGTV model. From the results of Figures 11–15 and Table 2, the noise standard deviation . We can observe that although the four models can effectively remove the noises while preserving the edges and details, the restored images by HYPTV model and LOGTV model have better visual effect with less staircase effects than by the ROF model, ATV model, and JIN’s model. The noise standard deviation . The visual effect of restored images by the ROF model and ATV model is particularly poor, but JIN’s model, HYPTV model, and LOGTV model can effectively remove the noises. Finally, Table 2 shows that LOGTV model has higher PSNR and SSIM values than other four models. This means that our proposed LOGTV model is available in reducing the speckle noise in some images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
4.5. Denoising Results of Real Ultrasound Images
In this subsection, we test some real ultrasound images. Figure 16 shows the experimental results of real ultrasound images by applying JIN’s model, HYPTV model, and LOGTV model. Table 3 shows the different iteration for the different test images by using the JIN’s model, HYPTV model, and LOGTV model. We find that LOGTV model is much effective than JIN’s model and HYPTV model in obtaining the satisfactory restored images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
5. Concluding Remarks
In this paper, we propose a new speckle noise restoration model based on adaptive TV method. Two new convex functions are introduced as the TV regularization term. By analyzing the diffusion performance of the proposed two models, one can see that the LOGTV model has faster diffusion speed than the HYPTV model. Moreover, we introduced two iterative numerical algorithms to solve the proposed models. The experiment results show the affectivity of our proposed model and the similarity between JIN’s model and HYPTV model. In addition, we compared the effect of the ROF model, ATV model, and JIN’s model with the LOGTV model, and the experiment results show high efficiency of LOGTV model in image restoration.
Appendix
The Proof of the Inequality of Theorem 6
Proof. For any , and , .
Here, we know ,
so, we have . Therefore, the proof holds to be true.
Data Availability
The experimental data are obtained by MATLAB R2017a, 2.93 GHz cup, 4 G RAM, and Windows 7.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
All authors typed, read, and approved the final manuscript.
Acknowledgments
This paper is partially supported by the Natural Science Foundation of Guangdong Province (2018A030313364), the Special Innovation Projects of Universities in Guangdong Province (2018KTSCX197), the Science and Technology Planning Project of Shenzhen City (JCYJ20180305125609379), the Natural Science Foundation of Shenzhen (JCYJ20170818091621856), and the China Scholarship Council Project (201508440370).