Abstract

We propose an adaptive total variation (TV) model by introducing the steerable filter into the TV-based diffusion process for image filtering. The local energy measured by the steerable filter can effectively characterize the object edges and ramp regions and guide the TV-based diffusion process so that the new model behaves like the TV model at edges and leads to linear diffusion in flat and ramp regions. This way, the proposed model can provide a better image processing tool which enables noise removal, edge-preserving, and staircase suppression.

1. Introduction

Image denoising is a fundamental task in the community of image processing, but there is always a dilemma for the denoising algorithms to simultaneously remove noise and to preserve edges. For example, the Gaussian filter can smooth noise effectively, but it also blurs the edges since it is just a low-pass filter which cannot discern noise and edges. Recent advances on this topic have brought considerable improvement in denoising performance, such as the kernel regression (LARK) [1], bilateral filter [2], patch-based method [3, 4], BM3D [5], and the partial differential equation (PDE) based methods [6]. In [7], the author presented a tutorial on these state-of-the-art denoising methods, and it has been shown that the LARK takes the bilateral filter and the nonlocal mean (NLM) [3] as special cases and is closely related to the anisotropic diffusion [8]. In this work, we intend to concentrate on the PDE-based methods.

During the past two decades, the diffusion partial differential equations (PDEs) which can be considered as stemming from the heat diffusion process have gained popularity in the community of image processing, with a particular emphasis on image denoising [6]. One typical algorithm is the anisotropic diffusion proposed by Perona and Malik (PM model) [8], which is able to reduce diffusion amount around boundaries and achieve a good trade-off between noise removal and edge preservation. In [9], the authors presented an improvement of the PM model using the difference eigenvalue. In [10], anisotropic second- and fourth-order diffusions are coined based on the gradient vector convolution. In [11], the second-order diffusion is combined with the fourth-order one fro image restoration, and, in [12], the convolutional virtual electric field is introduced into the diffusion model. The directional Laplacian is also introduced into the diffusion equation for image restoration [13]. The total variation model is another typical model which minimizes the following functional [14]: where the is the image domain, is the gradient operator, and is a weight parameter. is the unknown image to be recovered and is the input noisy image. The norm is the regularization term and the norm is the fidelity term. The regularization term measures the amount of oscillation found in the function and allows for discontinuities while disfavours oscillations; as a result, the TV model performs very well on preserving edges and removing noise.

Since its debut in 1992, there has been a flurry of works devoted to the TV model, such as the color TV [15], total generalized variation [16], Bregman iteration [17], L1-norm data fidelity [18], edge oriented TV [19], and the theoretical analysis [20], among others. It is interesting that a MTV model is introduced in [21]. However, the TV model, as well as the Perona-Malik anisotropic diffusion model [8], suffers from the so-called staircasing effect; that is, it transforms the ramp regions into piecewise constant regions. These staircases are visually unpleasant and can be falsely recognized as edges. A natural way to overcome this shortcoming is to increase the order of derivatives in the diffusion model, and the fourth-order PDEs have been of special interest in recent years [2224]. Although the fourth-order PDEs are free of staircasing effect, there is still staircasing effect in the second-order PDE models themselves [8, 14]. Another typical improvement of the TV model is proposed using a variable exponent in the TV energy by Blomgren et al. (BCMW) [25] as follows: where and with monotonically decreasing. This is a nonconvex minimization problem [26]; it behaves like TV model at edges and like linear heat diffusion in flat regions . After this model was proposed, there have been several works focused on this model by proposing different functions; for example, the authors in [27, 28] proposed to employ piecewise functions for ; the difficulties of these works are that how to set the thresholds in the piecewise functions. In [27], the author also suggested to use a smoothed version of the noisy image in , that is, . This choice makes the minimization problem convex [26]; however, the gradient magnitude can not characterize the edge and ramp region very well. Very recently, Chen et al. proposed an ATV model by designing an exponent function using the difference curvature (i.e., second-order directional derivative difference) [29]; , and are the second-order derivatives along and parallel to the gradient direction , respectively. However, the variant is sensitive to noise owing to the second-order derivatives. What is more, it is difficult to control the diffusion amount since the is time varying. There are also other works concerning the staircase reduction including the Gauss curvature driven diffusion [30], the linear regression method in the neighborhood filters [31], and the modified structure-based diffusion [32].

In this paper, our purpose also aims at devising a more effective exponent for the BCMW model in (2). Motivated by the fact that the steerable filter proposed in [33] performs very well in orientation analysis and edge detection [33, 34], the local energy measured by the steerable filter is employed to characterize the edges, ramps, and flat regions. Based on this local energy, we devise an exponent continuously changing from 1 to 2 to regularize the TV diffusion and also devise an adaptive weight for the fidelity term; this corresponding model is referred to as STV (steerable filter guided TV) model. The proposed STV model is mathematically convex and behaves like the TV model at edges and like the heat diffusion in flat regions. Comparative results on both synthetic and natural images demonstrate that this STV model not only has a good performance on avoiding the staircases but also performs well on preserving the edges.

The remainder of this paper is organized as follows. In Section 2, the steerable filters are introduced in brief and the local energy measured by the steerable filter is presented. In Section 3, the STV model is presented based on the local energy. In Section 4, the performance of the STV model is demonstrated by experiment and comparison. Finally, conclusions are drawn in Section 5.

2. Steerable Filter and Local Energy

Oriented filters are useful in many early vision and image processing tasks, such as texture analysis and edge detection [3537]. In [33, 34], Freeman and Adelson proposed the framework named steerable filter to create an oriented filter by describing it as a linear combination of basis filters. Under this framework, they designed the steerable quadrature pairs using the nth derivative of a Gaussian and its Hilbert transform. This steerable quadrature filter measures local energy and, based on the local energy, methods have been devised to analyze orientation and study structures like contours and edges [33, 34]. Here we employ the local energy to indicate edges and flat regions and to guide the diffusion of the TV model. In the following, we describe the steerable quadrature filter which follows the description in [33].

Generally, the steerable filter can be written in the following form: where is the interpolation function, is the basis function, and is the number of the basis functions. This expression tells us that the steerable filter can be written as a linear combination of rotated versions of itself . The steering theorems in [33, 34] answer the three questions: what functions can satisfy (3), how many terms, , are required in the sum, and what the interpolation functions, , are.

Under the steerable filter framework, the nth derivative of a Gaussian at arbitrary orientation using - separable basis filters is presented in [33], and its Hilbert transform is also approximated by finding the least square fit to a polynomial times a Gaussian. and form a steerable quadrature pair, which allow for analyzing spectral strength independent of phase and allow for synthesizing filters of a given frequency response with arbitrary phase [34]. In this paper, the second derivative and its Hilbert transform are employed to measure local energy. The takes the following form: where , , , , and   . Similarly, the reads where , , , , , , , and . Therefore, the local energy is measured by Here, we are more interested in the maximal response of at certain orientation : It was shown that the measured orientation and response accurately reflect the oriented structures of the input image [33, 34]; consequently, is employed to devise an effective exponent for the BCMW model in (2).

In previous works, intensity gradient [27] and difference curvature [29] are employed as the indicator to discriminate edges and ramps. Comparison between our new indicator and those is shown in Figure 1. From Figure 1, we can observe that the difference curvature is sensitive to the structures in image, including the object boundaries and the false edges caused by inhomogeneity and noises. The difference curvature in [29] is calculated from the being denoised image in each iteration; this makes the difference curvature less sensitive to noise, and, however, it also makes the ATV model difficult to control the diffusion amount. The gradient also captures the object boundaries and is less sensitive to the false edges stemming from inhomogeneity and noise than the difference curvature; however, it is calculated in advance and the false edges also affect the denoising effect. Meanwhile, the local energy from steerable filter is immune to inhomogeneity and robust to noise and captures the boundaries accurately. Although the local energy is not single-pixel wide, the max response characterizes the step edges very well (see the red line in Figure 1(a)), and the values nearby the max response are very small. Thus, the new indicator is a good choice to regularize the TV model for image denoising.

3. STV Model

Based on the new indicator derived from the steerable filter in (7), we devise a new exponent function and an adaptive weight and propose the STV model as follows: where the functions and are where is the normalized energy ,   and are the maximal and minimal values of throughout the entire image, respectively, and is a positive constant parameter. Since , the function monotonically decreases from 2 to 1, in flat and ramp regions, , , at sharp edges, , . Thus, the STV model approximates the TV model with the weight of the fidelity term near edges and approaches the linear heat flow away from the edges. Since the STV model can adapt the exponent function and the weight of the fidelity term in accordance with the image feature, it can preserve edges and fine details and prevent the staircasing effect simultaneously.

Let , , ; then and , ; that is, is an increasing convex function, and the problem (8) has a unique solution. Using the calculus of variation, we can obtain the solution of (8) by solving the following Euler equation: where . According to the orthogonal decomposition in [38], (10) can be further written in the following form: where and represent the second derivatives in the normal and tangent directions of the level sets of function , respectively. Therefore, the first and the second terms denote the diffusion in the normal and tangent directions of the level sets of function with different diffusion coefficients. Let , ; the diffusion ratio in the two directions is It can be seen from this diffusion ratio that, when the image gradient is small, approaches 2 and ; that is, the diffusion is isotropic. This property implies that the STV model can smooth the image and prevent staircases in flat and ramp regions. When the image gradient is high, approaches 1 and ; that is, the diffusion is mainly in the tangent direction; thus, the STV model can smooth the image and preserve edges.

Finite difference method is employed to solve (11); the forward difference is used for the time derivative and central differences are used for spatial derivatives. These difference operators are given by Thus, the discrete approximation of the directional derivatives of , that is, and , is simply derived after direct substitution of the difference operators listed above into and . For the measurement of convergence, the “normalized step difference energy” (NSDE) [30] was calculated at each iteration. It is defined as , where and denote the image at and iteration, respectively. The image dimension is . In Figure 2, we demonstrated the effectiveness of the proposed STV model and its NSDE graph. It can be observed that the result is clear and free of staircase, and the edges are sharp. The NSDE graph also shows that the STV model converged efficiently. In fact, when the iteration reaches 25, the NSDE is less than .

4. Experimental Results

In this section, we demonstrate the effectiveness of the proposed STV model by experiment and comparison on both synthetic and natural images. Nine well-known models are employed for comparison, including the TV model [14], the PM model [8], YK model [23], the ATV model [29], the DED model [9], the Hajiaboli’s fourth-order model (Hajiaboli) [24], the LARK [1], the BM3D [5], and the NLM [3]. The peak signal-to-noise ratio (PSNR) and the mean structure similarity (MSSIM) [39] are employed as object indexes to evaluate the image quality of the filtered images. The MSSIM ranges from 0 to 1 and if the filtered image is identical to the noise-free one, it is 1. We also visually evaluate the denoised images. The results of the NLM are yielded from the IPOL (http://www.ipol.im/pub/algo/bcm_non_local_means_denoising/) website, so the parameters for the NLM are default. We adopt the software packages of the LARK (http://users.soe.ucsc.edu/~milanfar/software/) and BM3D (http://www.cs.tut.fi/~foi/GCF-BM3D/index.html#ref_papers) and the parameters for both models are unchanged as in the package. The parameters for the other models are shown in Table 1. The time step and (or ) are adjusted to be as close as possible to those in other literatures such as [23]. It is well known that the iteration number is also critical to control the diffusion amount. We calculate the PSNR and MSSIM values at each iteration for the iteratively solved models, including the YK, PM, TV, ATV, STV, Hajiaboli, DED, and LARK models; the algorithms are terminated when the associated MSSIM values reach maximum. The runtime of each model for each experiment is also listed in Table 1; it is obvious that the BM3D is the fastest, and then the ATV and STV models; the LARK and DED models are the most time consuming.

The first experiment is a noisy synthetic image of ramps, which is shown in Figure 1(a). There are no complicated structure and texture in the image; however, the ramps and the sharp edges should be preserved. Figure 3 shows the results of the ten models. Visually, the PM and BM3D models perform the best on preserving the sharp edges and, apart from the DED model, the YK and LARK perform the worst. However, there are serious staircases in the results by the PM and TV models; there are also flow-like structure in the results by the LARK and BM3D models. The flow-like structure is more obviously in the Lena experiment for the LARK model; this will be shown in Figure 4. The result by the NLM model is somewhat foggy; see Figure 3(g). The result by the YK model is staircase-free, but there are dark spots; see Figure 3(b). The Hajiaboli model is also a fourth-order model and the result is much better than that of the YK model due to its anisotropic nature and comparable to that of the STV model; see Figure 3(j). The results by the STV and ATV models are visually comparable and staircase-free, but the PSNR and MSSIM values shown in Table 2 manifest that the STV model is superior to the ATV model. The result of the DED model is the worst from both visual inspection and PSNR and MSSIM indices; the reason is that there is no parameter playing a role similar to that of the parameter in the PM model. From Table 2, we also see that the BM3D has the best performance, the STV and Hajiaboli model are the second best, and the NLM, YK, and the DED models are the last. Clearly, the STV performs the best among the diffusion based methods.

The popular Lena and Pepper images are employed as the second and third examples. The noisy images in Figures 3~5 are yielded from the IPOL website by specifying the noise deviation to 20. The results of the ten models on Lena image are listed in Figure 4 and those on the pepper image are in Figure 5, but only the face region of the Lena image is shown for the sake of clarity. From Figures 4 and 5, it is obvious that, for the YK model, the results are misty apart from the speckles; for the PM model, there are also speckles and staircases which are visually unpleasant. For the DED model, since there is no parameter playing the role of the parameter in the PM model, the results are the worst from both visual inspection and PSNR and MSSIM indices. Similar results for the YK and PM models were also reported in [23]. Although there is no speckle for the TV model, there are many staircases, particularly, in the face region. While for the ATV model, there is no staircase, but there are dark spots as if there are acne and pimple in Lena’s face. We also observed acne and pimple in Lena’s face in the result of the Hajiaboli model. The results of the NLM are also misty. The LARK model suffers from the flow-like structure in Lena’s face, especially around the eyes and mouth. On the contrary, the proposed STV model yields clear and staircase-free result. The results of the BM3D model are also clear and staircase-free, except that there is slight flow-like structure around Lena’s chin. The PSNR and MSSIM values of each model in Figures 4 and 5 are reported in Tables 3 and 4, respectively. From these two tables, one can see that, although the BM3D and LARK models yield better PSNR than the STV does, the associated MSSIM values are comparable. The STV model outperforms the other models in terms of the MSSIM, including the NLM method, and performs the best among the diffusion based methods.

5. Conclusion

In this paper, we have proposed a nonlinear diffusion process (STV) for image filtering. The proposed STV model is coined by introducing the local energy measured by the steerable filter into the total variation (TV) model. The main result is that the STV model behaves like the TV model at edges and like the linear heat diffusion in ramp and flat regions, so that the STV model can simultaneously remove noises, preserve edges, and suppress staircases. Experiments have been conducted on both synthetic and real images, and comparisons have been launched with the classical and state-of-the-art models such as the YK, PM, TV, ATV, Hajiaboli, DED, NLM, LARK, and BM3D models. We have evaluated these models from the running time, PSNR and MSSIM indices, and visual inspection. Overall, the proposed STV model yields promising results and we believe that the local energy derived from the steerable filter acts as an effective indicator to discriminate edges and ramps and it can also be used in other PDE based approaches for image filtering.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors would like to express their gratitude to the editor and anonymous reviewers for their constructive comments that lead to this paper’s improvements in quality and representation. This work was supported in part by the Chinese Society of Neuro-oncology, Neuro-oncology Research Project CSNO-2013-MSD003, and the Program from the Tianjin Commission of Technology of China under Grant 11JCZDJC15600.