Abstract
The purpose of this paper is to develop an effective edge indicator and propose an image scale-space filter based on anisotropic diffusion equation for image denoising. We first develop an effective edge indicator named directional local variance (DLV) for detecting image features, which is anisotropic and robust and able to indicate the orientations of image features. We then combine two edge indicators (i.e., DLV and local spatial gradient) to formulate the desired image scale-space filter and incorporate the modulus of noise magnitude into the filter to trigger time-varying selective filtering. Moreover, we theoretically show that the proposed filter is robust to the outliers inherently. A series of experiments are conducted to demonstrate that the DLV metric is effective for detecting image features and the proposed filter yields promising results with higher quantitative indexes and better visual performance, which surpass those of some benchmark models.
1. Introduction
Image denoising is an important and fundamental issue in the field of computer low-level vision and has received considerable attention from scholars and practitioners. The goal of such task is to remove noise while preserving potential image features and further acquire a clean image which can be reliably used for subsequent vision tasks such as edge detection and object segmentation and fusion.
In the last two decades, the topic for image denoising has been well studied in the literature (see, e.g., the work of Tomasi and Manduchi [1], Buades et al. [2], Dabov et al. [3], Zhang et al. [4], He et al. [5], Zuo et al. [6], Zhang et al. [7], and Dong et al. [8] and references therein). Apart from the mentioned methods, a kind of anisotropic diffusion models based on nonlinear partial differential equations are also attractive and effective for image denoising. These methods are pioneered by Perona and Malik [9] for proposing the well-known PM model. Since then, growing interests have focused on its alternatives (see the works of Catte et al. [10], Alvarez et al. [11], Black et al. [12], Weickert [13], You and Kaveh [14], Gilboa et al. [15], Lysaker et al. [16], Gilboa et al. [17], and Wang et al. [18]). Recently, anisotropic diffusion models are still popular in the academic community, e.g., Chen [19] proposes an adaptive smoothing via local and contextual discontinuities. Chen et al. [20] present the ramp preserving PM model which is an effective tool for ramp preservation and speckle reduction. Chao and Tsai [21] and Li et al. [22] show that the local gray-level variance is an effective edge indicator for detecting image features by which they propose their modified PM models for feature preservation, respectively. Hajiaboli [23] and Chen and Gao [24] develop their anisotropic fourth-order diffusion models for feature preservation and addressing staircase artifacts caused by most second-order anisotropic diffusion models. Lefkimmiatis et al. [25] formulate the regularization model via Hessian-based norm for medical image denoising. Guo et al. [26] design an adaptive PM model called the dynamic -PM (D--PM) model which switches between isotropic diffusion and anisotropic diffusion in terms of an edge indicator based on local spatial gradient. Yang et al. [27] use nonlocal means theory to revise the PM model for feature preservation and mitigating staircase artifacts. Prasath et al. [28] propose a spatially varying edge coherence exponent-based Tikhonov total variation for achieving similar goals as in [27]. Chen and He [29] establish a novel filtering mechanism via an adaptive weighted anisotropic diffusion model. Siddig et al. [30] and Yang et al. [31] study their adaptive fourth-order diffusion models to reduce the over-smoothness for image features. Yao et al. [32] study a fractional diffusion equation for removing multiplicative noise. For more anisotropic diffusion models, we refer the reader to the comprehensive survey [33].
Most mentioned anisotropic diffusion models merely use the spatial gradient as their sole edge indicator. Nevertheless, noise corruption can generate discontinuities as well; hence, the local spatial gradient lacks robustness due to ambiguity between noise and image features. Chao et al. [21] and Li et al. [22] investigate that the local gray-level variance (LV) metric is a good alternative to the spatial gradient and can be effectively used to capture some important image features such as high-contrast edge features and low-contrast fine features, and they further propose their modified anisotropic diffusion models using this metric. However, the LV metric has two inherent drawbacks as follows. First, it is essentially isotropic and cannot indicate the orientations of image features. Moreover, it suffers from so-called dilution effect in the computing window, which degrades the performance of such metric for some low-contrast fine features. In this paper, we aim to address these drawbacks by introducing a directional local variance (DLV) metric and develop an image scale-space filter based on anisotropic diffusion by simultaneously utilizing the spatial gradient and DLV metric for image denoising.
The rest of the paper is organized as follows. In Section 2, we briefly describe some related works in this paper. In Section 3, we present the DLV metric and develop the aimed image scale-space filter for image denoising. In Section 4, we conduct experimental comparisons with some benchmark models. In Section 5, we conclude the paper.
2. Related Works
In this section, we describe some related works in this paper.
2.1. PM Model
To simplify notations, we denote image intensity of pixel at iteration as , the four-nearest spatial neighborhood of pixel as , and the magnitude as . Then, the PM model introduced by Perona and Malik [9] can be expressed aswhere for , is the time step size satisfying , and the diffusivity function with the threshold parameter has two commonly used forms:
It is well known that the spatial gradient as an edge indicator can sensitively capture any local intensity change, and this merit plays a crucial role in the PM model and its most variants. Nevertheless, such an edge indicator is not reliable and robust as mentioned before, and this fact would degrade the performance of the PM model and most variants, e.g., some outliers are falsely recognized and preserved as image features while some potential fine features are mistaken as noise and then smoothed out. Consequently, the PM model equipped with sole spatial gradient needs to be modified into another form which considers both spatial gradient and more robust edge indicator.
2.2. LV-PM Model
Let be a square spatial neighborhood of fixed radius and centered at the pixel . Then, we can compute the local variance (LV) of as (see [21])where the local mean is given by
Utilizing the local variance metric (4), Chao and Tsai [21] propose an adaptive threshold for the PM model as follows:where is a positive constant and is the normalized local variance, and it is formulated aswhere and denote the minimum and maximum values of local variance across an image at iteration , respectively.
The local variance metric (4) is shown to be an effective edge indicator for capturing contextual features such as textures and low-contrast fine features; hence, it is a good alternative to the spatial gradient [21]. Specifically, this edge indicator has two inherent merits as follows. First, more neighboring pixels around a given pixel are participated to compute the value of the local variance; they can offer more contextual information; thus, it is not like the spatial gradient that merely considers the difference of two adjacent pixels. Furthermore, the local mean involved in (4) can be used to measure the average intensity magnitude of neighboring pixels, whereas the spatial gradient has no such special form. Hence, these facts indicate that the local variance metric is more robust than the spatial gradient and able to reduce ambiguity between noise and significant features. Nevertheless, this metric has two main inherent drawbacks as follows. First, it is essentially an isotropic edge indicator which cannot indicate the orientations of image features. On the contrary, it may fail to some low-contrast fine features due to the dilution effect which happened in the computing window.
3. Methodology
3.1. Directional Local Variance
To simplify notations, let denote the eight-nearest spatial neighborhood of pixel and denote 1-dimensional (1D) directional spatial semineighborhood of fixed radius and started at the pixel along the -direction, where . Then, we propose the directional local variance (DLV) metric, , as follows:where is the directional local mean and defined by
Apart from the merits inheriting from the traditional local variance metric (4), the proposed metric (8) has two main inherent merits. First, it is an anisotropic edge indicator which is essentially different from the isotropic one (4). Consequently, the proposed metric can detect any local intensity changes in all directions and offers a “road map” to specify the orientations of important image features. Additionally, unlike metric (4), the dilution effect is anticipated to be mitigated by the proposed metric.
Below, we use three types of typical regions to intuitively illustrate the characteristics of the DLV metric. Patch-1 in Figure 1(b) shows that all DLV values are smaller while they are very similar. Hence, this patch more likely corresponds to a relatively homogenous region. Patch 2 in Figure 1(c) shows that the DLV values along the NE, E, SE, S, and SW directions are smaller, whereas the DLV values along other directions are larger. Hence, the central pixel and its neighbors more likely locate in an interobject boundary (edge) which stretches along the NE and SW directions. Patch 3 in Figure 1(d) shows that all DLV values are similar to each other while they are relatively larger. Consequently, the central pixel and its neighbors more likely correspond to the fine feature.
(a)
(b)
(c)
(d)
3.2. Image Scale-Space Filter via Anisotropic Diffusion
Motivated by the paradigm of anisotropic diffusion [19], we combine the spatial gradient and directional local variance defined by (8) to formulate the desired image scale-space filter as follows:where is the variable time step size defined by the local variance metric with the tunable positive constant and noise variance estimated from image , is the weighting function defined by the directional local variance , and is the weighting function defined by the magnitude of spatial gradient with the threshold . Furthermore, the generic nonnegative monotonically decreasing function used in (11), (12), and (13) is given by (2) or (3). The weighting function is simple to understand without further explanations, and the rationalities of the weighting function can be explained using statistical theory. From the law of large numbers in probability theory (see e.g., [34]), the value of the local variance in the ideal homogenous region generally behaves like if the size of the region is large enough. Consequently, we can roughly use as a threshold to achieve the separation of significant features and homogenous regions under noisy circumstance. Alternatively, a pixel more likely locates in the potential homogenous region when holds. Otherwise, it more likely corresponds to the potential image feature.
From (10), we know that the proposed image scale-space filter (10) are driven by the overall effects of two weights and in which embeds the effect of robust directional local variance for contextual features while embeds the effect of sensitive local spatial gradient for local features. Compared to the conventional anisotropic diffusion models such as the PM model (1), below, we show that the proposed filter has a different diffusion behavior. Without loss of generality, we simply consider 1D case and denotewhere and denote arbitrary image intensities and represents an outlier which means or . Furthermore, assume that the ground truth of is . Then, the proposed filter showswhich reads as the weighted average of and with adaptive weights and , respectively. Since holds, hence, goes to its ground truth in an iterative manner. Specifically, approaches with higher convergence rate for fast noise removal due to the facts and if the pixel being an outlier locates in the homogenous region, and approaches with lower convergence rate for edge preservation due to the facts and if the pixel locates at an edge. Factually, the above discussions are not limited to the outliers and applicable to more general situations. Furthermore, the variable time step size in (11) behaves like due to the fact for a desired filter so that the diffusion rate of the proposed filter (10) temporarily tends to be slow for inhibiting over-smoothness when noise magnitude becomes smaller. Contrarily, the PM model showswhich holds due to the fact for an outlier. Consequently, the PM model cannot effectively handle the outliers except for larger threshold which yet causes the problem of undesired over-smoothness, and these facts have been confirmed by the experimental results in [26].
Finally, we show that the proposed filter (10) maintains unconditional stability. To this end, we denote the maximum and minimum intensities across an image by and , respectively. Since always holds, we derive that
Also, we obtain
Hence, we obtain which confirms the unconditional stability of the proposed filter.
4. Experimental Results
In this section, we conduct several experiments to assess the effectiveness of the proposed filter and compare it with some competitive benchmark models: local variance-based PM model (LV-PM) [21], D--PM model [26], and recent fourth-order anisotropic diffusion model (FAD) [30]. To achieve the comparison, five commonly used digital images which are given in Figure 2 are used for the experiments. Furthermore, we use additive white Gaussian noise (AWGN) with noise variance to generate corrupted images and set for computing both LV and DLV metrics throughout the experiments.
(a)
(b)
(c)
(d)
(e)
4.1. Test for the DLV Metric
Before assessing the proposed filter for image denoising, we first test the effectiveness of the DLV metric proposed in this paper for detecting image features and compare it with the LV metric. The experimental results for the noisy Lena image are illustrated in Figures 3(b)–3(j), where brightness indicates potential image features and the label denotes DLV map along the -direction.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
It is evident that the edges along the S and N directions within extracted Patch 1 and Patch 2 in Figure 3(c) are much cleaner than the corresponding patches in Figure 3(d) whose edges are desirably masked out by the DLV metric; these facts confirm that the proposed DLV metric is an effective and anisotropic edge indicator which enables an anisotropic diffusion filter to achieve less normal smoothing for edge preservation and achieve more tangential smoothing for sharpening edges. It can be seen from Figure 3(b) that some edges (see, e.g., edges within Patch 2) are almost blurred due to the dilution effect caused by the isotropic LV metric in its computing window . Hence, we can conclude that the proposed DLV metric outperforms the classical LV metric for capturing image features.
4.2. Test for the Proposed Filter
In this experiment, we compare the proposed filter with the mentioned benchmark models for image denoising. To achieve that, we set , use the algorithm in [35] to estimate the noise variance , and adjust the parameter for a specific application. The whole experiment consists of three tests in which the first test uses the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) (see, e.g., [36]) to evaluate the models. The PSNR index is defined aswhere , here represents the image domain with size (i.e., the number of pixels) and is the ground-truth image. The SSIM index reads aswhere and are the estimated mean intensities of two windows and , and are the standard deviations, is the covariance between and , and and are the stabilization parameters. The SSIM value lies in the range where larger value means higher structural similarity between two images, and more details about this index can be seen in [36]. To be fair, the maximum iteration number is set to be 1000 so that the maximum scores of the quantitative indexes can be reported for each model. The evaluated scores of quantitative indexes are given in Table 1, from which we observe that the PSNR and SSIM values of the proposed filter are higher than those of the benchmark models in most cases. Thus, we can conclude that the proposed filter outperforms the benchmark models with respect to the objective quantitative indexes.
The purpose of the second test is to show the evolutions of PSNR and SSIM indexes using noisy Lena image with . From Figure 4, we observe that the proposed filter, compared to the benchmark models, uses less iterations to achieve its maximum PSNR and SSIM scores. Furthermore, the stability of the proposed filter is also validated.
(a)
(b)
The third test aims to illustrate the conceptual performance of the proposed filter for the corrupted images with . The denoising results are given in Figures 5–8, where Figure 6 for a fragment of Lena image is to more clearly demonstrate the conceptual performance. From Figures 5–8, we can conclude the following facts. Although the LV-PM model can preferably preserve most image features, it tends to leave some significant outliers at the interobject boundaries before blurring some low-contrast fine features. The reasons for this poor performance are just that the LV metric inherently belongs to an isotropic edge indicator that causes the dilution effect in the computing window as shown earlier. Although the D--PM model can preserve and even enhance the high-contrast edge features, it causes over-smoothness for some low-contrast fine features as more intuitively seen in Figure 6(d). One possible reason for this fact is just that the model is a hybrid one that combines an isotropic diffusion model and the PM model through an adaptive variable exponent-based diffusivity function which cannot effectively recognize the low-contrast fine features by using Gaussian prefiltered spatial gradient as an edge indicator. For the FAD model, it shows similar behaviors to the D--PM model as more clearly shown in Figure 6(e) since the employed Gaussian prefiltering would blur the low-contrast fine features prior to the spatial gradient estimation. Finally, we can see that the proposed filter equipped with directional local variance metric and local spatial gradient achieves more impressive conceptual performance among the comparative models with clearer edges (high-contrast edges and low-contrast fine features) and cleaner homogenous regions which can be more intuitively seen in Figure 6(f).
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
5. Conclusion
In this paper, we proposed an effective edge indicator for detecting image features and developed an image scale-space filter via anisotropic diffusion for image denoising. First, an anisotropic and robust edge indicator called directional local variance (DLV) metric was proposed for detecting image features. Unlike the traditional isotropic local variance (LV) metric, the proposed DLV metric has no so-called dilution effect and can be used to indicate the directional structures. Then, we formulated an image scale-space filter by combining two edge indicators (spatial gradient and DLV metric) and incorporated the modulus of noise magnitude into the targeted filter to trigger time-varying selective filtering. Furthermore, the inherent robustness of the proposed filter to the outliers was theoretically demonstrated. Experimental results illustrated that the DLV metric is anisotropic and robust and able to indicate the image directional structures and also confirmed that the proposed filter yields promising denoising results with higher quantitative evaluation indexes and better visual performance, which outperform those of the benchmark models. In the future, we will consider some prevalent techniques such as entropy theory and spectral analysis (see, e.g., [37–39]) to explore more robust edge indicators and employ them for image denoising.
Data Availability
The test images used for experiments are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
The first author was supported by Open Research Fund Program of Data Recovery Key Laboratory of Sichuan Province (Grant no. DRN19013).