Abstract

Impulsive noise removal for color images usually employs vector median filter, switching median filter, the total variation method, and variants. These approaches, however, often introduce excessive smoothing and can result in extensive visual feature blurring and thus are suitable only for images with low density noise. A marginal method to reduce impulsive noise is proposed in this paper that overcomes this limitation that is based on the following facts: (i) each channel in a color image is contaminated independently, and contaminative components are independent and identically distributed; (ii) in a natural image the gradients of different components of a pixel are similar to one another. This method divides components into different categories based on different noise characteristics. If an image is corrupted by salt-and-pepper noise, the components are divided into the corrupted and the noise-free components; if the image is corrupted by random-valued impulses, the components are divided into the corrupted, noise-free, and the possibly corrupted components. Components falling into different categories are processed differently. If a component is corrupted, modified total variation diffusion is applied; if it is possibly corrupted, scaled total variation diffusion is applied; otherwise, the component is left unchanged. Simulation results demonstrate its effectiveness.

1. Introduction

Noise reduction is a fundamental issue in image processing. In order to reduce noise, a great number of works have been presented over the past several decades. The overwhelming majority of these works cope with the gray-scale images. With the increase in use of color images, however, more and more works to reduce noise for color images are rapidly growing.

Indeed, color images often are corrupted by various types of noise. In this paper, impulsive noise is only considered, containing salt-and-pepper noise and random-valued impulses where they present themselves as occurring isolated chromatic points. To yield better images from their noisy versions, a series of various methods have been proposed.

Vector median filter (VMF) [1] and variants, based on reduced ordering, are popular methods. VMF calculates the cumulative sums of the Euclidean distance from every pixel to other pixels in a filtering window and outputs the pixel corresponding to the minimization sum of distances. The output pixel is called vector median, since each pixel is a vector containing multiple components. To achieve better performance in the variants, a few dissimilarity measures are used other than the Euclidean distance. For example, Basic Vector Directional Filter (BVDF) uses the angular distance [2]; Directional Distance Filter (DDF) employs the combination of the magnitude and directional processing [3]. In addition, -Trimmed Vector Median Filter () only considers alpha pixels to calculate the dissimilarity [4], rather than all pixels within a filtering window. These methods process the noisy and noise-free pixels uniformly. Therefore they introduce extensive smoothing and blur visual features extensively and are only suited to low density noise.

To overcome this limitation, various kinds of switching vector filters have been proposed in the rich literature, which aim to only replace the corrupted pixels. This type of filters usually consists of noise detection and noise removal. The former identifies the corrupted pixels, and the latter applies some algorithm to those corrupted pixels. The noise detection is crucial. If a detector fails to identify corrupted pixels, then they will be left unchanged, resulting in poor filtered images. If the detector classifies a corrupted pixel correctly but also identifies noise-free pixels as corrupted, the tiny details may be lost. Therefore, accurate and complete identification determines the quality of filtered images.

Currently, there exist many techniques to identify noisy pixels. Peris-Fajarnes et al. [5] use the rank-ordered differences statistic method. Smolka [6] uses a soft switching technique that assigns trimmed cumulated distances to the central pixel, where the trimmed sum of distances serves as an indicator manifesting the corrupted extent of the current pixel. Morillas et al. [7] utilize the local self-adaptive fuzzy filter. Ananthi and Balasubramaniam [8] use the technique of the interval-valued fuzzy set. Jin et al. [9] employ the quaternion theory. Malinski and Smolka [10] use adaptive switching technique. Particularly, after the concept of a peer group is introduced, many schemes based on it have been presented; for example, Morillas et al. [11] use the fuzzy peer groups, and Malinski and Smolka [12] utilize the fast averaging peer group.

Apart from the filters mentioned above, there exist a family of variational and partial differential equations. This type of methods usually adopts a variational energy minimization model to obtain the solution, such as the well-known Rudin-Osher-Fatemi (ROF) model [13], the hybrid model [14], the modified total variation diffusion method [15], and the total variation model [16]. The main advantage for them is to use the gradient of images to represent the model, and thus they have the capability of edge-preserving. Unfortunately, this type of methods has the drawback similar to the VMF; that is, they only work well on the low density noise because they do not distinguish between the corrupted and the uncorrupted pixels.

Inspired by switching filters and variational methods, in this paper a decision-based marginal diffusion method is proposed to reduce impulsive noise for color images. The proposed method, in contrast to vectorial methods emphasizing the correlation between the channels, independently treats each channel in a color image and implements diffusion operations on every corrupted component rather than every vector. In addition, the proposed method divides components into different categories based on different noise characteristics. If an image is corrupted by salt-and-pepper noise, the components are divided into the corrupted and the noise-free components; if the image is corrupted by random-valued impulses, the components are divided into the corrupted, noise-free, and the possibly corrupted components. Components falling into different categories are processed differently. If a component is corrupted, a modified total variation diffusion is applied; if the component is possibly corrupted, a scaled total variation diffusion is applied; otherwise, the component is left unchanged. Experimental results show that the proposed method is robust to different noise strengths and suitable for different images, with strong noise removal capability as shown by PSNR/MSSIM/FSIM results as well as the visual quality of restored images.

The rest of this paper is organized as follows. In Section 2, the details of noise removal are introduced. Section 3 reports experimental results and comparisons, and conclusions for this paper are drawn in Section 4.

2. The Proposed Denoising Method

In this section, the details of reducing salt-and-pepper noise and random-valued impulses are described, containing denoising structures, noise models, noise detection, and noise removal.

2.1. Noise Removal Structure

Two denoising structures are devised based on two noise characteristics, respectively. As shown in Figure 1, inset (a) is associated with salt-and-pepper noise and inset (b) is associated with random-valued impulses. Both structures consist of noise detection module and noise removal module, corresponding to detection stage and removal stage. For salt-and-pepper noise, a detector divides components into the corrupted and the noise-free components; for random-valued impulses, another detector divides components into the corrupted, noise-free, and the possibly corrupted components. Based on the two detection results, two mask arrays are built where each entry is a label indicating which category the corresponding component belongs to. Components falling into different categories are processed differently in separate channels. If a component is corrupted, modified total variation (MTV) diffusion is applied; if the component is possibly corrupted, scaled total variation (STV) diffusion is applied; otherwise, the component is left unchanged. To achieve better results, diffusion operations are iteratively implemented following the corresponding mask array instructions. Especially, for random-valued impulses, a hierarchical scheme is adopted in removal stage; the corrupted components are first processed and then the possibly corrupted ones, as in inset (b).

2.2. Noise Model and Analysis

RGB images to be processed are only considered in this paper, which contain three channels, the red, green, and blue, and a color image is defined as a two-dimensional matrix consisting of a certain number of pixels. Let denote a pixel at the position in the matrix and components for denote the RGB channel intensity values; a contamination image can be modeled aswhere and denote the original component and the contaminative component, respectively. In this model, the intensity of component is a random variable. If takes the value 0 or 255 with equal probability, it is salt-and-pepper noise model, denoted by SPM; if takes any value from the range , then it is random-valued impulsive noise model, denoted by RDM, assuming 8-bit per channel image representation.

The probability of a noise-free component processed incorrectly is analyzed in the vectorial and marginal methods. Assuming detectors can correctly judge whether every component is corrupted or not, then the marginal methods can correctly process all components in the removal stage. The vectorial methods, however, may identify noise-free components as noisy to process, because the processing is based on the pixel unit rather than the component unit. If a component is corrupted with probability , then a pixel is not corrupted with probability since a pixel may have corrupted 1, 2, or all 3 channels. Therefore, in vectorial methods a noise-free component is processed incorrectly with probability As shown in Figure 2, with the increasing probability , the probability of a noise-free component processed incorrectly is decreased. In an extreme case, only when all components are corrupted, can every component be correctly processed. Just from this viewpoint, the marginal methods are superior to the vectorial methods.

2.3. Noise Detection

Two detectors are used in the SPM and RDM, respectively. Both work in separate channels, based on the features of local neighborhood. In other words, a component is detected within the channel it lies in. Let be a current component centered in the sliding window; its neighborhood is defined aswhere is the neighborhood radius quantified into the positive integer domain and denotes the th channel domain. Obviously, a neighborhood contains samples.

The Detector Used in SPM. This detector takes components with intensity 0 or 255 as the noisy candidates. Let be a candidate, the absolute differences of intensities between it and every sample within the neighborhood are calculated, and then the number of samples, whose absolute difference is larger than a predefined threshold value, is determined, expressed aswhere is the predefined threshold value, denotes the cardinality of the set, and is the number of the samples that satisfy the threshold condition.

The number is used to determine if a current candidate is corrupted. The determining function isIn this formula, if , then candidate is corrupted; otherwise, it is uncorrupted. When all candidates are complete, a binary mask array can be built; each entry is either 1 or 0.

The Detector Used in RDM. This detector takes all components as the noisy candidates, which is devised based on the Rank-Ordered Absolute Differences (ROAD) [19]. It assigns an index to each candidate. Based on these indices, the corresponding candidate is judged as noise-free, corrupted, or possibly corrupted. Let be the current candidate; the ROAD of is defined bywhere is a integer, andThen, the index of iswhere and are predefined threshold parameters. The index , for , is a determining function, which divides candidates into three categories. If , then is declared as noisy-free; if , is declared as the corrupted component; otherwise, it is a possibly corrupted component.

2.4. Noise Removal

In the removal stage, diffusion operations are implemented in separate channels. The main reason is twofold. First, each channel in a color image is contaminated independently, and contaminative components are independent and identically distributed. Second, in a natural image the gradients of different components of a pixel are similar to one another, where Figure 3 illustrates this property.

Moreover, components falling into different categories are processed in different diffusion ways. In SPM, the components are divided into the corrupted and noise-free components. If a component is corrupted, the MTV diffusion is applied; otherwise, the component is left unchanged. Let be the th channel of the noise-free image, be its noisy version, and be the th distorted channel domain where is the th channel domain and ; then the MTV diffusion is expressed as Applying Euler-Lagrange to (9), the corresponding descent flow is where and div are the time step and the divergence operator and , where is a positive lifting parameter avoiding the vanishing.

In RDM, the components are divided into the corrupted, noise-free, and the possibly corrupted components, and a hierarchical scheme is adopted that first processes the corrupted and then the possibly corrupted ones. As shown in inset (b) in Figure 2, in the first stage, if a component is corrupted, the MTV diffusion is applied; otherwise, the component is left unchanged. In the second stage, if a component is possibly corrupted, the STV diffusion is applied; otherwise, the component is left unchanged. Let denote the corrupted domain of the th channel; the descent flow of MTV in RDM is as follows: Let denote the possibly corrupted domain of the th channel where and denotes the th estimated channel by the MTV diffusion; then the descent of STV is where is the scaled parameter, and the parameter value is obtained according to (8): Discretization of the divergence must be discussed. As shown in Figure 4, the techniques of central finite difference and half-pixel resolution discretization are used. Let be a current component; the divergence at the current is expressed aswhere and are the first-order derivatives in the x and y directions in the th channel, respectively.

At the half-pixel in the th channel, the following three formulas hold: In addition, ; then where Similarly, at the other three half-pixels,where Substituting (16) and (18) to (14), the divergence at the current can be discretized aswhere , denote the 4-neighbors of in the th channel; that is, Then, the explicit discrete schemes associated with (10), (11), and (12) are, respectively, as follows: In these equations, is the time step-size, and the superscript denotes the th iteration.

3. Experiments

In this section, three metrics used to evaluate denoising performance are first introduced containing Peak Signal to Noise Ratio (PSNR), Mean Structural Similarity Index Measure (MSSIM), and extended Feature Similarity index (FSIMc), then the setting parameters are discussed, and finally the experimental results and comparisons are reported.

3.1. Three Metrics

PSNR, given in decibels (dB), is a measurement based on mean pixel intensity errors between the noise-free and the restored images. Higher PSNR means better denoising capability. Let and be the noise-free and the restored images, respectively, denote the number of pixels in , and denote the Frobenius norm; the calculation of PSNR isMSSIM is a measure based on structural similarities [20]. It is bounded in the range , where the closer the value is to 1, the better the denoising scheme is implied. MSSIM is mean value of the structural similarities between the blocks of the noise-free image and the restored image . Therefore, the computation involves two blocks, denoted by and , respectively. Let , be the mean values of and , respectively, and the variances, and the covariance, and let and denote two stabilization variables; then the calculation of SSIM isFSIMc is a measurement based on feature similarities [21]. Similar to MSSIM, it is also bounded in the range , where the closer the value is to 1, the better the denoising performance is implied. This index is a combination of Phase Congruency (PC) and Gradient Magnitude (GM). Let be a pixel and be a parameter. and , where symbol means similarity, and the subscripts and denote chromatic channels; the calculation of FSIMc is

3.2. Parameter Setting

The Parameters in SPM. Five parameters are set in the proposed SPM method; are used in salt-and-pepper noise detection, and are used in diffusion operations. The parameters, and thr, are two thresholds where the former judges the relationships between a component and its neighbors and the latter decides whether the component is corrupted or not. The influence of both on the PSNR denoising results for different intensity noise is shown in Figure 5. For all noise densities, , and were set to 3, 25, 3, and 0.8, respectively. The parameter is the desirable iterative time that depends on noise strength and the time step-size . Figure 6 illustrates that the PSNR and MSSIM results for fixed strength noise depend on the number of iterations when . From inset (a) in Figure 6, as the number of iterations increases, the growth of PNSR tends to slow and ultimately achieves the stable value when meets or exceeds the desirable number , for any fixed noise. The same conclusion for the MSSIM measurement can be drawn from inset (b) in Figure 6. In our experiments, the values of are shown in Table 1.

The Parameters in RDM. Seven parameters are set in the proposed RDM method; are used in random-valued impulse detection, and are used in diffusion operations. For all the noise densities, , and were set to 2, 14, 150, 320, and 0.5, respectively. The parameter is the desirable number of iterations that is used in processing the corrupted pixels; the parameter is the desirable number of iterations that is used in processing the possibly corrupted pixels. Both parameters depend on noise strength and the time step-size . When , the values of two parameters used in our experiments are shown in Table 2.

3.3. Experimental Results and Comparisons

Experimental Results and Comparisons in SPM. A test set, denoted by , was built to validate the proposed SPM method. This set contained contaminative versions of seven images, and each image was corrupted by ten different densities of salt-and-pepper noise. The seven noise-free images are shown in Figure 7, taken from the CVG-Granada database. Using the values of parameters mentioned in SPM in Section 3.2, the proposed SPM method was applied to the test . The PSNR, MSSIM, and FSIM results for the Lena image are reported in Table 3, and the visual results for Lena, Pepper, and Baboon are shown in Figures 8, 9, and 10, respectively.

To augment the evaluation, the proposed method is compared with other five methods including MMF [22], AMMF [23], VMF [1], TVL1 in [18], and TVL1 in [17]. The MMF filter used the sliding window of size . In the AMMF, when the noise densities were 0.05, 0.1, 0.2, 0.3, 0.4, 0.5 0.6, 0.7, 0.8, and 0.9, the filtering window radius values were set to 1, 1, 2, 3, 3, 4, 4, 5, 6, and 7, respectively. The VMF used the sliding window of size . The source codes of TVL1 in [18] and TVL1 in [17] were taken from their authors, and the parameters used in the experiments were those recommended by the authors. The five methods were also applied to the same test set . The PSNR, MSSIM, and FSIM results for the Lena image are also reported in Table 3. Visual comparison results for the Pepper and Baboon images are shown in Figures 9 and 10, respectively, and the information about these images is labeled sideways.

In addition, the mean PSNR, MSSIM, and FSIM results for fixed noise are calculated for noisy images, the MMF, AMMF, VMF, TVL1 in [18], TVL1 in [17], and the proposed SPM method, respectively. The calculations are as follows:In the above three equations, denotes the noisy images with the same density noise in the test and , , and denote the PSNR, MSSIM, and FSIM values of the th image with the density noise, respectively. For example, if the MMF is used, denotes the mean FSIM value of all the images with the 0.1 density noise that is associated with the MMF. The mean values of the three metrics for different methods on different noise densities are plotted in Figure 11.

Experimental Results and Comparisons in RDM. By using the seven noise-free images in Figure 7, another test set was built for the proposed RDM method, denoted by , where each noise-free image was corrupted by different seven random-valued impulses. Using the values of parameters mentioned in RDM in Section 3.2, the proposed RDM method was applied to the test . The PSNR, MSSIM, and FSIM results for the London image are reported in Table 4, and the visual results for the Pepper and Baboon images are shown in Figures 12 and 13, respectively.

For comparisons, other five methods, MMF, AMMF, VMF, AVMF [24], and TVL1 [18], were applied on the test set . Both of the MMF and VMF used the sliding window of size . In both of the AMMF and AVMF, when the noise densities were 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, and 0.6, the filtering window radius values were set to 1, 1, 2, 2, 3, 3, and 4, respectively. The source codes of TVL1 [18] are taken from their authors, and the parameters used in the experiments are those recommended by the authors. The PSNR, MSSIM, and FSIM results for the London image are also reported in Table 4. Visual comparison results for the Pepper and Baboon images are also shown in Figures 12 and 13, respectively, and the information about these images is labeled sideways.

Similar to the comparisons in SPM, the mean PSNR, MSSIM, and FSIM results for fixed noise are calculated for noisy images, MMF, AMMF, VMF, AVMF, TVL1 [18], and the proposed RDM method, respectively, according to (28). The mean values of the three metrics for different methods on different noise densities are plotted in Figure 14.

Observations and Conclusions. The following observations and conclusions can be drawn from the quantitative measurements and visual comparison results. First, the proposed method achieved the best results in every case tested. The proposed SPM method achieved 14.05 dB, 10.68 dB, 14.72 dB, 17.93 dB, and 8.27 dB improvements over the MMF, AMMF, VMF, TVL1 [18], and TVL1 [17] on average, respectively, in the PSNR results, and achieved 0.472, 0.278, 0.519, 0.554, and 0.293 improvements in the MSSIM results and achieved 0.254, 0.164, 0.274, 0.398, and 0.169 improvements in the FSIM results. The proposed RDM method outperformed MMF, AMMF, VMF, AVMF, and TVL1 [18] by 3.56 dB, 4.16 dB, 4.80 dB, 3.73 dB, and 2.45 dB on average, respectively, in the PSNR results, and by 0.200, 0.182, 0.286, 0.231, and 0.135 in the MSSIM results and by 0.064, 0.097, 0.095, 0.102, and 0.071 in the FSIM results. Second, the proposed method has a strong capability to preserve detail. The proposed method reconstructed more image details from noisy images than the comparative methods. Third, the proposed method was more robust to different densities of noise than the comparative methods. In summary, the proposed method shows strong capability of reducing noise not only in SPM but also in RDM, in terms of the PSNR/MSSIM/FSIM results, in addition to producing a higher visual perception quality in restored images, in comparison with results from other methods.

4. Conclusions

In this paper, a decision-based marginal total variation diffusion is proposed for impulsive noise removal. In contrast to vectorial methods, the proposed method only treats distorted components rather than distorted vectors. Furthermore, the proposed method divides components into different categories based on different noise characteristics. Components are divided into the corrupted and noise-free components in SPM and divided into the corrupted, noise-free, and the possibly corrupted components in RDM. Components falling into different categories are processed differently. If a component is corrupted, modified total variation diffusion is applied; if it is possibly corrupted, scaled total variation diffusion is applied; otherwise, the component is left unchanged. To achieve better results, a hierarchical scheme is adopted in RDM; the corrupted components are first processed and then the possibly corrupted ones. A total of 119 noisy images are tested. Experimental results show that the proposed method is robust to different noise strengths and suitable for different images, with strong noise removal capability as shown by PSNR/SSIM/FSIM results as well as the visual quality of restored images.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work is partially supported by Foundation Science and Forefront Technology of Chongqing Science & Technology Commission under Grant no. cstc2016jcyjA0571 and also supported by Ph.D. Cultivation Foundation of Chongqing University of Posts and Telecommunications under Grant no. RC 2016002.