Abstract
To solve the problem that the similarity calculation between neighbors was easily disturbed by noise in the traditional nonlocal mean (NLM) denoising algorithm, a dual-core NLM denoising algorithm based on neighborhood multifeatures and variable-size search window was proposed. The algorithm first proposed to use the eigenvalues of the structure tensor to classify the region where the target pixel points were located and used different sizes of the search window to search for similar neighborhoods for target pixel points in different categories of the region, thus effectively avoiding the problem of oversmoothing or inadequate denoising of the image caused by the use of the global size. Then, the gradient features between image blocks were defined and combined with grayscale features and spatial features to measure the similarity of neighborhood blocks, which solved the problem of noise interfering with the search of similar blocks. Then, an adaptive algorithm with Gaussian–Sinusoidal dual kernel function and quantitative estimation of the optimal values of the filtering parameters was designed to calculate the neighborhood similarity weights to improve the accuracy of image denoising. Finally, the similarity weights were used to weight and average the search neighborhood of the target pixel points to achieve the denoising of the target pixel points. To test the effectiveness of the algorithm, denoising tests were performed using multiple standard grayscale images with different levels of Gaussian white noise added and compared with several advanced denoising algorithms. The experimental results showed that the algorithm was effective. The algorithm improved the image peak signal-to-noise ratio by more than 56.54% on average when Gaussian white noise was removed, and the structural similarity reached more than 0.701 on average. Compared with the traditional NLM algorithm and other improved algorithms, the algorithm proposed in this paper had strong denoising ability, better protection of edges and texture details, and the quality of the image was greatly improved, which had a good application prospect.
1. Introduction
As a kind of information carrier, the digital image was an essential auxiliary technology in daily life, teaching, and research. Due to the influence of objective factors such as equipment and working environment, the digital image inevitably mixed with noise, which was not conducive to the application of the digital image. The noise in most images comes mainly from Gaussian noise and impulse noise. Gaussian noise was caused by sensor noise and electronic circuit noise under low illumination or high-temperature conditions. Gaussian noise was mainly characterized by a relatively high density and a wide range of intensity fluctuations. Impulse noise generated by the signal noise intensity was a collective term for discrete types of noise that appeared in communication, also called pretzel noise. When the real signal of the image itself was mixed with the noise signal, it would cause a degradation of the image quality, which would seriously affect the subsequent processing of the image and the application of the image. Therefore, removing noise from digital images was an important way to improve image quality.
Commonly available image denoising algorithms included spatial domain denoising algorithms, such as mean filtering algorithm [1], median filtering algorithm [2], Gaussian filtering algorithm [3], bilateral filtering algorithm [4], etc.; and transform domain algorithms, such as K–L transform algorithm [5], partial differential equation algorithm [6], wavelet transform algorithm [7], independent component analysis-based algorithm [8], various joint denoising methods and improved algorithms [9], etc. The abovementioned algorithms targeted individual pixels in an image as the research target, which was a denoising method in local mode and suffered from the problem of local feature leakage, such as image structure texture, which was difficult to meet the current digital image processing requirements.
To further improve the image denoising effect, Buades et al. [10] proposed the nonlocal mean denoising algorithm (nonlocal means, NLM); however, the algorithm was highly complex, computationally intensive, and prone to noise interference, so the denoising effect was not very satisfactory. In response to these problems, various improvement algorithms have been proposed; for example, Vignesh et al. [11] used Gaussian and sinusoidal functions instead of the exponential kernel function in the traditional NLM function. Due to the limitations of the algorithm principle, details such as the edge texture of the image were damaged. Yuzheng et al. [12] used multiangle gradient features to solve the problem of noise interference in similar block searches; however, the appropriate parameters need to be tested several times. Haiqing and Hongwei [13] used an improved hybrid robust weight function to calculate the image block similarity weights and combined it with a two-stage denoising framework to achieve the denoising purpose. The predicted image parameters were difficult to find accurately, making the denoising results need to be improved. Han et al. [14] combined neural networks with nonlocal data of images as the input of the network and used the trained network to achieve noise removal. Due to the convergence speed and accuracy of the network, its effectiveness could be further improved. Zhuhua et al. [15] proposed the detail-preserving iterative NLM method, which achieved better results; however, due to the variability of regions, the above method could be improved for removing noise at retail locations such as edges/textures. Shervan et al. [16] proposed an approach for noise reduction in texture images, which was based on real word spelling correction theory in natural language processing. The proposed approach first generated the most similar pixels to noisy desired pixels in terms of textural features using local binary patterns. Then, the best one of the candidates was selected based on two-gram algorithm. Asem [17] proposed an image noise removal method based on collaborative filtering and singular value decomposition. It used the singular value shrinkage to guarantee the elimination of highly noisy pixels and used the random matrix to practically choose the level of the singular value threshold. Khmag [18] used a generative adversarial network model and a semisoft thresholding method to achieve image noise removal [18], but it was susceptible to interference from external factors when carrying out noise removal due to the inability to produce diverse results when considering independent samples alone. Khmag [19] used a patch-based estimation technique to estimate noise level and applied it to the proposed blind image denoising algorithm. Since the principal component analysis was used to estimate the noise level of the selected patch, the method may underestimate the noise level, which can affect the denoising effect. In summary, the NLM algorithm still had certain limitations, particularly in the complex structure of digital image processing; there was a large room for improvement.
This paper improved the traditional NLM algorithm idea to increase the denoising ability of the algorithm and the ability of image texture and detail preservation. The main contributions were as follows:(1)A method for classifying image regions based on the eigenvalues of the image structure tensor was proposed to select different sizes of search windows and adaptive smoothing filtering parameters for denoising images in different categories of regions, thus reducing the discrepancy between the denoising results and the original image.(2)Based on other literature studies, this paper enriched and improved the definition of the gradient feature and combined it with the grayscale feature and spatial feature to measure the similarity of neighborhood fast, which solved the problem that noise interferes with the search of similar blocks.(3)A Gaussian-sinusoidal dual kernel function was proposed to calculate the similarity weights, and an adaptive method was designed to quantitatively estimate the optimal values of the filtering parameters, which improved the accuracy of image denoising.
Simulation results showed that the method in this paper was superior to the traditional NLM method and other improved NLM methods in terms of enhancing image quality and improving denoising effect.
2. Overview of NLM Image Denoising Algorithms
The pixel gray value change was a direct reflection of the image color change and a quantitative display of the image texture detail transformation, which was the most direct way to measure the similarity [20], so the traditional NLM method used the neighborhood grayscale feature to measure the neighborhood similarity [21], whose formula was shown as follows:
Here represent the pixel neighborhood block centered at pixel point , pixel point , respectively, and represents the grayscale similarity value of two-pixel blocks.
The denoised estimate of an image pixel point affected by noise could be expressed as follows:
Here represents the gray value of point in the noisy image, represents the search neighborhood centered on pixel , and represents the similarity weight between the two neighborhoods centered on pixel and pixel , respectively, which was expressed as follows:
In Equation (3), represents the smoothing parameter that determines the degree of filtering of the algorithm.
The traditional NLM algorithm used an exponential function to calculate the weight value of a similar neighborhood. The exponential function had higher weights in the region of high similarity, but its weights decay too fast with the increase of Euclidean distance, which reduced the accuracy of weight assignment and led to the blurring of denoised images. To solve this problem, literature [11] used the Gaussian function, sine function, and other kernel functions to replace the exponential function in the traditional NLM algorithm. The kernel function formula was as follows:
Here was the smoothing parameter; represents the grayscale Euclidean distance between pixel neighborhoods as shown in Equation (1). represents the similarity weight between the two neighborhoods centered on pixel point and pixel point computed using the Gaussian kernel function, and represents the similarity weight between the two neighborhoods computed using the sinusoidal kernel function.
The Gaussian kernel decreased rapidly with increasing distance, and when the Euclidean distance of the neighborhood was larger, the Euclidean distance had less influence on the similarity weight, and the similarity was close to zero, whereas the sinusoidal kernel had a slow decreasing trend within a certain Euclidean distance, and when the Euclidean distance was larger than a certain threshold its similarity weight was zero, and the neighborhoods with lower similarity were filtered out.
The improved NLM algorithm using several different kernel functions proposed in literature [11] improved the problems of the traditional NLM algorithm. However, the experimental results showed that the algorithm obtained a better filtering effect than the traditional NLM algorithm when the noise intensity was weak. But when the image noise was strong, the improved NLM algorithm with Gaussian kernel function would reduce the denoising performance due to insufficient weighting, while the improved NLM algorithm with sinusoidal kernel function would easily cause blurring of the image due to overweighting. Therefore, the above algorithms need to be further improved.
3. Improved Nonlocal Mean Denoising Algorithm
3.1. Selection of Variable Size Windows
It was found that using a small search window in the smoothed region would result in a larger difference between the pixel variance of the denoised result and the original image. Based on the above understanding, different search window sizes should be used for different regions to reduce the difference between the filtered results and the original image.
The structure tensor of an image can effectively reflect the current distribution of details, and for each pixel point of the region, the structure tensor matrix was defined as follows:
In Equation (6), represent the gradient in the X and Y directions, respectively, and represents the Gaussian kernel in terms of standard deviation, the eigenvalues of the structure tensor can be expressed as follows:
Pixels in smooth areas had small differences in eigenvalues; pixels in edge/texture areas had larger differences in eigenvalues. Thus, image region classification can be achieved by detecting differences in the eigenvalues of each pixel. Define a region feature value and let , then the whole image region can be divided into n categories (e.g., detail region, transition region, smooth region, etc.), and the classification judgment expression was as follows:
Here represent the maximum and minimum eigenvalues of the entire image structure tensor. represent the range of regions for different categories, with smaller indicating smoother regions and vice versa, and represents the number of categories. Based on the region classification results, different sizes of search windows were selected in different regions. Theoretically, the number of classified regions should be as large as possible so that the image partition can be divided more carefully. In practice, when n was larger than 4, the variation of the denoising effect tended to be unchanged. Therefore, in this paper, the number of partitions n = 3 was chosen in the experiment.
3.2. Neighborhood Characteristics
Literature [12] defined gradient features for 0° direction, 45° direction, 90° direction, and 135° direction, but the gradient features for other directions were not included in the similarity measure; therefore, this paper added gradient features for 22.5°, 67.5°, 112.5°, and 157.5° to improve the accuracy of the similarity measure, and its specific directions are shown in Figure 1.
Based on the way 0°, 45°, 90°, and 135° were defined, the gradient equations for the other four directions (the dashed arrow directions in Figure 1) were defined as follows:
Here represents the coordinates of any pixel point in the noise-containing image. was the size of the neighborhood, and represents the gray value of the pixel point with as the coordinates.
Based on the above definition, then the new gradient feature formula for pixel point can be defined as follows:
Gradient features reflect changes in the brightness of neighborhood pixel gray values, and spatial features were used to measure changes in detail. Therefore, the neighborhood multifeatures of a pixel point can be characterized by combining the pixel grayscale feature, the gradient feature, and the spatial feature as follows:
Here represents spatial similarity features and represents grayscale similarity features, the formula for multifeature similarity measure between the neighborhoods of two-pixel points and can be expressed as follows:
3.3. Dual Kernel Function and Smoothing Parameter Adaptive Computation
In high-intensity noise, the Gaussian kernel function had poor denoising performance but maintained good detail, while the sinusoidal kernel function had good denoising ability but tended to overweight the results, resulting in blurred denoising. To combine the advantages of both Gaussian kernel functions and sinusoidal kernel functions, the improved kernel function can be expressed as follows:
Here , represent different values of the smoothing parameter and represents the multifeature similarity between neighborhoods.
The dual kernel function can effectively avoid the interference of less similar neighborhoods and achieve a balance between the Euclidean distance and similarity weights, thus effectively enhancing the denoising effect and improving image detail protection. In image smoothing regions, where pixel redundancy was greater, a larger value could improve the degree of smoothing; in edge/texture regions, where pixel redundancy was less, a smaller value was good for maintaining edges, so the smoothing parameters could be selected adaptively according to the image region classification results. Based on the approximate relationship between the filtering parameter and the noise variance under the Gaussian distribution characteristics, the optimal filtering parameters for the three different types of regions for images with a gray level of 255 can be defined as follows:
Here represents the noise variance and represents the standard deviation of the noisy image, represents the edge/texture region, represents the transition region, and represents the smoothed region.
3.4. Image Denoising
Determining the neighborhood search window size and the smoothing filtering parameters, the improved NLM formulation in this paper was as follows:
Here represents the filtered gray value of pixel point in the noise-containing image, represents the gray value at point of the noise-containing image, denoted the coordinate domain of the pixel point in the search neighborhood centered at , and represent the dual kernel function.
In summary, the block diagram of the algorithmic scheme proposed in this paper is shown in Figure 2.
The steps can be summarized as follows: Step 1: Inputting noise-containing images; Step 2: Use Equation (9) to implement region division and determine the search window size for different regions; use Equation (18) to calculate different regional filtering parameters; Step 3: Use Equation (15) to calculate the multifeatures between neighbors; Step 4: Calculation of similarity neighborhood weights using Equations (16) and (17); Step 5: Normalizing the processing weight values and traversing the neighborhood pixels for steps (3) and (4); Step 6: Noise removal using Equation (19) for the target pixels; Step 7: Applying steps (3)–(6) to the entire image, down to the last pixel; Step 8: Output the resulting image.
The flowchart of the algorithm is shown in Figure 3
For a noisy image of size , the computational complexity of the algorithm for denoising using this algorithm was which assumes that the size of the neighborhood used was , and the size of the search window was
4. Algorithm Testing and Application
4.1. Experimental Setting
The equipment configuration for this experiment was Intel(R) Core(TM) I5-7500 with 8.00 GB of RAM, Windows 10, and the processing software was MATLAB R2022b. Eight standard grayscale images (Jetplane 512 × 512, Boat 512 × 512, Lena 512 × 512, Parrot 512 × 512, Barbara 512 × 512, C.man 256 × 256, Lax 256 × 256, Peppers 256 × 256) shown in Figure 4 were used as the test image set, which were added to additive Gaussian white noise with mean 0 and different noise standard deviations . They were tested by the traditional NLM method, the Gaussian kernel function NLM method, literature [12–15], and the method of this paper. For the convenience of analysis and evaluation, the conventional NLM algorithm and Gaussian kernel function NLM algorithm, literature [12–14] used a neighborhood window size of 9 × 9 and a search window size of 15 × 15 in successive simulation experiments and literature [15] used a neighborhood window size of 9 × 9 and a search window size of 9 × 9, with several iterations of 3 and a step size of 6. The algorithm in this paper had a neighborhood window size of 9 × 9 and a search window size of 9 × 9 (edge/texture region), 15 × 15 (transition region), and 21 × 21 (smoothing region) for the three sizes, respectively.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
4.2. Determination of the Parameters
As shown in Equation (17), the filtering parameters affected the denoising results; therefore, the optimal parameters needed to be determined. In this paper, the filtering parameter with the maximum peak signal-to-noise ratio (PSNR) value was selected as the optimal parameter. The was calculated adaptively using Equation (18); thus only the optimal value of needed to be determined. When testing , the value of was fixed at 0.02, and the value of was varied from 0 to 0.1, and then the optimal value of the filtering parameter was determined according to the PSNR variation curve of the test image. Four grayscale images (jetplane, C.man, boat, lax) shown in Figure 4 were used for simulation tests. The images were added with Gaussian white noise with mean and standard deviation , then the relationship between and PSNR changes as shown in Figure 5.
From Figure 5, it can be seen that the PSNR transformation trend of the test image was increasing and then decreasing, and the best denoising effect was achieved when the PSNR value was maximum at . Therefore, was tested with the above value in the subsequent simulation.
4.3. Experimental Results and Their Comparative Analysis
To analyze the effect of noise intensity on different denoising methods, Gaussian white noise with mean and standard deviation was added to the six standard grayscale images shown in Figure 4, and then the denoising process was tested by using seven methods. The values of PSNR and structural similarity (SSIM) for 126 simulation results are shown in Tables 1 and 2, respectively.
From Tables 1 and 2, it can be seen that at low noise , literature [15] obtained a better denoising effect with maximum PSNR and SSIM values. The PSNR and SSIM of the improved algorithm in this paper were closest to the maximum value, although not the maximum value. However, as the noise level increased , the method in this paper then obtained a better denoising effect, and its PSNR and SSIM were better than the comparative literature methods. In this case, at , the PSNR value of the traditional NLM method was increased by an average of 38.571%, and the SSIM value reached an average of 0.654. The PSNR value of the Gaussian kernel function NLM results was increased by an average of 41.07%, and the SSIM value reached an average of 0.669. The PSNR value of literature [12] method results were increased by an average of 44.38%, and the SSIM value reached an average of 0.682. The PSNR values of the results of the method of literature [13] increased on average by 47.19%, and the SSIM values reached an average of 0.694. The PSNR values of the results of the method of literature [14] increased on average by 49.97%, and the SSIM values reached an average of 0.712. The PSNR values of the results of the method of literature [15] increased on average by 53.78%, and the SSIM values reached an average of 0.745. The PSNR values of the results of the method in this paper were improved by 56.53% on average, and the SSIM values reached 0.778 on average, as shown in Figures 6 and 7.
From the above experimental data, it can be seen that with the increase of noise intensity, the results of this paper’s method have been maintaining higher values of PSNR and SSIM, indicating that this paper’s algorithm had a better denoising ability and can maintain better image quality while removing noise.
To have a visual impression of the denoising effect of each algorithm on the noisy images, the denoising results of four noise-containing images with noise mean and standard deviation were shown. Noise-containing images and denoising results are shown in Figures 8–11.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
From Figures 8 to 11, it can be seen that although the traditional NLM algorithm removed the noise contained in the image, the excessive smoothing led to the loss of some detail information in the image, resulting in blurring of the details and the regions with rich edge information, as shown in Figures 8(b), 9(b), 10(b), and 11(b). The denoising effect of the Gaussian kernel function NLM algorithm was relatively good, but the denoising results showed some “bar” textures, as shown in Figures 8(c), 9(c), 10(c), and 11(c), which, combined with the original noiseless image, indicated that the above textures were extraneous noise. Literature [12] method improved the quality of the results, but the layout of the “ringing” phenomenon, as shown in Figures 8(d), 9(d), 10(d), and 11(d), affected the quality of the image. The local details of the results of literature [13] method were relatively blurred, as shown in Figures 8(e), 9(e), 10(e), and 11(e). Literature [14] method resulted in poor denoising of regions with lower brightness and blurring of local details, as shown in Figures 8(f), 9(f), 10(f), and 11(f). Literature [15] and the results of the method in this paper had the best noise removal, the closest to the original noise-free image, and the details were kept the clearest, especially the letters in the jetplane image as well as the fuselage, hull and other positions in the boat image were more easily recognized.
To further analyze the denoising effect of the methods in this paper, the noise residuals of the seven methods were displayed, as shown in Figures 12–15.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
From Figures 12 to 15, it can be seen that the traditional NLM noise residuals contained more edge texture information, especially in Figures 12(a), 13(a), and 14(a), the contours of the aircraft, the camera frame, the hull of the ship can be seen, which showed that the algorithm lost a large amount of detail information while removing the noise. The Gaussian kernel function NLM algorithm removed the noise residuals with a similar “bar” texture, as shown by the wireframes in Figures 12(b), 13(b), 14(b), and 15(b). Detailed information such as edges/textures were also present to varying degrees in the noise residuals of literature [12–14] and in the noise residuals of literature [15]. In the noise residuals of this paper’s method, the detail information, such as edge texture, was the least, and there were almost few signals that were continuous or had a certain morphology, indicating better protection of detail information, as shown in Figures 12(g), 13(g), 14(g), and 15(g).
To better evaluate the quality of the denoised image, the denoising results can be evaluated using a completely nonreferential algorithm (natural image quality evaluator, NIQE), and the expert evaluation values (in percentage) were added together to obtain the evaluation averages of the seven methods as shown in Table 3. As can be seen from Table 3, the denoised image obtained using the denoising algorithm proposed in this paper was of the highest quality and was able to retain the details and texture features of the image better.
Next, the time spent by different processing methods to process different images was counted, and the average time was derived using 30 experiments, respectively, as shown in Figure 16. From Figure 16, it can be seen that compared to other denoising methods, the running time of the method proposed in this paper was not the shortest but still better than the methods proposed in literature [13–15].
5. Summary
In this paper, a dual-kernel adaptive NLM algorithm based on the variable-size window and neighborhood multifeature was used for digital image noise removal, which could effectively reduce the interference of neighborhood noise on similarity computation, thus improving the accuracy of neighborhood pixel similarity computation. The algorithm applied a variable-size search window to determine the search window size and smoothing parameters of the current neighborhood and adapted the Gaussian–Sinusoidal dual-kernel function for similarity weight calculation, which effectively improved the denoising effect and the protection of edge texture based on effectively improving the image quality and signal-to-noise ratio. After simulation and verification, the method in this paper had a better denoising effect and edge/texture protection ability, which had a certain value of popularization and application in digital image processing. However, the experimental images used in this study were the standard test images in the field of image processing. How to extend this research to hyperspectral images and achieve real-time processing results in a short period will be a pressing issue to be addressed in the next research. Meanwhile, how to combine this algorithm with the latest research techniques, such as sparse representation and neural networks, to obtain more stable and better denoising results is also an area that needs to be further worked on in future work.
Data Availability
All the data used to support the findings of the study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors thank the professors and students at the University of Kitakyushu for their generous help with the full paper, from the investigation to the checking process. This study was supported by the performance-based salary distribution system in local colleges and universities, project no.: 21JK0007; Teaching Reform of Software Engineering Course under the Background of New Engineering and Engineering Certification, project no.: JG202225; Ankang City Science and Technology Plan Project in 2023 (project name: Research on Mixed Noise and Its Removal Algorithm in Digital Images).