Abstract
Effective noise removal has become a hot topic in image denoising research while preserving important details of an image. An adaptive threshold image denoising algorithm based on fitting diffusion is proposed. Firstly, the diffusion coefficient in the diffusion equation is improved, and the fitting diffusion coefficient is established to overcome the defects of texture detail loss and edge degradation caused by excessive diffusion intensity. Then, the threshold function is adaptively designed and improved so that it can automatically control the threshold of the function according to the maximum gray value of the image and the number of iterations, so as to further preserve the important details of the image such as edge and texture. A neural network is used to realize image denoising because of its good learning ability of image statistical characteristics, mainly by the diffusion equation and deep learning (CNN) algorithm as the foundation, focus on the effects of activation function of network optimization, using multiple feature extraction technology in-depth networks to study the characteristics of the input image richer, and how to better use the adaptive algorithm on the depth of diffusion equation and optimization backpropagation learning. The training speed of the model is accelerated and the convergence of the algorithm is improved. Combined with batch standardization and residual learning technology, the image denoising network model based on deep residual learning of the convolutional network is designed with better denoising performance. Finally, the algorithm is compared with other excellent denoising algorithms. From the comparison results, it can be seen that the improved denoising algorithm in this paper can also improve the detail restoration of denoised images without losing the sharpness. Moreover, it has better PSNR than other excellent denoising algorithms at different noise standard deviations. The PSNR of the new algorithm is greatly improved compared with the classical algorithm, which can effectively suppress the noise and protect the image edge and detail information.
1. Introduction
The noise image is mainly caused by the imperfect system and equipment. In the process of transmission, the image polluted by noise will affect people’s visual sense to varying degrees, sometimes even leading to the loss of many image features, making the image blurred, affecting the useful information of the image, and thus hindering people’s normal recognition. The image polluted by noise will have a great adverse effect on the subsequent image processing, mainly including image segmentation, extraction, detection, and recognition. Therefore, it is very necessary and important to use a good denoising algorithm for image denoising.
Image denoising can improve the accuracy of human visual recognition information and is a necessary condition for us to correctly identify images. In nature, due to various internal or external reasons, it is almost impossible to find natural pictures that are completely free of noise pollution, and the images are more or less polluted. This will make it harder for us to identify the images. In particular, some noise images, such as medical images and security images, which affect our significant judgment, need us to carry out image noise reduction to obtain clear image features. For those heavily polluted images, they lose the original content of the image, so effectively improving the accuracy of image recognition is a necessary condition for image recognition. Image denoising can improve image quality and is a prerequisite for further image processing. In image processing, image denoising is often the first and most important step. Obtaining high-quality and high-definition images through noise reduction is a strong guarantee and good foundation for subsequent image processing.
Traditional image denoising methods have been proposed for a long time and have been used for a long time now, but most of these algorithms are not very satisfactory, especially in the process of denoising, the details of the image will be lost, and most of the denoising performance and algorithm complexity are to be improved. Although the deep learning technology for image denoising has many scholars’ research, neural network research, due to the barriers of hardware, has not been developed, making the neural network technology no longer have complex network model of computation for too much worry, because high-performance GPU multicore parallel computing is well suited for the neural network model and is a necessary prerequisite for the rise of deep learning. Although a lot of people are beginning to study abroad, after all, the direction is the forward direction, and theory and technology are not very mature. Although the use of the image denoising aspect really has achieved good results, there are still many problems, so continuing research and perfect image denoising theory and improving the effect of denoising are very necessary. Based on diffusion equation and deep learning (CNN) algorithm, this paper adopts multifeature extraction technology to study the richer features of the input image in the deep network and designs an image denoising network model based on deep residual learning of convolutional network, which has better denoising performance. From the comparison results, it can be seen that the improved denoising algorithm in this paper can also improve the detail restoration of denoised images without losing sharpness. Under different noise standard deviations, the PSNR of the proposed algorithm is superior to other excellent denoising algorithms.
The first part is the introduction, the second part is related work, the third part is the Adaptive Diffusion equation and deep learning algorithm for image dryness, the fourth part is example verification, and the fifth part is the conclusion.
2. Related Work
Gaussian filtering first introduces the diffusion equation into image processing [1]. On the basis of some theoretical and numerical operations, differential operators can be obtained by transforming local filter operators. Differential operators can be transformed by local filter operators. In the case of two-dimensional images, the general processing method is to treat the pair diffusion as a uniform linear diffusion process; that is to say, the diffusion coefficients at all points of the image are the same. However, there are still loopholes in this diffusion equation [2], whose uniform diffusion makes it impossible to retain important details such as edges while removing noise. In view of the defects of the uniform diffusion characteristics of Gaussian filtering, it is natural to come up with an ideal method: reduce the diffusion at the edge of the image according to the prior information of the image, so as to remove the noise from the image and protect the information at the edge [3]. Therefore, we use the gradient operator as the operator of the edge detection to construct a monotone decreasing function [4] in which the diffusion coefficient changes with the gradient of the original image; that is, the magnitude of the gradient is inversely proportional to the diffusion coefficient. This adaptive denoising method can not only remove the noise in the flat area of the image but also maintain the protection of the edge of the image, which is called the nonuniform diffusion equation. However, this method still has loopholes and shortcomings. First, the gradient at the edge is large, so the diffusion coefficient is small [5]. The smaller the diffusion coefficient is, the better the edge is maintained, but the effect of noise removal cannot be achieved. Then, the gradient will produce deviation for image edge detection affected by noise.
A nonlinear diffusion equation [6] is proposed, which takes the gradient operator obtained in the previous step as the edge detection operator in the iterative process to reduce the impact of noise on edge judgment. However, this method cannot deal with the noise of the image edge. The diffusion coefficient is improved and designed as a matrix, which makes the diffusion coefficient larger in the tangential direction of the edge and smaller in the vertical direction of the edge, thus overcoming the shortcomings of the PM model [7]. A common problem with low-order models is the “ladder effect.” In view of the characteristics of the low-order model, a fourth-order diffusion equation is introduced [8]. In this model, Laplace is used to measure marginal areas, so there is no “ladder effect” in vision. However, the edge protection ability of the model is not good, and it is easy to produce speckle noise. In [3], an improved fourth-order diffusion equation model is proposed, which uses the modulus of a gradient to replace the absolute Laplace value as the operator of image edge detection. This model has a faster convergence speed and better denoising effect but blurred image edges. An anisotropic diffusion fourth-order denoising algorithm is proposed [9], which diffuses in different degrees in the normal and tangential directions and preserves the details of the image well, but this model has gradient artifacts. At present, the research on image denoising algorithms based on diffusion equations is also making continuous progress, mainly focusing on second-order diffusion equations, fourth-order diffusion equations, and higher-order diffusion equations [10]. The high-order weighted gradient variation model for image denoising, after the convolution of a ladder as a weighted function of the second derivative, newly established a high-order variational function and get a fourth-order partial differential diffusion model [11]; the model is effective in the noise, eliminates the staircase effect, and can have good protection effect on the edge. An improved semiadaptive threshold anisotropic diffusion filter denoising algorithm is proposed [12, 13]. In this model, the local difference method is adopted to distinguish damaged pixels from noiseless ones, and some damaged pixels are replaced by predenoised pixels of the Gaussian filter. Then, an anisotropic diffusion model with a semiadaptive threshold in the diffusion coefficient function is used to obtain the restored image. In order to achieve the semiadaptive threshold value of each diffusion, the gradient value of the destroyed pixel is introduced into the threshold value [14], which makes the diffusion of the smooth region larger and the diffusion of the boundary region smaller. This method can improve PSNR by 30% and structure similarity (SSIM) by 5%. It has a good effect on edge protection and noise removal. It overcomes the defect of blurring image edges by the fourth-order isotropic diffusion equation. An adaptive image denoising algorithm based on the fourth-order diffusion equation [15] is proposed, which uses the image gradient model to construct the measure function of the image feature information, and uses the feature detection function to adjust the normal and tangential image diffusion coefficient indexes adaptively according to the different features of the image. Isotropic diffusion is used to remove noise in flat and inclined regions, and anisotropic diffusion is used to protect the features of image edge points. Experimental results show that both denoising and edge preservation are taken into account [16]. A high-order nonlinear diffusion image smoothing method based on curvature mode is proposed [17]. Methods such as the combination of gradient and curvature, total variational coupling, and quasi-normal distribution diffusion are applied to image denoising.
Diffusion equation and deep learning use Local Receptive Field structure designed for image data [18], compared with Plain Multilayer Perceptrons (MLPs); this structure enables diffusion equation and deep learning to obtain good results while greatly reducing the parameters. This method is very useful when the amount of training data is relatively small because the total number of parameters is relatively small, so it is more difficult to overfit the training data. On the other hand, multilayer perceptron networks have more potential than diffusion equations and deep learning: multilayer perceptron networks can be used to approximate arbitrary functions [19], while diffusion equations and deep learning can only learn specific types of functions due to structural constraints. Another class of algorithm denoising autoencoder also uses the neural network as a denoising tool [20]. Denoising autoencoders are special neural networks that learn by unsupervised learning. Features are represented by the learned units of the hidden layer. Since the input-output of this neural network can be easily generated, it only needs to add noise and other pollution processing to the input, and good data features can be learned, so denoising autoencoder has become a very important algorithm in deep learning. However, the purpose of designing this kind of neural network is mainly to extract data features through unsupervised learning and to train the deep neural network layer by layer, rather than for denoising. Another difference is the fact that the noise added to the network during learning is usually not Gaussian white noise but pepper and salt noise or the Dropout of some of the input nodes. For stacked denoising autoencoder [21, 22], which is commonly used in deep learning, the noise will be added to the output node of the previously learned layer, which is different from the scheme that only adds noise to the initial input and then learns all parameters at the same time in denoising applications. In fact, the most effective neural network in the field of denoising is the most basic multilayer perceptron model. As a popular solution to denoising problems, methods based on image priors can extend various methods based on image priors [23], such as BM3D [24]. These methods are used to obtain image prior knowledge directly from the input image for image denoising. Although some denoising effects have been achieved, there are still some limitations. Firstly, these methods make use of the prior noise image, which will cause some error, so it is difficult to obtain all the features of the image, resulting in the limitation of denoising performance. Second, most methods use only the internal information of the input image and do not use any external information, so there is still room for improvement. In addition, for denoising methods based on discriminant learning [25, 26], especially those based on diffusion equation and deep learning [8, 27, 28], denoising networks with paired training data sets are trained and image noise distribution is learned in the hidden layer, thus improving denoising performance. In order to remove the known Gaussian noise level, pairs of training samples are used to train the denoising network to achieve advanced denoising performance.
3. Adaptive Diffusion Equation and Deep Learning Algorithm for Image Dryness
3.1. Image Denoising Based on Diffusion Equation
The process of recovering a clear image from an image containing noise is called image denoising so that the image information after denoising is as close as possible to the original clear image information, and the information error between them is as small as possible. Therefore, image denoising is an inverse process of image processing, as well as image deblurring and superresolution reconstruction. The noise model of the image is shown in Figure 1.
The low rank of hyperspectral images can be explained from the perspective of the linear spectral mixture model. Because of the high correlation between spectral features, each spectral feature can be represented by a linear combination of a small number of pure spectral endmembers, that is, a linear spectral mixture model. If each band of hyperspectral image data is expanded into a vector, then all bands together form a matrix X.
Diffusion equations are related to many unknown multivariate functions and their partial derivatives. For fixed positive integer k, we use the symbol Dku to represent all k partial derivatives of u.where (x1,x2,…, xn) is any permutations of k elements in the set {1,2,3, n}. Therefore, we can regard Dku as a vector in n-dimensional Euclidean space and write down its length as follows:
In particular, when k = 1, we call the n-dimensional vector:
That is the trace of u’s Hessian matrix, the sum of the diagonal elements of u’s Hessian matrix. Write the divergence of F as
Let the general form of the diffusion model be
The AOS format of the deformable diffusion model is
The block-based denoising algorithm divides the image into small blocks for denoising, and because the noise level of each image block is different, the selection of the denoising threshold is also different. In the process of denoising, noise mainly corresponds to a small singular value. Therefore, the larger the singular value is, the less it should shrink as the singular value shrinks.
3.2. Diffusion Equation Denoising Algorithm
Based on the PM model, the fitting diffusion coefficient was established to overcome the defects of texture detail information loss and edge degradation caused by excessive diffusion intensity. Then, in order to enable the threshold function to control the threshold value according to the maximum gray value and iteration times of the image, the threshold function is designed adaptively. Based on the advantages and disadvantages of the above two diffusion coefficients, a new diffusion coefficient is established, as follows:
The diffusion coefficients of each fit are shown in Figure 2.
In the process of diffusion, pixels’ gray value of the area and adjacent area has a kind of relationship, usually paying a predetermined constant gradient threshold k and adopting the predetermined good constant gradient threshold, which is not conducive to certain areas on the edge of the image detail information and images, according to the pixel location of the dynamic design gradient threshold k. Through the above analysis, a one-dimensional gradient threshold function k is designed which varies with the diffusion time and the number of diffusions.
In the tangent direction, the image needs strong diffusion in both the edge and the inner region, where noise can be eliminated and the interrupted edge can be connected on the edge. In the normal direction, the inner area of the image also needs strong diffusion to eliminate noise, but in the edge, no diffusion is needed as much as possible to maintain the edge features. Therefore, the diffusion coefficient B is always set as 1 in the tangential direction, and a new diffusion coefficient c is designed by excluding one ill-conditioned condition in the normal direction combined with the diffusion coefficient of Tv flow:
The neural network and deep learning model are very suitable for learning the function from picture block to picture block due to their strong expression ability. The frame of denoising using a neural network is shown in Figure 3. The hidden layer activation function is distinguished from the output layer activation function because the choice of the output layer activation function depends on the needs of the problem.
Specifically, we first randomly select a clean image block knife from the image data set and then artificially add Gaussian white noise to generate the corresponding noise image block X. Then, we take the vector-induced noise image block X as the input of the neural network and the corresponding vector-induced clean image block as the output of the neural network, update the parameters of the neural network through backward propagation, and gradually learn the required model through iteration.
3.3. Adaptive Image Denoising Algorithm Based on Diffusion Equation and Deep Learning
The initial value of the weight matrix of the neural network has a significant influence on the training process and the final result. For multilayer networks, we want the initial values of the network to satisfy randomness while ensuring that the input outputs of each hidden layer have the same statistical characteristics as possible. In order to achieve this, it is necessary to associate the initial value of parameters of each layer with the number of nodes of the corresponding layer and adjust it adaptively according to the number of nodes of each layer.
All training samples were checked in a cycle, and the parameters of the whole network were updated by backward conduction every time one training sample was checked. The advantages are that the updating speed of the weight matrix is greatly accelerated, and the network parameters are easier to escape from the subideal local minimum region because of the strong randomness. The disadvantage is that the error curve fluctuates more in the training process, the convergence conditions are more complex, and it is more difficult to converge. Relatively speaking, these shortcomings of stochastic gradient descent are easier to solve. For example, it is a good solution to solve the problem that the error is difficult to converge by gradually decreasing the learning rate with the increase of the number of iterations. Due to the advantages of training speed, a stochastic gradient descent algorithm has been widely used.
The setting of learning rate: a larger learning rate may make the neural network model learn faster and may also make the neural network diverge. In this paper, the initial learning rate was selected by the experimental method: 0.1 was initially tried, 0.05 was selected if the neural network diverged, and then 0.02 was tried to gradually reduce the initial learning rate. If the network converges but the training speed is slow, then we try to increase the learning rate. In order to ensure the convergence of the neural network, the current learning rate is multiplied by 0.99 in each cycle when the small-batch processing algorithm is implemented. Inertia parameter is a common acceleration method in neural network training, which can help the neural network to leave the flat region in function space faster. The specific operation method is to add part of the last weight update to each update of the weight matrix, namely,
The relation between the activation value of the element and input current is as follows:
As shown by the similarity measure,
The EPLL algorithm first learns the prior knowledge of the image and obtains the edge texture characteristics of the image through sample learning, which is constrained by the regular term of image denoising to obtain the denoising image with obvious texture characteristics. On the other hand, the image blocks of the default denoised images in the EPLL algorithm have significant similarities with the image blocks in the samples. When the EPLL value is large, the denoised image is considered to conform to the characteristics of the sample library, and the subjective vision is also better. The sum calculation method of likelihood probability is as follows:
The logarithmic likelihood probability of all image blocks is summed to obtain the total likelihood probability of the denoised image and sample database image. The image degradation process is known, and the properties of likelihood probability are understood. The image denoising problem is transformed into a solution formula:
The solution of the target equation is approximated by introducing the auxiliary variable splitting equation into semiquadratic partition:
The adaptive image drying model has higher denoising intensity, which solves the “blocky” effect in the denoising process. In order to further protect the edge texture and other details and give consideration to denoising intensity, an adaptive orthogonal diffusion filtering model was proposed to overcome the disadvantages of high denoising intensity of the orthogonal diffusion model and the loss of edge texture and other details. The experimental results show that the PSNR model improves about 30 dB compared with the classical model (TV flow diffusion model). Compared with the TV stream diffusion model, the new model is more flexible than the TV stream diffusion model, which can adjust the diffusion system according to different parts of the image processing, control the smoothness degree, and process clear images more reasonably.
4. Example Verification
In order to verify the rationality and effectiveness of the above algorithms, Lady (600 × 600) and Nudist graphs with Gaussian random noise were analyzed, and experimental simulation was carried out with Matlab software and semi-implicit additive operator splitting numerical algorithm, and their MSE and PSNR were compared. In order to study the suppression effect of diffusion coefficient on noise and the retention degree of important details such as image edge texture, noises of different sizes and variances were first added to the image, and the results are shown in Figure 4.
Based on the PM model, the diffusion coefficient in the diffusion equation is improved, and the fitting diffusion coefficient is established to overcome the defects of texture detail loss and edge degradation caused by excessive strength. Then, according to the maximum gray value of the image and the number of iterations, the threshold function is designed automatically. Experimental results show that the peak signal-to-noise ratio of the new algorithm is improved by about 16 dB compared with the classical algorithm. The proposed algorithm can effectively suppress the noise and protect the edge and detail information of the image.
Notice that the side length P of the noisy image block can be inconsistent with the side length Q of the denoised image block. Such setting is similar to the convolution operation of the image. For the pixel in the middle of the image block, the information of surrounding pixels can be used more than that of the pixel from the corner of the image block. Therefore, the prediction of the original value of the pixel in the middle part of the image block will be more accurate, while the error of the edge part will be relatively larger. The actual situation in the image block is an example of mean square error (MSE) in each position as shown in Figure 5; in the case where the original noise block size is 12 × 12 images, we use the fourth chapter putting forward the algorithm of denoising model of training, more after denoising images after denoising and ideal cleaning image blocks of error, and perform statistics after getting the picture.
As can be seen from Figure 5, the mean square error in the middle of the image block is the lowest, and the mean square error becomes higher toward the edge, among which the mean square error at the corner is the highest. Numerically, the mean square error (MSE) in the corner of the image block is approximately 25% greater than that in the middle. On the other hand, in many algorithms, the edge length of the noisy image block is the same as that of the denoised image block. Considering that there will be some overlap in the process of splitting and aggregation, the same size of denoised image block and noise image block means that the same number of noise image blocks provides more estimates of the clean value of each pixel.
Figure 6 records the curves of the mean square error of denoising image blocks in the training of the two models during the first 150 cycles with the number of iterations. It should be noted that from the 150th cycle to the 1000th cycle, the shape of each curve is basically the same as that from the 100th cycle to the 150th cycle.
It can be seen from Figure 6 that the model using linear rectification function can fit the training data set better than the model using hyperbolic tangent function. Firstly, in the whole training process, the model using linear rectifying function has a lower mean square error in both training set and test set than the model using hyperbolic tangent function. Secondly, it only takes a few iterations for the former to stabilize the mean square error on the test set, and then the error on the training set continues to decline, while the error on the test set slowly rises; that is, slight overfitting appears. However, the latter fluctuates more and the range is larger in the training process, and there is no overfitting trend in the whole process. As the learning rate decreases (multiplied by 0.99 per cycle), it gradually converges.
Based on the diffusion equation and deep learning, the Adam algorithm is used to replace the traditional gradient descent algorithm in the backpropagation algorithm like the DnCNN algorithm, multifeature extraction technology is adopted for feature extraction of the first layer of the neural network model, and the improved linear rectifier function is used as the activation function. Deep learning technology promotes the training speed and convergence speed of the whole model and has great advantages in convergence and speed.
As shown in Figure 7, we can find from the curve that the loss value of the blue line with low noise standard after convergence is much lower than that of the blue line with high noise standard after convergence, which conforms to the objective law, because the bigger the noise is, the greater the interference between pixels is, and the more difficult the model optimization is. In addition, when the noise is 35, we found that after 316 times of iteration the curve almost tends to be convergent, while a curve in high noise for 75, almost 1000 times or so, tends to be convergent, and the iteration is only a small drop, tens of thousands of times compared with diffusion equation and deep learning model with almost 4000 times. The loss function curve converges and becomes stable. The improved diffusion equation and deep learning model based on adaptive dryness in this paper have advantages in convergence speed and model training speed.
5. Conclusion
In this paper, the diffusion coefficient in the diffusion equation is improved and the fitting diffusion coefficient is established to overcome the defects of texture detail loss and edge degradation caused by excessive diffusion intensity. Then, the threshold function is adaptively designed and improved so that it can automatically control the threshold of the function according to the maximum gray value of the image and the number of iterations, so as to further preserve the important details of the image such as edge and texture. Finally, the simulation results show that the peak signal-to-noise ratio of the new algorithm is greatly improved compared with the classical algorithm, which can effectively suppress the noise while protecting the image edge and detail information. On the basis of image denoising based on neural network denoising, a linear correction function is proposed as the activation function of the neural network hidden layer, and the parameter setting of the model is discussed in detail. Experimental results show that the denoising performance of this algorithm reaches the best level at present, especially suitable for image denoising in a high noise environment.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
There are no conflicts of interest in this article.
Acknowledgments
This study was supported by the Industrial Support and Guidance Project of Colleges and Universities in Gansu Province: Research and implementation of digital comprehensive display system for Dadiwan prehistoric civilization based on virtual reality (Grant no. 2019C-09).