Abstract
The image enhancement for the natural images is the vast field where the quality of the images degrades based on the capturing and processing methods employed by the capturing devices. Based on noise type and estimation of noise, filter need to be adopted for enhancing the quality of the image. In the same manner, the medical field also needs some filtering mechanism to reduce the noise and detection of the disease based on the clarity of the image captured; in accordance with it, the preprocessing steps play a vital role to reduce the burden on the radiologist to make the decision on presence of disease. Based on the estimated noise and its type, the filters are selected to delete the unwanted signals from the image. Hence, identifying noise types and denoising play an important role in image analysis. The proposed framework addresses the noise estimation and filtering process to obtain the enhanced images. This paper estimates and detects the noise types, namely Gaussian, motion artifacts, Poisson, salt-andpepper, and speckle noises. Noise is estimated by using discrete wavelet transformation (DWT). This separates the image into quadruple sub-bands. Noise and HH sub-band are high-frequency components. HH sub-band also has vertical edges. These vertical edges are removed by performing Hadamard operation on downsampled Sobel edge-detected image and HH sub-band. Using HH sub-band after removing vertical edges is considered for estimating the noise. The Rician energy equation is used to estimate the noise. This is given as input for Artificial Neural Network to improve the estimated noise level. For identifying noise type, CNN is used. After removing vertical edges, the HH sub-band is given to the CNN model for classification. The classification accuracy results of identifying noise type are 100% on natural images and 96.3% on medical images.
1. Introduction
Digital images are part of everyday life. Applications such as satellite television, medical imaging, geographic information systems, and astronomy use digital images. Regarding the information revolution, one of the most intriguing features is the capacity to communicate and receive sophisticated data beyond traditional written language boundaries. In images, noise modeling is influenced by the capturing instruments and by problems with the data acquisition process. Other factors that influence noise modeling in images include radiation sources, transmission media, and image quantization.
An image is defined as a function with two dimensions f(xi, yi) xi and yi are coordinates and f(xi, yi) is the gray value. There are two types of images: grayscale image with one gray channel and RGB image with red, green, and blue channels.
Many medical imaging modalities are available. Ultrasound or sonogram is the cheapest and safest medical imaging for investigating abdominal and pelvic organs and checking fetal development. CT is costlier than the US and more affordable than MRI. They detect tumors, bone fractures, cancer identification, and internal bleeding. MRI is used to capture images of anatomical and physiological body regions. In all these imaging modalities, during its acquisition process, noise increases because of the capturing procedure, electrical fluctuations, body movement, body fat, and liquid present in the body. For example, in asymptomatic diseases such as brain aneurysms in their first stage, their features are similar to pixels of white noise. If missed during MRI imaging, it results in subarachnoid hemorrhage (SAH). SAH discharges blood into the subarachnoid space, it has 40% risk of life threatening [1]. A noisy image may not detect aneurysm leading to human mistakes in its detection. Similarly, the presence of noises reduces the performance of segmentation, registration, classification, or tensor estimation [2]. MR images get noise due to scanned objects, electrical fluctuations during acquisition, noise due to RF coils and conductors, and hardware components [3]. Noises in MRIs are often considered as Racian distribution with low levels (SNR < 2) [4] and the Gaussian distributions with SNR > 2 [5]. For denoising, many filters such as spatial domain filters, transform domain filters, anisotropic filters, nonlocal means, and wavelet transform approaches are used [6–9].
The noise unfavorably affects the image quality and may also damage postimage processing procedures. For improved observation and analysis of medical images, noise estimation, type identification, denoising, and image augmentation becomes crucial. Noise’s standard deviation is estimated and used as image denoising parameters and thereby improving images’ visual quality. Thus, the noise type should be predicted to generate denoised images. Several methods are present to eliminate noise, but there is a need for accurate noise estimation and its type identification. Figures 1(a) and 2(a) are original images of natural and medical image datasets, respectively.
Type of the noises:(1)Gaussian noise is the random intensity values added to the image, and it occurs when the random fluctuations from the source happen. On natural or medical images, it degrades the image by masking the fine details of the image. Figures 1(b) and 2(b) show their effect on natural and medical images.(2)Motion artifact: a noise that adds to the image during the acquisition of the image because of patients’ movement of fluids in the body and breathing. This blurs the image by smudging the boundaries. Figures 1(c) and 2(c) show their effect on natural and medical images.(3)Shot noise or Poisson noise: modeled using the Poisson process. This adds up to the image from the electric fluctuations. This is also similar to Gaussian noise. Figures 1(d) and 2(d) show their effect on natural and medical images.(4)Salt-and-pepper noise or impulse noise: may be created by small and unexpected disturbances in the image signal. It expresses itself by randomly occurring white or black (or both) dots over the image. On the natural or medical images, it appears as sparsely appearing black and white spots, which may vanish the micro details of the image. Figures 1(e) and 2(e) show their effect on natural and medical images.(5)Speckle noise is represented as multiplicative values. This adds to the image from the interference of many waves of the same frequency. This also appears same as Gaussian noise. Figure 1(e) and 2(f) show their effect on natural and medical images.
Effects of the noises on the medical images:(1)The detection of the boundary of the objects becomes tough when the noises are present in the image(2)Due to the wrong placement of the boundaries, the area measurements of tumors or any other clinical measure give the wrong results(3)The contrast and angle changes in the images can introduce the more complexity in differentiating between the healthy and unhealthy cells
On natural scene images:(1)The noise factors on the natural image make the machine learning model to give the wrong decisions.(2)While training the machine learning model for the object detection needs the clearer images. Noised images may train the system wrongly thereby whole model goes wrong.
In the machine learning approaches, the accuracy of the model depends on the feature-choosing methods, if the choice of features goes wrong the accuracy of the model is affected. Whereas in the deep learning model, the feature selection is automatic, based on the images of the dataset the features are selected which enhance the correctness of the model. Hence, the deep learning model provides the higher results compared to the machine learning models.
Highlights of this paper are:(1)Noise present in the medical image is estimated initially using the Rician energy equation and then refined using the neural fit model.(2)A novel CNN approach is designed to identify the noise type. The accuracy of classification is 100%.(3)The same approach is applied to medical images. Obtained accuracy of classification is 96.3%.
2. Literature Survey
Noise estimation is performed in the spatial and transformation domain. Spatial domain noise is estimated using PCA [10], a fuzzy model in MRI images [11]. In transformation, domain noise is estimated using DWT coefficients [12]. Both spatial and transformation domain are used to find the noise amount [13].
The presence of noise was identified by applying the DCT to obtain the kurtosis. The kurtosis values decrease with an increase in the noise density. The threshold value is computed by observing the kurtosis value for every image in the dataset. The absolute deviation for the noisy and noise-free images is calculated to decide image is noisy or not. They have considered the SIPI MISC dataset of natural images. The noise type detected is impulse, achieving an accuracy of 97% [14]. Subashini and Bharathi [38] extracted the statistical features such as kurtosis and skewness to determine the Gaussian, speckle, and salt,-and-pepper noises using the minimum distance pattern classifier. The experiment is conducted on satellite, X-ray, MRI, and digital images [15]. The Gaussian and impulse noise identification and removal by using the intensity equalization technique where the author calculates the distance between the histograms, the maximum distance becomes the threshold depending on the value of the type of noises identified the noises and the adaptive filters are employed to remove the noises [16]. Kumar and Nagaraju [17] designed a methodology to denoise the grayscale image by collecting the features such as entropy, information gain, and skewness and carried out a comparative analysis on the six classifiers and achieved the PSNR, SSIM, and SDME values of 47.27, 0.97 and 61.63 dB, respectively. The regression methods such as the kernel greedy algorithm are employed to detect the Gaussian noise using the orthogonal matching pursuit algorithm and achieve a mean square error of 0.033 [18]. The speckle noise is determined using the CNN, which has the two cascaded CNN models designed. The first stage will estimate the noise and give the input to the second module along with the noise image to remove the noise. Four layers in both the sets of CNN are designed with input dimension 40 × 40 × 3. The dataset: 1,000 CT images of size 256 × 256 (National Biomedical Imaging Archive) is employed and achieved the PSNR of 23.05 [19]. To estimate the presence of noise, a fuzzy model and DWT coefficients are used [20, 21]. Very few authors have contributed in identifying noise type. Most of the authors concentrate on denoising the specific type of noised images [10, 22–27].
The CNN model is employed to identify the noises such as impulse, Gaussian, and Poisson noises. They have used the SIPI MISC natural image dataset, noised with one and multiple combinations of noises of the dataset size 12,650 image, for training 11,000 and testing 1,650. To reduce the computation time, PCA filters are used at every layer; 21 layers are present in the model and achieved an overall accuracy of 96.3% [28]. Table 1 represents the literature review summary.
2.1. Summary of Literature Survey
The nature of the noise is preassumed based on the imaging modality used. These assumptions have saturated the performance of the filters. Many of the authors have proposed different techniques to identify impulse noise in a window of varying sizes. Very few authors have contributed to identify the noise type of the image. Dataset used is of natural images, and size is less. Hence, there is a need for detecting noise type of an image. After identifying noise type, a better denoising technique can be applied. Knowing the nature and distribution of noise plays a vital role. Hence, it is essential to characterize the noise type and noise level present in the images.
3. Proposed Methodology
The proposed approach for estimating noise and identifying noise types is shown in Figure 3. It has four main parts: the input image is transformed into DWT coefficients in the first part. The second part detects the edges from the input image and then down samples. The third part estimates the noise, and the fourth part identifies the noise type. The proposed algorithm for noise estimation and identification is shown in Algorithms 1 and 2.
| ||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||
3.1. Noise Estimation
Using the high-frequency components of the image noise is estimated. HH sub-band obtained from DWT has high-frequency components as well as diagonal edges. Diagonal edges are important features of the images; hence, they are not noises. So, these diagonal features have to be subtracted from the sub-band. For this, the borders of the whole image are found using the Sobel and then downsampled by 2. In order to remove diagonal edges, Hadamard operation is performed. Initial noise level of this image is determined utilizing noise using the Rician energy equation and then fed to the neural fit method to improve the estimated noise accuracy and resilience even more.
The blocks of the proposed system are briefed in this subsection.
3.2. Wavelet Transformation
The wavelet transform’s fundamental technique uses a function whose initial wavelet is zero. The fundamental wavelet may be expanded and translated to provide a function. The expression of the Haar wavelet and the scaling factor equation are as shown in Equation (1). The components are shown in Figure 4.
3.3. Edge Detection using Sobel Operator
Sobel filter is a basic approximation to the notion of a gradient with smoothing. The gradients in the X and Y axes are often detected using the 3 × 3 convolution mask. The operator consists of a pair of 3 × 3 kernels, as shown in Figure 5.
The kernels may be applied individually on the input image to obtain gradient components (name them Gx and Gy). The size and direction of the gradient at each position may be determined by combining these values using Equation (2).where G is the gradient value, Gx and Gy are the gradient components, respectively.
3.4. Initial Noise Estimation
After applying wavelet transform, HH sub-bands standard deviation is found using Rician energy equation Equation (3).where is the standard deviation of the noisy HH sub-band without edge, k is the constant equal to 4, N is the size, and Jne is the nonedge image. The combined image of Sobel and HH sub-band obtained image is shown in Figure 6.
3.5. Noise Estimation using Neural Fit
The purpose of the proposed system is to perform the noise estimation using the Rician energy equation and neural fit [33]. The initial noise is found by using Equation (3). For the adaptive noise estimation, the Rician noise is employed, and it is applied to the HH subcomponent [21].
The neural tool MATLAB is employed to design the neural fit model. The GUI-based training model has ten hidden layers, a sigmoid activation function, and a feed-forward network [34]. The features are divided into 60, 20, and 20 for the training, testing, and validation. The Levenberg–Marquardt algorithm is used for training with 10 hidden layers. The error is measured using mean square error. The parameters involved in training are as shown in Table 2.
The visualization of the neural fit model is shown in Figure 7, the input is standard noise calculated from Rician noise equation and the output is fitted using the neural-fitting model [35].
3.6. Noise Type Identification
Instead of directly providing images to CNN model, they are dimensionally reduced by half the size of the image and dimensionally reduced by performing Hadamard multiplication of HH sub-band and downsampled image [36]. Thus, obtained images are of high intensity. Hence, they are considered noise features. These images with noise features are fed to the CNN model for identifying noise types. Each layer of CNN architecture has four convolutional layers with a 3 × 3 kernel and filters of 8, 16, 32, and 32. Between each convolutional layer, batch normalization, ReLU is applied with max pooling, followed by fitting tool SoftMax with learning rate 1 × 10−4 and 40 validation frequency. The details of every layer are shown in Table 3.
It is known that Convolutional layer performs the convolution operation and Batch Normalization standardizes and normalizes the input from one layer to another. This helps to reduce the number of epochs [37]. The dimensionality of the feature map is reduced by the pooling layer [38].
4. Results and Discussion
The experiment is run to estimate the noise and to identify the noise type of the given image. Two datasets are used: standard 44 natural images from USC-SIPI image database [34]. The medical images from the cancer imaging archive TCIA [35] and local hospitals are shown in Table 4. These images are noised with Gaussian, motion, Poisson, salt-and-pepper, and speckle. After adding noise, natural images dataset size are 1,100 images and the medical images are 2,890 images. Details of data acquisition are given in Table 4.
Sample images from these databases are shown in Table 5. For experimentation, MATLAB 2019® is used. The dataset is divided into training, testing, and validation set in the ratio of 60 : 20 : 20.
The result and discussion have two subdivisions: noise estimation and noise identification.
4.1. Noise Estimation
The estimated noise level for motion-noised image is shown in Figure 8. The estimated noise level is 3.6755.
The estimated noise for natural and medical images with different noise is shown in Table 6.
4.2. Noise Identification
For noise type identification, CNN model without max pooling layer is used for the proposed methodology (Figure 1). The experiment is carried out on natural images and medical images.
The performance analysis of the defined CNN model to identify the noise is shown in Figure 9. The training process is shown in Figure 10 with validation accuracy of 97.05%.
The confusion matrix for natural images and medical images are shown in Figures 11(a) and 11(b), respectively. This shows the predicted class and the true class.
(a)
(b)
The results of the confusion matrix of natural images show accuracy of 100%. Hence, when we applied the proposed model to the medical images, the results show accuracy of 96.3%. The performance analysis of the proposed system for natural images and medical images is shown in Table 7.
4.2.1. Comparative Study
A comparative study shows the accuracy of the proposed algorithm is more accurate for identifying the noise type present in the natural and medical images as shown in Table 8.
The graph in Figure 12 shows the performance of this algorithm with other methods. The graph shows that this method of CNN using edge-subtracted HH subcomponent has good accuracy compared to other authors’ algorithms.
5. Conclusion
The nature of noise type is preassumed based on the imaging modality used. With these assumptions, filters performance is saturated. Hence, an intelligent image noise estimation and noise type recognition method are needed. Once the estimated noise and noise type corrupting an image are identified, an appropriate denoising filter with estimated noise can be applied. For denoising, it is important to characterize the noise type and the estimate noise based on their characteristics of distribution. Here, an experiment was carried out on the dataset of natural images from SIPI and medical images collected from TSIA and local Hospitals. The noises considered for identification are Gaussian, motion, Poisson, salt-and-pepper, and speckle. In the proposed system, instead of using images directly, the only noise component of the image is fed to the CNN for classification. HH sub-band of the DWT transformation has high-intensity values and also edges. From the HH band, edge components are removed by performing Hadamard operation with the Sobel edge-detected downsampled image. For noise estimation, the neural fit model is used, and for classification CNN model. The CNN architecture was designed without max pooling. Experimenting on natural images with five noises gives 100% accuracy, and on medical images with the same five noises gives the accuracy of 96.3%. The images contain multiple noises; hence, in future, the experiment of noise estimation and noise type identification can be carried out on these images. Also, further enhancement can be carried out on different datasets with different noise types.
Data Availability
The data that support the findings of this study are available on request from the corresponding author.
Conflicts of Interest
The authors declare that they have no conflict of interest.
Funding
The authors extend their appreciation to Taif University for supporting current work by Taif University Researchers Supporting Project number (TURSP - 2020/257), Taif University, Taif, Saudi Arabia.