Image Defogging Algorithm Based on Sparse Representation

Fan, Di; Guo, Xinyun; Lu, Xiao; Liu, Xiaoxin; Sun, Bo

doi:https://doi.org/10.1155/2020/6835367

Complexity

On this page

Abstract Introduction Conclusions Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Learning and Adaptation for Optimization and Control of Complex Renewable Energy Systems

View this Special Issue

Research Article | Open Access

Volume 2020 | Article ID 6835367 | https://doi.org/10.1155/2020/6835367

Image Defogging Algorithm Based on Sparse Representation

Di Fan,¹Xinyun Guo,¹Xiao Lu,¹Xiaoxin Liu,¹and Bo Sun²

Guest Editor: Shubo Wang

Received29 Apr 2020

Accepted02 Jun 2020

Published11 Jul 2020

Abstract

Aiming at the problems of low contrast and low definition of fog degraded image, this paper proposes an image defogging algorithm based on sparse representation. Firstly, the algorithm transforms image from RGB space to HSI space and uses two-level wavelet transform extract features of image brightness components. Then, it uses the K-SVD algorithm training dictionary and learns the sparse features of the fog-free image to reconstructed I-components of the fog image. Using the nonlinear stretching approach for saturation component improves the brightness of the image. Finally, convert from HSI space to RGB color space to get the defog image. Experimental results show that the algorithm can effectively improve the contrast and visual effect of the image. Compared with several common defog algorithms, the percentage of image saturation pixels is better than the comparison algorithm.

1. Introduction

In fog and haze weather conditions, the image taken by using imaging equipment is prone to quality degradation. The degraded image brings difficulties to the application of image in military, traffic, driverless vehicles, and other fields [1]. Therefore, dehazing the foggy image has important practical application value.

In recent years, many scholars at home and abroad have done a lot of research on image dehazing algorithms, among which the representative algorithms are mainly divided into three categories. The first is the method based on image enhancement, which achieves the purpose of dehazing by improving the contrast of the image, mainly including Retinex algorithm [2], wavelet transform algorithm [3], and histogram equalization algorithm [4]. In view of the problem of detail loss and uneven dehazing effect of the adaptive histogram equalization algorithm with limited contrast, Wang et al. [5] proposed a contrast adaptive clipping histogram equalization algorithm, which enhanced image contrast and enriched image detail information. This method does not consider the cause of image degradation; the image loses some information while enhancing the contrast. The second is the image restoration algorithm based on the imaging model, which constructs the physical model of image degradation, then estimates the unknown parameters in the imaging model through the prior knowledge, and finally recovers the fogless image, for example, dark channel prior algorithm [6], color attenuation prior algorithm [7], atmospheric dissipation function algorithm [8], and so on. He et al. [9] obtained the dark channel prior theory by analyzing statistically a large number of outdoor fogless images, which can be used to better recover fogless images. However, the image after the dehazing is easy to show halo effect, especially in the sky region which does not meet the dark channel prior theory which has serious color distortion. Many scholars put forward improvement measures. Han et al. [6] isolated the sky region of the image for processing by setting the threshold value. In order to improve the real-time performance of the algorithm, the image was reduced, which reduces the time required to figure out the transmittance map, and then interpolation method was used to restore the transmittance map to the size of the original image, and the resulting images achieved a better dehazing effect. Guo et al. [10] divided the image into three regions and adopted different dehazing algorithms for each region, which could well avoid false edges at the junction of sky and scenery. Ti et al. [11] proposed to divide the foggy image into sky region and non-sky region and introduced tolerance parameter correction transmittance to the sky region that does not meet the dark channel prior theory. Fan et al. [12] proposed a dehazing model combining atmospheric scattering model and color attenuation prior, which greatly improved the clarity and contrast of the recovered image. Peng et al. [13] eliminated color distortion while restoring contrast by adding color adaptive correction. Xu et al. [14] proposed an image dehazing algorithm based on the combination of sparse representation and dark channel prior based on K-SVD, which has a good dehazing effect. Liu et al. [15] refined the medium transfer map according to the sparse features of fog gray images and then reversely solved the fog degradation model. Finally, the processing advantage of the sky region in the fog image was obvious. Weijing [16] applied the sparse representation algorithm to the image dehazing. By comparing the dark channel prior model, the new algorithm has a better dehazing effect and a higher fidelity. The third is the neural network dehazing algorithm based on machine learning [17–20]. This method uses multi-scale convolution operation to extract the feature map of the image, then uses the fog image and its corresponding fogless image as the training set to train different network structures, and finally uses the trained model to image dehazing. However, this method does not consider the degradation of the image, so the phenomenon of incomplete dehazing occurs.

Aiming at the above problems, this paper proposes an image dehazing algorithm based on sparse representation on the basis of analyzing and summarizing the characteristics of existing methods. Firstly, the image is converted from RGB space to HSI space, and the two-stage wavelet transform extract feature extraction of the I component is used. Then, the K-SVD algorithm for dictionary training of the extracted features is used, the sparse features of the fogless image are learned, and the I-component of the fogless image is reconstructed. Then the S-component is nonlinearly stretched to improve the image saturation. In this paper, the proposed algorithm is tested and a good dehazing effect is obtained.

2. Defogging Algorithm Based on Sparse Representation

2.1. General Idea of the Algorithm

This paper proposes a single image defogging algorithm framework based on sparse representation as shown in Figure 1, which is mainly divided into two processes. One is the dictionary training process, first using image pairs (foggy images and their corresponding non-foggy images) to build a training set, then extracting the characteristics of the image pair I-component through wavelet transform, and using the K-SVD algorithm to sparsely combine a large number of image pairs dictionary training to get a dictionary of foggy images and non-foggy images. The other is the reconstruction process, first inputting the I-component of the image to be reconstructed, then extracting the features in the wavelet domain, and obtaining the sparse coefficient through the orthogonal matching pursuit (OMP) algorithm. The sparse representation coefficient gets the I-component of the reconstructed image. Compared with fogless image, the pixel value of S-component of foggy image is lower. In order to improve the image saturation, the nonlinear stretching function is used to stretch the image saturation. Finally, the processed HSI space image is converted into RGB color space to obtain the defogged image.

2.2. Feature Extraction in Image Wavelet Domain

Wavelet transform is the development of Fourier transform. Because of its good time-frequency localization ability [21], it can extract the effective information of the image well, so it is widely used in the field of image processing. In this paper, Haar wavelet is used as an orthogonal function to perform two-layer decomposition of two-dimensional images. Figure 2 shows the decomposition diagram of two-level wavelet decomposition.

Among them, HL_1, LH_1, and HH_1 represent the horizontal, vertical, and diagonal high-frequency components after first-order wavelet decomposition; LL_2 represents the low-frequency component of second-order wavelet decomposition. HL_2, LH_2, and HH_2 represent the horizontal, vertical, and diagonal high-frequency components after the second-order wavelet decomposition, respectively.

In this paper, two-level wavelet transform is used to extract the features of the I-component of the image in the training set. Firstly, the image is first-level decomposed to obtain the feature maps of horizontal, vertical, and diagonal directions and low-frequency components, and then the low-frequency components are decomposed to obtain high-frequency and low-frequency feature maps. The dictionary training is performed through the extracted feature map to recover the I component. Figure 3 shows the extracted feature map after two-stage wavelet decomposition.

(a)

(b)

2.3. Sparse Representation

Signals generally have sparse characteristics. The sparse representation of a signal represents the original signal through a series of basic signals in the dictionary or a linear combination of some atoms, which makes it easier to perform subsequent processing on the signal, such as encoding, compression, and amplification.

For image signals , a linear combination of sparse dictionary can be used to express [22]. A column in the dictionary is called an atom. Sparse representation is to use a linear combination of a small number of atoms in the dictionary to represent the signal, and its mathematical expression is [23]

Among them, represents the coefficient matrix. If the coefficient matrix is sparse, most of the elements in the matrix are zero, and there are only a few non-zero elements. The linear combination representation is a sparse representation.

3. Dictionary Training and Reconstruction

3.1. Dictionary Training Based on K-SVD

Dictionary training is the core of the sparse representation of the image. The quality of the dictionary after training will directly affect the effect of the sparse representation. The smaller the residual between the atoms and the signals in the dictionary, the more the structural features match, and the higher the quality of the reconstructed image. Since the images to be reconstructed are very different and the texture details of the images are quite different, the dictionaries obtained by different training methods will have different effects on the reconstruction of the images. Common training dictionary algorithms include MOD algorithm (Method of Optimal Directions, MOD), K-SVD algorithm (K-Singular Value Decomposition, K-SVD), and ODL algorithm (Online Directions Learning, ODL). The K-SVD algorithm uses alternate iteration to update the dictionary, which is more efficient than other algorithms. Therefore, this papers uses the K-SVD algorithm for dictionary training. The training process is shown in Figure 4.

The K-SVD algorithm was proposed by Aharon. This algorithm improves the decomposition efficiency by reducing the number of atoms in the dictionary; the atoms retained in the dictionary can also represent the original information. The specific implementation process is as follows [24]:

means complete dictionary, and means sparse coefficient moment:

Assign initial values to the complete dictionary and sparse matrix, and then update a column of atom in ; the cost function is where is the error matrix, which represents the errors that other atoms than the column have on the image. Then, equation (3) can be expressed as

For the above formula, if the singular value decomposition value of is directly used to update and , the updated value will not be sparse, so transform and , and only retain the non-zero values in :

Select all columns in the error matrix that satisfy the set to form , and then perform singular value decomposition:

The updated value is

Among them, is the largest singular value of , and and are the first column vectors of and , respectively. After traversing each column in the dictionary, the optimized dictionary is obtained. The process of K-SVD algorithm is shown in Table 1.

3.2. Image Defog Reconstruction

Signal sparse representation also involves another key issue, namely, sparse coding. The so-called sparse coding refers to the process of solving the sparse coefficient α when the overcomplete dictionary is known. The solution of this coefficient is an NP-hard problem. The common methods to solve the sparse coefficient include MP algorithm (Matching Pursuit algorithm, MP), OMP algorithm (Orthogonal Matching Pursuit algorithm, OMP), and SOMP algorithm (Simultaneous Orthogonal Matching Pursuit algorithm, SOMP). Atoms selected the OMP algorithm are orthogonal, which ensure that the signal and the selected atoms meet the orthogonal condition at each step of the iteration. Meanwhile, selected atoms cannot be selected again in the next iteration, which greatly reduces algorithm processing time and the number of atoms to decompose the original signal. In view of the above advantages, this paper uses the orthogonal matching pursuit algorithm [24] to solve the sparse coefficients.

For an image with fog, in the HSI color space, the fog is mainly distributed in the I-component. This paper reconstructed the I-component to improve the image brightness to achieve the purpose of fog removal. After dictionary training, a dictionary of the I-component of the foggy image and the I-component of the nonfoggy image is obtained, and the trained dictionary is used to reconstruct the foggy image I-component. The process is shown in Figure 5. First, feature extraction is performed on the I-component of the foggy image to obtain image feature blocks, and then OMP algorithm is used to obtain the sparse representation coefficient α in combination with the trained dictionary of the foggy image. The dictionary of the I-component of the fog-free image is brought into equation (1) to obtain the wavelet domain features of the reconstructed image I-component, and finally the inverse wavelet transform is used to obtain the reconstructed image I-component.

In order to improve the image saturation and make the image color more vivid, it is necessary to stretch the saturation. Due to uneven illumination at different locations of the same image, different regions have different saturations. In order to achieve different stretching effects under different lighting conditions, this paper uses a piecewise function to perform nonlinear stretch. Assuming that the minimum brightness value of the saturation component of the foggy image is and the maximum brightness value is , the interval is evenly divided into three segments, and the length of each segment is , using the following formula for saturation stretching:

In the formula, represents the value of the S-component of the original image and represents the value of the S-component after stretching.

In the HSI color space, the H-component determines the essence of the color of an object. It is determined by the wavelength reflected by the object, and different wavelengths reflect different colors. For a color, the hue of the image before and after defogging is approximately invariant. Therefore, this article does not process the H-component to keep the original hue. Combine the processed S-component, I-component, and original H-component and convert to RGB space to get the final defogging image.

4. Experiment and Result Analysis

4.1. Experimental Platform and Data Set

The experiment in this paper is carried out on Windows 7 operating system and MATLAB 2017a development platform, using the D-HAZY data set as the training set. The data set includes 80 RGB foggy and non-foggy images in indoor and outdoor scenes, some of which are shown in Figure 6. In order to improve the speed of the algorithm, the size of the image is standardized to 512 512.

(a)

(b)

4.2. Experiment and Result Analysis

The five images in the data set were compared with literature [5], literature [6], literature [10], and the algorithm of this paper. The results are shown in Table 2.

It can be seen from Table 2 that, after [5] algorithm processing, the image contrast is improved, the color is brighter, and the fog removal is more thorough, but the fourth image is overenhanced, resulting in white light on the wall area. After the algorithm [6] processed the image, the whole image appeared blue, the contrast was reduced, the color was dark, and there was a color disorder. For example, the first and second images were severely distorted due to excessive enhancement. The image processed by the algorithm in [10] can achieve good results. The overall color of the image is natural, but part of the image is not completely dehazed. For example, in the first image, there is a layer of fog in the distant trees. After the algorithm is processed in this paper, the effect of defogging is better for different images, the brightness of the image is moderate, and it is full of layering. For images that are not well-processed by other algorithms, such as the second and third images, the algorithm does not appear dark or distorted.

In order to objectively compare several algorithms, this paper uses the blind contrast enhancement evaluation method, using three indicators , , , SSIM (structural similarity, SSIM), and PSNR (peak signal-to-noise ratio, PSNR) to compare the image defogging effect, where is the ratio of the newly added visible edges of the restored image, is the ratio of the average gradient, and is the percentage of saturated pixels (pure black or pure white) in the restored image, and their calculation method is as follows [25]:where represents the number of visible edges in the original image , represents the number of visible edges in the defog image , is the set of visible edges in the fog-free image , and is the i pixel of the visible edges in the fog-free image. is the ratio of the gradient of the non-foggy image and the foggy image at , is the number of pure black or pure white pixels, and and represent the width and height of the image, respectively.

In general, when , , SSIM, and PSNR are relatively large and the value is relatively small, it means that the image has good dehazing effect. The objective evaluation results of the defog quality of the five images in Table 2 are shown in Table 3. It can be seen that the percentage of saturated pixels of the image processed by the algorithm in this paper is significantly lower than that of the other three algorithms, which is not much different from the algorithm in [10]; the newly added visible edge ratio is slightly lower than the other three algorithms, and SSIM and PSNR values are higher than the other three algorithms. From the average point of view, the and values of the algorithm in this paper are slightly lower than those of the other three algorithms, but the values are significantly better than other algorithms. The PSNR value is significantly higher than the other three algorithms.

5. Conclusions

Based on the image sparse representation theory, this paper proposes a new image defogging algorithm. The algorithm uses two-stage wavelet transform to extract image features for the I-component of the image, uses the extracted features for dictionary training in the sparse domain, learns the sparse features of the fog-free image, reconstructs the I-component of the fog image, andapplies the nonlinear stretching method to increase image brightness. Experimental results show that the algorithm can effectively improve the contrast of the image and improve the visual effect of the image. Compared with the common defogging algorithm, the percentage of saturated pixels, SSIM, and PSNR in the image is better than the comparison algorithm. Although the algorithm in this paper has achieved good results, it is relatively simple in feature extraction, resulting in partial image dehazing and color cast problems. The next step will be to study how to extract more useful features for dictionary training and improve the algorithm effect.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This research was supported by the following projects: Scientific Research Project of National Language Commission (YB135-125); Key Research and Development Project of Shandong Province (2019GGX101008 and 2016GGX105013); Natural Science Foundation of Shandong Province (ZR2017MF048); Science and Technology Plan for Colleges and Universities of Shandong Province (J17KA214).

References

R. Ma and S. Zhang, “An improved color image defogging algorithm using dark channel model and enhancing saturation,” Optik, vol. 180, pp. 997–1000, 2019.
View at: Publisher Site | Google Scholar
M. Yamakawa and Y. Sugita, “Image enhancement using Retinex and image fusion techniques,” Electronics & Communications in Japan, vol. 101, no. 8, pp. 360–368, 2018.
View at: Publisher Site | Google Scholar
D. Jingwei, Z. Chunli, and H. Bo, “Research on image defogging algorithm combining homomorphic filtering and wavelet transform,” Journal of Harbin University of Science and Technology, vol. 24, no. 1, pp. 66–70, 2019.
View at: Google Scholar
Y. T. Liang, L. Li, K.-B. Zhao, and J.-H. Zhao, “Defogging algorithm of color images based on Gaussian function weighted histogram specification,” in Proceedings of the 2016 10th International Conference on Software, Knowledge, Information Management & Applications (SKIMA), pp. 364–369, Chengdu, China, December 2016.
View at: Publisher Site | Google Scholar
Q. Wang, S. Wang, and B. Jia, “Curvelet transform and contrast adaptive clip histogram equalization-based image defogging algorithm,” The Journal of China Universities of Posts and Telecommunications, vol. 5, no. 2, pp. 100–108, 2018.
View at: Google Scholar
G. Han, X. Xiaoting, and L. Bo, “Research on image defogging method based on dark primary color prior,” Acta Optics, vol. 38, no. 4, pp. 113–122, 2018.
View at: Publisher Site | Google Scholar
Q. Zhu, J. Mai, and L. Shao, “A fast single image haze removal algorithm using color attenuation prior,” IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 3522–3533, 2015.
View at: Publisher Site | Google Scholar
S. Li, D. Zhang, and M. Ju, “An image dehazing method based on atmospheric veil,” in Proceedings of the 2016 International Forum on Mechanical, Control and Automation (IFMCA 2016), pp. 595–600, Shanghai, China, March 2017.
View at: Publisher Site | Google Scholar
K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 33, no. 12, pp. 2341–2353, 2011.
View at: Publisher Site | Google Scholar
X. Guo, P. Sun, X. Lu, S. Gao, X. Liu, and D. Fan, “Single image defogging algorithm based on three-region division,” Journal of Shandong University of Science and Technology (Natural Science Edition), vol. 39, no. 1, pp. 120–128, 2020.
View at: Google Scholar
X. Ti, Z. Li, and X. Guo, “Improved dark channel prior defogging algorithm to distinguish sky,” Information Technology and Informatization, no. Z1, pp. 100–103, 2018.
View at: Google Scholar
D. Fan, X. Ti, and Q. Meng, “Adaptive defogging algorithm based on color attenuation,” Computer Measurement and Control, vol. 26, no. 9, pp. 200–204, 2018.
View at: Google Scholar
Y. T. Peng, K. Cao, and P. C. Cosman, “Generalization of the dark channel prior for single image restoration,” IEEE Transactions on Image Processing, vol. 6, no. 27, pp. 2856–2868, 2018.
View at: Publisher Site | Google Scholar
X. Ziang, A. Sheng, and L. Weidang, “Image dehazing algorithm based on sparse expression and dark channel,” Journal of Zhejiang University of Technology, vol. 45, no. 3, pp. 315–319, 2017.
View at: Google Scholar
L. Kun, B. Duyan, W. Shiping, H. Linyuan, and G. Shan, “Single image dehazing based on sparse feature extraction,” Acta Optics, vol. 38, no. 3, pp. 298–306, 2018.
View at: Publisher Site | Google Scholar
W. Weijing, “Image denoising and smog removal algorithm based on sparse representation,” Zhejiang University of Technology, Hangzhou, China, 2017, Master’s thesis.
View at: Google Scholar
B. Cai, K. X. Xu, and D. Tao Qing, “DehazeNet: an end-to-end system for single image haze removal,” IEEE Transactions on Image Processing, vol. 25, no. 11, pp. 5187–5198, 2016.
View at: Publisher Site | Google Scholar
D. Engin, A. Genc, and H. K. Ekenel, “Cycle-Dehaze: enhanced CycleGAN for single image dehazing,” in Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 938–946, Salt Lake City, UT, USA, June 2018.
View at: Publisher Site | Google Scholar
J. Na, B. Wang, G. Li, S. Zhan, and W. He, “Nonlinear constrained optimal control of wave energy converters with adaptive dynamic programming,” IEEE Transactions on Industrial Electronics, vol. 66, no. 10, pp. 7904–7915, 2019.
View at: Publisher Site | Google Scholar
S. Wang and J. Na, “Parameter estimation and adaptive control for servo mechanisms with friction compensation,” IEEE Transactions on Industrial Informatics, 2020.
View at: Publisher Site | Google Scholar
Q. Mou, “Image denoising algorithm based on improved two-dimensional Haar wavelet,” Journal of Chongqing University of Technology (Natural Science), vol. 33, no. 6, pp. 177–183, 2019.
View at: Google Scholar
P. Yang, L. Gao, J. Wang, and L. Zi, “Color image denoising algorithm based on sparse representation and dictionary learning,” Computer Engineering and Science, vol. 40, no. 5, pp. 842–848, 2018.
View at: Google Scholar
C. Shen and M. Zhang, “Image sparse denoising algorithm based on dictionary learning,” Journal of Detection and Control, vol. 40, no. 2, pp. 128–134 + 140, 2018.
View at: Google Scholar
M. Aharon, M. Elad, and A. Bruckstein, “$rm K$-SVD: an algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Transactions on Signal Processing, vol. 54, no. 11, pp. 4311–4322, 2006.
View at: Publisher Site | Google Scholar
N. HautièreBruckstein, J.-P. Tarel, D. Aubert, and É. Dumont, “Blind contrast enhancement assessment by gradient ratioing at visible edges,” Image Analysis & Stereology, vol. 27, no. 2, pp. 87–95, 2011.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Di Fan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1162

Downloads

1101

Citations