Abstract
Given that a single remote sensing image dehazing is an ill-posed problem, this is still a challenging task. In order to improve the visibility of a single hazy remote sensing multispectral image, we developed a novel and effective algorithm based on a learning framework. A linear regression model with the relevant features of haze was established. And the gradient descent method is applied to the learning model. Then a hazy image accurate transmission map is obtained by learning the coefficients of the linear model. In addition, we proposed a more effective method to estimate the atmospheric light, which can restrain the influence of highlight areas on the atmospheric light acquisition. Compared with the traditional haze removal methods, the experimental results demonstrate that the proposed algorithm can achieve better visual effect and color fidelity. Both subjective evaluation and objective assessments indicate that the proposed method achieves a better performance than the state-of-the-art methods.
1. Introduction
Remote sensing images are widely applied to various fields because of its high spatial resolution and stable geometric location [1–3]. However, the process of remote sensing images acquisition is vulnerable to the atmospheric conditions (e.g., hazy or foggy), resulting from the fact that light is absorbed and scattered by the turbid medium such as particles and water droplets in the atmosphere [4–6]. Consequently, the technique of haze removal has received more attention in improving the visibility of satellite imagery. Nevertheless, how to achieve a single remote sensing image haze removal is still a challenging task since the regions spoiled by haze contain both ground features and haze components.
In the past decades, along with the development of computer and computer vision, the investigations on haze removal for satellite imagery have got some progresses. Chavez et al. [7, 8] improved dark-object subtraction (DOS) method to correct optical data for atmospheric scattering. This method assumes that the reflectivity of the pixels is very low, but, owing to the existence of haze, the number (DNs) in these pixels is nonzero. Therefore, the values of the DNs can be taken as the haze thickness. Then the DNs value is subtracted from a hazy image to achieve haze removal. Based on the researches by Chavez, Makarau et al. [9] searched for dark objects locally in the whole image to construct a haze thickness map (HTM). Therefore, subtracting the HTM from a hazy image leads to haze removal. Zhang et al. [10] proposed a haze optimized transformation (HOT) algorithm, to achieve haze region removal based on analyzing the features of visible-band space. But this method is not suitable for bright surfaces. Ni et al. [11] developed a method based on using linear intensity transformation (LIT) and local property analysis (LPA). However, this method strongly depends on the accurate estimation of LIT and LPA for haze removal.
Recently, the processing of single hazy image has a significant progress. Some methods based on polarization have been developed. Schechner et al. [12] created a scene’s distance map by utilizing the fact that one of the sources of image degradation in haze is partially polarized. Wang et al. [13] proposed a single image dehazing method based on adaptive wavelet fusion, which could preserve the most discriminant scene depth. Liu et al. [14] developed a polarization dehazing method by processing the low spatial frequency parts and the high spatial frequency parts separately. Regarding the image fusion method, Li et al. [15] proposed a single image dehazing approach based on a multiscale pyramid fusion scheme. In addition, with the development of deep learning, some researchers employed convolutional neural networks for single hazy image processing. Ren et al. [16] created a multiscale convolutional neural network model to learn the transmission map. After that, Li et al. [17] present a truly end-to-end network, combining the transmission map with the atmospheric light to produce the results. Nevertheless, these methods are complex and time consuming. Tang et al. [18] analyzed four types of haze relevant features and utilized Random Forests to learn the transmission map. Jiang et al. [19] proposed a method for gray-scale image dehazing, but it was not suitable for inhomogeneous scenes. Gu et al. [20] developed a dehazing method based on average saturation prior, which improved the atmospheric scattering model to cope with the inhomogeneous atmospheric light.
Some single image dehazing methods based on the physical model also have been promoted. Tan et al. [21] firstly proposed a single image dehazing method by maximizing the local contrast of the image. Although Tan’s approach is able to produce color over saturation with images with dense haze, Fattal et al. [22] removed the haze from color images based on Independent Component Analysis (ICA), but the approach is not suitable for the gray scale image haze removal. He et al. [23] summarized a rule that was called dark channel prior (DCP) based on observing the statistical characteristics pertaining to a large number of hazy free images. Then the real scene radiance is recovered based on DCP. Among the above methods, He’s method is simple and effective. However, the DCP method is based on the statistics of the outdoor hazy images, while remote sensing images have a different imaging distance with the outdoor images. As a result, the color drift is easily caused when applied to the remote sensing image.
In this paper, we proposed a novel haze removal method for single remote sensing images. By analyzing the haze relevant features (including luminance component, saturation component, and saliency component), we have established a linear regression model with multiple variables. The correlation between the hazy image and its corresponding transmission map is detected effectively based on learning the coefficients of the linear model. In addition, the atmospheric light was estimated by an improved method, which can reduce the influence of highlight areas on the atmospheric light acquisition. With the recovered transmission information and the estimation of the atmospheric light, the haze-free image can be recovered. Compared with the traditional haze removal technology on remote sensing images, the algorithm can achieve better results.
The remainder of this paper is organized as follows. Section 2 provides the details of our method, including physical hazy image degradation model, atmospheric light estimation, and linear regression model training. The experimental results are listed in Section 3; we present both subjective evaluation and objective assessments. Finally, conclusion is present in Section 4
2. Proposed Method
2.1. The Physical Hazy Image Degradation Model
Nayer and Narasimhan [24, 25] have a detailed description and derivation of the atmospheric scattering model, as shown in Figure 1, and the model is widely referenced by later researchers. They divide the influence of the light reflected by the atmosphere into two parts: direct attenuation and veiling light. The formation of a hazy image can be described aswhere represents the position of the pixel in the image, is the observed hazy image, is the scene radiance, is the global atmospheric light usually assumed to be constant, and is the medium transmission, which describes the radio of the light that is not scattered and gets to the camera. The medium transmission can be expressed aswhere d(x) is the scene depth and is the scattering coefficient of the atmosphere. The goal of dehazing is to estimate the ; we can obtain the real scene since we have an estimate of and by
(a)
(b)
2.2. Estimation of the Atmospheric Light
Most of the haze removal algorithms are based on the pixels associated with a single image to obtain an estimation of the atmospheric light. When the step of remote sensing image acquisition is under the condition of hazy weather, we commonly ignore the influence of sunlight. Tan et al. [21] utilized the maximum value of the pixels in the hazy image as estimation of atmospheric light, while the maximum values of the luminance component may belong to the highlighted object regions. In He’s work [23], the top 0.1% pixels in dark channel are taken as the atmospheric light. Although this method is robust, only taking one point into account may cause that the value of each channel is too high to lead to color drift. Conventional ways tend to be hard to achieve a satisfactory result when the highlight areas exist in image.
Based on the work of He and others, we developed a method that can reduce the influence of highlight areas on the atmospheric light acquisition. First of all, it is required to take the minimum channel map of the degraded remote sensing images. The minimum channel map is calculated asBased on the weighted quad-tree search method [23], the minimum channel map of the hazy image is divided into four average areas by location and then the score of each region is obtained:where is the index of each region, is the score of region , represents the mean value of the region , and represents variance in the region . Then take the region with the highest score as the candidate iterative region, and it is further divided into four smaller regions. This process continues to iterate until the size of the candidate region is smaller than the preset size threshold. The average of each channel in the last candidate region is selected as the result of .
2.3. Transmission Estimation
According to eq. (1), we can recover the real scene if we have acquired the estimation of atmospheric light and the transmission map. Thick haze can cause high brightness, flatness, and unsaturation in the image. We developed a linear regression model to estimate the haze concentration based on investigating various haze relevant features. Saliency represents which regions stand out from the neighbors and are the most attractive [26]. Due to the influence of haze, a large amount of information (color, edge, etc.) in the image is damaged, and the saliency of the target is also greatly affected. So the luminance component , the saliency component , and the saturation component have a strong relationship with the distribution of haze [15, 26, 27]. The concentration of haze is described as follows: where , , represents the nonnegative coefficients of the linear regression model with multiple variables in eq. (6). As a result of transmission is inversely proportional to the haze density, a linear model between transmission and the haze concentration feature was developed as follows: where is a nonnegative coefficient; for computational and description convenience, we rewrite Eq. (6) aswhere , , , =. The gradient descent method is applied to learn and a cost function , which is the sum of squares of all modeling errors; the cost function can be described aswhere m represents the total number of training samples and indicates the actual value of the sample. Then, by minimizing the cost function, we can obtain the series of coefficients. At the beginning, we randomly select a combination of parameters () to calculate the cost function. Then we traced the next combination of parameters to ensure that the value is reduced at the most rapid rate. Continue to do the above steps until the cost function reaches a local minimum. The derivative of the cost function is calculated as follows: And according to the gradient descent algorithm we haveIn the gradient descent algorithm, is updated during gradient descent. Then we can acquire the following expression:where the notation represents setting the value in the left side of the equation to be the value of the right side and is the learning rate, which determines the efficiency of the gradient descent algorithm. If learning rate is too small, we would need a large number of steps to reach the global minimum. Conversely, the cost function is unable to converge when learning rate is too large. Thus, it is significant to choose the right learning rate, and the calculation of the learning rate is shown as follows:where is initial learning rate, indicates the attenuation rate of each round of learning, represents the current number of learning steps which is equivalent to how many times we put batch into the learner, and is the number of steps per round of learning which is equal to the total number of samples divided by the size of each batch.
2.4. Training Data Preparation
In order to learn the coefficients in eq. (6) accurately, we prepared the training data based on the method proposed by Tang et al. [18]. Tang assumed that the image content is independent of scene depth or media transmission, and depth is locally constant. We illustrated the procedure of generating the training data in Figure 2. At first, we generated a random depth map with equal size for each unblemished image, and the pixel values in the synthetic depth map were extracted from the standard uniform distribution in the open interval (0, 1). Then we generated random atmospheric light A (m, m, m), where m is between 0.8 and 1. Finally, according to equation (1) and equation (2), we can obtain the hazy image I by utilizing the random depth map d and the random atmospheric light A. For transmission to be revealed better, we randomly collect 600 haze-free remote sensing images from the Google Earth for generating the training data. The procedure of preparing training samples is shown in Figure 2. The best learning result is that = 0.1217, = 0.9571, = 0.7806, = -0.54138, = 0.82178. Due to the fact that acquisition of these coefficients is based on the statistical characteristics of the training data, these coefficients ultimately represent the common characteristics of remote sensing hazy images. Through a large number of experiments on the real remote sensing hazy images, we found that these coefficients can be applied for a single remote sensing image to achieve thin haze removal and obtained good results. And we will give a discussion of the experiments in Section 3.
(a) Haze-free images
(b) Random depth map
(c) Artificial hazy images
2.5. Scene Radiance Restoration
Since the transmission map and the atmospheric light were known, the scene radiance can be recovered according to (1). However, the direct attenuation term can be very close to zero when the transmission is close to zero. That will bring a lot of noise. Inspired by Zhu et al.’s method [27], we restrict the transmission to an extent. Thus, the scene radiance restoration can be expressed as
3. Experiments and Evaluation
In this section, in order to verify the effectiveness of the method, multiple hazy visible images of several satellites data were tested. The performance of the proposed method is compared with DCP [23], HOT [10], Qin et al.’s [3], Liu et al.’s [4], and Multiscale Retinex [28] methods both qualitatively and quantitatively. All the experiments are carried out on the MatlabR2014a environment with a 2.5GHz PC and 4GB RAM.
3.1. Subjective and Comparative Evaluation
The transmission map results are compared between He’s original dark channel prior [23] results and our algorithm, as shown in Figure 3. As we can see, the transmission map obtained in this paper can capture sharp edge discontinuity points and outline the outline of the object, and halo and block artifacts have been effectively suppressed.
(a)
(b)
As shown in Figure 4, these hazy images data are collected from Google Earth and NASA Earth Observatory website. Figure 4(a) represents the input hazy images with uniform haze. Figure 4(b) represents the results of Multiscale Retinex method, which can achieve the effect of haze removal. But when the original image does not satisfy the gray-scale assumption, it will lead to color distortion. Figure 4(c) is the DCP method results; constrained by the inherent problem of dark channel priors, He’s algorithm cannot be applied to regions where brightness is similar to atmospheric light. Therefore, the DCP method is often unreliable when dealing with nonhomogeneous images. Figure 4(d) is the HOT method results; most of hazes have been removed, but the recovered image has a slight color distortion, and the resolution of the image will be lower than that of the original image. By observing the images in Figures 4(e) and 4(f), we can find that the results of Qin’s [3] and Liu’s [4] are blurred, along with color distortion. Figure 4(g) is our method results; as can be seen, the mist is effectively removed and the color fidelity is preserved well. What is more is that our method achieves a better visual effect.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
Figure 5 shows the comparison of dehazing results with the five state-of-the-art dehazing techniques [3, 4, 10, 23, 28] on remote sensing images with uneven haze distribution. To evaluate the availability of this method in dealing with uneven distribution haze, several representative images were selected. Compared with the results of Qin’s [3] and Liu’s [4], our method achieves better results in removing haze. Similarly, as can be seen from Figures 5(b) and 5(d), MSR [28] results and HOT [10] results have a common phenomenon of color distortion. Figure 5(c) shows He’s [23] results; the visual effect of the hazy images has been improved, but the color shift phenomenon still exists in the region with white objects. Furthermore, as shown in Figure 5(g), our algorithm has better performance in terms of haze removal ability, the overall contrast, and color fidelity. Nevertheless, our method is not suitable for thick haze removing.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
3.2. Objective and Comparative Evaluation
In order to objectively evaluate the algorithms, we select some classical evaluation indexes, including mean squared error (MSE), the ratio of new visible edges (), the gain of visibility level , the structural similarity (SSIM), the peak signal to noise ratio (PSNR), and the fog aware density evaluator (FADE). The MSE value can be utilized as a more convenient way to evaluate the degree of data variation [29]; a high value of MSE indicates that the haze removal algorithm is not effective while a low value of MSE represents that the recovered image is valuable. The value of evaluates the ability of haze removal method to recover the edges which are not visible in a hazy image. The value of represents the average ratio of gradient specifications before and after dehazing [30]. The high SSIM value indicates that the haze removal image is highly similar to the real image on the ground [27]. PSNR is calculated based on the error of the corresponding pixels, and the larger PSNR value indicates the slighter distortion [31]. Fog Aware Density Evaluator [32] (FADE) is a contrast descriptor which indicates the visibility of a foggy scene through measuring the statistical regularity deviation of hazy images and haze-free images. A low value of FADE implies better performance of visibility enhancement. The values of MSE, SSIM, PSNR, , , and FADE are listed in Table 1. Four images belonging to Figures 4 and 5 are selected for illustration. To sum up, our method achieves better results of MSE, FADE, , SSIM, PSNR, and . Therefore, the results of these experimental data indicate that our algorithm achieves better performance on contrast enhancement, visible edge enhancement, and haze removal.
When the linear coefficients are obtained, we display the time consumption comparison with DCP [23], HOT [10], Qin et al.’s [3], Liu et al.’s [4], and Multiscale Retinex [28] methods. As shown in Table 2, our research is faster than others.
4. Conclusion
In this paper, a novel and effective dehazing algorithm was developed to achieve single remote sensing image haze removal. A linear regression model with multiple variables is established and the gradient descent method is applied to the coefficients of the linear model. Then a hazy image accurate transmission map is obtained. In addition, we proposed a more valid method to estimate the atmospheric light, which can restrain the influence of highlight areas. Compared with the traditional methods, the experimental results demonstrate that the developed algorithm has a good performance in thin haze removal and color fidelity remaining. Both subjective evaluation and objective assessments indicate that the proposed method can recover a haze-free remote sensing image with good visual effect and high quality. Furthermore, our future research will turn to the removal of thick haze.
Data Availability
The data are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work is partially supported by the National Key R&D Program of China (Grants nos. 2016YFB0500100).