Abstract
Images taken by UAVs have shadows due to terrain factors. The image pixel brightness of the shadow areas is compressed, and the information is deficient, which impacts the recognition of image information and thus limits the subsequent image application. Therefore, the shadow removal of the image is crucial. Image enhancement algorithm is capable of improving the whole and partial contrasts of images, highlighting detail information, and removing shadows. Three classical optical image enhancement algorithms are analyzed. The analysis results show that image would be enhanced excessively after the histogram equalization algorithm to the shadow image enhancement. The pixel brightness are compressed by the Mask homogenization algorithm enhancement and uneven brightness in some areas after the enhancement of the traditional Retinex algorithm. Using the Retinex enhancement algorithm, this study proposes a combination algorithm to remove the shadow of the UAV remote sensing image. The proposed algorithm integrates the Retinex algorithm with the two-dimensional (2D) gamma function to remove the brightness colour of the UAV image, so it is capable of removing the shadow area of the UAV image and correcting the uneven darkness attributed to the image enhancement. The acquired UAV image is used to perform the experiment, and it is integrated with the LOG algorithm to extract the enhanced image features. As indicated by the experimental results, the integrated algorithm is proved with better performance to remove the UAV image shadow. The shadow areas of the features cannot be extracted in the original image, but after using the new algorithm to remove the shadow, the ground edge features can be clearly extracted.
1. Introduction
Due to terrain factors, the image acquired by a UAV flying at low altitudes will have shadow areas. The pixel brightness of the shadow areas will be compressed, and the information will be lost, which will limit the subsequent application of the image. Image enhancement aims to highlight the useful information of images and eliminate or weaken the interference information. After image enhancement, the previous unclear or interesting features required to be highlighted are enhanced, and the image quality is improved, which is more consistent with the requirements of the visual requirements and image analysis and processing [1–3].
The spatial domain method and the frequency domain method are common image enhancement methods. Spatial image enhancement follows direct operation, which processes pixels for enhancement directly. After the enhancement, the gray level is distributed evenly in the image, thereby expanding the image contrast. Using template and image convolution operation will cause some features to be suppressed, or prominent, which enhances the visual effect of the image. Frequency domain image enhancement is considered a type of indirect operation. Before image enhancement, images should be transformed into the frequency domain space for filtering, and the enhanced image is obtained after inverse transform procession of frequency information. As the image edge feature belongs to high-frequency information and the image background pertains to low-frequency information, high-pass filtering or low-pass filtering can be exploited to sharpen the enhanced image.
In 1971, Land and Mc proposed an adaptive Retinex enhancement algorithm from three aspects (i.e., colour balance, edge area enhancement, and gray level change), and it has been extensively used in image enhancement [4]. Professor Pal et al. of India built four complex nonlinear functions with parameters and simulated the corresponding typical nonlinear mapping [5]. In accordance with the optimization criteria, 12 parameters were adjusted adaptively. As a result, the fuzzy image was enhanced, and the satisfactory results were achieved. Uncertainty, complexity, and instability are considered the characteristics of images. Anzueto-Rios et al. applied fuzzy set theory for image enhancement and achieved good results [6, 7]. Gu et al. used the Retinex algorithm for image enhancement processing to improve the recognition degree of image details [8]. This method is capable of estimating the brightness and reflectivity of the image and performing enhancement processing directly on the image, which can achieve a relatively ideal effect. Kou et al. proposed a gradient-domain guided filtering algorithm using deep convolutional network for image restoration and image superresolution enhancement and achieved effective results [9]. Liu et al. used adaptive segmentation to correct the gray scale inhomogeneity of the image [10]. Xiong et al. combined different image enhancement algorithms for adaptive parameters adjustment and then applied the adjusted parameters to different linear enhancement areas, which led to significant results [11]. Jiang et al. decomposed the original image into images in different regions and then distributed different radiation coefficients to the regions, so the illumination compensation in image enhancement could be solved [12]. Hu et al. proposed an algorithm for image brightness correction using the bilateral gamma function [13]. Mao et al. developed an adaptive bidirectional logarithmic change image enhancement algorithm [14]. Yu et al. built a defogging degradation model to enhance the shadow image. The built model could improve the brightness and visual effect of the image, whereas it could not effectively suppress the noise influence in darker areas [15]. Song et al. decomposed the image frequency and divided the image into high-frequency images and low-frequency images [16]. The Retinex algorithm was adopted to enhance the low-frequency band, which can improve the effect of insufficient illumination on the image, whereas this method cannot display the overall details of the image. Han optimized the existing gamma correction function and developed an adaptive gamma algorithm for image enhancement processing of panoramic images under low illumination. The developed algorithm is capable of improving the brightness of the image, suppressing the brighter areas in the image, and improving the detailed features of the image area [17]. Zhang et al. proposed a multiscale Retinex algorithm for colour protection and image enhancement. However, after the image was enhanced, the image illumination was not uniform, and the brightness area turned out to be brighter, while the dark area was darker [18]. Li et al. adopted multiscale gradient domain-guided filter brightness enhancement algorithm and histogram adaptive brightness correction algorithm, which can be more suitable for colour image enhancement under weak light source [19].
The shadow removal method of UAV images in mountainous areas is investigated in this study. The identification of ground information in this shaded region is prone to error due to the occlusion of the mountain, the shadow of the UAV image, as well as the pixel brightness compressed and information gap of the image in the shadow areas during the image acquisition process for the drone. The UAV images were enhanced using the histogram equalization algorithm, the Retinex algorithm, and the Mask uniform light algorithm. Subsequently, the experimental results of the classical algorithm analysis were analysed and then compared using qualitative and quantitative methods. The shadow on the image could be removed using the Retinex algorithm, whereas the removal results were limited by partial problems (e.g., the uneven light and dark distribution of image, colour aberration phenomenon, and the image texture distorted to affect the visual effect). The Retinex algorithm was optimized to convert the RGB colour of the drone images to the HSV colour, and the optimized Retinex algorithm was used to enhance the brightness area after colour conversion and improve the image quality. The optimized Retinex algorithm can have high performance in the image shadow area removed, better image texture characteristics preserved, and the phenomenon of uneven image light and shade eliminated.
In the rest of this study, four general sections form the body of this study. In Section 2, the comparison of the UAV image enhancement algorithm is described. In Section 3, the employed intelligent methods are presented. Afterward, in Section 4, the obtained results are presented and discussed. Lastly, Section 5 is the conclusion giving a brief report of the results of this study.
2. Comparison of UAV Image Enhancement Methods
2.1. Comparative Analysis of Traditional Enhancement Algorithms
The grayscale frequency and brightness range of pixels in an image can be reacted by a histogram. Histogram equalization is performed using the cumulative function to correct the gray scale values, distribute the grayscale values uniformly, and enhance the image through nonlinear stretching. The pixel values are again matched for the stretched images. Thus, after the redistribution, the difference between the pixel values is not sharp, i.e., the original image is transformed into a well-distributed histogram by equalization the histogram [20].
The gray scale of an image can be considered a random variable of the interval [0,1], which can be represented using a probability density function. Assuming that the gray scale value of the image element is , the pixel gray level after the transformation is , and , are denoted as the probability density function of the random variable and , respectively, and the transformation function is ; the equation is written as [21]:
A gray level is generated for the respective transformed image element gray level on the original image. The transformation function satisfies with that: (1)It is single-valued and monotonically increasing in the interval(2)When , to ensure that the output gray scale has the same range as the input gray scale, it yields:
Substituting Equation (3) into Equation (1), it yields:
A continuous function transformation equation is presented above. When the digital image processing is used, if the digital image grayscale is ordered: where ; represents the image gray scale; is the total number of pixels; is the number of pixels in the th grayscale layer; is probability density in the th grayscale layer; is the pixels state function in the th grayscale layer; is the final transformation result.
Retinex theory is a model first proposed by Edwin Land on how the human visual system regulates the colour and brightness of perceived objects, Retinex (abbreviation for Retina and Cortex) [4]. The Retinex algorithm can be balanced in the grayscale dynamic range compression, edge enhancement, and colour constant qualitative. It can achieve a balance in three aspects, i.e., gray scale dynamic range compression, edge enhancement, and colour identity, so it can adapt various images enhancement automatically. In essence, the Retinex algorithm is an image enhancement algorithm based on lighting compensation. Currently, the SSR (single-scale Retinex) algorithm and MSR (multiscale Retinex) algorithm have widespread applications for light compensation for close-range shooting images. Moreover, it also plays a good dodging effect for the long-distance remote sensing image, especially for the coloured images, which can maintain the colour information of the image while removing uniform brightness. The image can comprise luminance components and reflection volume, and the imaging model can be represented as [22]: where is denoted as an image; represents the reflected light component. In addition, the magnitude of illumination intensity does not affect the reflected light component. denotes the incident light component, which determines the image gray-scale dynamic range. Thus, according to the principle, if the luminance component affected by outdoor light intensity in the image can be estimated, followed by the removal of the luminance component. The final result is the reflected light component of the object with the object’s own reflection ability in the algorithm for image enhancement. The Retinex algorithm process can fall into the steps below.
The 1st step: images are represented in Equation (6), where is the incident ray component and is the reflected light component.
The 2nd step: the image is converted to a log domain.
The 3rd step: is obtained by filtering processing .
The 4th step: according to the above equation, the original image minus the incident light component to obtain the reflected light component .
The 5th step: to take the antilog of and finally get the enhanced image.
The Mask algorithm is a common dodging method, mainly used for analysing the image from the frequency domain. However, it is considered that the intensity change is relatively slow, so the information reflecting the brightness change trend should be positioned in the low frequency part of the image. It is considered the interference part of the image if the part only reflects the changes in brightness without excessive image information. Instead, the information that reflects the scene reflection properties characteristics lies in the high frequency part unaffected by light, and it is the image with uniform brightness [23].
In Equation (10), where images with uneven illumination are , the image after the dodging treatment is , expressing the background images during processing as . The original images and the acquired noise images are subtracted. Subsequently, the gray scale offset is added to the light equilibrium result. It can be expressed as: where the is constant since images processing aims to maintain the average brightness of the original image, so the average brightness of the original image is usually taken as . The contrast ratio of the image after the dodging treatment should be adjusted to improve the brightness value of the image.
The UAV is used to obtain the images of Longtoushan Town, Ludian County. On that basis, the enhanced effect on the UAV image of the above four methods is analysed. To facilitate the operation, the image size of the UAV is adjusted, with the adjusted size of . Figure 1 illustrates the original image of the UAV. The histogram equalization enhancement algorithm, the Retinex enhancement algorithm, and the Mask uniform light enhancement algorithm are adopted to enhance the image, respectively, with the results presented in Figures 2–4.
According to the visual results, the image quality after enhancement is significantly improved, with buildings clearly visible, and the outline of the target object is defined clearly. Houses, roads, slope, and vegetation can be clearly displayed.
2.2. UAV Image Quality Evaluation Index
On the whole, the UAV image quality evaluation falls into two parts, i.e., subjective evaluation and objective evaluation. Subjective qualitative evaluation is also known as visual evaluation. In accordance with previous scales and evaluation standards and combined with existing experience, the quality of both images before and after processing is analysed, and the conclusions are drawn. The evaluation method primarily complies with experience, and it has low accuracy since different images and standards may produce different evaluation results.
Objective evaluation is evaluated using the mathematical model. Some properties of the image are quantitatively evaluated by quantifying the index factors and adopting some mathematical models. It is of important significance for image evaluation to evaluate the changes of these properties before and after image processing and build an evaluation system.
In brief, using subjective and objective quality evaluation comprehensively can evaluate the quality of image processing scientifically and reasonably. Furthermore, there are numerous objective evaluation indicators of image quality, in which variance, information entropy, and mean gradient are commonly used.
2.2.1. Variance
The number of image details can be compared by the gray variance, reflecting the discrete of the relative gray average of each image to a certain extent, which can be used to evaluate the size of image information volume. Scattered image gray scale distribution, large image contrast, and more information can be seen under a large variance. Conversely, small variance means low contrast and less image details. Therefore, the higher the image variance in the image comparison, the richer gray scale level, the higher the image quality will be, and vice versa. where represents the size of the image; is the pixel value of the selected image; represents the average value of image pixel.
2.2.2. Average Gradient
The image definition is mainly affected by weather conditions, UAV flight status, camera parameter setting, and other factors. The evaluation of the image definition is critical. Image definition is largely indicated in image blur, poor saturation, and tone difference. The common definition evaluation index is the average gradient, which can reflect subtle contrast changes in the image. The higher the average gradient, the better the image definition will be. In contrast, the smaller the average gradient, the worse the image definition will be. The mean gradient is formulated as:
In Equation (13), the gray value of the line and the list is , and the size of the image is .
2.2.3. Information Entropy
Information entropy, a measure of the amount of information in the image, is proportional to the amount of information contained in the image. The higher the information entropy, the more information the image will contain, i.e., the more detailed the image will be. Moreover, the smaller the information entropy, the less information the image will contain. The information entropy is calculated as:
In Equation (14), the gray scale of image is num, and probability of the gray level occurrence is .
Based on the quantitative analysis, the average gradient value, variance, and information entropy are applied for image enhancement comparison [24]. The comparison results are listed in Table 1.
According to Table 1, the original image is compressed due to insufficient exposure, and the average gradient, variance, and information entropy of the image are relatively low. Three indicators have been significantly improved using the three kinds of classic enhancement algorithm for image enhancement. The analysis results show that the histogram equalization algorithm shows excessive enhancement after the shadow image enhancement and pixel brightness compression after the Mask homogenization algorithm enhancement; after the enhancement, value, IE value, and MG of Retinex algorithm are higher than the others in three methods, thereby proving that the Retinex algorithm is the optimal. In the enhanced image, the ground objects and the texture of the landslide reinforcement and buildings can be seen clearly, and the vegetation texture on the mountain is relatively clear compared with that of the other three algorithms. According to the mentioned experiments, the Retinex algorithm is capable of achieving a better enhancement effect for the underexposed images.
The edge feature of the image contains considerable information regarding the image, and it can significantly indicate the reaction image clarity. The LOG (Gauss-Laplace Algorithm) operator comprises the Laplace edge detection algorithm and the Gaussian filtering algorithm, in which the Lapura operator primarily aims to highlight the edge and contour parts of the gray scale in the image and reduce the areas with slow changes in gray scale, and the Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise, with wide applications in image denoising. The Gaussian filtering function is defined as:
Convolution operations are performed between and , and is the smoothed image:
The Laplace operation is performed on the smooth images.
The LOG operator takes the form of . Edge feature points are the zero intersection points between the template and the image convolution operations. The LOG template, with the typical size of , is presented as:
The LOG operator is used to extract the edge features of the ground objects after the optimized Retinex algorithm. Moreover, it is compared with the original image, and the result of the ground objects edge feature extraction is presented in Figures 5 and 6.
According to Figure 5, the edge features of the original image are extracted poorly, and a large area of empty appears in the mountain. In Figure 6, the Retinex algorithm can be applied for image enhancement, which can enhance the underexposure image under weak light source. This algorithm improves the image features recognition degree and extracts abundant edge features. Furthermore, considerable edge information is extracted from the mountain.
3. Optimized Retinex Algorithm for Shadow Removal
According to the previous experiment, the Retinex algorithm has the optimal enhancement, and this algorithm is adopted to remove the shadow area of the image. The original image is presented in Figure 7. The Retinex algorithm is used to remove the shadow region, and the result is shown in Figure 8.
According to Figure 8, if the algorithm is adopted to remove the shadow area, the shaded part of the image will be relatively clearer, whereas the image appears the phenomenon of halo artifacts. The bright area increases excessively, and the dark area enhancement is insufficient. The whole image appears uneven distributions of light and shade, colour distortion, and image texture distortion, which will also affect the after processing.
To solve the problems above, the original Retinex algorithm is improved, which converts the enhanced RGB colour into the HSV space for processing. This algorithm uses the existing 2D gamma algorithm for brightness image processing. Subsequently, reverse operation is performed for the corrected image to restore the UAV image.
3.1. HSV Colour Space
It is difficult to ensure that each colour channel is enhanced or attenuated in the same proportion if the image enhancement algorithm is used to correct the colours in the three RGB channels, which leads to the colour distortion after enhancement. The calculation of the three channels simultaneously requires complex calculation. To perform targeted image brightness correction, this study chooses to correct colour images with uneven light in HSV colour space [25].
HSV model can reflect colour and saturation and conduct clustering calculation for various colours. It also can extract gray level and brightness information from the colours clustering, eliminate the effect of brightness and colour separation, and make the program more robust and have a better recognition effect than the RGB model. Chromaticity and saturation can reflect the colour type accurately, which are not very sensitive to the change of external lighting conditions. The colour conversion from RGB to HSV is nonlinear [26].
In Equations (19) and (20) above, the gray values of the three primary colours of the image are , , and , and the hue, saturation, and value of the image are , , and .
3.2. Two-Dimensional Gamma Function
To achieve uniform illumination image, 2D gamma function is used for colour correction, and parameters are adjusted according to the distribution characteristics of illumination components. For the input image, a 2D gamma function is constructed based on the extracted light component, and its expression is stated as follows [27].
In Equation (21), the brightness value of the corrected output image is , the parameter of brightness enhancement is , and is the average brightness value of the illumination component.
When the light value of a pixel is lower than the average value, the 2D gamma function operation can improve the brightness value of the pixel. If the light value at a point is 64, and the brightness value of the input image is 120, the corrected brightness value of the output image will be 149. Thus, the output image performance is improved when the original image illumination is too low. When the light of a pixel value is higher than the average, the brightness value at a certain point will be 120. Assuming that the light value at this point is 192, and the brightness value of the corrected output image is 108. As a result, the brightness of the image decreases when the light in the original image is too high.
4. Experiments and Analyses
We used the DJI Phantom 4 Pro UAV to capture images since its flight platform is more stable than the others and easy to operate. Its take-off weight was at 1388 g and had a maximum flight time of nearly 23 min. 1-inch CMOS effective pixels were 20 million, and its image resolution was up to . Moreover, the lens focus distance was 35 mm with the autofocus f/2.8-f/11 aperture. It had a maximum rise speed and a maximum flight height of 6 m/s and of 6000 m, respectively, as well as a GPS/GLONASS dual satellite positioning mode. The image of the acquired UAV was shaded by the occlusion of the mountain during the UAV flight.
The UAV image was loaded (Figure 9). First, the Retinex algorithm was used for image enhancement, and the results are presented in Figure 10. In the enhanced UAV image, the shadow region was significantly improved. The brightness of the image in the shadow region was significantly improved, and the object in the shadow region could be clearly identified. However, the enhanced image underwent texture distortion and colour distortion.
After the image enhancement, the colour transformation was performed, which converted the RGB colour to the HSV component. The image after colour correction of the inverse operation synthesized into a new image (Figure 11). After the 2D images underwent function correction, the brightness was uniform, and texture feature of the image was clear.
The gauss-Laplace operator was adopted to extract the edge features of the ground object surface after the correction of the image brightness. Moreover, the edge features of the original image were compared with those of the original image. The original image edge features are presented in Figure 12, and the corrected image edge features are illustrated in Figure 13.
According to Figure 12, no shadow existed in the original image, the texture was rich, and the edge features were relatively obvious. In the areas with shadows, there was emptiness in the extracted edge features, which revealed that the edge features were unclear and that the target object contour could not be extracted. According to Figure 13, the corrected UAV image could extract more obvious edge features from the original shadow area, which demonstrated that the method significantly impacted the removal of shadow areas and the correction of brightness.
5. Conclusions
When the UAV acquires the image, the acquired UAV image has underexposure and shadows due to the blocking of mountains and the influence of light. The image brightness value of the shaded region is compressed, and the information is lost. Vital information is contained in the image of the shadow region. To remove the shadow region of the image, an optimized Retinex algorithm is used to process the shadow. First, the Retinex algorithm is used for image enhancement, whereas the brightness of the image after enhancement is uneven, which reduces the quality of the image. To solve the problem above, the image can be converted from RGB colour to HSV space. The brightness of the HSV colour space is corrected by 2D gamma function, and the correction image receives inverse operation. Combined with the actual experiment for the UAV image, the optimized Retinex algorithm has a better effect in shadow area removal. After the shadow removal, the ground feature details of the shadow area of the optical image can be clearly identified, and the image quality improved. The algorithm works better on areas with lighter shadows, but less so with darker shadows. It is also the direction to be studied in the future.
Data Availability
(1).The (data type) data used to support the findings of this study are included within the article. (2) The (data type) data used to support the findings of this study are included within the “Figures information file(s)”.
Conflicts of Interest
The authors declared that there is no conflict of interest in presenting this manuscript.
Acknowledgments
An amended opinion was presented by Arun Kumar Sangaiah. We acknowledge the support provided. We would like to express our gratitude to the reviewers for their constructive comments on improving the quality of this study. This work was supported by the Yunnan Provincial Science and Technology Department Fund (202001AT070059).