Abstract
Oil spill accidents have gradually increased due to the continuous development of marine transportation and petroleum processing industries. Monitoring and managing marine oil spills present important economic, social, and practical implications in preventing offshore oil pollution and maintaining ecological balance. Unmanned aerial vehicle (UAV) has become a suitable carrier for low-altitude oil spill detection because of their fast deployment and low cost. Thermal infrared remote sensing images are used as the research object in this study. A method around histogram of gradient (HOG) features combined with a support vector machine (SVM) is proposed for identifying oil spills at sea to improve the accuracy of offshore low-altitude oil spill recognition and realize all-weather monitoring of offshore oil spills in offshore waters. Steps for extracting HOG features and basic principles of the SVM classification are first investigated. Image preprocessing is then performed on collected thermal infrared image data to produce samples. HOG features of samples are extracted, and the radial basis function is selected as the kernel function for training the SVM classifier. HOG features of the infrared image to be tested are calculated and then sent to the classifier for identifying the oil spills. In addition, the proposed method is compared with the back propagation(BP) neural network method and local binary pattern (LBP) combined with the SVM classification method for analysis. The results show that the oil film recognition method based on the HOG feature and SVM has a recognition accuracy of 91.3% in the environment of small infrared oil film samples, which is significantly better than the BP and LBP-SVM recognition methods, and obtains a shorter training time. The method proposed in this study has obvious advantages in terms of small sample size and processing efficiency, can meet the requirements of all-weather inspection of oil film pollutants by UAV in offshore port areas, and has great application potential in the field of maritime supervision informatization in the future.
1. Introduction
Oil may leak into the sea during its mining, refining, water transportation, storage, and transportation, which cause damage to the ecological environment and even threaten the sustainable development of human society. Increased human activity around oceans, coasts, and offshore areas due to the continuous development of the marine economy has increased the risk of occurrence of marine oil spills. According to statistics from the International Tanker Owners Pollution Federation (ITOPFF), more than 240 oil spill accidents have occurred annually around the world in the past 40 years. Oil spills of more than 50 tons have occurred in waters of 112 countries worldwide during the same period [1]. Oil spill events severely damaged the ecological environment of the sea area and threaten the industrial and fishery production in coastal areas of the pollution source. The identification of marine oil spills presents essential economic and social practical implications for the monitoring and governance of oil spill pollution [2–5].
At present, developed countries generally use a combination of high-altitude satellite and low-altitude aviation detection for oil spill monitoring. Satellite remote sensing is utilized to monitor large-scale oil spills, while low-altitude aerial monitoring is applied to verify the monitoring results of satellite remote sensing and observe the oil spill area in detail. Unmanned aerial vehicle (UAV) monitoring is an effective tool for identifying offshore oil spills due to its long range, low cost-effectiveness ratio, flexible maneuverability, and high-definition cameras [6].
To date, many researchers have conducted research on marine oil spill detection. Fingas and Brown [7] outlined a variety of remote sensing methods which were used to detect offshore oil spills and pointed out that these technologies can be installed in aircrafts or satellites for the inspection of large areas. Radar detection of oil spills has been widely used in aviation patrol aircrafts, and synthetic aperture radar (SAR) can effectively distinguish oil, algae, and plankton [8–10]. However, a miniature and lightweight SAR which can be installed in small UAVs is still under development, and oil spill detection results remain untested [11]. Another detection method is based on fluorescence. The light source is usually an ultraviolet (UV) laser directed at the target when fluorescence is used [12]. Duan et al. [13] constructed a detection system for monitoring laser-induced fluorescence in an underwater environment by UAV. The researchers conducted a demonstration measurement of remote sensing at a fixed distance and recorded the amount of natural water marked with fluorescence, slick oil, and dye at a height of approximately 10 m. However, this technology is unsuitable for daytime detection. Fang et al. [14] demonstrated the different reflection coefficients of water and oil film in the UV wavelength range. Huang et al. [15] showed that the sensitivity of UV reflection images used for oil spill detection is remarkably higher than that of visible light. However, UV images are disturbed by many factors, such as sunlight, slick oil, and biological materials. Some algorithms, such as that proposed by Zhan et al. [16], can be used to reduce interference. Andrea et al. [17] utilized threshold technology in near-infrared spectroscopy (750–1000 nm) to detect oil spills although false negatives are prone to occur. Hou et al. [18] recorded reflection data in the visible wavelength range provided by multiple satellites, such as the seascape wide-field sensor radiometer of SeaStar satellite. The use of multispectral data is an advantage of recording images in the visible range. Dong et al. [19] showed that spectral indexes of hydrocarbons and seawater are complementary. The spectral index of hydrocarbons can be used to detect continuous true-color oil films while the spectral index of seawater is used to detect seawater. Jiao et al. [20] used an RGB camera installed on an UAV for detection and proposed a method to automatically detect oil spills. The study results showed this method is feasible although it cannot be used to detect oil spills in the dark. We demonstrated that thermal infrared image monitoring is suitable for detecting oil spills at sea by combining these technologies and specific requirements because the oil absorbs the light in the visible area and reradiates part of light in the thermal infrared spectrum. Oil will appear cooler than water given that it shows lower infrared emissivity. When the infrared camera is used to monitor the sea surface with oil film, the irradiated temperature can be displayed to determine the distribution of oil pollution on the sea surface [21]. Based on this principle, an oil spill detection method using an UAV thermal infrared image is proposed in this study to complete all-weather inspections of oil films in offshore waters.
In the recognition algorithm of oil spill remote sensing images, Brekke and Solberg [22] proposed a Bayesian classification scheme for the recognition algorithm of oil spill remote sensing images by combining prior knowledge. Fiscella et al. [23] used a linear discriminant analysis method based on Mahalanobis distance. Nirchio et al. [24] utilized multiple linear regression to classify oil spills. Topouzelis et al. [25] applied artificial neural networks to approximate the relationship between dark area feature and class label and realize the recognition of the oil film. However, additional conditions of the first three schemes are very harsh, and the recognition results are insufficiently accurate. ANN demonstrates low learning efficiency and slow convergence speed; its selection of related parameters based on experience results in insufficient recognition accuracy. Deep learning-based recognition methods have successfully solved some problems in remote sensing image recognition in recent years. These methods are suitable for recognition in complex environments and can successfully summarize characteristics of the recognition content [26–28]. However, the accuracy and rapidity of remote sensing based on deep learning completely depend on the quality and quantity of initial samples. If the number of samples is insufficient, then erroneous results can quickly occur. Dalal and Triggs [29] proposed the introduction of Histogram of Gradient (HOG) features into the field of pedestrian detection which has been widely used in the recognition of ships, automobiles, and traffic signs. HOG combined with the support vector machine (SVM) classification algorithm are applied to the offshore oil spill identification method in this study. An infrared camera is attached to a UAV to collect thermal infrared images rapidly and identify marine oil spills in the near seaport area accurately which can be good for the marine management department quickly making decisions on oil spill cleanup.
The rest of this paper is structured as follows. In Section 2, data sources for this study are presented. In Section 3, the technical route of this study is discussed. In Section 4, the design process of the experiment is introduced. In Section 5, the methods used in this study are described. In Section 6, the results of oil spill remote sensing image recognition and corresponding discussion are analyzed. Conclusions and future research direction are presented in Section 7.
2. Data Sources
We use UAV equipped with thermal infrared sensors to realize all-weather identification of marine oil spill accidents in the offshore area and detect oil spill images. The infrared sensor detects according to thermal radiation. The oil slick on the water surface absorbs solar radiation then emits part of radiation and solar radiation. Since the heat energy is mainly concentrated in the infrared hot region (8-14 μm), the emissivity of oil in the hot infrared area is lower than that of water, so the oil appears dark compared to the background water in the infrared images taken in polluted waters.
In order to take into account the influence of weather conditions on the observation results, the time of the observation experiment lasted from November 17, 2020, to November 27, 2020. We selected different weather conditions such as sunny days, cloudy days, and fair to cloudy days to continue multiple observation experiments in order to collect more information. As shown in Figure 1, the location of the oil spill observation experiment is located on the beach near Yugang Park in Zhanjiang City, China (110.42°E, 21.22°N). Every time we carry out an experiment, we place a large water tank on the beach to simulate an oil spill. After each experiment, we use linoleum to treat it to prevent pollution of the marine environment. Figure 2 shows the oil film sample of the oil spill aerial photography experiment carried out in the seaside tank at noon on a sunny day.
3. Technical Route
SVM classification method is suitable for remote sensing because of its satisfactory generalization ability. This method can also achieve satisfactory recognition even when training samples are limited. SVM classification method has been widely used in land cover identification research [30]. Feature vectors extracted from samples are used to train the model and recognize oil and nonoil films in the infrared image according to the training model in this study. Digital image processing, including filtering of speckle noise and image enhancement, is performed after obtaining a large number of infrared images through the UAV. Samples of oil and nonoil film images are then stored separately [31]. The technical route of oil spill identification implemented in this study is shown in Figure 3. Sorted samples are identified using the detection algorithm that combines HOG features and SVM. The identification process can be divided into the following modules: (1) The training module mainly includes extracting HOG features of stages. In this process, the cell unit takes pixels, and the block unit takes cells, and the gradient direction angle is divided by 9 in the process. The obtained HOG features are used for training the classifier template. (2) The detection module first calculates HOG characteristics of the sample to be tested and compares the SVM classifier sent to it with the template used in the detection. The recognition result is maintained, and the accuracy rate is analyzed after the oil films are identified. Parameters of the SVM classifier are debugged and selected to determine appropriate parameters.
4. Experimental Setup
DJI ROYAL 2 UAV was used to collect remote sensing images of the test area. The UAV can fly at a maximum speed of 6 m/s and 5 m/s under automatic flight and manual operation, respectively. It is equipped with a thermal imaging camera with a resolution of and a visible light camera with a resolution of 48 million pixels using a 1/2 inch CMOS sensor. It can perform spot and regional temperature measurement and supports one-click switching between visible, infrared, and split-screen images. In the aerial photography of images, UAV flight height was set as 10 m, ground sample distance (GSD) was set as 0.53 cm/pixel, the pin-top shooting angle was set as vertical ground, the heading overlap rate was 80%, and the side overlap rate was 70%. After the experiment, a batch of infrared remote sensing images of was obtained. Image data used in the experiment are infrared images obtained through single-lens reflection under sunny weather conditions. As shown in Figure 4, a large amount of image data was collected using UAVs after sprinkling waste lubricating oil in the experimental water tank to obtain multiple-angle photography and shooting. The data were repeatedly collected during the day and night.
Preprocessing of the collected image data, including image translation, flip, rotation, correction, and filtering, is required in the experiment to produce usable sample data. Image preprocessing is an important step in the identification of marine targets because the phenomenon of noise points and inconspicuous edge boundaries in remote sensing images will interfere with the subsequent oil spill identification process. Therefore, processing the image, eliminating noise, using edge enhancement algorithms to strengthen edges of the target and the background, and performing preliminary preparations for the subsequent target recognition are necessary. Image preprocessing is commonly used to improve the robustness of the SVM classification. We artificially create images using image data enhancement to match real-life conditions. Based on these principles, we first preprocess the collected thermal infrared images mainly for denoising, cropping, correction, and image enhancement. The visualization of a certain image data processing is shown in Figure 5.
(a) Original image
(b) Grayscale image
(c) Rotate the image by
(d) Rotate the image by
(e) Denoised image
(f) Enhanced image
5. Methods
Image feature extraction plays an important role in object recognition in remote sensing images, such as object surface defect detection, pattern recognition, and medical image analysis. In this study, a machine learning method was introduced to conduct remote sensing monitoring of thermal infrared oil spill images, considering factors such as the discrimination performance, computational complexity and antinoise, rotation, and sample complexity of remote sensing images [32].
5.1. HOG Feature Extraction
HOG feature is disscription operator used for target detection in image processing, which constructs feature by calculating and counting the HOG of the local area of image and can ignore changes caused by image geometry and optical changes. This HOG feature has good geometric and optical invariance [33]. The process of HOG feature extraction can be expressed as the following steps.
5.1.1. Standardization of Color Space
The entire image is subjected to gamma standardization processing. Gamma represents the compression coefficient, and its value is usually 0.5. This compression processing can quickly reduce the sensitivity of the algorithm to lighting and improve the stability of the algorithm to changes in illumination. The mathematical formula of gamma is expressed as:
5.1.2. Calculate Gradient Information
Gradient direction values of each pixel are obtained consecutively after gradient values of horizontal and vertical directions of the image are solved separately. The derivation operation can not only learn the shape and part of the texture information but also weaken the interference caused by the light again.
The gradient is defined under the vertical direction and horizontal direction operators as follows: where , , represent the pixel gray, the horizontal gradient value, and vertical gradient value, at the pixel in the input image degree value. The direction at this pixel and gradient magnitude is expressed as:
5.1.3. HOG of the Cell
The image is split into multiple cells of the same size, and cells are composed of several unit pixels. The gradient information of a pixel in cells is collected using a histogram of nine bins. Each direction of the block is 20°, all pixels in the cell are multiplied by the gradient amplitude to project the gradient direction in the histogram. The sum of their projections is obtained to realize the HOG of a cell [34].
5.1.4. Normalized Gradient Histogram within a Block
The block unit comprises several cells with possible overlapping between different blocks. The HOG feature of the local image area is composed of all its blocks. Normalizing within the block is necessary mainly to avoid local exposure and other factors that cause an excessive change in the gradient [35]. The normalization process is expressed as follows:
In the formula, represents the normalized result, represents the unnormalized feature vector of a block, and is an imported parameter with a small value to prevent the denominator from being equal to 0.
5.2. LBP Feature Extraction
Local binary pattern (LBP) is a characterization method of local texture features, which can effectively reflect the texture characteristics of images [36]. The LBP operator is that the pixels to be detected are the center of the circle, and the number of pixels in the neighborhood is the radius of the circle. A method of calculation is adopted to evenly select pixels on the circle, and then, the corresponding measurement method is adopted to describe the corresponding relation between the pixels in the center of the circle and the selected pixels on the circle. At this point, the value obtained by the LBP operator is LBP eigenvalue, which can reflect the texture information of the corresponding region [37].
The LBP operator is defined as:
In the formula, and represent the abscissa and ordinate of the point to be detected corresponding to the circle center, represents the gray value of the pixel to be measured, represents the sample pixel on the rim, is the gray value of the sample pixel on the circumference, and is the sign function:
Set the coordinates of selected pixel points as :
The above formula represents the circumference radius of the center of the circle of pixels to be measured, and the size is the number of pixels between the selected pixels and the center of the circle of pixels [38].
5.3. Classification Principle of SVM Algorithm
The core concept of SVM is to search a classification hyperplane in the feature space as a decision function, separate data sample points of different types, and maximize the interval between points of different categories to achieve the purpose of classification [39]. SVM is based on the principle of minimizing structural risks to avoid overfitting problems and achieve strong generalization ability.
The above method is feasible for linearly separable samples but cannot do anything for linearly inseparable samples. Choosing an appropriate kernel function can map the nonlinear sample to a high-dimensional area to solve the problem of linear indifferentiation of the sample. The mathematical relationship of the objective function is:
The essence of the kernel function is to search a function that makes the result of the operation in the low-dimensional space the same as the result of the inner product mapped to the high-dimensional space. In this way, the calculation in the high-dimensional space is successfully avoided and the final result is equivalent. So, the classification function can be defined as:
The precondition for defining the kernel function is the function is symmetric. Hence, , where is positive semidefinite. Using the radial basis function kernel RBF function as the kernel function, the advantage is that the local can affect the SVM. The RBF is expressed as follows: where is the width parameter of controlling the radial range.
5.4. BP Neural Network Classification Algorithm
The back propagation(BP) neural network is a multilayer feedforward network trained according to the error inverse propagation algorithm [40]. The transmission of its neurons is -type function, and the output is a continuous quantity between 0 and 1. It can realize any nonlinear mapping from input to output. By learning the case set with correct answers, it can automatically extract reasonable solving rules. Its learning rule is to use the gradient descent method, through back propagation to constantly adjust the weight and threshold of the network, to minimize the sum of squares of error of the network. In practice, through training and comparison of different neurons, a network model with an appropriate scale is determined based on minimizing the system size, shortening the system learning time, and reducing the system complexity [41].
5.5. Accuracy Evaluation
In order to visually verify the classification results of the oil film and obtain the accuracy of the classification results, the definition of the accuracy of oil film image classification is:
In the above formula, is the number of correctly classified samples and is the number of samples in the test set.
6. Results and Discussion
In order to prevent excessive repetition of sample data, samples were taken at intervals of two data sets, and poor-quality images were eliminated. After image processing, 250 infrared remote sensing images were finally obtained. Two hundred images with improved quality are selected as the training set, and some images are randomly sampled as test set from the sorted samples. Positive and negative sample data are produced after marking sample data. In this experiment, the numbers of positive samples and negative samples are both 100. Sample data are imported into the SVM classifier for training. Other test data can be identified after training. Relevant parameters of the SVM classifier are subjected to repeated experiments and tests after setting. Some test sample images are randomly selected for identification inspection to check the detection performance of the proposed method [42]. The identification results are shown in Figure 6.
(a)
(b)
The set of positive and negative samples in the training set is used while adjusting the width parameter of the RBF to 0.1, the classification model based on the RBF is obtained. Twenty oil film samples from the test sets introduced above are selected for identification research. The SVM classifier was used to identify 20 oil film samples, 18 were oil films, and 2 were nonoil films, with an accuracy of 90%. In addition, the extracted HOG feature vector is still used to train the support vector machine, and linear, polynomial, and RBF kernel functions are utilized to identify the test sample data sets. When RBF is selected as the kernel function, the recognition accuracy reaches 90%, which is higher than other types of kernel functions, and the detection speed is breakneck. Numerous experiments demonstrated that the thick oil slick becomes warmer while the thin oil slick appears colder than the nearby seawater in the daytime. This phenomenon is the opposite at night due to thermal infrared. The image distinguishes the oil on the sea surface through thermal comparison. Thermal contrast is produced because of the different emissivity of water and oil.
The accuracy of the model is verified by using the test set described above. The training set is expanded by mirror transformation, brightness change, and translation transformation in the sample data of the test set, and 100 pictures are randomly selected for classification testing. Figure 7 shows the confusion matrix of oil film classification results. Except for the percentage row and column in the graph, the ordinate of the confusion matrix is the real result, the abscissa is the predicted result, and the percentages on the bottom and right represent the recognition accuracy for a certain class of test set. The sum of the data not on the diagonal is the number of discrepancies between the actual value and the predicted value, and the percentage at the bottom right indicates the overall recognition accuracy. If the sum of the first column is 48, which is the number of oil film samples in the test set, it means that 45 are correctly classified as oil films, and the other 3 are nonoil films. The recognition accuracy of the method in this paper is 93.7%, and the error rate is 6.3%. The sum of the data on the diagonal is 92, which means that there are 92 oil film images predicted, so the recognition accuracy of 100 infrared image samples containing oil films in the test set is 92.0%.
In order to further analyze the characteristics of this algorithm and perform performance comparison analysis with other algorithms, two aspects of recognition accuracy and false detection rate are discussed. The BP neural network, LBP-SVM classification algorithm, and the proposed algorithm were used for calculation and comparative analysis. In the sample data of the test set, a batch of regions are randomly cropped from the images, and the test set is expanded to 150 images by image preprocessing. The BP neural network and LBP-SVM classification algorithm in this paper are used to identify test samples to test the applicability of various methods. As shown in Table 1, the recognition accuracy of BP and LBP methods on infrared oil film images is 84.7% and 86.7%, respectively, while the recognition and classification accuracy of the method proposed in this paper is 91.3%. The results of the comparison show that the recognition accuracy rate of the proposed method is higher than the BP algorithm and LBP-SVM classification algorithm. The proposed algorithm improves the detection rate and reduces the false detection rate compared with BP and LBP-SVM algorithms. Repeated tests and experiments demonstrated that the proposed algorithm of this study improves the accuracy rate of oil spill recognition while satisfying the basic real-time recognition and presents an ideal target detection effect. Compared with the deep learning algorithms in the field of remote sensing, the method in this paper does not have high requirements on the quantity of sample data, the training time of the samples is fast, and the computing power of the hardware is not high. Therefore, it has huge application potential in the field of maritime supervision informatization in the future and is suitable for remote sensing monitoring of oil film pollution in offshore waters in combination with unmanned equipment.
7. Conclusions
A new oil spill detection method that uses both UAV and infrared camera is proposed in this study. This detection method can realize real-time and efficient detection of small oil spills in offshore areas, such as harbors, which may be ignored by port authorities. Early detection is essential in the effective cleanup of oil spill events. Compared with other observation methods, such as manned aircrafts, satellites, or maritime patrol ships, the proposed method presents low cost and shorter deployment time and can achieve all-weather detection in a small area, thereby saving on manpower while facilitating night patrols and ensuring the safety of personnel. However, it should be pointed out that the monitoring of offshore oil spills by UAV is greatly affected by weather and sea conditions, and it is not suitable for flying UAV in windy and heavy rain weather. Meanwhile, the endurance of UAV is limited, so multiple UAVs need to cooperate in all-weather monitoring.
Aimed at the problem of the accuracy rate of offshore oil spill identification, this paper proposes an oil spill identification method based on HOG-SVM algorithm which chooses RBF as the kernel function and realizes efficient detection of offshore oil spills. Compared with the recognition method of BP and LBP-SVM, the new method achieves higher accuracy and rapidity. Repeated experiments and tests exhibited the fast training speed, high accuracy, and satisfactory generalization ability of the model. Rapid all-weather detection can be realized when an oil spill accident occurs in an offshore port area.
Future study plans include the following aspects: (1) Images will be collected over a long time and in more weather conditions to improve the stability of classification model. A massive number of sample sets are collected in a long time can be used for offshore oil spill identification research based on deep learning algorithms. (2) Experimental conditions will be further improved in subsequent investigations. The experiment carried out in this study simulates an oil spill accident that occurs under calm sea conditions in a water tank by the beach without considering sea conditions, such as actual waves and currents. (3) Offshore oil spill detection with the help of UAV is seriously affected by meteorological conditions. Blurring in image data due to the presence of sea fog is not conducive to detecting oil spill accidents. Our follow-up investigation will focus on conducting a sea fog removal study on the image data collected by UAV.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
No conflicts of interest are reported regarding the present study.
Acknowledgments
This study was supported by the Fujian Provincial Science and Technology Planning Project (2020H0018 and 2021H0020) and the Zhanjiang City Science and Technology Development Special Fund Competitive Allocation Project (No. 2021A05034).