Abstract

Floods are one of the main natural disasters, which cause huge damage to property, infrastructure, and economic losses every year. There is a need to develop an approach that could instantly detect flooded extent. Satellite remote sensing has been useful in emergency responses; however, with significant weakness due to long revisit period and unavailability during rainy/cloudy weather conditions. In recent years, unmanned aerial vehicle (UAV) systems have been widely used, especially in the fields of disaster monitoring and complex environments. This study employs deep learning models to develop an automated detection of flooded buildings with UAV aerial images. The method was explored in a case study for the Kangshan levee of Poyang Lake. Experimental results show that the inundation for the focal buildings and vegetation can be detected from the images with 88% and 85% accuracy, respectively. And further, we can estimate the buildings’ inundation area according to the UAV images and flight parameters. The result of this study shows promising value of the accuracy and timely visualization of the spatial distribution of inundation at the object level for the end users from flood emergency response sector.

1. Introduction

Floods are the most frequently occurring and damaging natural disasters in the world. The average annual deaths and economic losses caused by floods are 1354.9 people and US $32.847 billion in China, ranked 4th and 2nd in the world, respectively [1]. Traditional flood monitoring uses hydrological processes simulation with precipitation data from surface hydrological stations, satellite observations, reanalysis data based on numerical models, and assimilation. These data have certain limitations in terms of temporal resolution, spatial resolution, and accuracy [2]. This signifies the need to quickly detect flood-affected areas with advanced technologies so that efficiently rescue activities can be initiated as soon as possible.

Satellite remote-sensing techniques have been widely used when nature disasters occurred. The main advantages of satellite data are its time and cost effectiveness, since satellite data of large areas can be gathered quickly and economically [3, 4]. The application of space borne remote sensing in flood detection varies across the large variety of spatial and temporal scales, as well as multisensor, operating in several platforms [5]. Numerous studies have been carried out for flood monitoring, inundation mapping, and loss assessment. However, the quality of satellite images is greatly affected by weather conditions such as clouds’ cover and shadow effect. Another important consideration is the long revisit time of most satellites that might be useful for long-term and large-scale floods monitoring [6].

In recent years, the UAV remote-sensing system has been used in disaster detection due to the advantages of real-time data acquisition and full-time observation [7]. This makes UAVs to be an efficient way to investigate high-risk areas that are unreachable by humans during disaster events. Flood rescue agencies can use UAV to quickly collect data and visualize flooded regions and grasp the state of the disaster. Some studies have investigated the applications of the UAV system for flood hazard modeling [810]. Lei used UAV remote sensing to investigate and monitor the severe cold rain, snow, and freezing disasters in southern of China in 2008 [11]. Schumann et al. evaluated the accuracy of a UAV-derived Digital Elevation Model (DEM) and assessed its reliability on flood mapping operation [12]. Annis et al. demonstrated the performances of the UAV data in simulating flood extension and depth [13]. Hashemi-Beni et al. adopted the UAV system for spatial flood assessment mapping and evaluating the extent of a flood event [14]. Most of the related works focus on the large-scale flood model integrated with DEM and GIS data. However, when flood occurred, transferring the trapped people in time is the focus of rescue work. Detection of submerged buildings would provide timely information for rescue efforts. To our knowledge, the applications of UAV to retrieve flood information at the object level are usually less investigated.

Flooded object recognition such as the buildings, crops, and infrastructures from aerial images can be analyzed to make disaster response decisions. UAV can obtain a large amount of image data when flood occurred. Remondino, Han, and other scholars have carried out research in flood data acquisition and disaster assessment using UAV remote-sensing technology [1519]. The pertinent literature shows that image processing such as edge detection, segmentation, and machine learning have been widely utilized for flood detection, but deep learning techniques are rare and not well investigated with for this purposes [20].

The images acquired with UAV have the characteristics of small picture format and image distortion, which increase the difficulty in image processing and information extraction [21]. In recent years, major breakthroughs have been made in the field of computer vision based on deep learning technology. In 2012, Hinton proposed a deep convolutional neural network algorithm and won the ImageNet competition champion in the field of computer vision [22]. Since then, companies such as Microsoft, Facebook, Google, and Baidu have successfully applied deep learning in image recognition and voice identification. In flood research, Chang utilized artificial neural network (ANN) to create a flood inundation forecast model [23]. Abbot optimized rainfall forecasting using ANNs [24]. Jiménez-Jiménez proposed an object-based approach for flood damage assessment [25].

This study proposes a method to detect flooded buildings and vegetation by integrating deep learning and UAV image processing. We use the YOLOv3 algorithm as a deep learning model on aerial images and estimate the area of submerged buildings through the relationship among the UAV flight parameters. The case study is the flood area of the Kangshan Levee of Poyang Lake. This study aims to estimate the affected area and potential trapped people by detecting the inundated buildings, analyze the disaster intensity, and provide real-time data for flood rescue departments to make decision.

The study is organized as follows. Section 2 introduces the study area, the Kangshan levee of Poyang Lake. Section 3 focusses on the research methodology, and the process of image acquisition is also elaborated. Section 4 illustrates data processing and experimental results. Finally, the overall achievements and limitations of the proposed approach are presented. The main contribution of this study is to combine UAV image data and deep learning algorithm for flood detection at the object level and provide timely visualization of the spatial distribution of inundation for flood emergency response sector.

2. Study Area

Poyang Lake is located at the north of Jiangxi Province, China. It is the largest freshwater lake in China and one of the main tributaries of the middle and lower Yangtze River. The lake is about 3,150 square kilometers, supplied by the Ganjiang River, Xiuhe River, Xinjiang River, Raohe River, and other water sources. It flows into the Yangtze River in Jiujiang City from south to north. The geographical location of Poyang Lake is shown in Figure 1.

In late June 2020, most of Jiangxi Province continued to rain heavily. Affected by heavy rainfall and upstream water, the total rainfall in northern Jiangxi was more than three times that of normal years. The area of the main body of Poyang Lake expanded by 352 square kilometers on July 8 compared with that on July 2. On July 14, the area expanded to 4,403 square kilometers, which was 2.5% larger than the historical average (3,510 square kilometers) during the same time period. The flood disaster has affected 499,000 people in 36 counties in Jiangxi. The flood control pressure of the local government is increasing along with the flooding area of Poyang Lake.

From July 2 to July 8, 2020, high-resolution satellite images with a spatial resolution of 10 meters were used for remote sensing monitoring by National Satellite Meteorological Center. The flooded areas of Poyang Lake are shown in Figure 2. The blue in the figure indicates the unchanged water body, the red indicates the expanded water body, and the green box is the study area, which is located at Kangshan Dike of Poyang Lake.

3. Data and Methodology

3.1. Data Acquisition

In response to the emergency floods in the Poyang Lake area, the aerial images were captured by Feima F200 UAV system. A total of 10 sorties were flown, covering an area of more than 50 square kilometers in total, and more than 3000 images were captured by the UAV platform. The forward overlap of the aerial image is 70%, and the lateral overlap is 65%.

The F200 UAV platform is a light fixed-wing UAV with a wingspan of 1.9 meters and a maximum flight time of 1.5 hours. The system has high stability and can repeat high-precision flight operations. It can be thrown off by hand, landed by parachute, easy to control, and can be operated by one person. Due to the limited load of the UAV, the size and weight of the sensor are limited. The UAV carries a miniature Sony ILCE-5100 camera. Its effective pixels are about 24.3 million, and the focal length is 20 mm. Figure 3 shows part of the trajectory planning of the F200 UAV equipped with a Sony ILCE-5100 camera to obtain flood image data. Table 1 shows the parameters of the Sony camera.

3.2. Methodology

This study uses the UAV image data and deep learning method to conduct flood automatic detection and estimation. The proposed methodology is shown in Figure 4. The methodology consists of three main steps.

3.2.1. Flood Image Dataset Construction

The purpose is to extract buildings and vegetation submerged by floods. Therefore, the flood images obtained by the UAV system need to be filtered out which contains buildings and vegetation submerged scenes. Since the forward and lateral overlap of the acquired images by UAV is as high as 65% to 75% of a single image and a large number of images are completely flooded areas, about 600 images have been screened as flood image samples for training and testing. We choose 500 images as the training data and 100 images as the test data. The dataset sample is shown in Figure 5 (Section 4).

The construction of the training dataset includes the collection of flood images and object labeling. At present, there are still no public datasets consist of aerial flood images, and image search engines such as Google and Baidu are used to collect aerial flood images. Totally, there are about 500 aerial flood images collected through the Internet, together with the images obtained by the UAV system, which form a training dataset. After that, the training samples are labeled and input into the deep learning model for feature learning.

3.2.2. Deep Learning Model

The research conducts the object detection based on YOLOv3 algorithm with Tensorflow framework. The model learns the characteristics of the buildings and vegetation inundated by flood through data labeling and training. Then, the inundated objects could be extracted using the trained model.

The deep learning framework Tensorflow has very powerful versatility. This study chooses Tensorflow as the operating environment. The experimental platform chooses the YOLOv3 algorithm to conduct object detection. YOLOv3 is a new peak in target recognition after the emergence of R-CNN series models. The object detection method with YOLOv3 [26] is shown in Figure 6.

The YOLOv3 algorithm divides the input images into S∗S grids; for each object and grid, it calculates the probability that the center of the objects falls within the grids. If the probability exceeds a threshold value, it is determined that there is an object in the grid. The boundary boxes are built for the grids with objects, and the confidence level of each box is computed simultaneously. Each bounding box contains five parameters: center of bounding box relative to the bounds of the tile (x, y), the width and height related to the entire image (w, h), and confidence level.

The output of each grid includes the position information, the confidence level of the boundary box, and the class probabilities. Thus, the loss function consists of three parts: the coordinate error (Errorcoord), the intersection-over-union error (Erroriou), and the classification error (Errorclass) [27]. The loss function is defined as follows:where Errorcoord represents the sum of the squared errors of the position information, Erroriou is the sum of the squared errors of the confidence level, and Errorclass represents the sum of squared errors of the classes. The formula of loss function is as follows:where S is the number of grid that input images are divided into, B is the number of bounding boxes predicted for each tile, Iiobj denotes if the target appears in tile i and Iijobj denotes that the jth bounding box predictor in tile i is responsible for that prediction, and λcoord and λnoobj are hyperparameters that separate the loss to loss from bounding box coordinate predictions and that from confidence predictions for boxes that do not contain targets. During the deep learning training process, when looking for the optimal parameters in the model, it is to find the parameters that make the value of the loss function as small as possible [28].

3.2.3. Flooded Buildings’ Estimation

The deep learning model could detect flooded object; after that, we can estimate the area of the flooded object. The flying height of the UAV is mainly related to the focal length (f) of the camera, the pixel size, and the image ground sampling distance (GSD). The relationship of the UAV flight parameters is shown in Figure 7 and the following equation:

The formula for calculating the area corresponding to a single pixel is as follows:where GSD is the ground sample distance, H is the UAV flight altitude relative to the ground level, f is the focal length of digital camera, and pix is the size of one single pixel on the CCD of the digital camera. Thus, the area of flooded buildings is the product of the pixel area and the number of pixels.

4. Flood Automated Detection Method

4.1. Data Preparation

Currently, there is no flood-related image data in the public datasets. The research in this study uses the Internet to search for aerial images of flooded houses and vegetation, together with the data obtained by UAV in the Poyang Lake flood area. The software Labelimg is used for labeling to create the training sample dataset. When UAVs perform more flood monitoring missions in the future, the acquired images can continuously fill the sample database to improve the accuracy and reliability of the model. Figure 7 shows part of the flood sample data.

4.2. Training Data Labeling

Deep learning methods need to train the model through a large amount of data. An important step is to label the data in the training dataset. In this study, the software Labelimg is used to label the training data.

Figure 8 shows the software Labelimg operation interface. Select the create-RectBox button to mark the flooded houses and vegetation on the right side. The green border in the picture is the effect after marking. If an image has multiple targets, repeat the above steps. After an image is marked, it would be saved as xml file.

The training data selected in this study contain about 1,000 images of houses and vegetation submerged by floods. In total, the identifiers are marked as building and vegetation. Table 2 shows the training data statistics.

4.3. Model Training and Optimization

Object detection based on deep learning is different from traditional methods. It no longer requires time-consuming steps such as feature selection and extraction; effective feature discovery can be made through deep learning networks. The main factors affecting the accuracy of the model are the richness and connotation of the training dataset. Figure 9 shows the training process.

The model is trained using YOLOv3 algorithm, and it took about 60 hours to get the parameters optimization. If the learning rate is appropriate, it should be ensured that the loss after each round of complete training is reduced and maintained at a small level after a period of training. During the training process, loss will continue to decline at the beginning and drop quickly; after a period of time, it will begin to converge, and finally, it will gradually stabilize. Figure 10 shows the loss curve of the model that when the number of iterations reaches about 5000 times, it starts to converge and gradually stabilizes and fluctuates around a fixed value.

5. Experimental Results and Discussion

5.1. Experimental Results

After the model training is completed, the model can perform target recognition on the test images. We select flood images from different scenes to verify the recognition effect of submerged buildings and vegetation. Figure 11 shows the detect effect of different scenes include images with good quality and illumination condition, images under insufficient light, images under occlusion, and only the building roof is exposed. The blue box represents building recognition, and the red box represents vegetation recognition.

Approximately 100 flood images were used for testing. For images with good quality and illumination condition, the recognition results are satisfactory. When the images are captured under insufficient light, the recognition accuracy is unsatisfactory for long-distance objects. In the case of images under occlusion, as long as the main features of the recognized object can be captured, it can be recognized. If the objects are severely occluded, it is difficult to identify the target. For images only the building roof is exposed, there are few test data that cannot be recognized due to incomplete features. However, in general, the detection and recognition rate are satisfactory.

5.2. Recognition Result Evaluation

The model was evaluated with the test results. There are 120 buildings and 230 vegetation in 100 test images, as shown in Table 3. The results show the model can effectively identify the inundated objects in the images from different scenes. The overall recognition rate can reach more than 85%. In terms of recognition speed, the detection time of each image only needs 2–4s. Experiments show that the UAV system can obtain flood data in time, and the proposed method can identify submerged buildings and vegetation effectively.

5.3. Inundated Building Area Estimation

We chose one flood image as an example to estimate the inundated building area. The pixel number of the example image is 1080∗810, and the pixel number of the flooded building area includes three sky blue labeling boxes, as shown in Figure 12, which is calculated as 44,3159 pixels.

The formula for calculating the area corresponding to a single pixel is shown in formula (4). In this example, the parameters are shown in Table 1:

After unifying the measure units, we can get the result of the pixel is 0.025 square meters, and the area of inundated buildings in the example image is 11078.98 square meters, which is shown in equation (5):

According to the area of buildings inundated by flood, the affected people and economic losses can be estimated, and scientific decisions can be made for evacuation and rescue work.

6. Conclusions

Floods cause great damage to the infrastructure and property, resulting in huge economic losses. Due to the lack of technologies that could automatically detect the flood affected at the object level, recovery services sometimes cannot be provided on time. Traditional remote-sensing satellites have demonstrated delayed response due to orbital period. At present, the UAV system has been widely used in the fields of natural disasters due to ideal for acquiring high-resolution images in a short period. The pertinent literature shows that UAV and deep learning techniques have been used for flood simulation model, but are rare and not well experimented with flooded object detection.

In this study, we introduce the deployed UAV remote-sensing system, deep learning method, and procedures for flood detection. The case study is adopted where the flood-prone area of Poyang Lake. We use YOLOv3 algorithm as a deep learning model on aerial images to detect the inundated buildings and vegetation. Experimental results show that flooded buildings and vegetation can be detected from the images with 88% and 85% accuracy, respectively. And then, we can estimate the inundated buildings area through the relationship among the UAV flight parameters. The experiment results have proved the feasibility and effectiveness of applying the UAV system for flooded region detection. Such timely flood inundation detection can provide visual disaster information in time and is crucial to efficiently rescue activities. For areas with severe floods, the rescue department will allocate resources according to the flooded buildings to ensure the safety of people's lives and property.

Nevertheless, there are still some items for improvement. For instance, we were unable to acquire more information such as the depth of floodwater and the destroyed degree of the buildings. For future work, we may employ UAV oblique photography or Light Detection and Ranging (LiDAR) equipment which could generate the three-dimensional (3D) model. More effort should be made to perform in-depth analyses by expanding the 3D image dataset using the UAV system. Considering the disaster scene, we would use lightweight algorithms such as YOLOv3_tiny and MobileNet to carry out future work. And furthermore, we try to integrate lightweight algorithms into the UAV flight control board for real-time target detection instead of data post-processing. This may improve the efficiency of the flooded objects detection and provide timely information for disaster emergency response and rescue.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

The work was supported by the Scientific Research Project of Tianjin Education Commission (Grant no. 2019KJ143).