Abstract
The height irregularity and complexity of steel structures bring difficulties to dynamic deformation monitoring. PDMS (photogrammetric dynamic monitoring system) can obtain the dynamic deformation of the steel structure, but the flexibility of monitoring is limited because the camera station can only be placed on the ground. In this study, UAV (unmanned aerial vehicle) -PDMS is innovatively proposed to be used in monitoring dynamic deformation of steel structures, and it is verified in the steel frame test and Jinan Olympic Sports Center Tennis Stadium test. To solve the problem that the attitude of UAV cannot be strictly maintained in the hovering process, the improved Z-MP (zero-centered motion parallax) method is used, and the monitoring results are compared with the original Z-MP method. The feasibility of UAV-PDMS applied to steel structure deformation monitoring and the feasibility of improving the Z-MP method to reduce UAV hovering error are verified. The monitoring results showed that the steel structures of the Jinan Olympic Sports Center Tennis Stadium were robust, and the deformations were elastic and within the permissible value.
1. Introduction
In recent years, with the development of the modern construction industry, steel structures have become highly irregular and complex, which bring difficulties to deformation monitoring [1]. Traditional nondestructive testing methods, including ultrasonic testing, radiographic testing, magnetic particle testing, penetrant testing, and TOFD testing, have low efficiency and low detection accuracy. When monitoring the deflection and deformation of long-span steel structures, the total station shall be used for fixed-point location. LiDAR measurement, as a new technology, has been applied to many fields such as cultural heritage protection and topographic survey with its advantages of high efficiency, high spatial resolution, uniform measurement precision, and noncontact. However, it is difficult to obtain the dynamic deformation of the long-span steel structure by using total station and LiDAR. Global navigation satellite system (GNSS) has the advantages of dynamic, high precision, and automation. It has become an important technique of structural health monitoring. However, many errors limit the high-precision positioning services of GNSS, and the signal receiver needs to be installed on the monitored object.
Photography, as a noncontact measurement technology, has attracted more and more attention in the past decades, mainly because of its nondestructive imaging characteristics of high precision and sensitivity. Some scholars have successfully applied photogrammetry technology to the deformation monitoring of engineering structures such as steel towers [2], bridges structures [3], masonry structures [4], and so on [5–7]. The DLT (direct linear transformation) method [8] and the MP (motion parallax) method [9] are two conventional methods used to obtain the deformation of the monitoring points. The DLT method is only suitable for a small range of measurements, and this method has high requirements for control points. The MP method is mainly used for obtaining the 2D plane deformation and the relative variation of monitoring points. However, during the process of measurement, the internal and external directional elements are always changing. The IM-MP (image matching-motion parallax) method is used to correct the influence of this phenomenon on the measurement results and to improve the measurement accuracy. The PSPST-IM-MP (photograph scale transformation-image matching-motion parallax) method to solve the problem of the reference points and the monitoring points is not on the same plane [6]. The Z-MP (zero-centered motion parallax) method optimizes the algorithm and improves the visualization effect of the deformation curve [10]. Currently, the instruments based on these algorithms are mainly industrial cameras or digital cameras. In order to further broaden the application scenarios of photogrammetry technology in dynamic deformation monitoring and improve the flexibility of monitoring, some scholars have used unmanned aerial vehicles and laser radar to monitor the deformation of steel structure by modeling. However, it belongs to the field of static deformation monitoring. In this study, we carry out the monitoring dynamic deformation of steel structure using the UAV-PDMS.
In this study, the UAV video recording method is used to collect image data, and then, the image is intercepted so that the data collection speed of 30 frames per second is realized.
In order to verify the feasibility of using UAV camera instead of ordinary digital camera in PDMS, we conducted a deformation monitoring test of a steel frame structure. As the test, we adopted the improved Z-MP method to solve the problem that the attitude of UAV cannot be strictly maintained after hovering. Finally, the dynamic deformation of the steel curtain wall of Jinan Olympic Sports Center was monitored by the UAV-PDMS based on the improved Z-MP method. The results show that the steel curtain wall is safe and reliable.
2. Related Work
To obtain the change of objects using UAVs, significant efforts have been made. Generally, we can categorize those changes into two types as (1) detect changes in the shape of an object and (2) detect changes in point positions of objects. The two-stage monitoring result can obtain those changes.
2.1. Detect Changes in Morphological Characteristics of Objects
Qinghua Han et al. proposed an image detection approach for corrosion. The proposed method could facilitate the early and comprehensive detection of corrosion along the entire route of the steel structure via integration with UAVs [11]. Norman Hallermann and Guido Morgenthal showed possible applications of UAVs in bridge inspection and the first steps in developing a semiautonomous inspection method for automatic damage detection in post-flight analyses [12]. Amir Mohammad Moradi Sizkouhi et al. presented an innovative approach to optimize the path planning during UAV flight in the monitoring of PV power plants. For this purpose, two algorithms were developed into static path planning and dynamic path planning [13]. Jinhong Chen et al. developed a UAV system, and its aerial image analysis method could evaluate the damage degree of earthquake area [14]. Chunsheng HUA et al. presented a method of detecting collapsed buildings with the aerial images which were captured by a UAV for the postseismic evaluation. The purpose of this work is to achieve the accurate detection of collapsed buildings in a small area from low altitudes [15]. Bo Yang proposed a real-time monitoring framework for a landslide susceptibility area based on a wireless sensor network using multiple UAVs [16]. These studies mainly monitor the morphological changes of objects using UAV.
2.2. Detect Changes in Point Positions of Objects
Qing Li et al. detected and assessed displacements of the rocks and cracks by edge detection based on Canny using UAV [17]. Jouvet et al. used a quadcopter UAV to land on a highly crevassed area of Eqip Sermia Glacier, West Greenland, to measure the displacement of the glacial surface with the aid of an onboard differential GNSS receiver. And it performed traditional UAV photogrammetry over the glacier and processed the resulting orthoimages by template matching in order to cross check the ice motion record with another well-established technique [18]. Hang Li et al. proposed a method of using an autonomous flying robot to explore an underground tunnel environment and built a 3D map [19]. Francesco Nex et al. combined photogrammetric techniques with deep learning algorithms to deliver a true orthophoto showing the position of building damages [20]. Shuhong Wang et al. used the SfM (structure from motion) method to build the DEM (digital elevation model) of the rock mass for real-time monitoring [21]. Zhonghua Hong et al. proposed a parallel processing approach for accelerating the speed of automatic three-dimensional (3D) building damage detection, using a preseismic digital topographical map and postseismic UAV images [22]. The above research studies tend to measure the displacement by formulating a model. These methods have resulted in low monitoring frequency. X. Wang et al. proposed a video analysis methodology for tracking the displacement response of buildings subject to dynamic loads using camera-equipped UAV platforms [23]. However, the study had not eliminated the errors induced by the camera movement. Yufeng Weng et al. used planar homography to eliminate the errors induced by the camera movement without the need of camera parameters. The structural displacement is estimated with the homography transformation matrix determined from the obtained homography features [24]. Still, the pixel accuracy of the method depends on the feature tracking accuracy, as well as the performance of the FNN. The pixel accuracy of the improved zero-centered motion parallax method is reliable. This method is not considered by most researchers.
2.3. Comparative Analysis with Related Work
We compared the three most related work, as shown in Table 1.
3. Unmanned Aerial Vehicle-Photography Dynamic Monitoring System
3.1. Introduction of Instruments
Figure 1 shows the UAV-PDMS. Figure 2 and Table 2 show the UAV and its main parameters, respectively.
3.2. The Improved Zero-Centered Motion Parallax Method
As shown in Figure 3, there are three virtual planes, which are the image plane, the object plane, and the reference plane. The camera sensor is located on the image plane. All the monitoring points are located on the object plane, which is also called the monitored plane. All the reference points are located on the reference plane. The function of the reference plane is to eliminate systematic errors. Ideally, the datum plane and the object plane are the same plane, but this condition is difficult to ensure in practical engineering.
It is necessary for improving the flexibility of PBMS to separate the reference plane and object plane. The displacement on the object plane is transformed into the datum plane, which indirectly eliminating the system error. The conversion process is shown below.
Suppose a monitoring point on the object plane move from position A to position B, which resulted in the displacement of the X-axis and of the Z-axis on the object plane. Meanwhile, the monitoring point is from position a to position b; the X-axis and the Z-axis also have displacements and . Equation (1) is the conversion process:
After the conversion is completed, the system errors can be eliminated. See [10, 27, 28], for the specific process of eliminating system error. However, the attitude of UAV cannot be strictly maintained after hovering in the practical engineering, which will lead to errors and wrong monitoring results. We try to reduce errors by using reference point and further improve the Z-MP method. The improved method is as follows.
According to the pixel coordinates of all reference points in each image, the barycentric coordinates are calculated aswhere n is the photo number and q is the total number of reference points.
Calculate the change value of the barycentric coordinates of the reference points of two adjacent images aswhere is the photo number.
Subtracting the change value and Equation (3) from the displacement and of the uncorrected monitoring point on the reference plane, as the corrected displacement and of monitoring point on the reference plane are as follows:
Finally, we convert back displacements and on the object plane which eliminates the systematic error.
4. Steel Frame Test
4.1. Steel Frame Fest Process
In order to verify the feasibility of using unmanned aerial vehicle to monitor dynamic deformation, we conducted steel structure test, and the side view of the steel structure test is shown in Figure 4. The specific test process is as follows: Step 1: control the UAV to rise to 7m above the ground. Step 2: after the UAV hovering, the airborne camera immediately began to record the vertical video of the steel structure. In the test, we gave a force F of 45N to the center of the steel structure and remained equilibrium. Step 3: remove the force F, and the steel structure gradually recovered its equilibrium in the back and forth shaking. The UAV video ended. Step 4: the above is a test process, and steps 2 and 3 are repeated many times.
Select a complete monitoring period from the videos, and extract one frame of the image every second for saving. Finally, 36 continuous images were selected as the data of this test.
4.2. Data Analysis and Discussion of Steel Frame Test
Figure 5 shows the layout of points, C0–C3 are the four control points, and U0–U19 are the 20 monitoring points. To make the following statements clear, monitoring points U8–U11, U5–U7 and U12–U14, and U0–U4 and U15–U19 are named group A, group B, and group C, respectively.
Based on Z-MP principle and improved Z-MP principle, the data are imported into software, and all the process data are obtained. We use four deformation graphs shown in Figure 6 to display the result.
(a)
(b)
(c)
(d)
Results 1: theoretically, the deformation curves in Figures 6(a) and 6(b) move back and forth around the value of 0 and approach to 0, but the deformation results based on the Z-MP method in Figure 6(a) approach to a negative value, indicating that the deformation curve are ill conditioned, while the deformation results based on the improved Z-MP method in Figure 6(b) are consistent with the theoretical situation, indicating good correction effect.
Result 2: according to Figure 6(b), the deformation interval of groups A, B, and C in the X direction gradually decreases, and the deformation values of the same group are similar, which proves that, after the displacement of group A, group C constrains group A through group B, which can be seen more intuitively in Figure 7.
Result 3: comparing Figure 6(c) with Figure 6(d), the results show that the modification of the improved Z-MP method is valid. It can be seen from Figure 6(c) that, after the force F is removed, the deformation curve did not produce displacement around the zero value, but an integral displacement.
Result 4: by observing the deformation result after correction, it can be seen that the steel frame structure has good elasticity.
5. Jinan Olympic Sports Center Tennis Stadium Test
The tennis court of the monitoring building is located in Jinan, Shandong Province. The tennis court covers an area of about 31400m square meters. Reinforced concrete frame shear walls are used in grandstand and function room. The folded plate structure in the form of willow leaves is adopted in the wall, and the rhombic lattice column is formed by combining with the steel column of curtain wall of the building. As shown in Figure 8, the combination of the folded plate structure and the steel columns of the curtain wall forms a diamond-shaped composite columns, which is helpful to improving the out-of-plane stability of the folded plate structure. The height of the steel structure decreases from south to north, with the highest elevation of 31.3 meters and the largest span of 38.5 meters [10, 29]. In the test, we used DJI Phantom 4 Pro to monitor the dynamic deformation of the southeast side of the tennis court.
5.1. Tennis Stadium Test Process
Figure 9 shows the layout of points; U0–U8 are monitoring points. Control the drone to fly close to the monitored building and then hover to record, as shown in Figure 10. After monitoring, UAV returns to land. In the test, the linear distance between the control points C0–C3 and the horizontal distance between the temporary points T0-T2 are measured with a steel ruler, which is shown in Figure 9.
Capture a recorded video at a rate of five frames per second. At last, 11 consecutive images were randomly selected as test data.
5.2. Data Analysis and Discussion of Tennis Stadium Test
Tables 3 and 4 show the pixel displacement values of the monitoring points U0–U8 in the X-axis and Z-axis, respectively. Table 5 shows the actual displacement values of the monitoring points U0–U8.
The monitoring results and comparison with the deformation generated by the Z-MP method using Camera Sony are shown in Figure 11. If the monitoring points are divided into three groups from high to low, U0–U2 group, U3–U5 group, and U6–U8 group, the following results can be obtained.
(a)
(b)
(c)
(d)
Results 1: the displacement range of the monitoring points with the same elevation is similar, and with the increase of elevation, the displacement range of monitoring points increases. That is, the deformation of the U0–U2 group is the strongest, followed by the U3–U5 group, and U6–U8 group is the last.
Results 2: among the nine monitoring points, the point U0 shows that the maximum displacement is 129.38 mm, and the deformation is within the allowable value. The deformation curve shows that the steel curtain wall structure of the stadium has good elastic performance.
From Figure 11(d), compared with the result of monitoring generated by Camera Sony, the displacement is also different because the monitored points are different. However, their elastic trends are similar, which shows that improved Z-MP using UAV has the same monitoring effect as the proven Z-MP method.
6. Conclusions
In the two tests, the UAV shows its portability and flexibility, provides a new monitoring perspective for PDMS, and further obtains richer monitoring data.
On the contrary, UAV is used in dynamic deformation monitoring, which also exposes some deficiencies, that is, the attitude of UAV cannot be kept strictly after hovering in actual engineering, which will lead to errors and eventually lead to wrong monitoring results. In order to solve this problem, we propose an improved Z-MP method. Experiments show that, in practical engineering, the improved Z-MP method can alleviate the ill-conditioned monitoring data caused by the attitude of UAV after hovering.
The monitoring results show that the maximum displacement of the steel curtain wall structure of Jinan Olympic Sports Center Tennis Stadium is 129.38 mm, the deformations within the permissible value [30]. The spatial structure of the tennis court is generally safe.
Data Availability
The image data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Authors’ Contributions
Mingzhi Chen, Chengxin Yu, and Yongquan Ge are the first, second, and third corresponding authors, respectively.
Acknowledgments
This work was supported by the Science and Technology Project of the Shandong Province of China (Grant no. 2010GZX20125).