Abstract
Ground deformation observation is wildly concerned in the field of earthquake engineering. This paper proposes a high-precision displacement measurement technology based on both computer vision and numerical simulation. During the earthquake, the vision-based testing system collects visual data of the target installed on the location to be observed. The visual data streams can be quantified to the dynamic relative displacement value automatically, by employing mathematical vision algorithms and then by taking the relative displacement as an intermediate quantity, which is brought into the numerical model for iteration. When the test result is close to the simulated one, the absolute ground displacement data could be obtained approximately. A series of experiments have been carried out to suggest that the proposed method presents an innovative and low-cost solution to ground measurement in high accuracy. The method not only realizes the real-time ground deformation observation; moreover, it also provides a wider range of reliable data support to understand deformation mechanism, investigate seismic source information, and recognize the ground motion characteristics.
1. Introduction
Ground deformation in earthquake has always been the focus in the field of earthquake engineering. Persistent monitoring and quantification of ground deformation before and after earthquake not only can establish significant amount of data for the mitigation of seismic hazard but also provide valuable insights into the evolution of the surface deformation accumulation phase and the ground motion properties [1, 2]. In spite of ground displacement response is known as the critical metric for ground motion evaluation, direct measurement also face a big challenge, especially for the dynamic measurements during an earthquake.
Early geodetic measurement relied on the ground-based optical or mechanical techniques, of which triangulation, trilateration, and leveling were the most common. However, it is difficult to provide a precise measurement in the complex geological environment consistently due to the discrete point monitoring and accumulative error [3]. The need for high performance in the field of intelligent sensing machines, data intelligent processing, real-time monitoring, and dynamic management have become the development direction of modern geodetic monitoring instruments.
However, the seismic observation networks make great contributions to the research of strong-seismic observation. The near-field main shock is still hard to be recorded completely owing to the sparse density of networks in pain areas [4]. In recent years, various space-geodetic techniques, especially the global positioning system (GPS) [1, 5] and interferometer synthetic aperture radar (INSAR) [6], have been extensively implemented to study the ground motion based on in-depth analysis of surface deformation. However, most geodetic methods usually have limitations in dynamic displacement measurements. GPS-based methods are restricted by the possible mismodeling of various intervening effects (such as ionospheric and tropospheric delay, multipath, and residual clock errors) [7]. Besides, the deviation caused by the influence of atmospheric, satellite orbit, and temporal decorrelation sensitivity will lead to the image interpretation error in the INSAR technology [8, 9]. All these limitations will increase the uncertainty and cost in the real ground motion measurements. In this paper, we use a computer vision-based observation technology which is an innovate approach through the previous observation techniques in recent years.
Computer vision measurement is an innovative technology [10]. Along with the unparalleled technological progress in digital image and computer intelligence technologies, vision-based technology is widely applied in various fields, such as artificial intelligence [11], structure detection [12], automatic assembly [13], and medical equipment [14]. In recent years, as a high-precision, noncontact, multipoint, and real-time measurement technology, vision technologies have significant potentials in civil engineering applications. Besides providing dynamic displacement information [15], vision-based technologies showed extensive implementation of damage detection [16] and structural health monitoring [17]. Vision-based technology also has great advantages on the measurements of ground motions compared with point-based methods. For measurement of a complex scene, structure from motion is an extensively researched topic, and 3D reconstruction is also widely used for remodelling of structures [18–20].
Moreover, the development of the city has made the number of urban monitoring equipment be in large scale. On this basis, this paper presents a new approach for measuring the absolute ground displacement using techniques of computer vision and numerical simulation, which is simple in design and has high accuracy. Besides presenting methods for each component in principle, a series of shaking table tests, such as target motion tracking and dynamic displacement monitoring, have been carried out to verify the reliability and accuracy of the proposed vision-based technology. The significance of this study is mainly reflected in two aspects. Firstly, it presents a new thought for the surface motion measurement. Secondly, it provides an effective basis to understand the ground deformation mechanism.
2. Ground Deformation Testing Method
2.1. Basic Principle of the Method
The basic principle of the ground deformation testing method is shown in Figure 1. From the figure, the camera (installed on the top of the rod) is used to obtain the relative displacement Δu, which equals to the camera deformation minus the ground deformation. Meanwhile, numerical simulation is performed to obtain the relative displacement Δu'. If the correlation coefficient between Δu and Δu′ is greater than a threshold value ε (in this paper, ε = 95%), we can get the approximate absolute ground deformation value by numerical simulation. Also, the iteration time is short due to the simplicity of single degree-of-freedom system (SDOF) to simulate the testing system. Based on the basic principle of structural dynamics, the visual testing equipment is simplified as a single degree-of-freedom system with centralized mass m supported by a mass-free structure with lateral stiffness k, which retains the original structural dynamic characteristics [21].
2.2. Test Instrumentation and Layout of Sensors
The camcorder used in this test, which has 1920 × 1080 pixels of resolution and is able to measure by 60 frames per second, the optical equipment (such as lenses, and cameras) and target size play important roles in the vision-based measurement system. Many sensors were deployed to record various parameters throughout the series of shaking table tests, such as acceleration and displacement. The layout of sensors in the test are shown in Figure 2, which includes 7 cameras, 2 accelerometers, and 1 gyroscope, denoted as Dc, A, and G, respectively.
(a)
(b)
2.3. Input Motions and Loading Conditions
The purpose of shaking table tests is to obtain the accuracy of the method on the ground deformation measurements. Therefore, the input motions should cover a wide range of frequency spectrum. Using the flat noise, 1 Hz, 3 Hz and El Centro and Taft ground motions as reference waves, the Taft ground motion was recorded at the Taft seismologic recording station during the Ms7.7 Kern County earthquake on 21 July 1952 in California, USA, with an original peak acceleration, fault distance, and duration of 0.152 g, 41 km, and 54 s, respectively. The acceleration time histories and Fourier spectra of the input motions are shown in Table 1. The flat noise inputs are used to obtain the inherent characteristic of the system, and 1 Hz (PGA = 0.1 g) and 3 Hz (PGA = 0.5 g) are used to verify the accuracy and precision of the vision-based ground deformation testing method.
2.4. Vision-Based Testing Method and Processing Method
The video was captured by the camera. The center coordinates and the radius of a target circle can be obtained by the circle fitting algorithm. The center coordinates of the target are derived from the sampled static image sequence. Therefore, the horizontal and vertical displacements of the target circle center in the image are obtained, and the real displacements are obtained by calibrating the relationship between the image pixels and coordinates of the actual objects (the 10 cm circle is used herein). The flow chart of the vision-based deformation test method is shown in Figure 3. Two key points of the method are the circle detection algorithm and calibration relationship. In this paper, the basic principle of circle detection based on the least squares method (LSM) is adopted, which uses the target circle’s edge point coordinates just doing one operation, and the target circle parameters (center coordinates and radius) are obtained. The algorithm flow is shown in Figure 4, in which edge is the edge of the target circle in the image and (Xi, Yi) is the edge point coordinates of the target circle. The asymptotic time complexity of the algorithm is O(n), which is the computational efficient.
The precision of the vision-based dynamic displacement testing is estimated by a small-scale shaking table test. The validation system is shown in Figure 5. Four target circles (radii of all circles were 10 mm) were set for the displacement test, and strain displacement meters were laid on the side of the table board to collect displacement data. The testing input excitations were 1, 3, and 5 Hz sine waves. Based on the calibration relationship, approximately 45.2 pixels represent 1 cm of actual space [22].
Under different excitations, the displacements for the selected mark A were obtained by displacement meter and vision-based displacement test method which were compared. The results are shown in Figure 6. It is clear that the displacement amplitudes and shapes are close to the curves obtained by the vision-based dynamic displacement testing method and the strain displacement meters. When input sine waves are 1, 3, and 5 Hz, the corresponding correlation coefficients of the displacement curves measured by the two testing methods are 0.994, 0.996, and 0.990, respectively.
2.5. Numerical Simulation and Verification
The whole testing system is simplified as a single degree-of-freedom model. Formula (1) is the governing equation of the vision-testing system. For the purpose of researching the feasibility and accuracy of the proposed analytical system, the same excitation loads were chosen to input into the numerical model. The analytical expression of the relative displacement of the system is calculated by the Duhamel integral method and is shown in formula (2), and the absolute ground displacement is obtained by formula (3):where m, k, , , and c are the quality, stiffness, frequency, damping coefficient, and damping ratio, respectively; and express the ground motion acceleration and displacement, respectively; is the analytical relative displacement between top of the rod and the ground surface; and is the vibration time, and means the time integral term.
The natural frequencies of the vision system are 7.96 Hz, 3.42 Hz, and 2.29 Hz, and the damping ratios of 1-meter pole, 2-meter pole, and 3-meter pole are 1.82%, 4.20%, and 6.32%, respectively. The relative displacement between experimental data and analytical solution (2-meter pole) is shown in Figure 7. The vision-based testing displacement result is close to the numerical simulation result in the frequency domain for both small and large PGAs, as shown in Table 2. However, in the time domain, the vision-based testing results are greater than the numerical results, the amplification factor are about 0.8120, 0.9244, and 0.8547 for the test cases of 1 Hz (PGA = 0.1 g), 3 Hz (PGA = 0.5 g), and Taft, respectively.
(a)
(b)
(c)
(d)
(e)
(f)
In addition to using graphics and quantitative parameters to intuitively reflect the correlation between the measured value Δu and the analytical solution Δu′, the Bland–Altman method is also used to verify the feasibility and accuracy of the numerical model. As shown in Figure 8, the mean is used as the abscissa and the difference d = Δu − Δu′ as the ordinate. The horizontal solid line in the middle is the mean line of difference d, which can be seen to be very close to the value of 0. Comparing the distribution of scatter points within the line of consistency limit (d ± 1.96Sd, Sd is the standard deviation), the differences of 95.941%, 99.200%, and 96.502% are located in the confidence interval under different working conditions, respectively. It shows that the results of the two methods are very close and consistent. The numerical model represents the dynamic response of real monitoring equipment well under complex ground motions.
3. Result and Interpretation
3.1. Ground Displacement
At present, the ground displacement generally is adopted by the macroseismograph (using the numerical integration method), and Figures 9–10 shows the displacement integral from acceleration to displacement in different test cases; it is clear that using acceleration integral to obtain the displacement requires corresponding processing of the acceleration signal, especially the filtering process, where the acceleration signal is not filtered, the displacement signal will be distorted, and under different filters, the displacement shows different characteristics. In addition, it can be seen from Figure 10 that, no matter what filter and parameter are adopted, the signal will be suppressed at some frequency domain, so that the displacement value generated by the integral is less than the value of ground truth.
In Figure 11, the ground displacement acquired by the new testing method is almost equal to the ground truth (in this test, we use Dc7, which is installed outside of the shaking table), where the relative displacement calculated by the numerical simulation is close to the measurements by the vision system. Moreover, the absolute displacements measured by the two methods coincide well in the frequency domain. From the BA chart of the two observation results, the scatter points are uniformly distributed in the standard deviation line, the mean line is close to zero, and the difference distribution in the confidence interval accounts for more than 95%. Hence, we can use the vision system and numerical simulation method to obtain the approximated ground displacements.
(a)
(b)
3.2. Concluding Remarks and Discussion
We have proposed and tested a simple but sophisticated new approach to estimate ground deformation. Major issues of the proposed method have been discussed in detail, such as visualized data processing and numerical methods. A series of shaking table tests were performed to investigate the feasibility and practicability of the proposed method. The new approach provides a ground displacement testing method with acceptable accuracy, in noncontact mode, being multipoint measured in real time and cost saving.
Based on our experiments and numerical simulation, the real-time ground displacement can be obtained, validated by the vision observations. Note that the same ground motion has been used when processing the ground displacement, which corresponds to the condition when the macroseismograph is installed at the bottom of the equipment. However, the acceleration is unknown under actual conditions; in such cases, we use the nearest macroseismograph data as a seed for input motion and use the relative displacement as an intermediate quantity, by repeated iteration to obtain the approximate ground displacement. The basic idea of our approach is to estimate the ground deformations, meanwhile, and the inverse method can be used to determine the absolute ground displacement by the following formula:where means relative displacement and can be tested by the vision-based method; m, c, and k are the mass, damping, and stiffness, respectively, and can be tested or calculated by the existing mature method.
Noted that the new ground displacement method can only track the motions in flat surfaces, the method cannot be performed on all axes (X, Y, and Z axes) that use binocular vision technology to recover the depth information and establish the three-dimensional displacement spatial field after earthquake. Meanwhile, the device parameters play a very important role in the test; in addition, we adopt the high-precision optimal circle fitting method, and with the development of computer vision algorithm, the precision can be improved continuously. Moreover, the results measured by direct integration and new testing method are both smaller than the real displacement, and the amplification factor is in the range of 1.2 to 1.3. Our method has the potential to provide large amounts of seismic data, and with the progress of optical equipment and VI (visual identity) algorithms, the accuracy of this method will be improved significantly.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors gratefully acknowledge the financial support for this study by the Natural Science Foundation of China (Nos. 51738001 and 61472373) and by the Project of the Central Level, Scientific Research Institutes for Basic R&D Special Business (DQJB17B03).
Supplementary Materials
This supplementary material is a video about how the software works and to show the process of the employed technique. The used technique can be seen in Section 2.4 of the manuscript. Using the software, we can recognize the target and obtain the displacement. In the video, the displacement is calculated for each frame, and the time series of the displacement is obtained after the whole video processed. (Supplementary Materials)