Abstract
For long-term monitoring of the midspan deflection of Songjiazhuang cloverleaf junction on 309 national roads in Zibo city, this paper proposes Zhang’s calibration-based DIC deflection monitoring method. CCD cameras are used to track the change of targets’ position, Zhang’s calibration algorithm is introduced to acquire the intrinsic and extrinsic parameters of CCD cameras, and the DIC method is combined with Zhang’s calibration algorithm to measure bridge deflection. The comparative test between Zhang’s calibration and scale calibration is conducted in lab, and experimental results indicate that the proposed method has higher precision. According to the deflection monitoring scheme, the deflection monitoring software for Songjiazhuang cloverleaf junction is developed by MATLAB, and a 4-channel CCD deflection monitoring system for Songjiazhuang cloverleaf junction is integrated in this paper. This deflection monitoring system includes functions such as image preview, simultaneous collection, camera calibration, deflection display, and data storage. In situ deflection curves show a consistent trend; this suggests that the proposed method is reliable and is suitable for the long-term monitoring of bridge deflection.
1. Introduction
It is very significant that monitoring bridge deflection can evaluate the health status of bridge and can provide the important reference base for the structural performance and operational status of bridge. Currently, there are two kinds of deflection measurement methods which are the traditional measurement method and the automatic measurement method. The traditional measurement methods include dial indicator, level, and total station. The principle of dial indicator measuring bridge deflection is very simple [1]. However, due to the great height of bridge, it is very inconvenient to install the dial indicator on main girder of bridge. The level is usually used to measure bridge deflection along with a scale [2], and this increases the complexity of measurement work. Furthermore, the total station is easily influenced by temperature and humidity and is not suitable for long-term monitoring in the wild [3]. In a word, the traditional measurement methods are rarely used to execute the long-term monitoring of bridge deflection, except for the temporary measurement.
The automatic measurement methods comprise accelerometer, microwave interferometer, GPS, connected pipe optoelectronic liquid level sensor, and so forth. The high-frequency component occupies a large proportion in the data collected by accelerometer, so the low-frequency component is drowned out when the displacement is obtained by computing the integral for acceleration data. However, the bridge deflection is exactly the low-frequency component. In the meantime, the double integral brings about errors. As a result, the accelerometer has lower precision [4]. The microwave interferometer can detect the bridge deflection according to the phase difference of reflected wave before and after deformation. Although the microwave interferometer has higher precision, this method is not suitable for the case that the transverse deformation and longitudinal deformation simultaneously occur on bridge [5]. GPS has very good practicability. Nevertheless, the cost of GPS is higher and the accuracy of GPS can only achieve centimeter level [6]. When the connected pipe optoelectronic liquid level sensors are used to detect the deflection of bridge, many pipes and pressure transmitters need to be mounted inside the main girder of bridge in advance, and the laying work is very complicated [7].
Based on the above comparative analysis, the CCD camera is chosen to monitor bridge deflection in this paper. The CCD image technology has some advantages such as fast speed, noncontact, high precision, and simple operation. With the development of image processing technology, the CCD inspection imaging technology is gradually applied to engineering practice. In 1999, Olaszek adopted the computer vision technology to detect the dynamic performance of bridge structure [8]. In 2002, Yu et al. utilized CCD cameras to measure the bridge deflection. However, the camera calibration was very complicated, and the five cross mark symbols installed on the bridge pier were used to calibrate the camera and measure the deformation. The three horizontal crosses were used to calibrate the horizontal ratio value, and the three vertical crosses were used to calibrate the vertical ratio value [9]. In 2007, Yoneyama et al. employed the DIC method to inspect the longitudinal deformation of a single-span simply supported bridge. The DIC method required spraying speckles along the longitudinal direction of bridge, and it was only suitable for detecting the small-span bridge [10]. In 2007, Chen et al. adopted the DIC method to measure the hookup that connects the two parts of a movable steel bridge. An artificial speckle pattern was formed on the hookup surface and an on-site camera system with a removable camera set was used to measure stress concentration at the hookup [11]. In 2015, Shan et al. presented a stereovision method to conduct the vibration test of a stayed-cable model, and the proposed stereovision method acquired 3D displacement curves and had a higher precision [12].
The above methods in [8–11] need to paste the scale on structural surface for converting the measurement unit from pixel into millimeter, and the prepared work is very complicated. The method in [12] employed Kalman filtering algorithm to track the circular target on the sequential images. Compared with the DIC method, Kalman filtering algorithm needs more time to search image for tracking of target and uses a lot of memory of computer. This has little effect on the short-term inspection. However, this influences the image capture in the long-term monitoring of bridge. Aimed at the monitoring requirement of bridge deflection, Zhang’s calibration-based DIC deflection monitoring method is proposed in this paper, and the corresponding bridge deflection monitoring system is integrated and adopted by Songjiazhuang cloverleaf junction.
2. Principle of Zhang’s Calibration Algorithm
According to the pinhole imaging model [13], the projection of a spatial point on the imaging plane of camera can be expressed aswhere is an arbitrary scale factor and is the intrinsic parameter matrix of camera without distortion. is the extrinsic parameter matrix of camera. and are the rotation matrix and translation vector from the world coordinate system to the camera coordinate system, respectively. is the 2D coordinate of point in the image coordinate system. is the 3D coordinate of point in the world coordinate system.
As shown in Figure 1, owing to the distortion of lens, there exists a certain deviation between the actual image point and the ideal image point. To eliminate the distortion of lens, a camera calibration algorithm based on a 2D planar pattern was proposed by Zhang [14]. Considering the radial distortion, a distortion model including two parameters was adopted in Zhang’s calibration algorithm. This distortion model can be expressed as follows [15]:where and is the normalized coordinate of image point under distorted condition. is the normalized coordinate of image point under undistorted condition. Both and are the radial distortions coefficients. The matrix in (1) and , are collectively called the intrinsic parameters of camera.
Zhang’s calibration algorithm only requires the camera to observe a planar pattern shown at a few (at least two) different orientations. Either the camera or the planar pattern can be freely moved. The motion need not be known. The detailed calibration procedure of Zhang’s algorithm is given as below.
2.1. Solving the Intrinsic and Extrinsic Parameters of Camera
As seen from (1), there exists the projective transformation relation between the world coordinate system and the image coordinate system. Suppose that the 2D planar pattern is selected as the model plane of the world coordinate system; namely, . Let us denote the th column of the rotation matrix by . From (1), we have
Let and ; we have where , where is an arbitrary scalar.
Computing is a process that solves the minimum residual error between the actual image coordinate and the image coordinate which is calculated by (4); the objective function is .
After solving , two basic constraints on the intrinsic parameters are obtained according to the orthogonality and given as follows:
Let
Note that is symmetric, defined by a 6D vector
Let the th column vector of be ; then we havewith
Therefore, (5) can be expressed by the following equation:
If images of the model plane are observed, by stacking such equations as (8), we havewhere is a matrix. If , we will have in general a unique solution defined up to a scale factor.
Once is estimated, we can compute all camera intrinsic parameters. The matrix is estimated up to a scale factor, that is, , with being an arbitrary scale. Without difficulty, we can uniquely extract the intrinsic parameters from matrix . Once is known, the extrinsic parameters for each image are readily computed. From (4), we havewhere .
2.2. Maximum-Likelihood Estimation
In general case, the distortion of lens cannot be ignored. Therefore, in the second step, the intrinsic and extrinsic parameters derived in the first step are regarded as the initial values to execute optimization.
We are given images of a 2D planar plane and there are points on the planar plane. Assume that the image points are corrupted by independent and identically distributed noise. For different point on a planar plane, the initial values of distortion coefficients , can be solved from (2) by the least square method. The detailed process can refer to the technical report [16].
These initial values including the intrinsic and extrinsic parameters and the distortion coefficients are substituted into the objective function to optimize camera. The maximum-likelihood estimate can be obtained by minimizing the following function:where is the projection of point in image according to (4).
Minimizing (11) is a nonlinear minimization problem, which is solved with the Levenberg-Marquardt algorithm [17]. It requires an initial guess of , which can be obtained using the technique described in the previous section.
Through the above two steps, the intrinsic and extrinsic parameters including nonlinear distortion coefficients of camera are all achieved.
3. Zhang’s Calibration-Based DIC Deflection Monitoring Method
The DIC method is widely used to measure the surface displacement or strain according to the correlation of light intensity of speckles before and after deformation, which are distributed on specimen surface randomly [18]. When the DIC method is employed to detect deformation, a single CCD camera is usually required to be perpendicular to the measured surface and the angle between the optical axis of CCD camera and the vertical line of measured surface cannot be larger than 5° [18]. In engineering, the angle between the optical axis of CCD camera and the vertical line of measured surface is often larger than 5°. As a result, the DIC method cannot be directly used to measure deformation when the skew angle is larger than 5°. To deal with this problem, Zhang’s calibration algorithm is introduced to figure out strict limit of the skew angle of CCD camera. Furthermore, Zhang’s calibration algorithm can eliminate lens distortion and convert measurement unit.
3.1. Principle of DIC
The correlation criterion needs to be firstly confirmed when the DIC method is used to calculate deformation. The zero-mean normalized cross-correlation (ZNCC) criterion has the most robust noise-proof performance and is insensitive to the linear scale and offset in lighting on images. Considering the requirement of actual application, the ZNCC criterion is chosen to conduct area matching in this paper. The ZNCC criterion is given as follows [19]:where is the gray value of reference image subset, is the gray value of target image subset, and , are the mean gray of reference image subset and target image.
The conics fitting method is used to conduct subpixel search in this paper. Assume that the pixel position of a point whose absolute value is the largest in the matrix of correlation criterion is . Total nine points composed of point and its eight nearest-neighbor points can locally constitute the quadratic surface, and the surface equation is given as below:
Utilizing the pixel positions of nine points and the correlation coefficients of corresponding positions, the values of in (13) can be solved, and the extreme point coordinates of surface can be obtained, that is, . This extreme point is exactly the center of subset on the deformed image; the center coordinate’s difference between the reference image and the deformed image is just the displacement of point , whose unit is pixel.
Before test, the traditional DIC method needs to paste the scale whose length is known on the measured surface for converting measurement unit from pixel into millimeter. In most cases, the scale may be the coordinate paper with a certain length or the geometry size of specimen. According to the ratio of the actual length of scale to the pixel length on image, the traditional DIC method can convert the measurement unit of displacement from pixel into millimeter.
3.2. Displacement Calculation of Measured Point
As shown in Figure 1, suppose that the coordinate of point in the world coordinate system is , and its coordinate in the camera system is . In the image coordinate system, the homogeneous image coordinate in millimeters is , and the homogeneous image coordinate in pixels is . After subpixel search of DIC, has been given. As a result, the relation between and can be described as follows [20]:where and are the physical size of a pixel on -axis and -axis, respectively, in mm/pixel, and these two parameters are the intrinsic parameters. is the coordinate of principal point, in pixels.
The relation between the coordinate in the camera coordinate system and the homogeneous image coordinate in millimeters is expressed as below:where is the focal length and is the scale factor.
The coordinate in the world coordinate system and the coordinate in the camera system meet the following equation:where is the rotation matrix from the camera system into the world coordinate system and is the translation matrix from the camera system into the world coordinate system. , can be acquired by Zhang’s calibration algorithm.
Equation (14) is substituted into (15), and (17) is given as below:
Combining (16) and (17), the coordinate in the world coordinate can be solved from the homogeneous image coordinate in pixels .
In (17), the scale factor must be obtained firstly. The value of is related with the selection of world coordinate system. In this paper, the world coordinate system is established on the measured surface. The plane of the world coordinate system coincides with the measured surface, and axis conforms to the right-handed coordinate system. Any three points in the word coordinate system are selected to establish the plane equation, and the equation is described as follows:where , , , are the coefficients of plane equation.
Equation (17) can also be expressed by the equation set as follows:
Equation (19) is substituted into (18); can be given by the following equation:
Through the above calculation, the homogeneous image coordinate in pixels can be converted to the coordinate in millimeters in the world coordinate system. Therefore, this paper implements the conversion of measurement unit from millimeters into pixels by Zhang’s calibration algorithm.
The sketch map of target image acquired by each CCD camera is shown in Figure 2. The solid circle surrounded by a rectangle is exactly white spot in Figure 2, which is the target tracked by each CCD camera. The rectangle in Figure 2 is the matching subset extracted by DIC method, which is used for calculating deformation. Assume that the image coordinate of matching subset’s center at is . Deformations are found on bridge under load, and the image coordinate of matching subset’s center at is . Thereby, the displacement of measured point on bridge at can be obtained by subtracting two coordinates, and the displacement equation is given by where is the transverse displacement of measured point on bridge at and is the vertical displacement of measured point on bridge at , which is exactly the bridge deflection in pixels.
(a)
(b)
Through the above conversion of measurement unit, the bridge deflection in millimeters can be acquired accordingly.
4. Comparative Test of Calibration Algorithms
To testify effectiveness and feasibility of Zhang’s calibration-based DIC deflection monitoring method proposed in this paper, the comparative test of two calibration methods is conducted in lab. One is Zhang’s calibration algorithm and the other is the scale calibration adopted by traditional DIC method.
4.1. Experimental Setup
As illustrated in Figure 3, the specimen is 300 mm in length and 50 mm in width. The specimen surface is coated with white paint, and some silicon carbides sprayed randomly on white paint are regarded as speckles. The specimen is fixed on the mobile platform, the translation knob in Figure 3 can control the horizontal movement of specimen, and the moving range from left to right is 20 mm.
To compare with the detection results of the proposed method in this paper, the dial gauge is pressed against the measured specimen. The measurement range of dial gauge is 25 mm, and the precision is 0.001 mm. The measured specimen moved horizontally along with the mobile platform; each movement distance is about 1 mm. In test, the dial gauge read is recorded and the specimen image is simultaneously captured by CCD camera along with each rotation of translation knob. The CCD camera is placed 1 m away from the specimen. The cameras can record the high-definition images with a pixel resolution of 1280 × 960 at 1 fps. The CCD camera is equipped with an optical zoom lens of F1.8–16 and focal length ranging between 12 and 36 mm.
During test, both Zhang’s calibration algorithm and scale calibration are adopted to convert the measurement unit of horizontal displacement in this paper. The scale selected by the traditional DIC method is the actual length of specimen, which is 300 mm, and the corresponding length in image is 647.93 pixels. Therefore, the scale factor between actual size and pixel size is 0.4630 mm/pixel, and this value is exactly the conversion coefficient of measurement unit used by the traditional DIC method.
As shown in Figure 4, a planar pattern consisting of eighty-one 30 mm × 30 mm black and white squares is used to calibrate CCD camera. The open software “Camera Calibration Toolbox for MATLAB” [21], which is compiled according to Zhang’s algorithm based on a planar pattern, is employed to execute the camera calibration in this paper. Before test, 9 images of the planar pattern in different postures are firstly captured by the CCD camera for camera calibration. Then, 8 × 8 corner points on each image, which are expressed by the red rectangle in Figure 4, are selected to execute camera calibration by the calibration software. Finally, the intrinsic and extrinsic parameters including nonlinear distortion coefficients of CCD camera are all obtained by the calibration software.
4.2. Experimental Result Analysis
The displacement curves acquired by two different calibration algorithms are shown in Figure 5. It can be seen that two displacement curves have the same trend and agree very well. The displacement data obtained by dial gauge are regarded as the truth-value of displacement in this paper, and the displacement errors of the other two calibration algorithms are analyzed accordingly.
Figure 6 gives the absolute error and relative error curves of two calibration algorithms. As can be seen from Figure 6, the displacement precision of the proposed method is higher that of traditional DIC method. The mean absolute error of displacement measured by the DIC method based on Zhang’s calibration is 0.033 mm, and the relative error is floated at 0.5%. The mean absolute error of displacement detected by the DIC method based on scale calibration is 0.050 mm, and the relative error is larger than that of the former.
(a)
(b)
Results of comparative test of two calibration algorithms indicate that Zhang’s calibration-based DIC deflection monitoring method has higher precision, and this testifies effectiveness and feasibility of the proposed method in this paper.
5. Deflection Monitoring for Songjiazhuang Cloverleaf Junction
Songjiazhuang cloverleaf junction is located on G309 National Highway, Zibo city, Shandong province. The total length of bridge is 670.06 m. The superstructures of bridge are 22 m × 30 m prestressed concrete simply supported box beams and have 12 box girders in the transverse direction. The substructures of bridge are column piers and bored pile foundation. The deck width is 32 m, and the layout of deck is 0.5 m + 15 m + 1 m + 15 m + 0.5 m; it is a six-lane two-way bridge. The bridge was opened to traffic in 2005. The monitored bridge in this paper is crossing through Jiaozhou-Jinan railway, which is located under Songjiazhuang cloverleaf junction.
5.1. Monitoring Scheme
According to the deflection monitoring requirement of Songjiazhuang cloverleaf junction, the midspan deflections of the 3rd, 4th, 9th, and 10th box girders need to be detected (see Figure 7). Four targets are installed on the midspan position of the 3rd, 4th, 9th, and 10th box girders. As shown in Figure 7(b), every two CCD cameras are mounted on the middle pier of corresponding deck for monitoring respective target and recording target image at each time.
(a)
(b)
As illustrated in Figure 8, a CCD camera is put 2 m away from the lower surface of the cap beam. The horizontal distance between each CCD camera and the corresponding target is about 15 m. In addition, the distance between the middle pier and the nearest box girder is 1.35 m under each deck. Thereby, the distance between each CCD camera and the corresponding target is about 15.19 m. For making CCD camera capture clear images, a spotlight is installed below each CCD camera and is used to provide illumination for the corresponding CCD camera at night.
Four targets are separately installed on the midspan position of 3rd, 4th, 9th, and 10th box girders, and the vertical surface of each target is perpendicular to the corresponding box girder (see Figure 8). Each target is made of stainless steel with 10 mm depth and is bent up to 90°. As shown in Figure 9, both the length and width of target’s vertical surface are 150 mm, and the size of target’s horizontal surface is same as that of vertical surface. The horizontal surface of target is fixed on the midspan position of corresponding box girder with expansion bolts. The background color of target is black, and there exists a white solid circle on the center of vertical surface, which is 100 mm in diameter. This white circle is exactly the target tracked by CCD camera at different time.
5.2. Deflection Monitoring System Based on CCD Cameras
Based on the deflection monitoring scheme for Songjiazhuang cloverleaf junction, the corresponding deflection monitoring system for Songjiazhuang cloverleaf junction is integrated in this paper. As shown in Figure 10, the deflection monitoring system for Songjiazhuang cloverleaf junction includes a computer, CCD cameras, lenses, spotlights, and targets. The computer is the core of deflection monitoring system, which is placed in the duty room and controls the operation of deflection monitoring system. Considering the long-term requirement of deflection monitoring for Songjiazhuang cloverleaf junction, the computer chooses 8 GB memory, 1862 GB hard disk capacities. The computer’s motherboard contains 5 cable interfaces, and four interfaces are connected to four CCD cameras through network cable.
The type of CCD camera used for monitoring deflection is AVT Company’s GT1290, which utilizes network cable to transmit image. The CCD camera has the image resolution of 1280 × 960 and the sample frequency of 1 Hz. The CCD camera is set to shoot black and white images when monitoring deflection, and its range of operating temperature is −20°C–60°C. The lens with 50 mm focal length is employed to capture clear target images when the object distance is farther, which is made in Computar Company. Moreover, it is noted that the white circle on target should always appear in the field of view (FOV) of camera within the allowable range of bridge deflection.
Based on the deflection monitoring scheme for Songjiazhuang cloverleaf junction, the deflection monitoring software for Songjiazhuang cloverleaf junction is compiled by MATLAB. This deflection monitoring software is a stand-alone executable program. Running the deflection monitoring software for Songjiazhuang cloverleaf junction only needs installing MCRInstaller, which is the dynamic linking library (DLL) of MATLAB; no MATLAB complier needs to be installed. The deflection monitoring software for Songjiazhuang cloverleaf junction has functions such as image preview, image collection, camera calibration, deflection display, and data storage.
As illustrated in Figure 11, the software interface includes five functional zones, namely, image collection, parameter input, operation button, deflection curve, and value display. For the convenience of adjusting the FOV size of CCD cameras, the area of image collection in software interface can display the FOV of four CCD cameras in real time. The areas of parameter input and operation button are the core part of deflection monitoring software, which are used for controlling image acquisition and processing. The area of deflection curve and value display can show deflection curves and calculate deflection values in real time. The operation flowchart of deflection monitoring software for Songjiazhuang cloverleaf junction is given in Figure 12.
5.3. In Situ Deflection Monitoring Results
The in situ deflection data of Songjiazhuang cloverleaf junction are shown in Figure 13. Three deflection curves measured on Sep. 10–17, 2014; Oct. 9–17, 2014; and Oct. 20–28, 2014, are given in this paper, and the time period of each curve is one week.
(a) Sep. 10–17, 2014
(b) Oct. 9–17, 2014
(c) Oct. 20–28, 2014
As can be seen from Figure 13, the deflection curves of 4 measured points in each subgraph have jitters. The reason of jitters is vehicles passing across the bridge deck. However, the deflection curves’ trends of four measured points are basically consistent. The trend consistency of four deflection curves indicates that four CCD cameras can work stably on site and the proposed method can accurately measure the deflection deformations of bridge, and the bridge works in the good condition. Experimental results verify the reliability of the proposed method, indicating that the proposed deflection monitoring method in this paper can meet the long-term requirements of bridge deflection.
6. Conclusions
Aiming at meeting the requirement of deflection monitoring for Songjiazhuang cloverleaf junction, Zhang’s calibration-based DIC deflection monitoring method is proposed in this paper. According to the proposed method, the deflection monitoring software is developed by MATLAB, and 4-channel CCD deflection monitoring system for Songjiazhuang cloverleaf junction is integrated accordingly. The in situ deflection data show that deflection curves of four measured points have the consistent trends, and this suggests that Songjiazhuang cloverleaf junction works in good condition, and the proposed deflection monitoring method in this paper is reliable and accurate. Compared with other deflection measurement methods, the proposed method is simple to operate and costs less and is convenient to conduct automatic measurement and long-term monitoring. This proves that this method will have the broad application foreground in engineering.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This study is supported by the National Natural Science Foundation of China under Grants nos. 51478148 and 51408261, the Natural Science Foundation of Heilongjiang under Grant no. E201434, and the foundation of Harbin City under Grant no. 2015RAQXJ028.