Abstract

The traditional inclination measurement method usually measures one-dimensional or two-dimensional inclinations of power transmission towers. This research proposed a method, which is feasible to measure three-dimensional (3D) inclinations using the stereo images of power transmission towers. The main work of this method is the 3D reconstruction of power transmission tower model using stereo images and the calculation of 3D inclinations. As for the 3D reconstruction, a method of calculating the matching cost using a combination of multichannel red–green–blue stereo pairs and Sobel edge detection stereo pairs is proposed to obtain accurate matching cost, and a method of invalid disparity judgment is proposed to detect the invalid disparities in small closed areas; as for the calculation of 3D inclinations, a 3D inclination calculation method is put forward to measure 3D inclinations by rotation and translation of different 3D reconstruction models. The experiment shows that the 3D reconstruction quality is good and the results of the 3D inclinations are accurate, which is of great value in the static or dynamic measurement of 3D inclinations of power transmission towers.

1. Introduction

Power transmission tower is an important truss structure for high-voltage power transmission lines. The inclination measurement of power transmission towers is of great significance to ensure the structural safety during construction, operation, and maintenance of power lines [1, 2]. One key problem is how to accurately measure the inclinations of power transmission towers caused by environmental factors such as foundation settlement [38].

At present, the methods of inclination measurement include plumb method, theodolite method, plane mirror method, total station prism-free measurement method, and laser scanning method [46, 914]. The plumb method has a wide range of applications and is measured by the principle of gravity, but it is easily affected by environment and human factors with large errors, low precision, and big risk; the theodolite method needs artificial prisms, and the instrument needs to be setup multiple times, which is greatly affected by human factors; as for the plane mirror method, due to the complicated setting of the plane mirror, it also requires a lot of manual adjustment, and the measured results need to be calculated twice, so the accuracy is low; as for the total station prism-free measurement method, the precision is mainly affected by environment and manual observation; as for the laser scanning technology, the internal processing time is long, and the equipment cost is relatively high, which is rarely used in the measurement of the inclination of iron towers in recent years.

In general, a common problem of these methods above is that only one-dimensional or two-dimensional inclinations of power transmission towers are measured, which has difficulty in three-dimensional (3D) measurement [4, 5, 11, 13]. Hence, in this research, a method is proposed to measure the 3D inclinations of power transmission towers. In this method, the 3D inclinations are measured using a stereo pair of photos taken by a stereo camera. The main work is the 3D reconstruction model of power transmission tower using stereo images and the calculation of 3D inclinations. Experiment tests are carried out to verify the accuracy of this method.

2. 3D Reconstruction of Power Transmission Tower Model Using Stereo Images

2.1. 3D Reconstruction Method

A stereo pair is a pair of photos (i.e., left and right photos) of a power transmission tower taken by a stereo camera, as shown in Figure 1. Stereo images contain 3D modeling information, which is feasibly used for 3D reconstruction.

The key work of 3D model reconstruction is the accurate calculation of disparity map for 3D reconstruction, which mainly consists of four steps [1517]. The first step is to calculate the matching cost of corresponding pixels in the rectified stereo photos, the second step is to calculate the cost aggregation, the third step is to carry out disparity optimization, and the fourth step is to carry out 3D reconstruction.

Step 1: Calculation of matching cost [1820]. The stereo pair is processed by census transformation to obtain a bitstream code as follows:where denotes the bitstream codes of pixel denotes a value 0 or 1 by comparing with denotes the sort of all the values 0 or 1 in a stream, which forms bitstream codes. The matching cost of pixel p with disparity d in each type of stereo pair is calculated using hamming distance as follows:where denotes the matching cost of pixel p with disparity d, denotes the bitstream code of in the left image and denotes the bitstream code of in the right image, and H denotes calculation of hamming distance [21].

Step 2: Calculation of cost aggregation [1820]. The matching cost is generally ambiguous, and wrong matches can easily have a lower cost than correct ones. Therefore, an additional constraint is added that supports smoothness by penalizing changes of neighboring disparities. Cost aggregation is calculated by referring to a semiglobal matching algorithm as follows:where denotes the cost aggregation of pixel p with disparity d, denotes the cost aggregation of pixel p-r with disparity d, denotes the cost aggregation of pixel p-r with disparity d-1, denotes the cost aggregation of pixel p-r with disparity d + 1, denotes the cost aggregation of pixel p-r with disparity i, and denotes the cost aggregation of pixel p-r with disparity k. P1 and P2 are constant penalties. The costs Lr are summed over paths in all directions r. The disparity of is the value of d corresponding to the minimum value of .

Step 3: Disparity optimization. Disparity optimization contains unique constraint and consistency check [1820]. As for unique constraint, the minimum cost aggregation of pixel p is not close to the other cost aggregations as follows:where denotes the minimum value of cost aggregation of pixel p with disparity is the value corresponding to the minimum cost aggregation, and θ is a constraint value. As for consistency check, the two disparities of pixel p in the left and right photos should be no more than 1 as follows:where denotes the disparity of p obtained from the left photo and denotes the disparity of p obtained from the right photo. If Equation (7) is not satisfied, the calculated disparity of p is ignored.

Step 4: 3D reconstruction of power transmission tower. By referring to the disparity map after disparity optimization, the 3D reconstruction model of power transmission tower is calculated as follows:where W denotes the coordinates in the world coordinate system (WCS), M denotes the coordinates in the camera coordinate system (CCS), and Q denotes the transformation matrix containing the information of intrinsic parameters and camera position parameters.

The result of 3D reconstruction model of a power transmission tower is shown in Figure 2. It can be seen that part of tower members is missing in the 3D reconstruction model. Hence, the current four-step method is not good enough for 3D reconstruction. The reason is that the shape of tower members is usually slender and narrow, the texture characteristics of tower members are poor, and the colors of tower members are similar, which are difficult to obtain a good 3D reconstruction quality.

In order to improve reconstruction quality, a method of calculating matching cost using a combination of multichannel red–green–blue (RGB) stereo pairs and Sobel edge detection stereo pairs is proposed to obtain accurate matching cost as following:where , , , and are the matching costs of red channel, green channel, blue channel, and Sobel edge detection channel and c1, c2, c3, and c4 are the combination coefficients. Usually, a disparity limit D is used to constrain the searching range of d, i.e., . The value of disparity limit D is the biggest disparity in the stereo photos, which is determined by manual identification. In detail, two corresponding pixels from left and right photos, which contain biggest disparity, are selected by hands to calculate the disparity limit D.

In addition, the four-step method is not effective to eliminate the invalid disparities in small closed areas. Hence, a method of invalid disparity judgment is proposed to detect the invalid disparities in small closed areas. In particular, if the differences of adjacent disparities in the same closed areas are less than the limit and the number of adjacent disparities is less than the limit as well, these adjacent disparities are treated as invalid disparities. A neighborhood discriminant formula is used to determine the invalid pixels as follows:where max denotes the maximum value, denotes the invalid disparities, denotes the value limit, denotes the number limit, and NArea denotes the number of invalid disparities in the same closed areas. These invalid disparities are eliminated from disparity map and refilled by interpolation of adjacent valid disparities instead.

The 3D model is reconstructed, as shown in Figure 3(a), where D = 200, , , , , , and . By comparison with Figure 2, Figure 3(a) presents better 3D reconstruction quality, verifying that the method improvement is effective. In addition, the proposed method is compared with three other methods, namely left–right consistency check method, uniqueness detection method, and subpixel fitting method [1520]. The results of the three methods are shown in Figures 3(b) and 3(d); by comparison, it can be seen that the proposed method displays clearer 3D spatial shape than the other three traditional methods, verifying good reconstruction effect of the proposed method.

2.2. Parameter Optimization Analysis

The parameters (i.e., the maximum searching disparity D, the value limit , the number limit , the combination coefficients c1, c2, c3, and c4) have a great impact on the reconstruction quality. Hence, it is important to determine the optimal values of these parameters. These parameters should satisfy the following inequality to ensure a good reconstruction quality:where denotes a function of reconstruction quality and denotes the minimum acceptable reconstruction quality. Bigger value of indicates better reconstruction quality, so the value of is determined according to the requirement of the reconstruction quality. Usually, the value of is not less than 0.90. In particular, the reconstruction quality E is described by the following equation:where denotes the number of 3D model points which satisfies the accuracy and denotes the total number of 3D model points.

A progressive search method is proposed to obtain the optimal values of D, , , , , , and . In this method, each parameter is assigned with many initial values, and the orthogonal test is carried out by combinations of different initial values of these parameters. The values of parameters which satisfies good reconstruction quality EH will be remained and subdivided for the next orthogonal test. Orthogonal tests will terminate when the accuracy Etarget is reached as follows:where Dn−1, , , , , , and denote the values of parameters in the (n−1)th orthogonal test which satisfy good reconstruction quality EH and En denotes the reconstruction quality in the nth orthogonal test; .

An experiment is carried out to give a clear illustration. In the first orthogonal test, the initial values of parameters are , , , , , , and . There are 216 groups of combinations in this orthogonal test through combinations of different values of these parameters. EH is set to 0.80, and is set to 0.90. The results of the first orthogonal test show that the values of parameters which correspond to good reconstruction quality EH are , , , , , , and . Then, these values are further subdivided and used for the next orthogonal test, , , , , , , and . After several orthogonal tests, the accuracy is reached, and the optimal values of parameters are , , , , , , and . The 3D model is shown in Figure 4. It can be seen that the reconstruction quality becomes better by comparison with Figure 4, verifying that the progressive search method is effective.

3. Calculation of 3D Inclinations Using Reconstructed 3D Model

3.1. Calculation Method

The reconstructed 3D models are the collection of 3D point clouds in the CCS, and the rotations measured by inclinometer in the following experiment test are in the WCS. In order to compare the inclinations measured by 3D model and inclinometer in the same coordinate system, the 3D inclinations in CCS are transformed to the values in WCS as follows:where P3D,WCS denotes the 3D points in the WCS, P3D,CCS denotes the 3D points in the CCS, RWC denotes the rotation matrix from CCS to WCS, and TWC denotes translation vector from CCS to WCS. The values of RWC and TWC are estimated using the point coordinates of the same points in two different coordinate systems [2225].

The spatial location of 3D model will change if the power transmission tower has inclinations (i.e., rotations) affected by foundation settlement. Hence, the 3D inclinations are calculated by comparison of the different spatial locations. In other words, the current 3D model (i.e., the 3D model after inclination) is feasible to be obtained through rotation and translation of the reference 3D model (i.e., the 3D model before inclination) as follows:where P1,Current denotes the 3D points of current 3D model, P1,Reference denotes the 3D points of the reference 3D model, , , and denote the rotation angles (i.e., inclinations) around Z-, Y-, and X-axis in the WCS, respectively, RCR denotes the rotation matrix from reference 3D model to current 3D model, and TCR denotes translation vector from reference 3D model to current 3D model. The values of RWC and TWC can be estimated using the coordinates of P1,Current and P1,Reference based on the “iterative closest point” (ICP) algorithm [2225]. Compared with the traditional inclination measurement method such as inclinometer, which only measures one or two directions of rotation angles, this proposed method can measure the 3D inclinations , , and simultaneously.

3.2. Experiment Test

Experiment test is carried out to verify the calculation accuracy. The experiment object is a small-scaled power transmission tower. The height is 1,000 mm, the width is 300 mm, and the length is 300 mm, as shown in Figure 5. The direction of Z-axis is vertical from bottom to top of experiment model, the direction of Y-axis is horizontal from back to front of experiment model, and the direction of X-axis is horizontal from left to right of experiment model. Four elevating feet are used to generate inclinations. The inclinations , β, and γ of power transmission tower during test are measured by a dual-axis digital angle inclinometer, as shown in Figure 5. The measurement resolution of inclinometer is 0.1°, and the measurement accuracy of inclinometer is 0.1°. Big pixel size and short distance between stereo camera and test model will increase the measurement accuracy. In this experiment, the number of pixels is about 4 million, and the distance between stereo camera and test model is about 0.4 m.

The first experiment test is carried out to verify the measurement accuracy of inclination rotating only in the Y direction. Four cases are studied, as shown in Figure 6. In the four cases, the values of inclinations α and γ are always 0, and the values of inclinations are 0, 5, 10, and 15°, respectively. The inclinations are controlled using the inclinometer. The reference 3D models (cases 2–4) and current 3D model (case 1) are reconstructed, as shown in Figure 6, and the 3D inclinations are calculated, as shown in Table 1. It can be seen that the calculated values are close to the measured ones, and the maximum measurement error is 0.11°, indicating that the method is accurate for 3D inclination calculation.

The second experiment test is carried out to verify the measurement accuracy of inclination rotating simultaneously in the X and Y directions. Four cases are studied (case 1 and cases 5–7), as shown in Figure 7. In the four cases, the values of inclination α are always 0, the values of inclination are 0, 5, 10, and 15°, respectively, and the values of inclination are 0, 5, 5, and 10°, respectively. The reference 3D models (cases 5–7) and current 3D model (case 1) are reconstructed, as shown in Figure 7. Then, the 3D inclinations are calculated, as shown in Table 2. It can be seen that the calculated values are close to the measured ones, and the maximum measurement error is 0.08°, indicating that the method is accurate for 3D inclination calculation.

The accuracy of disparity map has a deep effect on the calculation results of 3D inclination. In this research, a combination of multichannel RGB stereo pairs and Sobel edge detection stereo pairs is proposed to calculate the disparity map. In addition, there are also some other methods, such as mean absolute differences (MAD) method, sum of absolute differences (SAD) method, sum of squared differences (SSD) method, mean square differences (MSD) method, normalized cross-correlation (NCC) method, sequential similarity detection algorithm (SSDA) method, and sum of absolute transformed difference (SATD) method. Hence, a comparison of these methods is carried out. The 3D models reconstructed by these methods are shown in Figure 8, and furthermore the values of 3D inclination calculated by these methods are compared, as shown in Table 3. By comparison of the maximum absolute errors of these methods, it can be seen that the proposed method has the smallest error of all, indicating good calculation accuracy of the proposed method.

4. Conclusion and Discussion

4.1. Conclusion

This research proposes a method of high-accuracy measurement of 3D inclinations using the stereo images of power transmission towers. The main conclusions are drawn as follows:(1)A method of calculating the matching cost using a combination of multichannel RGB stereo pairs and Sobel edge detection stereo pairs is proposed to obtain accurate matching cost, and a method of invalid disparity judgment is proposed to precisely detect the invalid disparities in small closed areas. The experiment result shows that the 3D model reconstructed by the proposed methods presents good 3D reconstruction quality by comparison with the other methods such as left–right consistency check method, uniqueness detection method, and subpixel fitting method.(2)A progressive search method is proposed to efficiently get the optimal values of the model parameters D, , , c1, c2, c3, and c4 for 3D reconstruction. The optimal values in the experiment test are D = 167, , , c1 = 1, c2 = 1, c3 = 1, and c4 = 1, respectively. The experiment result shows that the 3D model presents better reconstruction quality after parameter optimization.(3)A 3D inclination calculation method is proposed to calculate the 3D rotation angles using different 3D reconstruction models. The experiment results show that the proposed method is more accurate for calculation of 3D inclinations than MAD method, SAD method, SSD method, MSD method, NCC method, SSDA method, and SATD method.

4.2. Discussion

(1)The size of power transmission tower does not influence the application of this method because this method can use part of the power transmission tower for calculation whether the tower is large or small. This means that if the size of tower is large, only a small part of this tower can be used for calculation.(2)The 3D inclinations can be static or dynamic by setting the frame rate of stereo images, so the sampling frequency of dynamic measurement is determined by frame rate, i.e., high frame rate indicates high sampling frequency. Because the processing time of a stereo image to calculate 3D inclinations is about several minutes, the measurement of 3D inclinations is offline measurement.(3)The 3D reconstruction model is built under the WCS, which has no relevance to camera poses, so the camera poses do not affect the 3D reconstruction model. Compared with traditional inclination measurement methods such as inclinometer, which only measure one or two directions of rotation angles at most, this proposed method can measure the 3D inclinations , , and simultaneously.(4)The result is tested under experiment environment, which is different from field environment. The future work is to further verify the proposed method in the field test, which will have great value in the construction, operation, and maintenance of power lines. The field environment is more complex than experiment environment, and the factors such as the light, vibration, and weather should be considered in the further research.

Data Availability

The data used to support the findings can be obtained by contacting via [email protected].

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

The authors gratefully acknowledge the 2021 Science and Technology R&D Project (Science and Technology Project of State Grid Jiangsu Electric Power Co., Ltd.) (no. J2021022).