High-Precision Volume Measurement of Potholes in Pavement Maintenance
Accurate three-dimensional measurement of potholes is a significant concern in road maintenance. However, the assessment of road potholes still relies heavily on human inspectors to make maintenance suggestions. To realize full-field measurement of pavement pothole automatically, a high-precision volume measurement method based on second-order Taylor expansion is proposed, where the second-order Taylor estimation of each point is converted into a convolution operation. On the one hand, this method discards the second-order fitting of the surface, which greatly reduces the computational complexity. On the contrary, the second-order Taylor estimation is not restricted by the surface shape because it only depends on the distribution of the adjacent points. Experiments on synthesized and real point sets demonstrate that the proposed method outperforms the state-of-the-art methods under various point cloud shapes.
Due to vehicle traffic and temperature change, roads expand/shrink and crack, leading to potholes, which can cause accidents, uncomfortable journey, and damage to the vehicle’s wheels [1, 2]. Therefore, timely maintenance and repair of pavement are necessary . Since the traditional artificial detection and evaluation are slow and expensive, pavement detection and evaluation gradually transit from artificial to intelligent [4–6]. An efficient and intelligent inspection system includes two parts: detection and measurement. Pothole detection helps to identify and determine a pothole position. Pothole measurement can be divided into 2D profile measurement and 3D size measurement, which can assist road managers in determining timely maintenance needs . The 3D size measurement results directly affect the amount of filling and the quality of pavement maintenance. With the rapid development of computer vision and sensors, many pothole detection [2, 4, 8] and measurement methods [5, 9, 10] have emerged. The maturity of pothole detection technology [2, 4, 11, 12] and 2D profile measurement [5, 9, 13, 14] has been driven by large computing power and data. However, little attention has been paid to high-precision 3D measurement, especially for volume measurement. Accurate volume information corresponds to an appropriate repair plan, which is more conducive to vehicle traffic. In this study, the precise volume measurement of potholes has been studied.
The volume measurement techniques are mainly divided into two categories: linear surface approximation and nonlinear surface approximation. For the former, the surface is mostly approximated by convex polyhedron. The main factors affecting the precision of such linear surface approximation is the division strategy of the point set, which includes grid partition and triangulation [15, 16]. Triangulation is flexible and can accurately divide various shapes. As triangulation was proposed, many variants of the standard procedure have been raised, such as 3D alpha shape (3DAS)  and triangulated irregular network (TIN) [18, 19].
The nonlinear approximation is divided into geometric primitives’ fitting [20–22] and nonlinear surface fitting [23, 24]. Furthermore, geometric primitives’ fitting includes geometries such as sphere, cuboid, cylinder, and cones, in which the volume of the fitted geometry is regarded as the point set volume. This method has a good supplementary effect on the incomplete point cloud data. In other words, this method is robust to points missing. With the development of deep learning, a learning-based framework has been proposed, in which the volume estimation is regarded as a general regression framework [25–27]. However, because of the variety of geometry and topology of real-world shapes, geometric primitives’ fitting has a narrow range of application. For nonlinear surface approximation, the surface is mainly approximated by polynomials [23, 28] or B-spline patches [24, 29]. Then, the nonlinear surface is used to calculate the point set volume. This method has less constraints on the shape of the point set. However, the application of the algorithm is restricted by the large computational complexity and the choice of the nonlinear function. Regardless of geometric primitives’ fitting or nonlinear surface fitting, the point set surface distribution needs to be informed in advance. The geometry needs to be known in geometric primitives’ fitting and the nonlinear function form must be set in advance for nonlinear surface fitting.
In this study, a second-order estimation method for point set surface has been proposed, which does not need to set the equation form in advance. Moreover, the second-order estimation is converted into a convolution operation, which greatly reduces the computational complexity. In detail, Taylor expansion is used to divide volume estimation into two parts: polyhedron volume estimation and second-order component estimation. The polyhedron volume can be estimated accurately by triangulation and the second-order component can be measured by convolution operation. According to Taylor’s remainder, the error range can be accurate estimated with division accuracy.
2. The Proposed Method
For any point set, let us set its projection plane is X-Y, and its points are regarded as the sampling points generated by a curved surface . Next, a high-precision volume estimation method between the projection plane and the surface is given.
For the surface , collect the terms up to second order:where is the Euclidean distance between reference point and adjacent point , is the partial derivative of with respect to , and is the mixed second-partial of at . Other partial derivatives are defined in the same way. The partial derivative and the second-partial, respectively, represent the linear and quadratic parts of the surface. is an infinitesimal of higher order than .
For the sake of clarity, the whole calculation process is divided into two parts:(1)Prove that the linear approximation of surface is the surface’s polyhedron approximation (see Appendix A, for exact details)(2)Estimate the second-order component of surface
Second-order estimation needs more neighboring points. A wide-ranging meshing is shown in Figure 1.
For the second-order component of , its integration in the region is
The second-partials are consistent with Newton's divided differences of partial derivative, which can be expressed as three-point centered-difference formula. By differential representation, the second-order component of volume can be approximated by the following formula:
For the other three points , , and , we process the second-order parts in the same way. Then, the average and symmetric approximation of second-order component is
Figure 2 gives a visual representation of (4). For display convenience, each coefficient is multiplied by 48. From the figure, we can see that the coefficients are mainly distributed in the lower right of the region . Moreover, the positive and negative coefficients are alternately distributed.
To increase the stability of the algorithm, the partial derivatives of the other three directions are added. The merging process is shown in Figure 3.
The coefficients on each node in the grid are superimposed, and the final coefficient distribution of second-order component is as shown in Figure 4. We call the final coefficient distribution the second-order estimation kernel (SEK). Because the coefficients in Figure 4 are the superposition of the four directions, the SEK needs to be divided by (48 4).
Because the boundary pixels do not completely overlap with the kernel, the boundary pixels cannot be convoluted. For this convolution boundary problem, a common and effective method is edge filling. In detail, add the edge points and set the depth value of the filled edge points to 0 before convolution. Due to the uncertainty of point distribution, edge filling with 0 may lead to curvature jump and bring some noise to the second-order component of surface. In order to reduce the influence of edge filling, the median value is used for noise suppression in the final second-order approximation.
3. Error Analysis
To clarify the accuracy improvement of this method, the second-order Taylor series expansion can be written compactly aswhere is gradient of and is Hessian matrix.
The volume in each mesh is same with the integration of in the mesh:
Obviously, the first-order term and are infinitesimals of the same order. The second-order term and are infinitesimals of the same order. Let be the meshing accuracy; then, the precision of polyhedron volume and are infinitesimals of the same order. The precision of our volume and are infinitesimals of the same order. For example, if is larger than 10, our method can increase volume accuracy by more than an order of magnitude.
Because SEK implements the second-order component of the volume, SEK reflects the curvature of the reference point and the adjacent points. For points on the boundary or near the creases of two bending planes, SEK will have nonsmooth fluctuations due to sudden fluctuations. Such some second-order estimation noise will be generated. To reduce noise, we use the average of second-order estimation as the final second-order component.
As can be seen from Section 3, if the partition accuracy is , the precision of the one-order volume estimation is the same-order infinitesimal of ; the precision of our method is the same-order infinitesimal of . In other words, in the case of accurate point cloud acquisition, the precision of the one-order volume is , and the accuracy of our volume is .
Take a hemisphere with a radius of 1, assuming it is embedded in the zero plane, as shown in Figure 5.
To give convincing results, four methods are included in our comparative studies such as 3DAS , TIN [18, 19], cubic polynomial fitting (CPF) [30, 31], and quartic polynomial fitting (QPF) [30, 31]. 3DAS finds the contour of the point set, and then, the volume of the contour is regarded as the volume of the point set. By triangulating the top and bottom surface of the point set, TIN achieves irregular 3D division of the point set. Calculating the volume of each division, the volume of the point set is the sum of all the divided volumes. For CPF and QPF, the point surface is given as bivariate function including higher-order polynomial terms. Then, the volume of point set can be measured by double integral of bivariate function.
The results of 3DAS, TIN, CPF, QPF, and the proposed method on the hemisphere are shown in Tables 1 and 2. Tables 1 and 2 record the measurement results with X-Y plane division accuracy of 0.1 and 0.01, respectively. The radius of the hemisphere is 1 and the volume is 2.0943951. The error in the table is the absolute value of the deviation between the measured value and the true value.
It can be seen from Tables 1 and 2 that the 3DAS and QPF have poor measurement results. 3DAS calculates the convex hull volume of the point set, which is sensitive to bumps or sudden transitions. Bumps or sudden transitions will cause the measured volume be larger than the real volume. QPF is worse than CPF, indicating that the quartic polynomial is not suitable for the hemisphere. Furthermore, the measurement errors of CPF and QPF are larger than that of TIN, which means that the cubic or quartic fitting is not as good as polyhedron. The proposed method has the highest accuracy, which adds the second-order component of surface into volume estimation. As can be seen from the table, the accuracy of the proposed method is an order of magnitude higher than the other four methods regardless of division accuracy.
The second-order component coefficient distribution of hemisphere is shown in Figure 6. It can be seen that, at the intersection of the road and the hemisphere, the second-order estimator has obvious fluctuations. This is because SEK reflects the curvature change of the points and surrounding points. The fluctuations can be positive or negative, which, respectively, reflect the convexity and concave of the surface. In mathematics, convexity means bulging downward and concave means bulging upward. With 0.1 partition accuracy, the second-order component coefficient distribution of hemisphere has an average value of 0.78, which reflects the magnitude of the curvature change of the point set. When the partition accuracy is 0.01, its average value is 0.0078, which is smaller than the average value with 0.1 partition accuracy because the finer division results in the smoother transition between points.
4.2. Various Geometries
To test the robustness of the proposed method, several geometries are tested as shown in Figure 7. These geometries include mutation edges, gradual edges, and concave.
For various geometries, Table 3 records the measured results of different methods with 0.1 partition accuracy of the X-Y plane. In the table, “True V” represents the true value of the volume. The average errors of 3DAS, TIN, CPF, QPF, and the proposed method are 4.2876, 0.0371, 0.5694, 2.5830, and 0.0217, respectively. Overall, the measurement results of our method are closest to the “True V.”
Theoretically, for a certain partition accuracy, the error of the proposed method will be less than a specific value. For example, the error should be less than 0.01 with 0.1 partition accuracy. However, for different geometries, the errors have large fluctuations and exceed the error threshold mentioned above. This is because the X-Y partition accuracy is not the same as the point set partition accuracy. For bathtub, the maximum distance of the point set partition is as high as 0.4359 with 0.1 X-Y partition accuracy. The division accuracy of 0.4359 corresponds to the error threshold of 0.8282. TIN, CPF, and the proposed method can all achieve this accuracy, and TIN and the proposed method can far exceed this threshold. This means that, in the case of uneven division, the error threshold estimation is inaccurate. In other words, the error threshold under this partition cannot be estimated as the nonuniformity of the partition. Although we cannot estimate the error threshold, we can see that the proposed method can effectively improve the volume measurement precision of the point set.
Table 4 shows the measurement results of different methods with 0.01 partition accuracy of X-Y plane. The average errors of 3DAS, TIN, CPF, QPF, and the proposed method are 4.425271, 0.000719, 0.589593, 2.918653, and 0.000461, respectively.
Compared with the 0.1 partition accuracy, the errors of 3DAS, CPF, and QPF are almost constant when the division accuracy is 0.01. This is because these three methods are based on the global distribution of the point set. Obviously, the fine division does not affect the overall distribution. On the contrary, the measurement accuracy of TIN and the proposed method has been greatly improved with the improvement of division accuracy. However, the error of the proposed method is smaller than that of TIN.
Compared with the other four methods, the proposed method has greatly improved the accuracy of volume measurement for various geometric shapes.
4.3. Real Pothole
This experiment was carried out by 3D-printed pothole, with a design volume of 0.0003220798 cubic meters and a printing error of 2 mm, as shown in the left of Figure 8. The right image of Figure 8 is the scanned point cloud of pothole by FaceGo, with a scanning accuracy of 0.1 mm. Assume that the true value of pothole is 0.0003220798 cubic meters.
For the pothole, the measurement results of 3DAS, TIN, CPF, QPF, and the proposed method are shown in Table 5. Both TIN and the proposed method have small measurement errors. Due to fine division, TIN has high measurement accuracy. However, the error of our method is smaller than that of TIN, which means that second-order component has been effectively calculated.
On the whole, whether for simulated or real pothole, the proposed method can effectively improve the measurement accuracy.
In this study, we proposed a high-precision volume measurement method, which calculates the first-order and second-order components of the volume separately. The second-order component can be used to describe the curvature of each point for point set segmentation or registration. Furthermore, the high-precision volume of the point set can be used in bi-ventricular volume estimation, disease surveillance, road maintenance, and so on. In the future, how to calculate the volume of any form of point cloud will be the main research direction.
In this study, a high-precision volume measurement method is proposed, which estimates the linear and second-order components of the volume separately. The second-order component of each point is converted into a convolution operation with 6 6 kernel, which greatly reduces the computational complexity and weakens the constraints on the surface shape. Extensive experiments on synthesized data and real data demonstrate that the proposed method outperforms the state-of-the-art methods under various point cloud shapes.
A. Polyhedron Approximation
In this section, we prove that the linear approximation of surface is the surface’s polyhedron approximation. For a surface , the grid partition of its projection plane X-Y is shown in Figure 9, where the blue lines indicate the division of the surface by the grid.
Using first-order Taylor polynomial of at point , the volume on the region iswhere the area of region is denoted by . The finite difference approximations of first partial derivatives are
So, the volume in the region can be approximated by
Similarly, for the other three points , , and , using first-order Taylor polynomial of at these points, the volume over the region can be written as follows:
So, using the first-order Taylor expansion of four points, which are the four vertices of region , a symmetric expression about the volume is obtained:
This is to say, the volume is equal to the ground area multiplied by the average height. It is simple to proof that this volume is equal to the polyhedron volume. In a word, the linear approximation of surface equals to the polyhedron volume.
The .m, .tdu xt, and .mat data used to support the findings of this study have been deposited in the HuangHuim/volume-measurement-of-point-cloud repository (https://github.com/HuangHuim/volume-measurement-of-point-cloud).
Conflicts of Interest
The authors declare no conflicts of interest.
Zuofeng Zhou and Jianzhong Cao conceptualized the study; Huimin Huang curated the data; Zuofeng Zhou carried out formal analysis; Mulong Liu and Huimin Huang carried out funding acquisition; Huimin Huang investigated the study; Huimin Huang developed the methodology; Mulong Liu and Huimin Huang administrated the project; Huimin Huang, Qingquan Wu, and Guoliang Hu collected the resources; Huimin Huang and Qingquan Wu helped with software; Zuofeng Zhou and Jianzhong Cao supervised the study; Mulong Liu and Guoliang Hu validated the study; Huimin Huang and Mulong Liu wrote the original draft; Zuofeng Zhou reviewed and edited the manuscript.
This work was supported by the Natural Science Basic Research Program of Shaanxi, with Grant no. 2022JQ-688, Chinese Universities Scientific Fund, with Grant no. 2452020264, Key Research and Development Program of Shaanxi Province, with Grant no. 2020ZDLGY04-03, and project “Research on rapid acquisition of remote sensing image and its application technology in commercial aerospace: ANSO-CR-KP-2020-15.”
N.-D. Hoang, “An artificial intelligence method for asphalt pavement pothole detection using least squares support vector machine and neural network with steerable filter-based feature extraction,” Advances in Civil Engineering, vol. 2018, Article ID 7419058, 12 pages, 2018.View at: Publisher Site | Google Scholar
P. Labatut, J. P. Pons, and R. Keriven, “Efficient multi-view reconstruction of large-scale scenes using interest points, delaunay triangulation and graph cuts,” in Proceedings of the International Conference on Computer Vision, pp. 1–8, Rio de Janeiro, Brazil, October 2007.View at: Publisher Site | Google Scholar
Z. Li, C. Zhu, and C. Gold, Digital Terrain Modeling: Principles and Methodology, CRC Press, Boca Raton, 2005.View at: Publisher Site
B. Twala, T. Marwala, and Y. Tekane, “Landscape aware miniature aerial vehicles (MAVs),” University of Johannesburg, Johannesburg, South Africa, 2020, Dissertation.View at: Google Scholar