BSV System with MSC and Its Application in the Measurement of Surface Cracks

Ma, Hua; Yang, Guang

doi:https://doi.org/10.1155/2023/5935284

Mathematical Problems in Engineering

On this page

Abstract Introduction Methods Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2023 | Article ID 5935284 | https://doi.org/10.1155/2023/5935284

BSV System with MSC and Its Application in the Measurement of Surface Cracks

Hua Ma¹and Guang Yang¹

Academic Editor: Francesco Cannizzaro

Received01 Aug 2022

Revised15 Jan 2023

Accepted19 Jan 2023

Published30 Jan 2023

Abstract

In this paper, we have proposed a flexible noncontact crack-size measurement method that can realize binocular stereo vision measurement with only a single camera. On the premise that the camera’s intrinsic parameters have been accurately calibrated, we use a camera to collect the image of the crack from two directions. Then, we calculate the motion parameters using the collected images from the camera in different positions. In addition, Canny algorithm is used to extract the edge pixels of crack images. Finally, we establish the binocular stereo vision model for crack measurement according to the camera parameters, the motion parameters, and the edge information of crack images. Thus, we can measure the crack size through this model. Experimental results show that the measurement error is less than 5% under a distance of 2 meters, which can effectively prove the precision of the proposed method. In addition, our method only uses a single camera. Compared with the traditional binocular stereo vision method, this method is not only flexible but also more economical.

1. Introduction

In this work, we are devoted to seeking a more flexible and economical crack measurement method based on computer vision under the premise of ensuring the measurement precision. Crack measurement plays a fundamental role in monitoring of structure stability and safety. The cracking process might also be expected during the serviceability of concrete structures, but when the fracture magnitude (width and linear extension) significantly increases, this could lead to a deterioration in steel reinforcement rods or it could be a presignal of forthcoming collapse. Until now, crack measurement has remained a common problem in railway engineering [1, 2], structural engineering [3], road engineering [4], and water conservancy projects [5, 6].

Traditional measurement methods for structural surface cracks include the microscopic method, ultrasonic method, and displacement sensor method. The microscopic method measures the crack by using a crack microscope [7]. When we use a crack microscope, we place the lens parallel to the crack. This will lead to inefficiency and inconvenient operation. The crack depth can be obtained by ultrasonic methods [8]. Similar to the microscopic method, there are still problems of inconvenient operation and low efficiency. The sensor method places a displacement sensor inside the material in advance so that the real-time data of surface cracking can be obtained [9]. Although the method is of high precision, it is often expensive and can only be used for one crack, which is generally suitable for real-time crack monitoring in key structures. In recent years, the crack measurement method based on multisensor fusion has been gradually applied to engineering practice. For instance, two methods for fusion-based damage diagnostics for stiffened composite panels have been proposed in literature [10], which demonstrates that the intrinsic capacity of the two techniques can be utilized, leading to synergistic effects for damage diagnostics. Literature [11] proposes a methodology to fuse signals from multiple ultrasonic sensors and detect cracks in the reinforced concrete reference structure. In essence, literature [10, 11] still belongs to ultrasonic methods which have the disadvantage of inconvenient operation.

The abovementioned methods have problems of inconvenient operation (microscopic methods and ultrasonic methods) or high hardware costs (displacement sensor methods). At present, with the development of digital image processing technology, surface crack detection technology based on computer vision has been widely used [12]. It refers to using the camera to collect the images of the crack, and then, the geometric dimension information of the crack can be obtained according to the image. This is a noncontact measurement method, which is fast and highly automated. It is often mounted on drones or vehicles to achieve rapid and accurate measurement of cracks.

However, in recent years, crack detection methods based on computer vision have mainly focused on how to identify cracks [13, 14], and there is little research on how to measure the size of cracks. Literature [15, 16] studied how to use digital images for size measurement. However, this method needs to strictly ensure that the camera is parallel to the plane of cracks; otherwise, the measurement error will be uncontrollable. Fortunately, binocular stereo vision (BSV) technology can be used to solve this problem. For the BSV method, it usually requires two cameras. In addition, the relative positions of the two cameras shall be kept strictly unchanged during the measurement process. This will adversely affect the flexibility and economy of measurement.

To further improve the flexibility and economy of BSV, we propose a crack measurement method using BSV with a moving single camera (MSC) in this paper. First, we use a camera to take crack images from two different directions. Then, we use the feature-matching method, camera motion estimation method, and image edge extraction method to realize the high precision and automatic measurement of crack width. This method uses only one camera, which significantly improves the flexibility of measurement. Compared with the traditional BSV, our method only needs one camera, so the flexibility is greatly increased. Besides, if the type of the camera is the same, our method can be half cheaper for imaging devices; compared with other monocular vision methods, our method does not need to keep the camera parallel to the crack plane and can achieve high shooting accuracy at different shooting angles. As to the ultrasonic method, acoustic information fusion method, microscopic method, and displacement sensor method, our method is a noncontact measurement method with a higher degree of automation.

2. Measurement Principles and Methods

2.1. Measurement Purpose

In general, we use the minimum distance method to calculate the crack width. On the premise that the spatial positions of the each point on the two edges of the crack are known, for each point on one edge, we calculate the minimum distance from the point to the other edge by

In Equation (1), denotes a point on the edge of one side of the crack and denotes a point on the edge of the other side. The maximum value of all calculated dis_i is the maximum width of the crack, which is the purpose of crack measurement.

After we have cleared the purpose of measurement, we need to know the spatial positions of each point on the two edges of the crack. In the following section, we introduce how to use BSV to calculate the 3D coordinates of spatial points.

2.2. Basic Principle of BSV with MSC

Essentially, BSV with MSC is a generalization of BSV. In this section, we take BSV as an example to explain this in detail.

As shown in Figure 1, if O₁ and O₂ are two cameras, we call it the BSV system. If they are just two different positions of the same camera, we call it the BSV system with MSC. In this paper, we only discuss the latter case.

The projection points of the point X on the imaging plane of camera position 1 and position 2 are x₁ and x₂, respectively, and x₁ and x₂ are the so-called corresponding points.

The optical centers of the camera at the two positions are O₁ and O₂, and the projection matrices of the camera at these two positions are M₁ and M₂, respectively.

We call the three-dimensional space, where point X is located, the world coordinate system, and the two-dimensional plane, where point x₁ and x₂ are located, is called the image coordinate system. In addition, there is a camera coordinate system in the BSV system, and O₁ and O₂ are the origin of the camera coordinate system at the corresponding position, as shown in Figure 1.

We write the coordinates of x₁, x₂, and X as homogeneous coordinates; that is, x₁ = (u₁, , 1), x₂ = (u₂, , 1), and X=(x, y, z, 1). The relationship between them can be described as

In Equations (2) and (3), s₁ and s₂ can be eliminated as intermediate variables. Essentially, Equations (2) and (3) are the equations of lines O₁x₁ and O₂x₂, respectively, and X is the intersection of the two lines. In this way, we can calculate the spatial three-dimensional coordinates of the point X through the projection points of the spatial points in the two positions of the camera. This is the basic principle of BSV, and the specific derivation process can be referred to literature [17].

The projection matrix M₁ can be decomposed as M₁ = K [R₁, t₁], and so is M₂. K denotes the intrinsic parameter matrix of the camera. R₁ denotes the rotation matrix of camera position 1, and t₁ denotes the corresponding translation vector. The matrix is defined as the rotation from the camera coordinate system to the world coordinate system, and the translation vector is defined as the translation from the camera coordinate system.

We set the camera coordinate system of position 1 to coincide with the world coordinate system, and then, we can get M₁ = K [I, 0], where I is the identity matrix. In this case, we set the rotation and translation from position 2 to position 1 as R and t, respectively; then, we can get M₂ = K [R, t]. R and t are called motion parameters; the process of solving motion parameters is called motion estimation [18].

2.3. Measurement Method

In Section 2.2, the basic principle of BSV with MSC has been explained, and in this section, we describe how to measure cracks in this method.

The schematic diagram of surface crack measurement based on BSV with the MSC system is shown in Figure 2. When we use the camera to capture the same one crack image from position 1 and position 2, respectively, we can obtain the three-dimensional coordinates of a point on the edge of the crack. Ulteriorly, we can complete the measurement of the maximum width according to Equation (1).

It has been previously described how to use BSV with the MSC system to measure cracks in detail. However, there are still two key problems: one is how to solve the intrinsic parameter matrix K and motion parameters [R, t], and the other is how to effectively recognize the crack edge from the image. In the following sections, we discuss these two problems in detail.

3. Key Technologies

3.1. Solution of System Parameters

According to the analysis in Section 2.2, the parameters mainly include the camera’s intrinsic parameter matrix K and motion parameters R and t. As for the matrix K, its solution method has been quite mature. In this paper, Zhang’s calibration method is adopted to solve it with a plane checkerboard [19]. In the following, we focus on how to solve the motion parameters of the camera.

Equation (4) describes the so-called “polar equation,” where F is the fundamental matrix, and it has the following properties:

We set t = [t₁, t₂, t₃]^T, and in Equation (5),

We setwhere E is called the essential matrix.

We set

We can get

The essential matrix is only related to the motion parameters of the camera. Therefore, the motion parameters of the camera can be solved by the essential matrix.

Typical methods for solving the essential matrix include the “5-point algorithm” [20] and the “8-point algorithm” [21]. In other words, if we can find at least 5 pairs of corresponding points, we can calculate the essential matrix. In addition, considering that data redundancy can reduce errors, we can use more corresponding points to calculate the essential matrix.

Once we have calculated the essential matrix, we can decompose this matrix by the method of SVD to get [t]_× and R [22].

However, the translation vector obtained by the abovementioned method can only obtain the direction of translation but cannot obtain the size of the translation vector. The translation vector and the real translation vector are proportional, and the ratio is called the “scale factor.” Usually, the scale factor is deduced according to some objects with known size in the image, such as a circle with known radius and a square with a known side length. The method of obtaining the scale factor can be referred to literature [23].

3.2. Image Edge Detection

In this paper, edge detection refers to the detection of edge pixels of crack images. Essentially, image edge is a discontinuous and abrupt form of image local features. Therefore, edge detection is actually performed to find the position of local feature mutation in the image and obtain the two-dimensional coordinates of this position for the image.

A gray level is a basic attribute of images, and the gradient of images describes the change rate of the gray level. Generally speaking, the gray level of the edge region changes faster. Therefore, we can design edge detection algorithms according to this characteristic. In this paper, we adopt the Canny algorithm to detect the crack edge in the image [24, 25].

The core steps of the Canny algorithm are image smoothing, gradient calculation, nonmaximum suppression, and dual-threshold detection. For a typical crack image of a surface, the Canny algorithm is used for crack edge detection, and the detection results are shown in Figure 3.

(a)

(b)

4. Crack Measurement Experiment

In this section, we describe that how we adopt BSV with the MSC system to measure the maximum width of a crack. The camera parameters used in the system are shown in Table 1.

In the following, we first describe the experimental process and then verify the precision of our proposed measurement method.

4.1. Experimental Process

We use the camera to collect an artificially constructed crack image from two different directions. In addition, in order to estimate the scale factor, a black-and-white checkerboard is placed near the crack, and the size of the individual squares on the checkerboard is 10 mm × 10 mm. The crack image we collected is shown in Figure 4.

(a)

(b)

The camera shooting angle in Figure 4(a) is 80° and that in Figure 4(b) is 70°. Here, we need to pay attention to the camera shooting angle, and Figure 4 shows only a schematic diagram, which shows shooting from different angles; we do not have strict requirements for shooting angles.

To obtain the corresponding points required to estimate the essential matrix E, we adopt the SIFT algorithm to match the feature points of Figures 4(a) and 4(b). The results are shown in Figure 5.

The intrinsic parameter matrix of the camera is K. Then, when the corresponding points and the grid size in the image are known, we can calculate the rotation matrix R and the translation vector t of the camera from position 2 to position 1 by the method described in Section 3.1.

In addition, we can use Equation (10) to calculate the homography matrix H from Figures 4(b) to 4(a) through K, R, and t.

In Equation (10), n denotes the unit normal vector of the imaging plane of camera position 2 in the coordinate system of camera position 1, d denotes the distance between the coordinate system origin of camera position 1 and the imaging plane of camera position 2, and n_d = n/d. For any pair of corresponding points, for instance, x₁ and x₂, H has the following properties.

After the camera motion estimation is completed, the Canny algorithm is used to detect the edges of the two images, respectively. The results of edge detection are shown in Figure 6.

(a)

(b)

We set x_2i (u_2i, ) to be the projection point of X_i (x_i, y_i, z_i) in Figure 4(b), where X_i is a point on the edge of a crack in the world coordinate system. Then, the corresponding point of x_1i in Figure 4(a) can be calculated by x_2iH, that is, x_1i = x_2iH.

Here, we need to pay attention that since the estimation results of camera motion parameters are generally slightly different from the real values, the coordinates of the corresponding points of the point x_2i in Figure 4(a) calculated by the homography matrix are often inconsistent with the real coordinates of the corresponding points. Then, we can take the edge point closest to the Euclidean distance of the point x_2iH in Figure 4(a) as the corresponding point of x_2i, as shown in Figure 7. The coordinates of the true corresponding point are recorded as x_1i = (u_1i, ).

Now, we have obtained x_1i (u_1i, ) and its corresponding projection matrix M₁ = K [I, 0] and x_2i (u_2i, ) and its corresponding projection matrix M₂ = K [R, t]. Then, we can calculate the spatial three-dimensional coordinates of X_i according to the method described in Section 2.2.

After the three-dimensional coordinate calculation of all points on the two edges of the crack is completed, the method in Section 2.1 can be used to calculate the maximum width of the crack.

4.2. Experimental Results

The solution results of the camera’s intrinsic parameter matrix K and motion parameters R and t are shown in Table 2.

For the crack shown in Figure 4, its known maximum width (7.032 mm) is used as a reference value. Then, under the conditions of a measurement distance of 0.5 m, 1.0 m, 1.5 m, and 2.0 m, the crack width is measured according to the experimental process described in Section 4.1, and the shooting angles of the two cameras at the two positions for the four distances are roughly 60° and 80°, respectively. The measurement results and measurement error are shown in Table 3.

It can be seen in Table 3 that, under different measurement distances, the BSV with the MSC system can achieve the measurement error within 5%. This shows that the proposed measurement method has good reliability. In addition, this method does not need contact measurement, so it has good convenience.

Besides, to verify the advantages of our proposed method over similar computer vision-based crack measurement methods, we compare the measurement precision of our method with the methods in literature [15, 16]. We set the measurement distance to 1 m and compare the measurement error under different camera shooting angles, and the cracks used for measurement are still shown in Figure 4. The experimental results are shown in Table 4.

As shown in Table 4, under the condition that the camera imaging plane is parallel to the plane of the crack, that is, the camera shooting angle is 90°; the measurement precision of the three methods is very close. However, when we change the shooting angle, only our method can maintain relatively reliable measurement precision. This fully shows that the proposed method has good reliability while ensuring flexibility.

5. Conclusion

In this paper, we have proposed a crack measurement method based on BSV with MSC using computer vision. This is an intelligent, fast, and highly automated measurement method. The main contributions in this article are as follows:(1)We only use one camera to realize the function of traditional BSV. Compared with the traditional BSV with two cameras, this method is more economical. Compared with the ultrasonic method, microscopic method, sensor method, and traditional monocular vision method, the flexibility of this method is greatly improved.(2)We construct a complete crack measurement system and realize the noncontact automatic measurement of cracks in a large space. The experimental results show that our method can ensure the measurement precision and has higher flexibility.(3)In this measurement system, we integrate the feature matching method, motion estimation method, and edge extraction method. It can directly calculate the crack size according to the collected image, which makes the measurement results more objective and less interfered by human factors.

The measurement precision of our method is independent of the camera’s shooting angle. In addition, compared with monocular vision measurement systems using the same imaging device, our method is more economical. Therefore, this method is economical, flexible, and has high precision. Furthermore, it can be combined with an unmanned aerial vehicle (UAV) so that it can measure cracks in complex or dangerous situations.

6. Future Work

In the measurement experiment, we use a relatively ideal example; that is, there is no interference of other adverse features in the measurement scene. In fact, when using the computer vision method to solve such problems, we must consider how to accurately segment the crack region in the image with other regions.

In future work, we plan to combine crack detection with crack measurement [13, 14, 26]. We will first use the current depth learning method to accurately find the location of the crack from the collected image and extract it. Then, for the extracted crack image, we use the method in this paper to measure its size. In this way, the system will be more intelligent and more widely used.

Data Availability

The data calculation results are provided in the manuscript.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This research was funded by the Key Scientific and Technological Projects of Henan Province (China), grant number 212102310904.

References

M. S. A. Hashmi, M. Ibrahim, I. S. Bajwa et al., “Railway track inspection using deep learning based on audio to spectrogram conversion: an on-the-Fly approach,” Sensors, vol. 22, no. 5, Article ID 22051983, 2022.
View at: Publisher Site | Google Scholar
H. Wang, J. Berkers, N. van den Hurk, and N. F. Layegh, “Study of loaded versus unloaded measurements in railway track inspection,” Measurement, vol. 169, Article ID 108556, 2021.
View at: Publisher Site | Google Scholar
S. Shim, N. Saltybaeva, N. Berger, M. Marcon, H. Alkadhi, and A Boss, “Lesion detectability and radiation dose in spiral breast ct with photon-counting detector technology: a phantom study,” Investigative Radiology, vol. 55, no. 8, pp. 515–523, 2020.
View at: Publisher Site | Google Scholar
D. R. Sulistyaningrum, B. Setiyono, J. N. Anita, and M. R. Muheimin, “Measurement of crack damage dimensions on asphalt road using gabor filter,” Journal of Physics: Conference Series, vol. 1752, no. 1, Article ID 12086, 2021.
View at: Publisher Site | Google Scholar
Y. Tang, Z. Huang, Z. Chen et al., “Novel visual crack width measurement based on backbone double-scale features for improved detection automation,” Engineering Structures, vol. 274, no. 274, Article ID 115158, 2023.
View at: Publisher Site | Google Scholar
Y. Tang, Z. Chen, Z. Huang, Y Nong, and L Li J, “Visual Measurement of dam concrete cracks based on U-net and improved thinning algorithm,” Journal of Experimental Mechanics, vol. 37, no. 2, pp. 209–220, 2022.
View at: Google Scholar
M. Alimoradzadeh and Ş. D. Akbaş, “Superharmonic and subharmonic resonances of atomic force microscope subjected to crack failure mode based on the modified couple stress theory,” The European Physical Journal Plus, vol. 136, no. 5, p. 536, 2021.
View at: Publisher Site | Google Scholar
J. Chakraborty, A. Katunin, P. Klikowicz, and M. Salamak, “Early crack detection of reinforced concrete structure using embedded sensors,” Sensors, vol. 19, no. 18, p. 3879, 2019.
View at: Publisher Site | Google Scholar
X. Zhou and Q. Yu, “Wide-range displacement sensor based on fiber-optic fabry–perot interferometer for subnanometer measurement,” IEEE Sensors Journal, vol. 11, no. 7, pp. 1602–1606, 2011.
View at: Publisher Site | Google Scholar
A. Broer, G. Galanopoulos, R. Benedictus, T. Loutas, and D. Zarouchas, “Fusion-based damage diagnostics for stiffened composite panels,” Structural Health Monitoring, vol. 21, no. 2, pp. 613–639, 2022.
View at: Publisher Site | Google Scholar
J. Chakraborty and M. Stolinski, “Signal-level fusion approach for embedded ultrasonic sensors in damage detection of real RC structures,” Mathematics, vol. 10, no. 5, p. 724, 2022.
View at: Publisher Site | Google Scholar
K. W. Chou, L. G. He, K. Hsu, W. C. Liao, and C. W. Huang, “Application of an unmanned aerial vehicle for crack measurement using image calibration supported by laser projectors,” Multiscale Science and Engineering, vol. 3, no. 4, pp. 225–235, 2021.
View at: Publisher Site | Google Scholar
Q. Zhou, Z. Qu, and C. Cao, “Mixed pooling and richer attention feature fusion for crack detection,” Pattern Recognition Letters, vol. 145, pp. 96–102, 2021.
View at: Publisher Site | Google Scholar
K. E. Tatsis, K. Agathos, E. Chatzi, and V. Dertimanis, “A hierarchical output-only Bayesian approach for online vibration-based crack detection using parametric reduced-order models,” Mechanical Systems and Signal Processing, vol. 167, Article ID 108558, 2022.
View at: Publisher Site | Google Scholar
E. S. Gadelmawla, “Computer vision algorithms for measurement and inspection of spur gears,” Measurement, vol. 44, no. 9, pp. 1669–1678, 2011.
View at: Publisher Site | Google Scholar
X. J. Xu and X. N. Zhang, “Crack detection of reinforced concrete bridge using video image,” Journal of Central South University, vol. 20, no. 9, pp. 2605–2613, 2013.
View at: Publisher Site | Google Scholar
T. R. Dahl and G. Spector, Binocular stereovision, 1991.
A. Zakhor and F Lari, “Edge-based 3-D camera motion estimation with application to video coding,” IEEE Transactions on Image Processing, vol. 2, no. 4, pp. 481–498, 1993.
View at: Publisher Site | Google Scholar
Z. Zhang, “A flexible new technique for camera calibration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330–1334, 2000.
View at: Publisher Site | Google Scholar
F. Shao, Z. X. Liu, and J. An, “A discriminative point matching algorithm based on local structure consensus constraint,” IEEE Geoscience and Remote Sensing Letters, vol. 99, pp. 1–5, 2020.
View at: Google Scholar
S. Bibin, A. Glittas, M. Sellathurai, and G. Lakshminarayanan, “Reconfigurable 2, 3 and 5-point DFT processing element for SDF FFT architecture using fast cyclic convolution algorithm,” Electronics Letters, vol. 56, no. 12, pp. 592–594, 2020.
View at: Publisher Site | Google Scholar
S. Borra and R. Thanki, “A frt - svd based blind medical watermarking technique for telemedicine applications,” International Journal of Digital Crime and Forensics (IJDCF), vol. 11, 2021.
View at: Google Scholar
S. Agarwal, M. K Chandraker, F Kahl, and D Kriegman, “Practical global optimization for multiview geometry,” in Proceedings of the Computer Vision - ECCV 2006, 9th European Conference on Computer Vision, Graz, Austria, May 2006.
View at: Google Scholar
J. A. Canny, “A computational approach to edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679–698, 1986.
View at: Publisher Site | Google Scholar
A. Lapušinskij, I. Suzdalev, N. Goranin, J. Janulevicius, S. Ramanauskaite, and G. Stankunavicius, “The application of hough transform and Canny edge detector methods for the visual detection of cumuliform clouds,” Sensors, vol. 21, no. 17, p. 5821, 2021.
View at: Publisher Site | Google Scholar
Z. Qu, C Chong, L Ling, and Y. Z Dong, “A deeply supervised convolutional neural network for pavement crack detection with multiscale feature fusion,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, pp. 1–10, 2021.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Hua Ma and Guang Yang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

180

Downloads

220

Citations