Abstract
Gradation has an important influence on the mechanical properties of earth and stone materials after compaction, and a reasonable gradation is the key to ensure compaction quality. In this paper, a set of earth and stone materials gradation detection system based on digital image processing was proposed, which was suitable for detection in the construction sites. The system obtained images which were taken from multiple directions of materials transport vehicle, used the thresholding method to segment the images, detected particle contours through edge detection algorithm of Canny to realize the image recognition of particle size, and drew the gradation curve of earth and rock materials finally. It was verified by selecting limestone aggregates of different gradations, and the results showed high accuracy. The system can realize on-site detection of the earthwork gradation rapidly and accurately at a dam construction site.
1. Introduction
Earth and stone materials are major construction materials used in hydraulic, highway, railway, and structural engineering. Gradation affects the mechanical properties of earth and stone materials after compaction. It is critical to adopt reasonably graded fillings to ensure high compaction quality with improved volume stability and permeability resistance. How to accurately and quickly detect and identify the gradation of earth and stone fillings has always been a focus for engineers. The traditional gradation detection method is to extract random samples and sieve the samples manually, or use a sieving machine. Gradation curves are obtained by sieve analysis manually [1]. Although this method is mature and widely used, it is time consuming and laborious. In particular, the method is not appropriate for currently adopted highly efficiently automated filling construction, which requires frequent and rapid gradation detection. There are few rapid gradation detection methods for a massive amount of earth and stone material despite the urgent need for gradation detection in construction. It is therefore imperative to develop an efficient and easy-to-apply method for gradation detection in a high-volume earth and stone material project.
There are some reliable methods for rapid gradation detection. Researchers have searched for the shape features of aggregate particles by combining three-dimensional laser scanning, laser profiling technology, and digital image analysis. Lee et al. [2] developed a system for collecting and analyzing the three-dimensional data of particles from the surface of coarse aggregates. As aggregate passed on a conveyor belt, the data of the upper hemisphere of particles were collected by laser triangulation and the three-dimensional size and shape of particles were calculated [2]. Kim [3] and Haas et al. [4] realized the automation of the rapid detection of aggregate properties, such as size, shape, angularity, and texture, through laser profile scanning, and various digital image analyses. Mgangira et al. [5] quantified differences among individual aggregate particles by means of three-dimensional laser scanning, allowing automated quantification of the shape features of aggregate particles. Mgangira [6] used information obtained from three-dimensional laser scanning and scanning electron microscopy to quantify and identify differences among aggregate particles. Han et al. [7] verified and applied the two-dimensional slice recognition of a three-dimensional mesostructure for different aggregate gradations and packaging algorithms and established equivalent three-dimensional numerical specimens containing cubic aggregate, according to the experimental results of the Walraven equation.
Many other researchers focused on digital image analysis, which involves the image acquisition of aggregates via photography, image recognition, and gradation detection. Their works focused on the detection of the shapes of sediments, composition of rock, particle size of rock, homogeneity of asphalt, and shape and particle size of concrete aggregate. Maerz [8] designed a visual-based system that quickly, objectively, and efficiently determines the shape features of aggregate particles. The VDG-40 Videograder system [9, 10], developed by LCPC, a public laboratory facility in France, automatically obtains the gradation of aggregate and the needle-and-plate content of aggregate particles. Bouquety et al. [11] analyzed 122 aggregate samples with a VDG-40 Videograder and verified the function of the VDG-40 with additional data. Applying a linear measurement and Hough transformation technology, Wilson et al. [12] established a shape index and quantified the shapes of aggregate. Kwon et al. [13] studied the physical and topological properties of unbonded aggregate and realized the recognition of bonded aggregate. Argenti et al. [14] used a fast texture algorithm for the cooccurrence matrix to conduct pixel-by-pixel classification with supervised learning and maximum likelihood estimation and proposed a single-pixel multiwindow classification scheme. Xu and Wang [15], through analysis of the gray values of digital images, introduced a new method of image segmentation. They optimized ordinary illumination and used infrared light instead of visible-wavelength lighting. At the same time, they designed an algorithm based on the histogram characteristics of coal and gangue. They finally realized the automatic recognition of segments of coal and gangue. Through automatic analysis of the gradation particle size, Buchanan and Brown [16] evaluated potential automatic grading devices and provided insights into their development and application prospects. Bruno et al. [17] detected the gradation of mixed aggregate and effectively analyzed a section image of asphalt through plane image analysis. Al-Rousan et al. [18] evaluated the most widely adopted image analysis of the shape features of aggregates and showed that this technology is affected by angle and shape changes. Kwan et al. [19], adopting digital image processing, analyzed the shapes of coarse aggregates and measured the shape features of main particles, resulting in stronger correlation and a higher detection rate compared with traditional manual measurement. Using an IBM-XT Microsoft computer, Barksdale et al. [20] collected and measured the shapes of aggregate, analyzed and expressed the shapes in the form of charts, tables, and histograms, and measured the shapes, surface areas, and roughness of aggregates. Zhang et al. [21], adopting the digital-image-based particle-size-distribution recognition of dam granular material (i.e., image acquisition), a large database, and a neural network, recognized several sizes and grading curves of dam particles on a small scale and thus obtained the particle size distribution of dam materials.
The rapid detection of the gradation of earth and stone material by image processing still faces issues, such as a complex detection system, low degree of artificial intelligence, and failure to rapidly determine whether materials meet requirements. Aiming to solve the issues encountered in the detection of gradation, a rapid gradation detection system for earth and stone materials was developed in the present study on the basis of the integration of image recognition. The system captures high-resolution images through a high-definition camera, obtains the diameters of earth particles through image enhancement, image contour processing, and image boundary processing, draws gradation curves, and decides whether the test results meet construction requirements. The aforementioned image processing takes only 1 to 5 seconds and the test results can be promptly uploaded to and stored in mobile and remote terminals, facilitating the real-time and on-site monitoring of earth and stone materials. A scaled prototype was assembled to investigate the stability and accuracy of the system proposed. The gradation of typical limestone material was tested and the results were compared with the results of traditional sieve analysis to verify the reliability of the proposed detection system. The proposed gradation detection system has many advantages over traditional sieve analysis, including (1) no requirement of complicated sieving equipment, (2) rapid real-time data processing, (3) the full use of and incorporation into existing construction facilities, requiring no additional procedures, and (4) the rapid, stable, and reliable gradation detection of earth and stone materials in practice.
2. Rapid Gradation Detection System for Earth and Stone Materials
Water is usually added for lubrication before compaction to ensure the compaction quality of earth and stone materials. The addition of water is usually completed during transportation. Specifically, the transport vehicle enters a spray room and spends time under a water sprinkler. The proposed gradation detection system was developed for this procedure. The idea is that a photograph is taken and analyzed as the vehicle loaded with earth and stone materials is stopped. If the results are within the desired gradation envelope, materials are considered qualified and sent for construction. Their position of offload is recorded by a vehicle-borne Global Positioning System device. The materials are considered unqualified if the results lie outside the envelope. The evaluation results indicating excessive or insufficient material within a certain range of particle size are recorded. The vehicle then returns to the stock ground and its load is mixed with lacking materials such that the requirements of desired gradation are met and the materials are ready for retesting. The on-site layout of the proposed system is shown in Figure 1 while the technical processes are shown in Figure 2. The proposed system has advantages including no requirement for additional procedures, high efficiency of filling, and a low cost for construction.
3. Prototype of the Gradation Detection System
3.1. Hardware
An image-recognition-based prototype of the rapid gradation detection system for earth and stone materials was assembled in accordance with the description of the gradation detection proposed in the previous section, providing a reference for subsequent on-site application. Figure 3 shows the exterior and interior of the prototype, including the test prototype and the overall structure made of aluminum alloy ①, water supply ②, which is to add water during the transportation of earth and stone materials to lubricate the surface to improve the rolling quality of the earth and stone materials, model of the transport vehicle ③, white plastic box simulating the carriage of the transport vehicle ④, light emitting diode (LED) lighting ⑤, industrial camera ⑥, detection software and storage terminal ⑦, sprinkler nozzle ⑧, and system control (for the entry and exit, lighting, and spray) ⑨. High illumination is desired owing to the high speed of the vehicle, and the LED lighting was thus customized to have high illumination, high lighting efficiency, high color rendering, and long service life. Specific parameters of the light source are a color temperature of 5500 K, color rendering index Ra of 80, average service life of 50,000 h, power of 2 × 140 W, and luminous flux of 2 × 3200 lm. To extract feature information, the rock camera is required to be stable and reliable and have superior technical functions. An acA1920-50gc camera, manufactured by Basler, was selected for the prototype. The camera has a complementary-metal–oxide–semiconductor photosensitive chip with dimensions of 11.3 mm × 7.1 mm, pixel dimensions of 86 μm (horizontal) × 5.86 μm (vertical), frame rate of 50 fps, and resolution of 2.3 megapixels. Figure 4 shows the software interface.
3.2. Algorithm and Software of the Detection System
The algorithm for gradation detection is based on the segmentation of a high-definition image of earth and stone materials. The feature sizes of particles are calculated according to the number of pixels in an image and the length represented by a single pixel. The geometric characteristic (surface) area S, perimeter , and round R0 of materials are identified and obtained so that the long axis a and short axis b of the ellipse matching the stone area on the outside are obtained. Finally, the particle size d is calculated. Perimeter of the aggregate particle is calculated as [22]where is the length of the boundary segment, is the per code segment, is the number of even digits in the sequence of boundary chain codes, and is the total number of codes in the boundary chain codes. Surface area S is calculated aswhere and are the ordinates of the target boundary in the image while and are the ordinates of the starting point of the boundary. Other quantities are expressed as
Using the results calculated with (3)–(5), the particle size d is calculated as
The particle shape is assumed to be ellipsoid and the length of the short axis d is taken as the sieving size in the analysis. The particles are counted in each size range and the volume of each particle is calculated as an ellipsoid volume. The density is assumed to be the same for all particles and the volume is converted to a mass. The number of particles in each range multiplied by the mass of a single particle equals the mass of the group, and gradation curves are drawn in accordance with the percentage contribution that each group makes to the total mass.
Figure 5 shows the seven steps of the segmentation algorithm for earth particles in an image: Step 1 is inputting the original photograph, applying a gray treatment, and then converting the image to a two-dimensional array. The resulting image is shown in Figure 5 ①. Step 2 is the equalization of gray levels. The histogram of the gray image is revised to be uniformly distributed to enhance contrast, which is particularly helpful when images have a background or foreground that is too bright or too dark; i.e., details in overexposed and underexposed photographs are better revealed. The equalization is as follows where n is the number of pixels in the image, l is the number of gray levels, is the number of pixels whose gray level is , is the probability of the occurrence of gray level k, and is the gray value of a pixel after image equalization. Step 3 is bilateral filtering, which involves removing noise, preserving edges, and removing distortion interference due to dust and uneven lighting to obtain [23]: where , is the pixel-sized neighborhood centering on , are the pixel coordinates of the neighborhood, is a weighted coefficient whose value is the product of the image brightness similarity factor and spatial proximity factor , is the pixel gray corresponding to the equalized gray image , and is the variance of the space factor and brightness factor. The enhanced image following gray level equalization (Step 2) and bilateral filtering (Step 3) is shown in Figure 5 ②. Step 4 is threshold processing, which adopts Otsu and local double-window threshold optimization to distinguish the target from the background and generate a binary image. The optimal threshold binarizes the original image to provide [23]: where is the average gray level, is the maximum variance between the target class and nontarget class, is the probability of the target class, is the probability of the non-target class, and are the average gray levels of the two classes, and l is the maximum gray level (usually set to 255). The resulting image is shown in Figure 5 ③. Step 5 is octet code processing, which scours the connected area, marks the inner and outer contours, fills target holes through the inclusion relationship between the inner and outer contours, and removes the inner-hole noise of the target from the binary image. The resulting image is shown in Figure 5 ④. Step 6 is applying an open operation to the binary image after hole filling, i.e., first corroding and then expanding to solve the adhesion of the targeted aggregate. The resulting image is shown in Figure 5 ⑤. Step 7 is applying Canny edge detection to the binary image after the open operation, extracting the boundary of the targeted aggregate, and drawing. The resulting image is shown in Figure 5 ⑥.
The interface of the software of the detection system is shown in Figure 4. The interface mainly includes a menu bar, display window, toolbar, and detection status. The menu bar provides step-by-step control options for the detection process. The display window displays images captured by industrial cameras (① in Figure 4), filtered images (② in Figure 4), octet-coded images (③ in Figure 4), and determined gradation curves (④ in Figure 4). The toolbox adjusts the parameters used in the algorithm. The process control bar indicates the status of detection. The processing software is highly efficient and fast and capable of recognizing an image and drawing the corresponding gradation curve within 1 second. These performances are advantageous in the recognition and processing of massive amounts of image data.
4. Tests and Results Analysis
4.1. Earth and Stone Materials
The amount of water on the surface of the earth and rock material also has an important impact on the quality of the photos. For different amounts of water, the camera parameters need to be adjusted to improve the quality of the photos. At the same time, many tests were conducted on earth and stone materials with different lithological properties. Results reveal that, owing to differences in color and shape among rocks, the testing of different materials using the same algorithm gives rise to a deviation from the actual gradation, suggesting that, as for rocks with different lithological properties, the recognition algorithm and parameters need to be adjusted accordingly. After adjusting the algorithm and parameters, this system can be applied to earth and rock materials with different lithology in minimum particle size of 5 mm.
Limestone exists widely in nature and is widely used as an important infrastructure material. The gradation of earth and stone materials composed of limestone affects the safety and stability of hydraulic engineering projects. The image acquisition and recognition of limestone are difficult owing to the dark color and clear edges of limestone. Limestone was thus selected in this paper as a typical material with which to conduct trial tests on gradation detection. There was a five-to-ten-fold scale difference between the larger on-site detection system and the smaller prototype system. It was necessary to reduce the size of material accordingly, to the extent that the maximum particle size was no greater than 60 mm. Test materials were sieved following a standard procedure. Ten samples of limestone aggregates with different gradations were selected for testing. Four samples of test materials are shown in Figure 6. Test materials were placed in the model carriage of the transportation vehicle, photographed, and analyzed by the software of the detection system.
(a)
(b)
(c)
(d)
4.2. Analysis of Test Results
A comparative analysis was conducted for the test results; i.e., gradation curves obtained by the image recognition detection system and by standard sieving analysis were compared and analyzed and characteristic particle size errors in the test results were taken as evaluation indicators. Standard sieve and image recognition were conducted for the 10 samples of mixed-grade limestone earth rockfill materials, and gradation curves were obtained for the 10 samples adopting two methods. Results are shown in Figures 7 and 8. The fitting degree of the detection results is rather high, and the gradation curves of the detection results are in good agreement. The indicators d60, d30, and d10 are usually used to evaluate gradation. The present paper uses these three indicators to compare errors for the two test methods. These three characteristic particle size indicators of 10 group tests are listed in Table 1. The maximum error value of d60 was 5.8 mm and the minimum errors were 0.7 mm. The maximum error value of d30 was 6.4 mm and the minimum errors were 0 mm. Values of d10 were small for the 10 groups, except for one group with an error of 11.7 mm; the maximum value for the other nine groups was 5.8 mm. Therefore, the overall operation of the image-recognition-based gradation detection system for earth and stone materials is relatively stable and the precision of the detection results is good.
The detection system was developed for complex scenarios. A comprehensive optimization algorithm based on local optimization is proposed for complex scenarios, such as a random composition of lithology, mixed gradation, and large color difference, which enhance the information within the imaging area. In the detection of rocks with random lithology composition, mixed gradation, and a large difference in color, the values of d60, d30, and d10 were well within allowable ranges and the detection time was only 1–5 s.
In practice, the gradation curve of earth and stone material generally varies within the envelope of the designed gradation, which means the characteristic particle size is allowed to vary within a large range. As an example, for rockfill materials, the literature has suggested a gradation envelope of d60 ranging from 80 to 250 mm, that of d30 ranging from 20 to 80 mm, and that of d10 ranging from 0.5 to 12 mm [24]. The error in image recognition detection proposed in this paper is smaller than the difference between the upper and lower limits of the corresponding characteristic particle size range, suggesting that the error in recognition does not substantially affect the determination of gradation and meets the requirements of construction.
5. Conclusions
An image-recognition-based rapid gradation detection system for earth and stone materials was developed. Necessary hardware and software were described and a test prototype was manufactured and assembled. Gradation was determined for 10 samples of limestone earth and stone filling materials with different gradation. The maximum difference in the characteristic particle size between the results of image analysis and those of standard sieve analysis was 6.4 mm, which is smaller than the measured particle size range and basically meets practical gradation requirements. The comparison indicates that the proposed gradation detection system provides test results that are fundamentally reliable with improved efficiency.
The present study found that currently used detection algorithms and parameters applied to earth and stone materials do not have universal applicability. A proper adjustment according to the lithological properties and shapes of aggregates is thus needed.
The image-recognition-based gradation detection system developed in this study has the characteristics of real-time detection, real-time transmission, and real-time calibration, and the system can thus be applied on a construction site. The number, parameters, and light source of industrial cameras should be selected and adjusted in accordance with specific lighting conditions and the size of transport vehicles on site.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The research work was supported by the National Key R&D Program of China (2018YFC0407103), the National Natural Sciences Foundation of China Fund Project (Nos. 51879285 and 41731289), and the Basic Research Project of the China Institute of Water Resources and Hydropower Research (GE0145B0112017). The authors thank Glenn Pennycook, MSc, from Liwen Bianji, Edanz Group China (http://www.liwenbianji.cn/ac), for editing the English text of a draft of this manuscript.