Research Article  Open Access
Crack Detection and Evolution Law for Rock Mass under SHPB Impact Tests
Abstract
In order to accurately identify and quantitatively calculate the surface cracks of rock mass under SHPB impact loading, an automatic crack detection algorithm was proposed and evaluated by the experiment. In SHPB experiment, cracks on the rock surface can effectively reflect its current state and better analyze the damage process. Firstly, the SHPB system was used to impact 12 groups of rock specimens under different impact velocities. A highframe camera with 50,000 FPS was used to capture the damage process of the rock mass; using the manual annotation method, we got a dataset of SHPB damage images including a total of 310 original images and 310 corresponding cracked annotations. Secondly, a deep convolution network model named CrackSHPB was designed based on a deep learning algorithm. The algorithm can automatically identify the crack on the rock surface during impact damage process and further provide a quantitative result of cracks, crack area. Finally, after the crack on the rock surface in each frame image was identified automatically through the model, cracks were quantitatively analyzed by the proposed algorithm, the growth rate of cracks was calculated, and their evolution law was concluded. The crack identification algorithm proposed in this paper can provide a more accurate quantitative method for rock damage by cracks on the rock surface, and evolution law can further explain the failure process of rock at high strain rate.
1. Introduction
SplitHopkinson pressure bar (SHPB), developed by Kolsky in 1950, is an experimental device that can effectively study the constitutive relations of materials and effectively analyze the dynamic mechanical properties of rock materials [1, 2]. By using a waveform collector to record the incident, reflected, and transmitted waves on the input and output rods, the dynamic mechanical stressstrain relationship of the experimental materials is obtained via a theoretical formula. Many researchers have employed this method to study the dynamic mechanical properties of rock specimens and achieved great research results [3–6].
In recent years, with the development of microelectronics technology, highframe camera technology has become popularized, and more and more researchers have adopted highspeed camera to record the entire process of rock failure. They studied the fracture characteristics of rock mass by the law of surface crack propagation. Zhao et al. [7, 8] studied the characteristics of dynamic tensile failure of coal body by a Brazilian disk splitting experiment on an SHPB impactloaded coal specimen. Through digital speckle image technology and a highspeed camera, the tensile strain field of the rupture of the rock specimen was analyzed preliminarily. Huang et al. [9] carried out a series of uniaxial compression tests about rocklike material with two unparallel fissures, in order to research the effects of preexisting fissures on the mechanical properties and crack coalescence process. The photographic monitoring was adopted to capture images to investigate the crack initiation, propagation, and coalescence process. Blair and Cook [10] proposed a new nonlinear rulebased model for the fracture in compression of heterogeneous brittle materials such as rock is presented, and the model produces nonlinear stressstrain behavior similar to that observed in laboratory tests through the image. Pan et al. [11] invented an algorithm of a rock discontinuous cellular automaton (RDCA) for modeling rock fracturing processes from continuous to discontinuous deformation under mechanical loading. The stability of the method is confirmed by the observation of crack propagation in several mechanical experiments. Li et al. [12] presented experimental and numerical studies on the feasibility and validity of using prismatic rock specimens in the splitHopkinson pressure bar (SHPB) test. The research results show that it will be possible to investigate the crack propagation and failure mechanisms from the flat side surface of the prismatic specimens by highspeed imaging technique in SHPB tests. Bai and Liu [13] used the algorithm of Wavelet transform to detect the edge of CT image and quantitatively measured the growth of rock crack to forecast the status of mine roof. They all identified cracks in rocks with digital image processing technology and studied the mechanical properties of rock.
However, the identification of surface cracks in rock mass is very challenging: (1) the gray value of a crack is very close to the gray value of the surface of the rock mass, which makes it very difficult for us to use the traditional method, which is based on threshold segmentation in traditional image processing algorithms for crack identification; (2) the surface of rock mass often contains a large amount of background noise, making the texture structure very complicated to analyze; (3) when the rock mass is damaged by impact, the crack morphology that is produced varies in topological structure, and it is very difficult to provide an exact definition of a crack.
In recent years, deep learning technology has made breakthrough progress in many fields [14]. Especially in the field of computer vision, the deep learning method has achieved satisfactory results in many image recognition competitions and even exceeded the level of human recognition. In short, deep learning is a specific subfield of machine learning and is vaguely inspired by information processing and communication patterns in biological nervous systems. It has the flowing features: (1) use a cascade of multiple layers of nonlinear activation units for feature extraction and transformation; (2) learn multiple levels of representations that correspond to different levels of abstraction. From a whole prepositive, deep learning can be classified into the flowing three categories: supervised, semisupervised, or unsupervised. Compared with other traditional image processing algorithms, it can better handle complex problems under real conditions [15]. For the identification of sandstone surface cracks, due to the impact damage of SHPB, the cracks in each frame of sandstone specimen are changing, and the surface cracks have thousands of morphological characteristics. Therefore, the traditional crack detection algorithm cannot perform an acceptable result for identifying cracks on rock surface under impact loading. Therefore, in this paper, a CrackSHPB model was proposed and evaluated by the experiment data captured by the camera. Finally, we study the damage law of rock under impact loading by using the identified cracks.
2. Experiment Work
2.1. Experimental System
The experimental system used the SHPB dynamic load system device in the State Key Laboratory for Geo Mechanics and Deep Underground Engineering at China University of Mining and Technology (Beijing). As shown in Figure 1, the SHPB experimental test system mainly includes the following parts: an impact power system (mainly highpressure gas cylinders and bullets), impact rods, input rods, output rods, absorption rods, dampers, speed test systems, parallel light sources, strain gauges, highly dynamic strain gauges, waveform recorders, highframerate cameras, and video storage systems. The experimental camera model used in the lab was Photron FASTCAM SA5 with a resolution of 512 ∗ 272 with an FPS setting of 50,000. The bar used in this SHPB system is a steel pressure bar with 206 GPa elastic modulus. In addition, the length and diameter of the input and the output bar in the system are both 2000 mm and 50 mm, respectively.
2.2. Rock Samples
A picture of the test specimens is shown in Figure 2, and the damage image of a specimen after SHPB impact loading is shown in Figure 3. According to the method of production recommended by the ISRM [16], the test piece is made into cylindrical plugs with a size of . The rock specimens used in this experiment were made from a large piece of rock by core, cutting and grinding process. And the large rock is produced in a quarry, Fangshan district, Beijing. In addition, in order to best satisfy the assumption of uniformity and remove friction, the end face and the circumference of each sample were precision machined and polished to make the ends nonparallel to be less than 0.02 mm and perpendicular to the longitudinal axis within 0.25°. Finally, the Vaseline lubricant is evenly applied to the contact surface of the test piece and the bar.
(a)
(b)
As illustrated in Figure 3, it is a postmortem of fractured image of rock specimen under 4.687 m/s. It can be found that the crack first appeared in the center of the specimen and propagated along loading axis direction. It is subjected to the tensile failure, and the crack is called main crack. Besides that, there are some cracks near the contact of the specimen which is called secondary crack. And it caused by the shear failure due to the further compression between the bar and specimen. Finally, the rock specimen is split into two parts along the impact loading direction (center line of specimen).
In the study, RatSnake software was used to mark the cracks in the image at the pixel level, which was an efficient software for image annotation [17, 18]. It can help users to quickly collect the regions of interest in the image, then export the complementary images of these regions to the computer, and annotate the different regions at the same time. First, we import an image to the software and set grid size. Then, we manually select the pixel values that belong to the crack and marked them with green color. Finally, we export the binarized image of cracks which is shown in Figure 4.
After labeling the image for the impact damage of each specimen, the recorded data need to be divided. In the experiment, 12 impact fracture experiments were performed on the same kind of rock samples, and 11 were successfully completed. In a traditional machine learning or deep learning application, the dataset usually splits into training dataset and testing dataset [19]. Specifically, the model is initially fit on a training dataset which used to fit the parameters. The testing dataset is, therefore, a set of examples used only to assess the performance. Therefore, three sets of videos are selected as the training set, and the remaining eight sets are used as test sets. Dataset establishment process is shown in Figure 5. Specifically, taking three videos as input data for the model training allows the model to learn complex patterns of cracks like humans. In a traditional machine learning task, the training dataset is used for fitting the parameters of the deep neural network and testing dataset is used for verify the model capability. As illustrated in Figure 6, the loss curve can reflect the training process; while exceeding a certain number of training steps, the decline rate of the curve becomes very slow which means the model has been well trained. Since the annotation process is very timeconsuming, we want to utilize minimal data to train a model. According to the method discussed above, a total of 74,206 images represent cracks, and 58,467 images represent noncrack images. The size of all images is set up to be 28 × 28 pixels.
2.3. Experimental Procedures
The detailed experimental steps are as follows. (1) Cut, polish, and process the collected test pieces into standard patterns. (2) Affix the strain gauges on the input and output rods and connect the highly dynamic strain gauges to the waveform recorders. (3) Fix the position of the rock specimens. Because the rock specimens are susceptible to slipping when subjected to an SHPB impact; it is necessary to evenly apply Vaseline to the contact portion of the SHBP rod and the rock specimens before the experiment. (4) Connect the trigger to the speed test system and the video acquisition system. (5) Adjust the power system and perform the first impact damage. When the destruction is complete, check the data in the waveform recorder and video acquisition system. (6) According to the previous steps, the impact destruction experiment is performed in sequence and the impact velocity of the bullet changes with the different impact volume of the bullet. The experimental process was repeated several times.
As shown in Figure 7, we present six sets (bxy1 to bxy6) of experimental data including incident, transmitted, and reflected waves according to the following equation:where is the strain, is the voltage measured by the waveform recorders, and is the calibration coefficient. In specific, before the experiment, we perform an impact calibration test without the test piece, obtain the strain according to the impact velocity , and compare with the voltage value . And the K value obtained by experimental measurement is 0.000194792.
3. Methodology
3.1. Image Filtering for Rock Surface
As illustrated in the Figure 8(a), the lower half of the image has many discrete calibration points (as shown in the red region). In our work, we use a nonlocal mean filtering method [20] to remove these points and remain original cracks as much as possible. And the filtering result is shown in Figure 8(a).where is the image and represent the location of the center pixels of each block (in this paper, we set block pixels).
(a)
(b)
3.2. CrackSHPB Model
The overall structure of CrackSHPB designed in this paper is shown in Figure 9. Each section will be described in detail below. For an input image with 272 ∗ 512 pixels, we perform a sequential scan from left to right and top to bottom for each pixel and select a block with 28 ∗ 28 pixels (Figure 9 yellow block), and then we input this block to the CrackSHPB model. If this block belongs to crack, the model will output “1,” otherwise it will give “0.” Therefore, we sum all pixels with “1” as the crack area and mark them with red color.
The largest part of CrackSHPB is the convolution kernel. The convolution operation is a kind of local operation, which can effectively extract various local information of a twodimensional image [21]. The traditional crack identification method is mainly manual design of convolution kernels. After the convolution operation is performed on the image, the crack recognition effect is achieved. However, the disadvantage of this method is that the performance of the algorithm depends entirely on the design of the convolution kernel because not all cracks can be recognized by manually designed convolution kernels. Not only can we learn a single feature, but similar information such as texture, direction, and gray value can be well represented. For a 2dimensional input image , we use a convolution kernel that is also 2dimensional, and the calculation formula for image convolution is
The activation function is mainly used to increase the nonlinear expression capability of CrackSHPB. Since the linear layer is added after the linear mapping, it cannot be fitted to a higherorder function. Therefore, the activation function layer is sometimes referred to as a nonlinear mapping layer. The traditional activation function uses . However, in recent years, has been proved to be able to better train the model. The CrackSHPB model uses as an activation function [22].
The main role of the loss function is to address the case in which the forecast output of the sample training output deviates from the actual result [23]. The deviation is evaluated, and the optimization algorithm uses it to learn the parameters that minimize the value of the loss function. In general, the smaller the value of the loss function, the higher the robustness of the model that we develop. The loss function is usually expressed by the following formula:where is sample numbers, and are the true and predict values, and is a regularization parameter that prevents overfitting.
In the design of the CrackSHPB model, the crossentropy loss function is chosen (crossentropy) [24]:
After the objective function has been designed, the optimization method must be chosen for solving the above equations. Because the definition of the damage function in the deep learning network is very complicated and there is no analytical solution to the optimization problem, we must find the optimal solution using numerical analysis methods. The stochastic gradient descent (SGD) is a simple but very effective method. For a traditional optimization problem, it can quickly find the minimum value of the function. In addition, SGD has been successfully applied to largescale and sparse machine learning problems often encountered in text categorization and image classification [25]. Therefore, in this paper, we use the SGD method to train the CrackSHPB model.
3.3. Training Result
After designing the structure of the model, the training data that were prepared in the first step are input into the model for training. When evaluating the CrackSHPB model, the value of in equation (5) needs to be observed at each time. When falls below a specified threshold, the CrackSHPB model is considered to be converged. To prevent overfitting during training, the original data are divided into training set (76 images with 272 × 512 pixels) and testing set (234 images with 272 × 512 pixels). The curve of the training set and the setloss function of the entire training process are shown in Figure 6.
From Figure 6, it can be concluded that the error values of the CrackSHPB model in the training set and the verification set all stabilize at approximately 5%, indicating that the CrackSHPB model has converged.
3.4. Parameter Visualization
As shown in Figure 10, to better analyze the calculations that were performed by each layer of the convolution kernel in the CrackSHPB model, a set of convolutions are randomly selected for visualization. On the left side of Figure 10 is the input image of a group of cracks. The information that is extracted from each convolution layer is different: some layers focus on the extraction of crack morphology, whereas other layers focus on the feature extraction of crack gray values.
4. Results and Discussion
4.1. Evaluation Index
In the field of artificial intelligence or machine learning, the confusion matrix [26] is a table layout that is used to visualize the results of predictive classification. Table 1 shows a confusion matrix which each row represents the label of the prediction instance, and each column represents the label of the actual sample that allows visualization of the performance for a crack detection algorithm.

We can think of the identification of surface cracks in rock as the twocategory problem of judging each pixel in the image, whether the current pixel belongs to a crack or not. Therefore, there are 4 different cases for each pixel: (1) this pixel belongs to a crack, and the test result is also a crack. The judgment result is true positive (TP); (2) this pixel belongs to a crack, but the detection result is noncrack. The judgment result is false negative (FN); (3) this pixel belongs to noncrack, and the detection result is also noncrack. The judgment result is true negative (TN); (4) this pixel belongs to noncrack, but the detection result is a crack. The judgment result is false positive (FP)
Among them, indicates how many of the identified cracks are real failure cracks, and indicates how many cracks we detected from the actual rock mass destruction images that accounted for the actual crack ratio. In general, and are contradictory indicators. For example, identifying all pixels as cracks in the recognition process will make reach 100%; however, this will make very low. In contrast, if only a few cracks are identified, this will result in a very high ; however, will be very low. Since errors will inevitably result in the labeling process, it may be appropriate to increase the allowable error from 1 to 5 pixels in the actual calculation. That is used to determine when the current pixel is predicted to be a crack. If the real image has a crack in its 1 to 5 pixels range, the current pixel's prediction result is a true result. Therefore, this article set the allowable error to 3 pixels for testing.
4.2. Crack Detection Results
In the experiment, there were 11 groups of sandstone specimens obtained for impact destruction video, and 3 sets (bxy1, bxy3, and bxy5) were used as training data to obtain the CrackSHPB model. Therefore, the remaining 8 sets of video images are used as test sets to verify the recognition effect of the CrackSHPB model. The detection results are shown in Table 2.

Due to space limitations, a set of SHPB impact video recognition results at different stages are given. As shown in Figure 11, the left column represents the original image of the specimen when damaged by the dynamic impact, the right column is the corresponding recognition result, and the corresponding frame number and the current crack quantification result are, respectively, given. Therefore, cracks can be quantified by using the number of recording pixels.
To visually demonstrate the effect of identifying cracks by CrackSHPB, some identification images with the quantized result in pixellevel are presented in Figure 11.
Besides the above evaluation index, we also compute the area under the receiver operating characteristic curve (ROC AUC). ROC curve is a commonly used graph that summarizes the performance of a classifier overall possible thresholds. For a ROC curve, the closer it is to the upper left corner, that is, the closer the AUC value is to 1, the better the classifier is [27].
As shown in Figure 12, we can obviously find that the curves are all in the upper left of the coordinates and the mean of ROC AUC is close to 0.840, suggesting that the model has a good classification effect for crack pixel and noncrack pixel.
4.3. Crack Propagation Rate
After cracks are identified and quantified, and it is possible to accurately analyze the rate of increase of cracks in rock specimens when subjected to impact damage. Figure 13 shows the variation curve of the crack quantification result when bxy10 is broken.
From Figure 13, we can see that the cracks grow fastest in the 6th to 7th frames of the specimen, and the cracking occurs at that time; after that, the crack growth trend of each frame meets the linear condition, and the growth rate is 371.42 pixels/frame. In specific, the blue scatter point represents the crack area of frames, and then we employ a linear regression technique (y = ax + b) to fit these points. Therefore, we adopt the slope “a” as the propagation rate of the crack area. Table 3 shows the results of crack recognition and quantification of cracks by the CrackSHPB model throughout the experiment.

4.4. Crack Evolution Law
In the process of impact failure, with the increase of strain, cracks appear at the center of the disc when its maximum stress value reaches the tensile strength of the rock mass. As the strain of the test piece continues to increase, the crack in the center of the test piece continues to expand in the axial direction and the main crack develops. Before the main crack penetrated, secondary cracks began to appear at both ends of the test piece due to the compressive and shear stresses acting on the contact part between the test piece and the rod. The secondary crack propagates along the two ends of the test piece to the inside, and finally breaks through with the main crack to form a broken surface, leading to complete destruction of the test piece. And the image of the crack penetrating and the image after impact damage are shown in the Figures 14 and 15.
5. Conclusions
In this paper, we set a SHPB experiment test system with highframerate camera, and 12 groups of rock specimens was damaged under different impact velocities and the damage process video of the rock mass was captured by the camera. Then, we proposed a CrackSHPB model to detect cracks on rock surface based on deep learning. Finally, we analyze the evolution law and damage process of rock at high strain rate. The following conclusions can be made through our experimental study:(1)The CrackSHPB model, established based on the deep learning method, can effectively identify cracks on the surface of sandstone. Accuracies of 89.5%, 87.4%, and 88.1% were obtained in terms of , , and , respectively.(2)Through the CrackSHPB model, the cracks on the surface of rock can be identified at the pixel level, and the accurate crack quantification result can be obtained by counting the pixels.(3)When the sandstone specimen is damaged by impact, the crack growth rate of the sandstone specimen after crack initiation accords with the linear model.(4)In the impact damage process, the main crack first appeared in the center of rock specimen and then expanded to the two ends with secondary cracks which randomly distributed.
Data Availability
Data used in this article are available through email from the corresponding author.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This research was supported by the National Natural Science Foundation of China (51404277 and 51274206). This support is greatly acknowledged and appreciated.
References
 W. Cheng and B. Song, Split Hopkinson (Kolsky) Bar Design, Springer, New York, NY, USA, 2011.
 H. Kolsky, “An investigation of the mechanical properties of materials at very high rates of loading,” Proceedings of the Physical Society, vol. 62, no. 11, pp. 676–700, 2002. View at: Publisher Site  Google Scholar
 K. Weinberg and M. R. Khosravani, “On the tensile resistance of UHPC at impact,” European Physical Journal Special Topics, vol. 227, no. 12, pp. 167–177, 2018. View at: Publisher Site  Google Scholar
 M. R. Khosravani and K. Weinberg, “A review on split Hopkinson bar experiments on the dynamic characterisation of concrete,” Construction and Building Materials, vol. 190, pp. 1264–1283, 2018. View at: Publisher Site  Google Scholar
 M. R. Khosravani, M. Silani, and K. Weinberg, “Fracture studies of ultrahigh performance concrete using dynamic Brazilian tests,” Theoretical & Applied Fracture Mechanics, vol. 93, pp. 302–310, 2017. View at: Publisher Site  Google Scholar
 A. Mardoukhi, Y. Mardoukhi, M. Hokka, and V.T. Kuokkala, “Effects of heat shock on the dynamic tensile behavior of granitic rocks,” Rock Mechanics and Rock Engineering, vol. 50, no. 5, pp. 1171–1182, 2017. View at: Publisher Site  Google Scholar
 Y. Zhao, G.F. Zhao, Y. Jiang, D. Elsworth, and Y. Huang, “Effects of bedding on the dynamic indirect tensile strength of coal: laboratory experiments and numerical simulation,” International Journal of Coal Geology, vol. 132, pp. 81–93, 2014. View at: Publisher Site  Google Scholar
 Y. Zhao, S. Gong, X. Hao, Y. Peng, and Y. Jiang, “Effects of loading rate and bedding on the dynamic fracture toughness of coal: laboratory experiments,” Engineering Fracture Mechanics, vol. 178, pp. 375–391, 2017. View at: Publisher Site  Google Scholar
 Y.H. Huang, S.Q. Yang, W.L. Tian, W. Zeng, and L.Y. Yu, “An experimental study on fracture mechanical behavior of rocklike materials containing two unparallel fissures under uniaxial compression,” Acta Mechanica Sinica, vol. 32, no. 3, pp. 442–455, 2015. View at: Publisher Site  Google Scholar
 S. C. Blair and N. G. W. Cook, “Analysis of compressive fracture in rock using statistical techniques: part I. a nonlinear rulebased model,” International Journal of Rock Mechanics and Mining Sciences, vol. 35, no. 7, pp. 837–848, 1998. View at: Publisher Site  Google Scholar
 P.Z. Pan, F. Yan, and X.T. Feng, “Modeling the cracking process of rocks from continuity to discontinuity using a cellular automaton,” Computers & Geosciences, vol. 42, no. 3, pp. 87–99, 2012. View at: Publisher Site  Google Scholar
 X. Li, T. Zhou, D. Li, and Z. Wang, “Experimental and numerical investigations on feasibility and validity of prismatic rock specimen in SHPB,” Shock and Vibration, vol. 2016, Article ID 7198980, 13 pages, 2016. View at: Publisher Site  Google Scholar
 C. Bai and X. Liu, “The edge detection technology of CT image for study the growth of rock crack,” in Proceedings of 2009 ISECS International Colloquium on Computing, Communication, Control, and Management, vol. 4, pp. 286–288, Sanya, China, August 2009. View at: Google Scholar
 A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” International Conference on Neural Information Processing Systems, vol. 60, no. 2, pp. 1097–1105, 2012. View at: Google Scholar
 J. Schmidhuber, “Deep learning in neural networks: an overview,” Neural Networks, vol. 61, pp. 85–117, 2015. View at: Publisher Site  Google Scholar
 ISRM, “Suggested methods for determining tensile strength of rock materials,” International Journal of Rock Mechanics & Mining Science & Geomechanics Abstracts, vol. 15, no. 15, pp. 99–103, 1978. View at: Google Scholar
 D. K. Iakovidis, T. Goudas, C. Smailis, and I. Maglogiannis, “Ratsnake: a versatile image annotation tool with application to computeraided diagnosis,” Scientific World Journal, vol. 2014, Article ID 286856, 12 pages, 2014. View at: Publisher Site  Google Scholar
 D. K. Iakovidis and C. V. Smailis, “Efficient semanticallyaware annotation of images,” in Proceedings of IEEE International Conference on Imaging Systems and Techniques, pp. 146–149, Batu Ferringi, Penang, Malaysia, May 2011. View at: Google Scholar
 G. James, D. Wtten, T. Hastie, and R. Tibshirani, An Introduction to Statistical Learning, Springer, New York, NY, USA, 2013.
 L. Yang, R. Parton, G. Ball et al., “An adaptive nonlocal means filter for denoising livecell images and improving particle detection,” Journal of Structural Biology, vol. 172, no. 3, pp. 233–243, 2010. View at: Publisher Site  Google Scholar
 G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, “Improving neural networks by preventing coadaptation of feature detectors,” Computer Science, vol. 3, no. 4, pp. 212–223, 2012. View at: Google Scholar
 V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” in Proceedings of International Conference on International Conference on Machine Learning, pp. 807–814, Washington, DC, USA, June 2010. View at: Google Scholar
 S. Ioffe and C. Szegedy, “Batch normalization: accelerating deep network training by reducing internal covariate shift,” in Proceedings of International Conference on International Conference on Machine Learning, pp. 448–456, Lille, France, July 2015. View at: Google Scholar
 K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, Cambridge, MA, USA, 2012.
 T. Zhang, “Solving large scale linear prediction problems using stochastic gradient descent algorithms,” in Proceedings of TwentyFirst International Conference on Machine LearningICML/04, p. 116, Banff, Alberta, Canada, July 2004. View at: Google Scholar
 D. M. W. Powers, “Evaluation: from precision, recall and Ffactor to ROC, informedness, markedness & correlation,” Journal of Machine Learning Technologies, vol. 2, no. 1, pp. 37–63, 2011. View at: Google Scholar
 T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, vol. 27, no. 8, pp. 861–874, 2005. View at: Google Scholar
Copyright
Copyright © 2019 Xie Beijing et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.