Abstract
As the degree of industrialization is getting higher and higher, the requirements for the accuracy of materials are getting higher and higher. Among them, the detection of round holes in materials is particularly important. Round hole inspection is one of the important methods for material forming and precision inspection. This paper studies the round hole detection method of composite chemical materials and aims at using deep learning image technology to provide an efficient and convenient detection method for round hole detection. This paper proposes a fast circular hole detection algorithm based on contour extraction and validity judgment. The algorithm can extract the circular holes on the material sufficiently and quickly, and the image recognition technology based on deep learning can effectively improve the accuracy and efficiency of circular hole detection. Whether it is in circular contour extraction, validity analysis, or parameter calculation, the improved algorithm has shown good results. The experimental results show that the improved algorithm is significantly better than the canny algorithm for the extraction of circular hole contours. In terms of effectiveness, the calculation time of the improved algorithm is lower than the original algorithm in different data sets, and the highest is 1.14 seconds lower than the original algorithm. The error in parameter calculation is also the lowest, and the error of a set of data is as low as 0.1%.
1. Introduction
1.1. Background
In today’s energy system structure, traditional fossil fuels still account for a large proportion. The problems of low energy utilization, overexploitation, and large amounts of toxic and harmful gases emitted by combustion have caused two major global problems, namely, energy crisis and environmental pollution. As an emerging chemical composite material, more and more applications are being used in the new century, especially in the manufacturing of chemical materials. And the improvement and upgrading of its functions have greatly met high requirements for materials. Because of this, the measurement of composite chemical materials is also very important, because it not only affects the accuracy of the material but is also an important step in material shaping.
As the most perfect geometric body, the circle design is adopted in many aspects of life. Therefore, as a circular detection technology, there are many applications in the fields of industry, robotics, and general science. Therefore, in order to develop an accurate and fast circle extraction method, great efforts have been made. There are many different methods for measuring the size of mechanical parts on the market. Different measurement methods achieve different results. Traditional measurement methods are mainly based on manual operation, using various testing instruments such as universal tool microscopes, coordinate measuring machines, and other man-made measurements. However, manual detection often has some small errors, which make it impossible to accurately measure the actual size of the circle.
The round hole detection method based on deep learning image technology has not been studied too much, which mainly focuses on the recognition of images. Now, the development of deep learning technology makes the application of deep learning technology more and more wide. There are more and more related researches on it, and there are many researches used in image recognition. But more attention is paid to image recognition and feature extraction, and there is no special detection and extraction technology for circles.
1.2. Significance
Nowadays, the application of chemical composite materials is more and more extensive, not only in daily life, but also in medical and health, aerospace, and other aspects. Since chemical composite materials possess excellent qualities that are not possessed by the original simple substance in nature or the original materials, more and more researches have been conducted on them. However, most of the domestic research on round hole measurement is still in the aspect of manual inspection tools, and there is no in-depth study on information technology. And foreign research has a certain history, such as Germany and the United States, there have been many standards and measurement systems for the research of instrument precision measurement. In addition, as a major industrial country in China, there is a great demand for industrial materials and equipment. Therefore, it is of great significance to study the circular hole detection technology of chemical composite materials. It can not only improve the precision of chemical composite materials but also provide a reference for the circular hole detection of other materials.
1.3. Related Work
The research on deep learning has always been a hot topic of research. Deep learning image technology is also an image processing technology based on the development of computer technology. There are still some related researches on this. Litjens et al. investigated the use of deep learning in image classification, object detection, segmentation, registration, and other tasks. And they provided a concise overview of research in each application area and discussed the open challenges and directions of future research [1]. Zhao and Du proposed a classification based on spectral spatial feature (SSFC) framework, which uses dimensionality reduction and deep learning techniques to extract spectral and spatial features, respectively. The experimental results on the well-known hyperspectral data set show that their proposed SSFC method is superior to other commonly used hyperspectral image classification methods [2]. Jian et al. proposed an alternative method for steganalysis of digital images based on Convolutional Neural Networks (CNN). Their model has achieved excellent performance for various payloads in all test algorithms [3]. In the measurement of round holes, Scitovski and Sabo et al. were inspired by the problem of identifying rod-shaped particles (such as bacilli), and they proposed an effective algorithm. The algorithm is based on a modification of the well-known -means algorithm with generalized circles as the clustering center. It is very important to have a good initial approximation. In order to identify the detected generalized circles, a QAD index was proposed, and he also proposed a new DBC index. It is specifically used in this situation. The recognition process is started by searching for a good initial partition using the DBSCAN algorithm. After that, he calculated the corresponding generalized circle cluster center for the obtained cluster. This can happen if the set of data points originates from the intersecting or touching generalized circles. This method has been illustrated and tested on different artificial data sets from many generalized circles and real images [4, 5]. Wang proposed an improved RHT combined with fitting subpixel circle detection algorithm. The improved RHT algorithm uses 1 point obtained by random sampling and the other two points obtained by horizontal and vertical search to calculate and accumulate parameters, respectively. This algorithm introduces soliton-removing points and edge maps of small areas, which increases the probability that the three points belong to the same circle. Then, the edge pixel set corresponding to the identified circle is fitted to reduce the deviation effect caused by using only three edge pixels to calculate the circle parameter. This improves the reliability of fitting and the accuracy of parameters while removing noise. It conducts experimental tests on detection performance, parameter estimation accuracy, and noise robustness. Compared with other methods, this method has stronger anti-interference ability and higher calculation accuracy [6]. Qiao et al. proposed a new angle auxiliary circle detection (AACD) algorithm based on random Hough transform to reduce the computational complexity of traditional random Hough transform. This algorithm improves the sampling method of random sampling points and uses the area proposal method to reduce invalid accumulation, thereby significantly reducing the amount of calculation. Compared with the traditional Hough transform, the algorithm is robust, suitable for multicircle detection under complex conditions and has strong anti-interference ability. In addition, the algorithm has been successfully applied to the inspection of car body solder joints, and the experimental results verify the effectiveness and accuracy of the algorithm [7]. López and Cuevas studied a circle detection method based on the recently proposed meta-heuristic technology: an optimization algorithm based on teaching, which is a group-based technology inspired by the teaching process. The algorithm uses the coding of three points as candidate circles on the edge image. In order to evaluate whether such a candidate circle really exists in the edge map, an objective function is used to guide the search. In order to verify the effectiveness of the proposed method, several tests were performed using noise and complex images as input, and the results were compared with different circle detection methods [8]. Generally speaking, for circle detection, a lot of focus is on accuracy, which is the precision of measurement. However, there are few researches on round hole recognition and detection.
1.4. Innovation
Predecessors have a lot of research on circle detection technology, most of which focus on the algorithms and manual measurement tools for circle parameter detection. In this paper, based on the development of deep learning technology, some innovations based on circular hole detection of chemical composite materials are proposed. (1) An improved algorithm for contour extraction, parameter calculation, and validity calculation in circular hole detection is proposed, and the experiment proves to be effective. (2) Deep learning image technology is used for round hole detection of chemical composite materials. This has a very good speed-up effect for round hole detection, because deep learning image technology will preprocess the original image.
2. Introduction to Roundness Evaluation Algorithm and Deep Learning
2.1. Roundness Evaluation Algorithm
The important part of the round hole detection is to calculate the roundness evaluation, which includes the extraction of the circle contour and the calculation of the circle parameters, which has a very important meaning for the roundness error [9]. Because it is difficult to extract a figure exactly the same as the original image for the extraction of a circle, there is bound to be an error, so the method can only be used to assess the error. Selecting the image algorithm with the smallest error as the optimal algorithm is widely used in the industry [10]. Where circles are needed in industry, such as model making and ring extraction, great attention is paid to the evaluation of roundness errors [11].
2.1.1. Roundness Evaluation
The roundness error evaluation methods include the smallest circumscribed circle method, the largest inscribed circle method, the least square circle method, and the smallest area circle method [12].
In Figure 1, (a) is the minimum circumscribed circle method, and the radius of the ring is Rmax-Rmin. Such an algorithm can retain the graphics to the greatest extent, but the error is relatively large [13]. (b) is the method of the largest inscribed circle. It can be seen that this method will leave a lot of blank areas, which is beneficial to the subsequent selection. The ring selection has volatility [14]. (c) is the method of least square circle, which has different values for the ring, take (Rmax-R)-(R-Rmin), so as to avoid the interference of other factors, and extract R as the intermediate value [15]. (d) is the smallest area circle method, which takes the smallest error method, but at the same time the loss of edge shape is more [16]. The following will introduce the mathematical model of the least square method, because the least square method is used in the article, so other methods will not be described too much.
(a)
(b)
(c)
(d)
2.1.2. Least Square Mathematical Model
In the least square model, because of the eccentricity, that is, the position difference between the fitted circle center and the actual circle center, the algorithm should be improved. In order to obtain the position difference, the characteristic function of the error evaluation can also be carried out, as in Formula (1), where (m, n) is the reference center coordinate value [17].
As shown in Figure 2, in a rectangular coordinate system, with the origin as the center, the positive semiaxis of is the polar axis and is the turning radius [18]; then,
In Formula (2), is the radius of the least square circle; Ei is the deviation of the sample point Pi from the least square circle in the direction; is the angle between the eccentricity and the polar axis [19].
In actual measurement, . So [20], there is Formula (3):
Combine the following Formula (4), Formula (5), and Formula (6),
In the Formula, is the cosine of the eccentricity, and is the sine of the eccentricity [21]. Formula (2) can be simplified to Formula (7):
Or Formula (8):
Suppose the objective function is Formula (9):
According to the principle of least squares, find the values of and , when is the minimum value [22], respectively, as Formula (10), Formula (11), and Formula (12).
Substituting the Formula (8) into the simplified Formula to sort out the Formula (13):
The Formula shows that if and angle are measured, the deviation Ei of each sampling point to the least square circle can be obtained [23]. Therefore, the roundness error evaluated by the least square circle criterion is Formula (14):
After the summary, the flowchart is shown in Figure 3:
2.2. Deep Learning Image Technology
Deep learning image technology refers to the neural network of biological cells. Simple cells are very sensitive to the edge information of objects in the input visual information, while complex cells have a larger receptive field than simple cells and are relatively insensitive to the location of features [24]. Simple cells output their processed visual signals to complex cells [25]. It can be seen that the cat’s visual information system processes the externally input visual signals in a hierarchical manner. It first uses simple cells to form low-level edge features and then uses complex cells to form more complex and stable high-level features [26]. Convolutional neural network simulates the process of visual information processing by cats [27]. Convolutional neural networks use multilayer convolution and pooling operations to simulate this process. The convolution operation simulates simple cells, and the pooling operation simulates complex cells. The low-level convolutional layer is sensitive to the local area in the image and is used to extract simple features in the image such as edge features, while the high-level convolutional layer is not sensitive to the local features in the image [28].
As a complex algorithm, the complexity is embodied in the powerful functions, the huge calculation data, and the complexity of the execution process. Deep learning requires more parameter models to support more complex and diverse training [29]. As an image recognition technology, it needs some parameters to support it. Among them, convolutional neural network, as one of the commonly used models of deep learning, has a lot of convenience.
Figure 4 shows that the Boltzmann machine in deep learning neural network is one of the first neural networks proposed. But it is too simple, so a restricted Boltzmann machine is proposed, and the convolutional neural network is also improved on its basis [30].
2.2.1. Convolutional Layer
Compared with the traditional neural network, the special feature of the convolutional layer is that it adopts the method of local connection and weight sharing to simplify the network [31–33]. The convolution of deep learning is an operation in which the convolution kernel slides on the original image to obtain the dot product, and a weighted average method is adopted to calculate the result more accurately. Therefore, a higher weight is assigned to the most recent measurement result. It is assumed that the weighting function () is adopted to realize the realization, where represents the time interval of the measurement result according to the current moment. If we use this weighted average operation at any time, we get the continuous estimation function , as in Formula (15):
This operation is called convolution and is usually written as Formula (16):
Normally, when using a computer to process data, the continuous time signal will be discretized, and the given value is the data at a specific time interval. Therefore, a more realistic assumption is that the sensor gives a measurement result every second, so that the time can only take an integer value. Assuming that both and are defined at an integer time , the discrete expression of the convolution operation is obtained as Formula (17):
In the application of deep learning, the input is usually a high-dimensional data array, and the kernel is also a high-dimensional parameter array generated by the algorithm. For example, suppose the input is a two-dimensional image , and a two-dimensional convolution kernel is also used to obtain the following convolution operation. The Formula is shown in (18):
2.2.2. Pooling Layer
In a convolutional neural network, a pooling layer is often connected after the convolutional layer. Generally speaking, the number of feature maps in the pooling layer is the same as the number of feature maps in the convolutional layer above it. The feature maps of the pooling layer and the convolutional layer have a one-to-one correspondence, and the two also adopt a local connection method, but the local receptive fields of the neurons in the pooling layer generally do not overlap with each other. From the introduction in the previous section, we know that the convolutional neural network can effectively adjust the size and the number of channels of the output feature map. However, due to the existence of local connections, it is inevitable that parameters necessary for convolution calculations will be introduced. Sometimes, we want to select features (combine features with similar semantics) without changing the output channel in advance but do not want to add additional parameters. At this time, we should consider using the pooling layer.
2.2.3. Fully Connected Layer
In a convolutional neural network, after multiple convolutional layers and pooling layers, one or more fully connected layers are generally connected. In a neural network, they are connected to each other as nodes. The neurons in the fully connected layer are fully connected with the neurons in the previous layer. The convolutional neural network integrates the features of the image through a fully connected layer. Neurons in the fully connected layer usually use the ReLU function as the activation function. The output of the fully connected layer is the input of the output layer. The input image is classified using Softmax logistic regression, which is generally called the Softmax layer. By comparing the advantages and disadvantages of the convolutional layer, we usually choose to place the fully connected layer in the last or two layers of the convolutional neural network. The fully connected layer will convert the high-dimensional feature map output by the convolution into a one-dimensional vector to facilitate subsequent classification or regression.
3. Deep Learning Image Processing
3.1. Convolutional Neural Network Pattern Recognition Training
Using the convolutional neural network to identify and analyze the image, so as to accurately find the position of the circular hole of the chemical composite material, and inspired by the principle of partial perception in biology, human visual cells are only related to the part of the current scene. Therefore, we are in a neural network, and there is no need to make full connections. At the same time, we let most of the neurons share weights, so as to further reduce the number of parameters, the structure is shown in Figure 5.
In Figure 5, the input is a two-dimensional tensor. The convolution kernel feature is extracted to C1, and the corresponding feature map is obtained at the C1 layer. The feature maps extracted in this way are subjected to a pooling operation to obtain a feature map (feature map) S2, and these maps are then convolved to obtain the C3 layer. In this way, two operations of convolution and pooling are performed in a loop. Finally, it is connected into a vector and input to the traditional neural network S4 to get the output.
The C1 layer is the feature extraction layer, which uses the convolution kernel method to slide and extract effective features on the original image. It includes some image edges, which reduces the network connection, reduces the amount of intermediate parameter storage, and reduces the amount of calculation. If the full connection method is adopted, the intermediate parameters of a picture are as high as 100 million. If a convolution kernel is used to extract features, the connections will be sparse. In addition, the convolutional neural network also has the characteristics of weight sharing, which further reduces the intermediate parameters.
For image recognition, it is very important to extract features from data images. Because the feature extraction of the original image is often more complicated, there are many impurities in it, which is not conducive to the generation of the digital image of the system. As shown in Figure 6, in this article, the data is grayed before the image feature extraction, and then the corresponding translation and rotation operations are performed. This will make it easier for the computer to recognize.
Figure 7 shows the correct rate and loss curve of the training set during the 40,000 iterations of the 80,000 pretraining model. The purpose is to prove the learning effect of convolutional neural networks. As can be seen from the figure, due to the good pretraining before, although the learning rate is set very small, the network still converges quickly. We believe that the design of the residual structure and the application of batch norms play a key role. A total of 40,000 iterations, in fact, the value of the loss function hardly changes after 20,000 iterations.
3.2. Laplacian Feature Map Dimensionality Reduction
Laplacian Eigenmaps (LE) is a nonlinear data dimensionality reduction method. This algorithm only involves some partial calculations and a sparse eigenvalue, so it is not only very simple in terms of calculation. At the same time, it can also reflect the intrinsic geometric structure of the manifold. Because of its advantage, in this step, the binary features obtained by the convolutional neural network are reduced in dimensionality, so that a 128-dimensional binary vector is obtained. This is an image segmentation technique. This algorithm adopts a very good method for calculating nonlinear dimensionality reduction which not only can be naturally connected to the cluster but also has the maintenance performance in terms of locality.
As shown in Figure 8, the figure on the left shows that there are two types of data points (the data is a picture). The middle image shows the position of each data point in the two-dimensional space after dimensionality reduction using the Laplacian feature mapping. The figure on the right shows the result of using Partial Autocorrelation Algorithm (PCA) and projecting in the first two main directions. In Figure 8, after the B-map dimensionality reduction, the two types of data points have a distinct hierarchy and have their own trends, but in C, the PCA algorithm makes the two types of data points mixed together. Obviously, for this classification problem, the result of the Laplacian feature mapping is significantly better than PCA.
(a)
(b)
(c)
4. Compound Chemical Material Round Hole Detection
Some current circular hole detection algorithms are mainly aimed at natural landscape images with circular holes. They first detect the edge of the image and then select a set of edge pixels with circular characteristics to complete the detection of circular holes. These algorithms can accurately detect round holes, but they are usually time-consuming. The detection of composite chemical materials needs to be more detailed, because most of these materials are used in precision instruments.
4.1. The Realization Process of Composite Chemical Material Round Hole Detection
4.1.1. Target Contour Extraction
Based on the image recognition technology and preprocessing in Section III, the image processing technology based on deep learning extracts the target contour. Since the application scenarios of image processing technology in Section 3 are relatively broad, the precision is not high enough for composite chemical materials to meet the requirements. It takes too much time to perform repeated measurements to extract the mean, and the effect cannot be achieved. This is not only a burden on the detection system but also a lot of time wasted. Therefore, in response to this problem, this paper proposes a contour extraction method suitable for composite chemical material images.
4.1.2. Construction of Round Hole Candidate Set
Using the contour extraction method of the composite chemical material image, many contours on the composite chemical material image can be obtained. Among these contours, there may be some circular holes that need to be detected eventually but more are the contours of noncircular holes. If the round hole validity judgment is performed on these contours one by one, a large number of invalid calculations will be added and the performance of the round hole detection algorithm will be affected. For this reason, it is hoped that some means can be used to reduce the number of contours that need to be judged for the effectiveness of the circular hole, so as to improve the performance of the algorithm.
The algorithm can exclude some obvious noncircular contours according to the roundness of the contours, thereby constructing a candidate set of circular holes. The specific steps are shown in Table 1:
In Table 1:
Step 1. initialize the circle hole candidate set as all the contours extracted by the target contour extraction method in the previous section
Step 2. for each contour in the candidate set (denote the current contour as ), calculate the total number of pixels Nk and use it as the perimeter of the contour, calculate the total number of pixels Sk surrounded by it, and use it as the area of the contour
Step 3. determine whether the perimeter Nk and area Sk of the contour meet the following conditions. If it is satisfied, keep the contour in the circle hole candidate set; otherwise, delete this contour from the candidate set
4.1.3. Judgment of the Validity of the Candidate Circle
In most of the circular hole detection algorithms, the effectiveness of the circular hole is a key means to exclude noncircular contours. It excludes noncircular contours by constructing an annular area. This constructed ring area can automatically adapt to the size and roundness of the candidate circle. When the size or radian of the candidate circle is smaller, the width of the annular area is smaller. Through this adaptive control factor, the validity algorithm of this article can adapt to candidate circles of different sizes, and the candidate circles with smaller roundness can be more strongly restricted to improve the accuracy of the judgment algorithm.
As shown in Figure 9, when the radius of the candidate circle is 10, the inner radius of the ring is 8.88, and the outer radius is 11.22. This adds a certain limit to the extraction of circles. For noncircular contours, they cannot completely appear in the ring, which can ensure the effectiveness of the extraction of circles without the appearance of extracting other shapes. When , the inner radius of the ring is 4.44 and the outer radius is 5.56, so the ring is smaller. It shows that the smaller the radius of the candidate circle, the higher the accuracy of the circle, and the more effective the extraction can be guaranteed.
4.2. Contour Extraction Performance Comparison
In the whole circle hole detection algorithm, contour extraction is an important step. The accuracy of contour extraction affects the accuracy of subsequent round hole validity determination and the accuracy of round hole parameter calculation. In the experiment, the figure of merit (FoM) is used as the evaluation index for the accuracy of contour extraction, and the definition of FoM is shown in Formula (19):
Among them, NI and NA, respectively, represent the ideal contour of the image and the contour actually detected by the algorithm, and di represents the Euclidean distance between the pixel on the actually detected contour and the closest pixel on the ideal contour.
It can be seen in Figure 10 that in the contour extraction of the composite image, the effect of the improved algorithm and the canny algorithm is not much worse. But the overall improved algorithm will be slightly better than the canny algorithm. And with the increase of noise, the gap between the effect of the improved algorithm and the canny algorithm becomes larger and larger. This also shows that in the contour extraction of composite images, for the extraction of complex images, the effect of the improved algorithm will be more obvious. For composite chemical materials, the improved algorithm is obviously more targeted. The canny algorithm is not efficient for contour extraction of composite chemical materials, and the FOX value is always lower than 0.75. And as the noise increases, the efficiency is still declining. Although the improved algorithm also has a downward trend, the starting point is higher. It has the same high effect as the extraction of ordinary composite image contours. Therefore, the improved algorithm has excellent performance for contour extraction of composite chemical materials.
4.3. Comparison of Effectiveness Judgment Performance
The validity judgment of the candidate circle is the core of the circle hole detection algorithm in this paper. It is a decisive factor that affects the detection rate of round holes, and to a certain extent, affects the efficiency of the algorithm. To illustrate the performance of the candidate circle validity judgment algorithm, in this section, the validity judgment experiment is carried out for the candidate circle contour in the actual chemical composite material image size ().
It can be clearly seen in Table 2 that when a large amount of data is analyzed, the improved algorithm has obvious advantages. For 4 sets of different data, the operation time of the improved algorithm is about 1 second faster than that of the original algorithm, the gap is up to 1.14 seconds, and the smallest is 0.67 seconds. This shows that the improved algorithm can show strong superiority under different data sets.
4.4. Comparison of Calculation Performance of Round Hole Parameters
The ultimate goal of circular hole detection is to determine its parameters, such as circular position and radius. In this experiment, two random circles were selected, one with a small depression and the other with a small bump. And using the center of gravity circle method, the least square circle method and the improved algorithm simulate the parameters of the circular hole. The results are shown in Table 3.
In Table 3, the front is the bump data, and the back [] is the defect data. It can be seen that the improved algorithm in this paper has a good positioning and measurement effect on abnormal circles such as broken holes and bulges. For the center of gravity method, the measurement effect of the radius of the convex and concave circles is good, and the error is about 1.2. This is not high in circular measurement, but the measurement of the position of the center of the circle is slightly unsatisfactory. The position error of the convex circle reaches 3 and 6, which is relatively large. The opposite is the least square method, the least square round method can operate on a circle, and it is good for the measurement of the center of the circle, and the error is not high. However, the measurement of the radius is very lacking, and the radius error of the concave circle reaches 10, which is unacceptable. The improved algorithm adopts a better neutralization and has a very good detection effect for the center position and radius of the circle, and the error is controlled within 0.2. This is already very good for the calculation performance of the parameters.
5. Conclusion
The rapid development of computer and network technology has made great progress in deep learning technology. Image processing technology based on deep learning technology has also developed greatly, but there are not many researches on round hole measurement. Most of them are still on manual inspection tools, and there are not many studies on round hole detection of composite chemical materials. This paper takes the advantages of deep learning image recognition technology. which combines with traditional round hole detection, and proposes a chemical composite material round hole detection technology based on deep learning images. For deep learning image recognition, convolutional neural network is mainly used. This article conducts image recognition training on it, makes simple preprocessing of the image, and performs grayscale and translation rotation operations. After that, based on image recognition, this paper conducts round hole detection on chemical composite materials and proposes improvement measures on the traditional detection algorithm. After the experimental detection, the improved algorithm is used for contour extraction and validity detection. The effectiveness of the algorithm has a good performance, which shows that the technology proposed in this paper has a good effect on the detection of round holes in composite chemical materials. However, there are deficiencies in the text. In terms of deep learning image technology, there is no code implementation for the training process in this article. This is because of limited space and insufficient knowledge of related technologies, which will be strengthened in the future.
Data Availability
No data were used to support this study.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this article.
Acknowledgments
This research was financially supported by Training Program of Innovation and Entrepreneurship for Undergraduates (No. X2020078, North China University of Science and Technology).