Abstract
Machine vision-based surface defect detection and classification have always been the hot research topics in Artificial Intelligence. However, existing work focuses mainly on the detection rather than the classification. In this article, we propose GSPSO-LRF-ELM that is the grid search (GS) and the particle swarm optimization- (PSO-) based local receptive field-enabled extreme learning machine (ELM-LRF) for the detection and classification of the surface defects on the magnetic tiles. In the ELM-LRF classifier, the balance parameter C and the number of feature maps K via the GS algorithm and the initial weight Ainit via the PSO algorithm are optimized to improve the performance of the classifier. The images used in the experiments are from the dataset collected by Institute of Automation, Chinese Academy of Sciences. The experiment results show that the proposed algorithm can achieve 96.36% accuracy of the classification, which has significantly outperformed several state-of-the-art approaches.
1. Introduction
The magnetic tile is an important component of the motor, whose surface defects directly affect the performance and the life of the motor. Therefore, defective surfaces on the magnetic tiles need be detected and analyzed during the production process [1]. The common types of surface defects of the magnetic tiles mainly include “break,” “crack,” “fray,” “uneven,” and “blowhole” [2], which are shown in Figure 1. These surface defects are used to be inspected by humans, which has inevitably suffered from several downsides such as the low detection efficiency, the poor detection consistency, and the high laboring cost. As a result, the automatic detection of such surface defects using the visual inspection and image processing attracts more and more attention [3]. However, conventional automatic detection methods either have low detection accuracy or fail to classify the detected defects, which have severely affected the following process of industrial production [4].
(a)
(b)
(c)
(d)
(e)
In recent years, a number of approaches based on machine vision have been proposed for improving the detection and classification of surface defects on magnetic tiles. Valavanisa and Kosmopoulos [5] proposed a method which uses the geometric and the texture features to detect and classify defects in the weld radiographs. This method improved the detection speed, but the extracted features were too complicated. Li et al. [6] used the fast discrete curvelet transform (FDCT) and texture analysis for the detection of “cracks” in magnetic tiles longer than 0.8 mm, but it could only detect single “cracks” whilst other types of defects were not considered. Yang et al. [7] proposed to use nonsubsampled shearlet transform for surface defects detection of the magnetic tiles, which could effectively remove “uneven” background, grinding texture and noise interference during defect detection rather than any other kinds of defects. He et al. [8] proposed a framework for the detection of steel surface defects, classification priority network (CPN), and a new classification network, multigroup convolutional neural network (MG-CNN). The framework has better classification performance, but there is the problem that the classification results are unstable in early training.
In summary, most of the existing approaches for surface defect detection of the magnetic tiles can detect a single type of surface defects and often have relatively low detection accuracy. After detection of the possible defects, their types are not further classified, which is not conducive to finding the cause of the defect for improving the subsequent production process. In this article, we aim to solve the aforementioned problems. We use the GS method to obtain the optimal parameter combination (C, K) more accurately and divide the GS method into two parts: rough optimization and fine optimization; PSO algorithm is proposed to optimize the initial weight Ainit of ELM-LRF and further classify the defect categories. The main contributions of our work can be highlighted as follows.(1)Because ELM-LRF has poor initial weights stability, we use the particle swarm optimization (PSO) to optimize the initial weights of ELM-LRF, which improves the classification accuracy of the classifier(2)In order to improve the performance of the classifier, the method of grid search (GS) rough optimization and fine optimization is used to optimize the balance parameter C and the number of feature maps K in ELM-LRF(3)Using the optimized ELM-LRF to classify the surface defect categories in the detected images and compared with some advanced multicategory classification algorithms, our proposed method has higher classification accuracy
The remainder of this article is organized as follows. In Section 2, the related work and technical background of ELM-LRF are introduced. Section 3 presents the proposed GSPSO-LRF-ELM algorithm, that is, the grid search and the particle swarm optimization optimized ELM-LRF. The experiments results and analysis are given in Section 4. Finally, Section 5 concludes the article along with future prospects for the next phase of research.
2. Related Work and Technical Background
There are six common types of the magnetic tiles, which are “break,” “crack,” “fray,” “uneven” “blowhole,” and “free,” on the surface of the magnetic tile. In order to further classify the test results, it is necessary to extract the feature information of each category as the input of classifier to realize the defect classification.
Vision-based defect detection and classification systems have great advantages for industrial production, which have promoted a large number of related work in relevant fields [9]. The general workflow of the system is illustrated in Figure 2.
2.1. Extraction of Region of Interest (ROI)
Firstly, the industrial camera and the video acquisition equipment are used to obtain the image of the magnetic tile. After background removal, the region of interest (ROI) is extracted as the input for next stage of processing.
2.2. Preprocessing and Image Segmentation
Image preprocessing includes enhancement, sharpening, and denoising of images. For image segmentation or detection of defects, several methods can be used. Commonly used image segmentation methods include region growing [10], mean iterative segmentation [11], maximum entropy segmentation [12], and Otsu [13]. In this article, considering that the surface of the magnetic tile image is dim and the image is complicated, an entropy weighted automatic threshold Otsu maximum interclass variance image segmentation method is chosen for the image segmentation [14].
2.3. Image Feature Extraction and Defection Classification
Some commonly used features include color, shape, texture, and spatial relationship. ELM-LRF with convolution layer and pooling layer can realize feature self-extraction of input image. For image classification, some classical algorithms include support vector machine (SVM), artificial neural network (ANN), Bayesian classification (BC), and K-nearest neighbor (K-NN). Zhou et al. [15] extracted features as input to SVM for classification of automobile surface defects. Kumar et al. [16] used the gray level cooccurrence matrix and the texture shape geometry as the features of the detected weld image, followed by using the ANN for detection and classification of the defects. Yapi et al. [17] trained BC to distinguish the defect-free fabrics from the defect ones and achieved good detection results. Cetiner et al. [18] used features obtained from wavelet distance as the input of K-NN to further classify wood materials.
With the development of deep learning, convolutional neural network (CNN) has been successfully applied in many different applications. Tao et al. [19] classified the detected metal defects through a compact CNN, which satisfies the robustness and accuracy of detection. Wang et al. [20] realized the function of automatically extracting image features in the case of less prior knowledge of the defect detection images.
Due to the advantages of CNN sharing convolution kernel and feature self-extraction, it has been widely used in current classification research. But CNN adopts traditional BP training, which has the disadvantages of slow convergence and being easy to fall into local optimum. In 2015, Huang et al. [21], inspired by CNN and the extreme learning machine (ELM), proposed a local receptive field-based extreme learning machine [22] (ELM-LRF). In this article, the ELM-LRF algorithm is used to detect and classify the surface defects of the magnetic tiles, where the grid search (GS) and particle swarm optimization (PSO) algorithm are used to optimize the parameters and weight in the ELM-LRF algorithm. The experimental results show that this proposed method has significantly improved the accuracy of the defect detection and classification.
After detecting and classifying the surface defects of the magnetic tile, the overall defect detection accuracy and classification accuracy of the different defect types are obtained and compared with existing approaches for performance assessment and evaluation.
2.4. Technical Background of ELM-LRF
In 2004, a new type of single hidden layer feedforward neural network (SLFNs) was proposed by Huang and his team as extreme learning ,machine [21], which is characterized by easy parameter selection, fast learning speed, and good generalization performance. In this article, a special ELM, namely, ELM-LRF, is adopted [22]. The feature of this method is to add a single layer convolution and pooling network similar to CNN on the basis of ELM to realize the self-extraction of image features and to classify the input by the output weights of ELM.
In order to make the input more adequate, K different feature maps can be obtained by using K different input weights in the ELM-LRF [23]. Its specific implementation is divided into the following three steps.(1)Randomly generating the initial weight Ainit. The specific calculation formula is as follows:where is the initial weight, K is the number of feature maps, is the size of the local receptive field, and each column in is a set of the orthogonal bases of . The input weight of the feature map kth is , which is arranged by the column . The initial weight is orthogonalized by the singular value decomposition (SVD), and the result of the orthogonalization is A. The value of the convolution node (i, j) of the feature map kth is calculated bywhere d × d is the input image size, is the size of the feature map, and is the input weight of the kth feature map at (m, n) point.(2)Square root pooling. The specific calculation formula is as follows:If (i, j) is out of bound, then , where the pooling size e represents the distance from the center of the pool to the edge [24]. In the ELM-LRF, the pooling map has the same size as the feature map, and both are . and represent the nodes (i, j) in the feature map kth and the combined nodes (p, q) in the kth pool map.(3)Calculating the output weight matrix. To calculate the corresponding feature map and pooling map for each input sample x, the row vector is formed by concatenating the combined nodes in the pooling graph and then connecting the row vectors of the N input samples to obtain the combined layer matrix . The final combined layer and the output layer are fully connected. The output weight is , which is calculated using regularized least squares analysis. The specific equation is as follows.(a)If ,(b)If ,where N is the number of input samples, K is the number of feature maps, is the size of the feature map, is the output weight, is the expected output matrix, and is the regularization parameter.
3. The Proposed Grid Search and Particle Swarm Optimization-Based Local Receptive Field-Enabled Extreme Learning Machine (GSPSO-LRF-ELM) Algorithm
The surface of the magnetic tile is curved and the curvature is different. The collected magnetic tile images have uneven illumination. In this article, we use the entropy weighted automatic threshold Otsu image segmentation method to segment magnetic tile images [14]. In view of the different local changes in the magnetic tile images, we use adaptive thresholds to segment uniform and nonuniform regions.
The GS is used to optimize the balance parameter C and the number of feature maps K in the ELM-LRF to find the optimal parameter combination (C, K). The PSO algorithm is used to optimize the initial weight in the ELM-LRF to find the optimal Ainit. The optimized ELM-LRF classifier is called GSPSO-LRF-ELM. The classification algorithm flow chart is shown in Figure 3.
3.1. Optimization of the Balance Parameter C and the Number of Feature Maps K by GS
In the ELM-LRF algorithm, the most important parameter combination is the balance parameter C and the number of the feature maps K. The selection of these two parameters directly affects the performance of the algorithm. Therefore, the GS is used to optimize the parameter combination (C, K) in the ELM-LRF. The optimization Algorithm 1 of GS for the parameters C and K is as follows:
|
The array matrix is obtained by combining the values of the arrays A and B. Each array in the matrix is inputted into the classifier to obtain the corresponding classification accuracy. Comparing the accuracy of the each classification, the parameters BestC and BestK corresponding to the highest classification accuracy are selected as the balance parameter C and the number of feature maps K in the classifier.
3.2. Optimization of the Initial Weight Ainit by PSO
The idea of PSO is derived from the foraging behavior of birds, in which each particle represents a set of possible solutions, and all particles form a group. The particles determine their speed and position according to their historical information and group historical information until the optimal solution is found. The iterative update equation is as follows:where and are the velocity and position of the number d dimension of particle i at the number k iteration and is the weight, respectively, and are the learning factor of the individual and the group, is the optimal position of particle i in the d dimension in the number k iteration, is the optimal position of the individual in the d dimension of the whole population, and , is a random number of [0, 1] intervals.
In order to make the PSO have the better global search ability in the early stage and the better local search ability in the later stage, the work adopts the nonlinear inertia weighting factor [25], as shown in the following :where and are maximum and minimum weights, respectively, and and are the current iteration number and the maximum iteration number.
The work uses the PSO algorithm to optimize the initial weight Ainit in the ELM-LRF algorithm. Firstly, D initial particles are generated, and the corresponding feature map matrix, pool graph matrix, and output weight matrix β are calculated; secondly, it uses the formula Hβ = T to calculate the prediction label T; finally, the classification accuracy of the image is taken as the fitness function, and the optimization goal of the PSO is to maximize the fitness function. The PSO for initial weight optimization of ELM-LRF Algorithm 2 is as follows:
|
4. Experiments and Results
4.1. Experiment Settings
The dataset used in the experiment was from the dataset on surface defect detection of the magnetic tile collected by Institute of Automation, Chinese Academy of Sciences [2]. The folder name of the dataset is magnetic-tile-defect-datasets (magnetic-tile-defect-datasets dataset acquisition address: https://github.com/abin24/Magnetic-tile-defect-datasets.). A total of 1344 images were collected. In order to make the experiment more reasonable and reliable, the experimental data were randomly selected as the defect and defect-free images by 1 : 1. The types and quantities of experimental data selected are shown in Table 1. Due to the different ROI of different magnetic tiles, the size of the image is different. For this reason, the image is uniformly converted into 64 dip × 64 dip size before preprocessing.
The classification accuracy of the surface defect of the magnetic tile is used as a criterion for judging the experimental results. The higher the classification accuracy is, the better the classification performance of the algorithm has.
In addition, in order to better analyze the experimental results, the false detection rate and the missed detection rate of each category are separately counted as follows:
All experimental environments in this article are operating system Windows 8.1 64 bit, processor Intel Core i5-4200M @ 2.50 GHz, memory (ARM) 8 GB, and software MATLAB R2018a.
4.2. Experiment Results and Analysis
We use ELM-LRF algorithm for defect detection and classification experiments. Dividing the dataset into training set and test set randomly, the types and quantities of images in training set and test set are shown in Table 2.
In the ELM-LRF algorithm, in order to analyze the influence of the balance parameter C and the number of feature maps K on the algorithm, the GS is used to optimize the two parameters. The parameter optimization is divided into two parts: rough optimization and fine optimization. In the rough optimization, the range of the parameter C is set to {10−4, 10−3, 10−2, ... , 102, 103}, the range of the parameter K is set to {10, 20, 30, 40, 50, 60}, and the rough optimization 3D map of the results is shown in Figure 4. It can be seen from Figure 4 that when lgC is taken as −2 and K is taken as 50, a rough optimal parameter combination (C, K) is obtained.
For the further fine optimization, according to the rough optimization experiment results, the C and K setting range and step size are reduced, and the range of the parameter C is set to {0.005, 0.006, 0.007, ... 0.019, 0.020}, the range of the parameter K is set to {45, 46, 47, ... 53, 54, 55}, and the fine optimization 3D map of the results is shown in Figure 5. It can be seen from Figure 5 that when C is taken as 0.016 and K is taken as 55, a fine optimal parameter combination (C, K) is obtained. At this time, the highest classification accuracy of parameter optimization is 98.04%. In the parameter optimization experiment, the test classification accuracy is adopted as the criterion for judging the experimental results.
In the following experiment, the population size was set first, followed by the number of iterations N, the maximum inertia weight and the minimum , the learning factors c1 and c2, the maximum particle velocity and the minimum , and the maximum particle position Xmax and the minimum Xmin. The selection of the balance parameter C, the number of feature maps K, the kernel size Ke, and the pool size e in the ELM-LRF algorithm are shown in Table 3.
The proposed classification algorithm counts the detection and classification results of six categories of the magnetic tile surface. The statistical results are shown in Table 4. As seen from Table 4, the classification algorithm proposed in this article can achieve 100% correct rate when identifying the defect and defect-free magnetic tiles. Among the five types of the defect categories, the classification rate of “break” is the highest, which is 100%; the lowest rate of “uneven” is 86.67%. In the experiment, the number of the defective magnetic tiles is falsely detected as the blowhole and the number of uneven defective magnetic tiles is missed from detection the most, both of which are 4 pieces. Based on the whole test results, the proposed algorithm achieves good results in the detection of surface defects of the magnetic tiles and can be applied to detect and identify surface defects of the magnetic tile in actual production.
4.3. Comparative Experiment
In order to verify the performance of GSPSO-LRF-ELM algorithm in image classification, the proposed algorithm was compared with four traditional classification algorithms, support vector machine (SVM), artificial neural network (ANN), extreme learning machine (ELM), and local receptive field-based extreme learning machine (ELM-LRF). The experimental results are shown in Table 5.
As can be seen from Table 5, the training accuracy of the proposed GSPSO-LRF-ELM algorithm is 99.07%, and the test accuracy is 96.36%, the highest among all five compared algorithms. The test accuracy of the proposed algorithm is 4.54% higher than that of the traditional ELM-LRF algorithm, and it is much higher than the other three classical classification algorithms. In terms of time consumption, ELM training time and test time are the shortest. SVM and ANN detection time is also very short. Comparatively speaking, the training and testing time of the proposed GSPSO-LRF-ELM algorithm in this article is longer. This is because the convolution and pooling layers are added to the algorithm, and the input weight is optimized by PSO, so the algorithm runs longer. Although the running time of the algorithm proposed in this article is slightly longer, its training and testing accuracy have been significantly improved, which is more consistent with the demand for detection efficiency in the offline defects detection of the magnetic tile.
In order to make the experiment more comprehensive, the accuracy, false detection rate, and missed detection rate of the five algorithms for six categories of the magnetic tile defect as shown in Figures 6–8.
As can be seen from Figures 6–8, the classification accuracy of the GSPSO-LRF-ELM algorithm for the magnetic tile defect category is higher than that of the other four algorithms, and the classification rate of “free” and “break” reaches 100%. In terms of false detection rate, the overall false detection rate of the proposed algorithm is low, and the false detection rate of “crack” and “uneven” is 0%. In terms of missed detection rate, the algorithm proposed in this article is significantly lower than the other four classification algorithms, and the missed detection rate of “free” and “break” is 0%.
In summary, the algorithm proposed in this article is outstanding in the classification of the magnetic tile defects, and the classification accuracy is higher than the other four algorithms.
5. Conclusion
In this article, we propose an optimized local receptive field-based extreme learning machine for detecting and classifying the surface defects of the magnetic tile. In the ELM-LRF classifier, considering the difficulty of selecting the balance parameter C and the number of the feature maps K, the grid search method (GS) is used to optimize the balance parameter C and the number of the feature maps K, and in order to obtain the optimal initial weight Ainit, the particle swarm optimization algorithm (PSO) is used to optimize the initial weight Ainit of the classifier. The optimized classifier is named GSPSO-ELM- LRF. After preprocessing and segmenting, the magnetic tile images are inputted to the GSPSO-LRF-ELM classifier, and the surface defects of the magnetic tile will be detected and classified. Through experimental comparison and analysis, the method proposed in this article has the highest accuracy and better detection efficiency in the detection and classification of the magnetic tile surface defects.
In the future, the online detection and classification systems for the surface defects of the magnetic tiles will be further researched to achieve real-time detection, classification, and analysis.
Data Availability
The dataset used in the experiment was from the dataset on surface defect detection of the magnetic tile collected by Institute of Automation, Chinese Academy of Sciences. The Magnetic-tile-defect-datasets master data used to support the findings of this study have been deposited in the https://github.com/abin24/Magnetic-tile-defect-datasets repository.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this article.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (61503271 and 61603267) and Shanxi Natural Science Foundation of China (201801D121144 and 201801D221190).