Abstract
Automatic defect detection is an important and challenging problem in industrial quality inspection. This paper proposes an efficient defect detection method for tire quality assurance, which takes advantage of the feature similarity of tire images to capture the anomalies. The proposed detection algorithm mainly consists of three steps. Firstly, the local kernel regression descriptor is exploited to derive a set of feature vectors of an inspected tire image. These feature vectors are used to evaluate the feature dissimilarity of pixels. Next, the texture distortion degree of each pixel is estimated by weighted averaging of the dissimilarity between one pixel and its neighbors, which results in an anomaly map of the inspected image. Finally, the defects are located by segmenting this anomaly map with a simple thresholding process. Different from some existing detection algorithms that fail to work for tire tread images, the proposed detection algorithm works well not only for sidewall images but also for tread images. Experimental results demonstrate that the proposed algorithm can accurately locate the defects of tire images and outperforms the traditional defect detection algorithms in terms of various quantitative metrics.
1. Introduction
Due to unclean raw materials and undesired manufacturing facilities used in the tire manufacturing process, tire components may be contaminated by various defects, such as metallic or nonmetallic impurities (e.g., steel threads, screws, and plastic fragments), bubble, and overlap. When a vehicle with the defective tire is at high speeds, these defects often lead to a blowout of the tire. Therefore, the nondestructive defect detection technique based on X-ray imaging is essential for tire quality assurance. The traditional quality inspection process is mostly performed by human inspection, which often occurs from inaccurate and undetected inspection results due to visual fatigue and leads to low efficiency with high labor costs [1]. As a result, computer vision based detection techniques have become an important and efficient tool to improve quality of the products and increase manufacturing efficiency [2].
Automatic quality inspection of industrial products has been a popular research topic in the image processing and computer vision communities. Many methods based on different theories, such as texture analysis and spectral analysis, have been proposed to address the limitations of manual inspection. In the texture based detection methods, the detection of defects is carried out by comparing texture features of different image patches. Therefore, a key issue for such methods is texture feature extraction. Latif-Amet et al. [3] used the subband cooccurrence matrices (CM) to characterize texture features of multiscale subbands of inspected images. A major disadvantage of CM is the high computational complexity for large-size images. In [4], Tajeripour et al. proposed a defect detection method which applies the local binary pattern (LBP) to extract texture features. However, for low-quality images, LBP has poor performance in the feature description.
Meanwhile, there also exist many transform-based methods for defect detection. Due to its capability of singularity analysis and noise immunity, the wavelet transform is well suited for finding the location of anomalies in textured images. Tsai and Hsiao [5] proposed a wavelet-based method for automatic surface inspection, which generates an image with enhanced local anomalies by reconstructing the selected wavelet coefficients according to a synthesis strategy. Serdaroglu et al. [6, 7] use independent component analysis (ICA) and topographic ICA to generate feature vectors of wavelet subbands. Then the defects are detected according to the Euclidean distance between feature vectors. In addition, the Gabor transform is also a popular tool for extracting local frequency information of textured images. In [8], Kumar and Pang utilized Gabor filters to detect fabric defects, in which a foreground image is firstly extracted and then the defects are located by segmenting it straightforwardly. However, this method is sensitive to the choice of filter parameters. Although the optimized filters have been developed for defect detection in [9], they have a high computational cost. Unlike the above methods that use fixed basis functions to represent images, Tsai et al. [10] proposed a method for defect detection in solar modules, which uses ICA to learn a set of basis functions from defect-free training images. Each image under inspection can be represented as a linear combination of the learned basis functions. By analyzing the reconstruction error between the image under inspection and the reconstructed image from representation coefficients, defects can be detected. Similarly, some detection methods based on sparse representation have been introduced in different application fields [11, 12]. These methods use a sparse constraint to learn an adaptive representation dictionary from test images. Unfortunately, the learning process is computationally expensive.
Most of existing automatic inspection techniques are focused on textile [1–4], solar wafer [10, 13], and flat steel [14]. Recently, a few research works for tire defect detection have been reported in the literature. In [15], Guo and Wei proposed a detection method based on the image component decomposition (ICD) technique, which exploits the local total variation filtering and the vertical mean filtering to separate defects from inspected images. From the point of view of edge detection, Zhang et al. [16, 17] detected defects by using the multiscale geometric transform and the edge detection operator. Based on the dictionary representation technique, Xiang et al. [18] proposed a dictionary based detection method for tire defects by analyzing the distribution of representation coefficients. However, these methods for tire defects detection were designed only for tire sidewall images. Consequently, all of them fail to work for tire tread images due to their complex structures. Although a density projection based method [19] was presented for detecting tire tread images, this method only provides the orientation information of defects and cannot accurately locate them.
The reason for the weakness of the above methods is that they do not effectively capture texture distortions of images. To address this problem, a simple and efficient detection method is proposed in this paper, which takes advantage of the feature similarity of tire images. Specifically, for an inspected image, the proposed method firstly estimates the texture distortion degree of each pixel by weighted averaging of the dissimilarity between this pixel and its neighbors within a local window and produces an anomaly map of the inspected tire image. Then the defects are located by segmenting this anomaly map with a simple thresholding function. Experimental results on tire X-ray images show that the proposed method can effectively detect the defects both in the tire sidewall and in the tire tread. To the best of our knowledge, this is the first work that can accurately locate the defects on tire thread images. In addition, as the computational core of the proposed algorithm, computing the feature dissimilarity of image pixels can be implemented independently in parallel, which makes the proposed method feasible for tire online inspection.
2. Characteristics of Tire Images
Due to imperfect raw materials and manufacturing process, tire sidewall and tire tread may contain various types of defects such as impurity, bubble, and overlap. Figure 1 shows the main defects of tire sidewall and tire tread in the quality inspection. Figure 1(a) displays a tire sidewall image that contains a metallic impurity, in which the region of impurity is darker than its neighbors. On the contrary, a bubble on the tire sidewall shown in Figure 1(b) is brighter than the surrounding pixels. Figure 1(c) is an example of the overlap which results in an irregular texture. Figures 1(d) and 1(e) further show two tire tread images, which, respectively, contain an impurity and an overlap.
(a)
(b)
(c)
(d)
(e)
From Figure 1, it is observed that tire tread images contain more complicated textures with lower contrast than tire sidewall images, which leads to difficulties in detecting defects in the tire tread. Apart from this observation, it is further noted that both tire sidewall images and tire tread images are dominated by a texture which manifests a high regularity and defects locally break the regularity of a texture. This regularity generally means a similarity between pixel features. Therefore, the defects can be located by analyzing the feature dissimilarity of a pixel and its surrounding ones. In the following section, the proposed detection method based on the feature dissimilarity analysis is described in detail.
3. The Proposed Method
The similarity between pixels of a textural image implies that there exists an implicit dependency between one pixel and its neighbors. Therefore, a pixel can be represented as a weighted linear combination of the surrounding pixels. Based on the above considerations, we propose a detection method by using weighted texture dissimilarity to measure perceptual texture distortion, in which the distortion of a pixel is defined as the dissimilarity between the original feature value and the represented one of pixels. Figure 2 shows a block diagram of the proposed method, which consists of three main steps: extracting texture features, evaluating structural dissimilarity, and segmenting the defects. Texture features of each pixel are extracted using the local kernel regression (LKR) descriptor [20] and represented as vectors of features. Then the anomaly of each pixel is determined by weighting the dissimilarity between this pixel and its local neighbors. Finally, the defects are located by segmenting the anomaly map with a simple thresholding process.
In order to determine the distortion of each pixel, we introduce an anomaly value for each pixel to quantize its distortion. Specifically, the anomaly value of a pixel in an image is defined as follows:where is a weight determined from the input data, is a spatial window centered at pixel , and is a feature dissimilarity metric between pixels and . The key issue in calculating the anomaly value is how to determine the feature dissimilarity metric . Therefore, we will describe the feature descriptor adopted to represent the texture structure of inspected images first, the definitions of , and afterwards.
3.1. Texture Feature Extraction
Different from the intensity-based metric for similarity measurement, the proposed method uses the feature correlation as a quantitative measure of the similarity between pixels and their neighbors, which is motivated by the following two reasons: on the one hand, texture feature extraction of inspected images is essential for measuring the perceptual texture distortion, which is an efficient way to avoid the negative influences of illumination changes for similarity measurement. On the other hand, we use the correlation metric to calculate patch similarity which has the advantage of being insensitive to outliers.
Due to its robustness to noise and other perturbations, we adopt the LKR descriptor to derive a set of feature vectors of an image, which acquires the textural structures of images by analyzing the local gradient information. Unlike scale-invariant feature transform (SIFT) and histogram of gradient (HOG) which use the quantization of oriented gradients to reduce computational cost, LKR computes the texture feature between oriented gradients without the quantization step, which leads to a better feature description power. In general, LKR is more invariant to the shift and rotation transformations than the conventional feature descriptors based on gradients and key points. For a pixel at position , the LKR feature vector is defined as a self-similarity between this pixel and its neighbors, which is derived by the following form:where is the spatial coordinates, is the number of pixels in a local window with size , and is a gradient covariance matrix. In the numerical calculation, is estimated by averaging a collection of spatial gradient vectors within the local window and can be written as follows:where and are the first derivatives along and directions, respectively.
From (3) it is seen that can be interpreted as averaging geodesic distances in a patch. Thus it is robust to noise and other perturbations. In addition, as a key part of LKR, can capture the singularities of images, which makes LKR suitable for describing the texture feature of images. For complete details, we refer the interested reader to the work of Takeda et al. [20]. In order to validate the effectiveness of LKR, the LKR magnitudes (for a LKR feature vector , we define its LKR magnitude as the square root of the sum of the squares of its elements; i.e., ) of two tire defective images are displayed in Figure 3. It can be observed that the LKR descriptor can effectively capture local texture distortion of tire images.
(a)
(b)
(c)
(d)
3.2. Feature Dissimilarity Evaluation
In general, the computation of pixel-based dissimilarity metric is quite sensitive to noise so that the resulting anomaly values of pixels are unstable in the presence of noise. Instead of making use of a pixel for similarity measurement, the proposed method exploits a patch-based metric to improve the robustness of measuring similarity (patch-based metric has been extensively used in the image processing community. Its robustness against noise has also been demonstrated in the literature. See [21, 22] for some application examples). Thus the anomaly model (1) can be generalized aswhere and are the patches of pixels centered at and , respectively. For the simplification of notations, we use instead of .
In (4), the dissimilarity metric plays a significant role in computing anomaly. In this paper, we exploit a correlation metric based on the LKR feature descriptor to measure the dissimilarity between image patches, which is defined as follows: where denotes a LKR feature extraction operator, and is an adjustment parameter. The correlation defined in (6), also called cosine similarity, is a measure of the cosine angle between and because the inner product depends exclusively on the angle between two vectors. Due to the advantage of robustness to noise, the cosine similarity outperforms the conventional metrics for the similarity measurement, such as Euclidean (L2) distance or Manhattan (L1) distance [23]. In fact, the cosine similarity is the uncentered Pearson correlation coefficient, which avoids subtracting the vector means and permits a simpler calculation. We evaluated the performance of the cosine similarity and Pearson correlation coefficient and found that the performance of cosine similarity is very close to Pearson correlation coefficient. Therefore, we use it to measure the similarity of LKR feature vectors.
In essence, (1) is equivalent to applying a weighted filter to the dissimilarity values between pixels. The simplest form of a weighted filter is the uniform weight that assigns the same weight to all data; for example, all weights set to 1. However, the uniform weight generally leads to an oversmoothed result. In the literature, there exists various weight functions such as Gaussian weight, Geman-McClure weight, and Hebert-Leahy weight [24]. For calculating the anomaly map, the weight value should be inversely proportional to the degree of dissimilarity. Thus, we adopt the following weight function due to its simplification:where , and is a small constant that avoids division by zero.
For each pixel , we can yield its anomaly value by substituting (5) into (4). The smaller the value of is, the smaller the possibility of belonging to a defect is. Therefore, we can use as a dissimilarity measure to distinguish defect pixels from the inspected image. In essence, defines an anomaly map of the image. Anomaly maps of six tire tread images are displayed in Figures 6(b) and 7(b). It can be noted that the entire defect regions are clearly highlighted in the anomaly maps, which facilitates an accurate segmentation of defects by using a simple segmentation algorithm.
3.3. Defect Segmentation
After generating the anomaly map, defects can be located by way of a thresholding process. Specifically, if the anomaly value of a pixel is larger than a given threshold , this pixel is considered to be a part of a defect and its value is set to 1. On the contrary, it is set to 0 if its anomaly value is below the threshold. This simple thresholding process can be formulated as follows:The output of the above process is a binary image, in which white pixels denote defective regions and black pixels denote defect-free regions.
Although the defects can be detected by using the above thresholding process, there exists a small amount of defect-free pixels which are detected as defective. To resolve this problem and improve the robustness of the proposed algorithm, the pixels contained within defective regions further determined the identity of a defect by the local variance analysis. The local variance of a pixel is calculated over a local window centered at , which can be formulated aswhere is a normalized Gaussian weight and is the local mean defined as By integrating the local variance into (8), the thresholding function is then modified aswhere is a given threshold.
The complete operational procedure of the proposed detection method can be algorithmically summarized in Algorithm 1.
| ||||||||||||||||||
4. Experiments
In order to validate the effectiveness of the proposed algorithm, this section gives our experimental results performed on a test dataset that is provided by Linglong Tyre Co. Ltd. This dataset consists of 40 defective sidewall images, 20 defect-free sidewall images, 40 defective tread images, and 20 defect-free tread images, in which different types of the defects like impurity, bubble, and overlap are included. The size of all test images is . The performance of the proposed method is evaluated by comparing with ICD-based method (ICDM) [15] and the improved wavelet-based method (IWaveM) [5]. It needs to point out that we improve the performance of the original wavelet-based method [5] by extracting local wavelet features from small image patches and introducing the vertical mean filtering as used in ICDM. These methods are implemented in MATLAB and performed on an Intel-i7 1.73 GHz computer system with 8 GB RAM. Although there are other methods for tire inspection in the literature, we cannot comprehensively quote them in the paper due to the page limitation. In addition, the codes of these methods are not available online. In all comparisons, the scale parameter for ICDM is set to 7, the Daubechies wavelet transform is exploited for IWaveM with four vanishing moments over three decomposition levels, and for the proposed method we utilize a local window with size and set the patch size , , .
4.1. Performance Evaluation
The performance of these detection methods mentioned above is quantitatively evaluated by detection rate (DR) [4, 10], precision (), recall (), and -measure () [25, 26], which are deeply related to the ROC analysis [27] and, respectively, defined as follows:where is the total number of test images, is the number of detected defective images, is the number of detected defect-free images, are the number of true-positive pixels, are the number of false-positive pixels, are the number of false-negative pixels, and we set as in [28]. corresponds to the anomaly pixels detection performance, while is the fraction of detected anomaly pixels to ground-truth. As an overall performance measurement, the -measure is the weighted harmonic mean of and [29].
In Table 1, the detection rates of different detection methods for test images are listed for quantitative comparisons. From this table we can observe that for tire sidewall images of the proposed method outperform ICDM and IWaveM in terms of DR; for tire tread images it also achieves high DR. However, ICDM and IWaveM are not applicable (N/A) to detect the defects of tire tread images. To further compare the performance of these detection methods, the average precision, recall, and -measure on tire sidewall images are shown in Figure 4. It is obvious that the proposed method offers better detection performance than ICDM and IWaveM. In general, ICDM and IWaveM perform well on simple texture images. When the images are with complex structures, they do not work as well. The major reason is that the low contrast and the complicated texture of tire tread images reduce the saliency of defects, which causes them to produce unsatisfactory results.
For visual comparisons, Figure 5 shows the detection results of these detection algorithms on tire sidewall images, respectively. As shown in this figure, all detection methods can locate the defects of tire sidewall images, and the proposed method obviously outperforms ICDM and IWaveM, which is also justified in Table 1 and Figure 4. To demonstrate the effectiveness of our method detecting the defects of tire tread images, Figures 6 and 7 illustrate the anomaly maps of two types of defects (impurity and overlap) of tire tread images and their detection results produced by our method, respectively. It can be observed that the proposed anomaly model is quite sensitive to texture distortion and the saliencies of defects are well highlighted in the anomaly maps.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(a)
(b)
(c)
4.2. Parameters Selection
There are four main parameters in the proposed method, the size of local windows and for feature dissimilarity evaluation and the thresholds and for defect segmentation. The parameter is used to segment the anomaly map which describes the feature dissimilarity of image pixels. Therefore, it ranges from to . A small value is sensitive to noise and may produce false alarms in defect-free images, whereas a large value results in a loose control limit and may lead to miss a subtle defect. Figure 8 shows the performance curves of the proposed method with varying from to . It can be observed that the proposed algorithm is insensitive to in the range . Thus, we choose in our experiments, which is a trade-off between and . For different types of inspected images, the parameter can be determined by using a manual tuning method. We find that its performance curves are similar in shape to the ones shown in Figure 8. In all experiments, we empirically set for tire sidewall images and set for tire tread images, which yield a well overall detection performance.
The size of local window has an important influence on the detection performance of our method. The optimal choice of this parameter depends on the size of defects. By observing defective images, we find that the pixels in the vertical direction are very similar and defects locally break the similarity of pixels. Therefore, a rectangular window with the size is preferred. In order to evaluate the influence of varying the window size , Figure 9 shows the change curves of three quantitative metrics , , and with varying from to . It reveals that our method works well when , that is, the size of local window being . In consideration of detection effectiveness and computational cost, we choose the size of for all experiments. In addition, we find that our algorithm is insensitive to , and we empirically set .
4.3. Computational Cost
To evaluate the computational cost of three detection methods, we compare the running time on the test dataset. The CPU implementations of ICDM, IWaveM, and our method require 2.6 s, 1.2 s, and 17.3 s on the average, respectively, which are performed on Intel Core i7 3.60 GHz. The proposed method for LKR and the dissimilarity metric is computationally expensive. In general, the LKR step takes approximately 18% of the execution time, whereas 73% of the time is spent in calculating the dissimilarity metric. However, our method is suitable for parallel processing because LKR and the dissimilarity metric for each pixel can potentially be calculated independently. In the practical application, therefore, we can accelerate it on the GPU.
5. Conclusions
Automatic quality inspection is strongly desired by tire industry to take the place of the manual inspection. This paper presents an efficient detection method for automatic quality inspection, which takes advantages of feature similarity of tire images and captures the texture distortion of each pixel by weighted averaging of the dissimilarity between this pixel and its neighbors. Different from the existing tire defect detection algorithms that fail to work for tire tread images, the proposed detection algorithm works well not only for sidewall images but also for tread images. Experimental results performed on real tire inspected images validate that the proposed detection method outperforms the current defect detection methods.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 61202150, Grant 61272245, Grant 61373078, and Grant 61472220; Key Research Development Program of Shandong Province under Grant 2015GGX101004; China Postdoctoral Science Foundation under Grant 2013M531600; and Program for Scientific Research Innovation Team in Colleges and Universities of Shandong Province.