Abstract
Identification of the highly reflective surface defects on roller parts is a requirement to assure high quality of parts. However, the highly reflective roller surface has the reflection characteristics, which easily lead to the missed detection or wrong detection of defect targets in the visual recognition process. For this problem, a new identification method based on image library matching is proposed. First, to protect the edge information of the defect target while eliminating noise, preprocessing of the measured initial images can be accomplished by using the accelerated optimized bilateral filtering. Second, the entropy and grid gray gradient are used to achieve rough segmentation of highly reflective surface defects on roller parts. Finally, a defect fine identification method based on the Hu invariant moment matching integrated with morphological classification is proposed for achieving image library matching and further quickly removing the pseudodefects. Experimental tests were conducted to verify the effectiveness of the proposed method in achieving accurate identification of highly reflective surface defects on roller parts. The proposed method has an accuracy of 98.2%, and the running time can basically meet the requirements of real-time performance.
1. Introduction
Because of its reflective properties, the highly reflective rotary surface (HRRS) can clearly reflect images of the surface of smooth precision rotary parts, of which the aerospace precision bearing roller is a typical example. Defects may occur in the production, postprocessing, and use, such as pits, scrapes, abrasions, and scratches (also called scratch strips) [1]. Defects on the roller surface can even result in fracture and failure of the bearing during operation, causing heavy economic losses [2]. However, there are various types of surface defects which are in different sizes and shapes and are randomly distributed on the roller balls [3]. They can be held responsible for the slow progress in automated testing. At present, nondestructive testing methods based on some physical characteristics are mainly used, such as photoelectric detection [4], eddy current testing [5], and ultrasonic testing [6]. The accuracy and efficiency cannot be balanced, and the cost is high. The vision sensor technology is widely used in industrial fields [7] and can provide noncontact measurement, human preference for visual information, and rapid development of computer image processing technology. Thus, many existing high-reflection surface defect detection methods are based on the collection of highly reflective surface defect images of vision sensors. However, because of external interference and other factors, the collected images are distorted or the defect information is blurred [8], which leads to missed detection or wrong detection. The core purpose of surface image processing of metal parts is to accurately identify the defect targets in the image. Therefore, defect identification is very important for surface quality assurance of metal parts with the HRRS.
With regard to feature identification, it includes methods based on the threshold segmentation method [9–11], edge detection technology [12–16], wavelet transform [17–19], morphological operation [20–22], and so on. Yuan et al. proposed an improved Otsu method in the field of global adaptive threshold segmentation algorithm [9]. The threshold weighting was used to improve the effectiveness of the segmentation threshold. It has achieved good performance in image segmentation of metal surfaces with gray peak distribution. However, the algorithm is less effective in metal surface images with complex gray fluctuations. Vasilic and Hocenski proposed a surface defect segmentation algorithm based on Canny edge detection and corner positioning [12], which can better extract defects such as scratches and pits, but the defect extraction effect for images with more complex backgrounds is not good. Zhou et al. proposed a method for identifying metal surface defects based on the fusion of shear wave and wavelet features [17], which can be effectively used for surface defect identification of different types of metals, but the training process requires a large number of metal surface defect sample images, the operation process is cumbersome, and the practicability is low. Sharif et al. analyzed the infected blood cells through the top-hat transform [20], and experiments showed that this method is more accurate than the traditional watershed method. Yan and Liu applied the adaptive morphological filtering method to defect identification of X-ray weld images and obtained satisfying results [21]. Mao and Zhang proposed a workpiece image object identification algorithm based on the top-hat transform sequence analysis [22], which has good antinoise and antibackground interference capabilities. However, most of these target identification methods based on top-hat transformation can hardly meet the integrity requirements of workpiece defect segmentation, and often there are omissions and over segmentation. Moreover, for different workpieces or images of different defects on the same workpiece, manual intervention is required. Its accuracy and automation are not high. Ai and Xu proposed an identification method of defect targets for metal surface based on Contourlet transform [23]. This method can achieve high-accuracy positioning of defect targets, but the obtained edge information is incomplete. Therefore, all the abovementioned feature identification methods are applicable or may not produce accurate results for the engineering applications discussed in this paper. As a result, it is desirable to develop a method that is able to deal with the problem of feature identification discussed in this paper.
In this paper, a novel method for identifying defects on the HRRS is proposed to reduce the limitations of existing identification methods. The proposed identification method consists of three parts: preprocessing, rough segmentation, and fine identification. Firstly, the bilateral filtering, which can construct a compromise filter that makes full use of spatial and range information, can be used to perform “edge preservation and denoising” processing on the original images. Secondly, an adaptive background rough segmentation method based on the entropy and grid gray gradient is designed to segment the image. Finally, the Hu invariant moment matching based on morphological classification is used to achieve image library matching and further quickly remove the pseudodefects. The experimental results show that the proposed identification method can improve the accuracy of defect recognition rate, and with the increase of image database samples, the recognition accuracy will be improved gradually, which has certain exploration significance.
The remaining part of this paper is arranged as follows: in Section 2, we describe systematic measurement and preprocessing. In Section 3, we propose a defect recognition algorithm. Section 4 provides experimental results with real data. Finally, concluding remarks are provided in Section 5.
2. Systematic Measurement and Preprocessing
2.1. Image Collection
The principle of measurement is shown in Figure 1. The defect detection system is composed of a vision sensor (CCD), an LED light source, several optical lenses, a digital micromirror device (DMD), and a high-precision turntable. During the measurement process, the part to be tested with surface defects is first placed on the turntable and adjusted until its center is on the rotary axis of turntable. Then, the illumination light generated by the collimated LED light source is incident on the total internal reflection (TIR) prism, and the incident light is reflected by the TIR prism and vertically irradiated on the DMD. It makes the modulated reflected light (illuminance uniform) pass through the TIR prism and then projected onto the highly reflective surface of the roller under test by a filter and a projection lens to obtain a measured image through a visual sensor (CCD). Finally, the obtained image information is analyzed and the defect is detected.
The use of DMD in the system enables the use of a common LED light source to produce a finely tunable illumination with uniform illumination, which is more sensitive to surface defects, thereby enabling ball surface defect detection to be possible. Meanwhile, the use of DMD in this system enhances the versatility of the illumination system.
2.2. Image Preprocessing
Bilateral filtering considers the pixel value variation in the image domain and the spatial domain at the same time and achieves satisfactory noise smoothing effect while maintaining the edge information [11]. But, the operational efficiency of the algorithm is reduced due to the construction of two Gaussian filter kernels. In this paper, based on the separation characteristics of the Gaussian function [24], the two-dimensional Gaussian function convolution on the image is divided into two steps, which improves the algorithm efficiency greatly without sacrificing precision.
The spatial domain filter coefficient template is determined by the geometric distance of the image space, and the pixel function of image is expressed as . c is the Gaussian weight of the image spatial domain filtering. s is a Gaussian weight based on the image value domain. The image output by Gaussian bilateral filtering is defined as follows:where c and s are given by equations (2) and (4), respectively:with
In equation (4),
It can be seen that, for the small variation region on the image, the range weight s is close to 1. At this point, the spatial domain weight plays a major role and the Gaussian filter works to smooth and denoise the image. But, in the large variation region, the domain weight s decreases. At this point, the smoothing filter plays a limited role and only serves the purpose of preserving image edge information.
According to the separation characteristics of the Gaussian function, it is separated as in the following equation:withwhere is a pixel in the image, is the two-dimensional Gaussian kernel function, and and are the one-dimensional Gaussian kernel functions of on the image columns and rows, respectively.
Based on the above equation, equations (2) and (4) are separated into one-dimensional functions as follows:
Suppose the image is 300 × 300 pixels, the filter kernel size is chosen to be 10 × 10, and the weight of the 10 × 10 points on the filter kernel can be calculated. The time-domain weight of the bilateral filtering will be calculated for 9 × 106. In this paper, the filtering method is adopted in which the two-dimensional Gaussian function convolution on the image is divided into two steps. The number of calculations of the domain weight is reduced to 1.8 × 106 times, a decrease by 80%.
3. Defect Identification Algorithm
After the image is preprocessed, a defect identification algorithm is designed to match the correctly identified defects in the database. The defect identification algorithm, which can achieve accurate identification of defect targets, mainly uses the invariant moment of the image and the morphological classification of defect features.
3.1. Defect Rough Segmentation
3.1.1. Image Meshing
When meshing is performed, too small a grid means heavy calculation and inaccuracy in calculating the gray value entropy. An oversized grid contains too much background redundant information, which leads to a poor defect contour shape in contour matching and low classification accuracy. Based on numerous experiments, the size of 30 × 30 pixels is used for mesh partition because the algorithm has the highest accuracy.
3.1.2. Rough Segmentation of Highly Reflective Surface Defects
This section mainly analyzes blurred boundary low-contrast defects that are difficult to extract, and it can be extended to the general significant defect targets. Different from the existing image segmentation algorithms, the algorithm proposed in this paper does not need binarization and the original grayscale image of the region after image segmentation is still retained. The image is first divided into blocks, and the entropy of the gray value of each block is calculated. The degree of dispersion of the gray value is counted to determine whether the block is a smooth background or a defect area with changed gray values.
The degree of change in the gray value of each block image is calculated by using the following equation:where
It is equivalent to finding the variance of the two-dimensional discrete function, as shown in the following equation:where is the gray value of the image at (i, j) and and represent the pixel size of the image. The gray value with the highest frequency is the background gray information of the nondefective area of the image. In equation (11), hmax is the gray value with the highest frequency in the image block, and therefore it has good adaptability and universality for the background separation of different images and can identify defective areas.
Experiment results show that the above algorithm works well for low-contrast defect images, but for some images with very small defects and high contrast, the effect is not obvious. Then, a second criterion is introduced to calculate the change gradient of the gray value of each pixel of each image. If the gray gradient change in the grid exceeds the set threshold, it indicates that the gray level of the image pixel in the area has a step change and contains defect information.
If the function has a first-order continuous partial derivative in the plane region D, then a vector can be defined for each point of D. This vector is called the gradient of the function at point P, as shown in the following equation:
For digital images, it is equivalent to calculating the gradient of a two-dimensional discrete function.where is the gradient of the gray value change at (i, j) in the image, and are the gradients of the gray value change in the horizontal direction and the vertical direction, respectively, and is pixel value interval for calculating the gradient.
As shown in Figure 2, when characterizing a defect, the connected image of the image defect grid is labeled by referring to the connected field label of the binary image. The defect feature image blocks identified by the algorithm are connected to form a labeled defect feature contour, which provides information (such as contour, circumscribed rectangle, centroid, and invariant moment) for subsequent matching recognition.
(a)
(b)
(c)
(d)
3.2. Defect Fine Identification
3.2.1. Components of Hu Invariant Moment
Hu invariant moment is a calculation method of shape invariant moment, which has invariance of translation, proportion, and rotation in image matching [25]. In this paper, the method is used to construct image matching invariant moment, which can improve the computational efficiency effectively.
Let the two-dimensional digital image be represented by , and its (p + q)th moment can be defined as
After discretization, the above equation becomeswhere C and R represent the column and row of the image, respectively.
In image matching, there may be problems such as translation, scale change, and rotation, and it is necessary to construct an invariant moment from the base moment. The translation invariance is determined by the zero-order origin moment and first-order origin moment, and the centroid coordinates of the target region can be obtained, as in the following equation:
The (p + q)th central moment can be defined aswhere
For two-dimensional digital images, x0 and y0 are their centroid coordinates, which represent the gray centroid of the image in the horizontal and vertical directions, respectively.
The center moment can be constructed by selecting the centroid of the target area as the center. Then, the calculation is always about the points in the target area relative to its centroid and is independent of the position of the target area, hence the translation invariance.
In order to minimize the influence of scale change on the central moment, the zero-order central moment is used to normalize the central moment of each order. The normalized central moment iswhere
It can be seen that the zero-order moment represents the mass (area) of the target region. If the scale of the target region changes, the zero-order central moment will decrease correspondingly, so it has scale invariance.
The Hu moment constructs rotation invariance. The following 7 invariant moment groups can be derived by using the second- and third-order central moments, which remain unchanged in image translation, rotation, and scale change:
Therefore, the Hu invariant moment of the image can be obtained by applying equation (21), and the distance between two images A and B can be calculated by using the following equation:withwhere and () are the 7 Hu invariant moments of the images A and B to be matched, respectively.
calculated by equation (22) is the similarity degree of defect images A and B.
3.2.2. Defect Fine Identification Process
Among the five surface defects of the roller, scratches and points are the most common and are differentiated from the other types in image morphology. The group point defects are composed of irregular point defects [10]. Therefore, the defect target short diameter to long diameter ratio Rb and the area S are introduced to perform rough classification before matching, and then the Hu invariant moment matching is performed with the defects in the database. It ensures the identification accuracy and further improves the efficiency as well.
The short and long diameter of the defect is represented by the distance between the two vertices. If the points , , , and are the four vertices of the minimum circumscribed rectangle of the defect feature rotation, then we have
In equation (25), Rb is the short diameter to long diameter ratio of the defect feature contour, and equation (26) is the defect area. When the defect is a strip type, Rb tends to be small; otherwise, the defect is square. In this case, the defect may be a point type when S is small, and it may be others types (pits, scrapes, scratches, and so on) when S is large. In this way, the description of the morphological classification of defect features can be achieved. Therefore, the fast and accurate recognition of defect images based on the Hu invariant moment matching principle and the defect feature morphological classification is shown in Figure 3. First, the short diameter to long diameter ratio of the D target contour identified is used to distinguish strip defects. Then, the area of nonstrip targets is judged to determine the defect type of those with the smallest area. Finally, the Hu invariant moment matching is performed on the corresponding types of defects in the database. The similarity degree of the two images is calculated by using the Euclidean distance to realize defect matching and recognition, and the threshold is experimentally determined.
4. Experiment Results and Analysis
The algorithm verification experiments and tests in this paper are conducted under the following circumstances: Intel Core i3 CPU, 8G memory, Python 3.6 compiler environment, and MySQL database. The research objects are the surface images of precision roller bearings with obvious scratch defects, two types of defects, and pits with blurred boundaries and low contrast, respectively.
According to the process of defect identification in this paper, a step-by-step algorithm experiment is performed on the images shown in Figure 4(a). The first line is an image with obvious scratch defects, and the second line is an image with two types of defects—points and scratches. In the third line, the image has pits in the center with an approximate circular boundary which is extremely blurred. In the fourth line, the image has pits with normal contrast and insignificant scratches. There are some high-frequency noise points in all the images.
(a)
(b)
(c)
(d)
(e)
Figure 4(b) shows that the accelerated bilateral filtering smoothes the grayed original image effectively. Figure 4(c) performs the gray value entropy statistical analysis, gray value gradient change calculation, and background segmentation with the mesh size of 30 × 30 pixel for each image. The image blocks with defects are obtained, and the complete defect feature contour is distinguished by the four-way domain mark as shown in Figure 4(d). The final defect feature information is as shown in Figure 4(e). It can be seen that the defects in the three images are identified completely and effectively, which verifies the strong robustness of the defect recognition algorithm proposed in this paper.
To verify the reliability and accuracy of the algorithm, it is compared with several common defect identification image segmentation algorithms [11, 26] in terms of accuracy and efficiency, as shown in Figure 5. As for the comparison of experimental results, this paper mainly discusses the accuracy of defect identification and avoidance of the missed detection problem rather than the accurate calculation of defect size. The methods proposed in recent years have higher accuracy for defect size identification, and there are problems of false detection and missed detection for the influence of boundary blurring and specular light. Therefore, according to the characteristics of boundary blurring and specular influence of the images to be inspected, this paper selects two representative defect recognition methods to compare with the results obtained by the algorithm in this paper. They are the optimal threshold processing method based on the Otsu operator, which has the best recognition in the statistical sense, and Canny edge algorithm with better segmentation effect in metal surface defect recognition. With the increase of image database samples, the accuracy of the method in this paper will gradually improve.
(a)
(b)
(c)
(d)
The three methods have good segmentation and recognition capability for these defects. The algorithm in this paper can completely retain the gray information inside the defects while identifying the defects. In the experiment of low-contrast pits with blurred boundary of the third and fourth lines, both the traditional Otsu segmentation method and the canny operator edge detection algorithm with adaptive threshold segmentation fail to complete the defect identification. In the third line of the image, the traditional Otsu segmentation method is unable to segment any effective information, but the method proposed in this paper can fully identify and preserve the complete gray information of the image. In the fourth line, both the traditional Otsu segmentation method and the canny operator edge identification algorithm with adaptive segmentation threshold can recognize the defects, but much information is lost and the defect shape cannot be determined. By comparison, the method in this paper can fully recognize the defects and retain the complete gray information of the image.
To further demonstrate the accuracy and efficiency of the algorithm, in the experiment, 50 images were selected, mainly including 54 defects such as pits, abrasions, and scratches. The contrast of defects in the images covered the whole range from being blurry to clear, with typical representation, and 5 of them had no defects, is identified by using the current algorithm and two other algorithms, as shown in Table 1. As can be seen, the traditional Otsu method has an accuracy of 72.2% and the canny operator edge identification algorithm with adaptive segmentation threshold has an accuracy of 75.9%. The accuracy of the algorithm proposed in this paper is 98.2%, which only fails to identify pit defects with an extremely low contrast and boundary barely visible to the naked eye. Its running time is between the two algorithms, which can basically meet the real-time requirements. Meanwhile, with the increase of image database samples, the accuracy of the method in this paper is gradually improved. In order to further prove the accuracy and efficiency of the algorithm in this paper, 650 images (including target defects) in the image database are taken as the tested images for image matching, with a matching accuracy of 99.5%, which further verifies the accuracy of defect recognition based on image database matching, which has certain exploratory significance.
5. Conclusion
By using image library matching, a new identification method for the HRRS is proposed. This method includes three stages: preprocessing, rough segmentation, and fine identification. For the first stage, a preprocessing method is proposed. It uses the accelerated optimized bilateral filtering. For the second stage, a defect rough segmentation method is proposed. It uses the entropy and grid gray gradient to automatically achieve rough segmentation of defects. The rough segmentation method is adaptive and has good generality. For the third stage, a novel defect fine identification method is proposed. It makes use of Hu invariant moment matching based on morphological classification to achieve image library matching and further quickly remove the pseudodefects. This identification process avoids the problem of the missed detection or wrong detection of defect targets in the visual recognition process of defects, which is caused by the reflection characteristics of the HRRS, and has a small amount of calculation. The proposed identification method is simple, precise, and rapid and can be an attractive alternative to existing identification methods. It has been successfully used in practical applications. As observed from the experiment, satisfactory results have been achieved.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the Aeronautical Science Foundation of China (ASFC) (grant no. 201603T6001), the National Natural Science Foundation of China (NSFC) (grant no. 51775433), and the Natural Science Basic Research Plan in Shaanxi Province of China (grant no. 2018JQ5180).