Abstract
Existing image segmentation and image enhancement methods are deficient in complex industrial environments. Therefore, an improved morphological band-pass filter algorithm is presented. The first step of the algorithm is obtaining two marker images by erosion operations for a test image with two kinds of structuring elements: one slightly larger and one smaller than the feature but similar in shape. The second step is obtaining an image only, including the background, and an image including the feature with the background, excluding noise. The final step is realizing the feature segmentation by carrying out a difference operation on the two images. Selection of the structuring elements in the algorithm and the computational cost reduction are also discussed for engineering applications. Experimental results show that the proposed algorithm achieves the accurate segmentation of the circle at a specific scale through the velocity-optimized morphological operation and features good real-time performance and high accuracy in complex industrial environments, which could meet the requirements of industrial online monitoring.
1. Introduction
In visual measurement systems, it is necessary to remove or suppress background noise and interference with image segmentation and enhancement. In a complex industrial environment, background noise and interference (e.g., vibration, water mist, dust, and illumination changes) are unlikely to be totally suppressed. Yet, they exert adverse effects on testing images. Simultaneous and accurate real-time performance of visual measurement systems in complex industrial environments are thus difficult to ensure. This problem limits the advanced promotion of vision measurement technology in the field of industrial testing [1–4]. Extensive and in-depth studies have been conducted by many scholars in relevant fields. The studies can be divided into two categories.
The first category includes studying better ways to weaken background information and highlight feature information. Grigoryan and Agaian [5] described a unified approach for signal thresholding by introducing the general concept of weighted thresholding using monotonic sequences for signal and image set-theoretical representation. The main advantage of this representation is that it allows for the performance of nonlinear operations by a weighted threshold, enhancing many geometric features present in the original signals and images via coefficient manipulation and weighting. Zhang et al. [6] presented a novel image enhancement method inspired by the Retinex framework, which simulates the human visual system. However, the proposed method is more suitable for underwater environments. Liu and Chen [7] proposed a new infrared image detail enhancement approach. It not only achieved the goal of enhancing the digital detail, but also processed an image quite true to the real situation. Sadreazami et al. [8] proposed a new image-denoising method in the contourlet domain by using the alpha-stable family of distributions as input to contourlet image coefficients. In terms of the peak signal-to-noise ratio and mean structural similarity index, as well as the visual quality of the denoised images, the proposed method outperforms other existing methods.
The second category includes improving accurate and real-time feature extraction algorithms. Hough transform (HT) is a robust method for feature extraction. However, its poor real-time performance and low precision render the method inefficient for industrial testing. In view of this disadvantage, Xu et al. [9] presented the randomized HT (RHT). Spratling [10] proposed a new method for implementing a voting process in HT, via a competitive neural network. Djekoune et al. [11] presented a new modification of the HT method developed for an automatic biometric iris recognition system. Jiang [12] optimized methods for determining sample points and finding candidate circles of the RHT method for circle detection. These studies improved the accuracy and real-time performance of HT, to a certain extent.
Additionally, some new feature extraction algorithms were proposed in recent years. Cai et al. [13] presented an efficient circle detector based on the region growth of gradient and histogram distribution of Euclidean distance. Experimental results demonstrated the ability to detect circular objects under occlusion, image noises, and moderate shape deformations with good precision. Sun et al. [14] proposed improved features from accelerated segment test (FAST) feature extraction, based on the random sample consensus (RANSAC) method. The proposed feature extractor not only effectively extracts features, but also reduces positioning error availably, making the proposed FAST feature extraction based on RANSAC feasible and efficient. De Marco et al. [15] presented a randomized iterative workflow, which exploited geometrical properties of image isophotes, selecting the most meaningful edge pixels and classifying them into subsets of equal curvature. The new method accurately detects circles within a limited number of iterations, maintaining subpixel accuracy, even in the presence of high noise levels.
The above studies all show advantages. However, in complex industrial environments, the first category cannot completely suppress all kinds of interference, and the second cannot fully meet the requirements of online monitoring, in terms of accuracy or real-time performance.
Morphological band-pass filtering is a method of subtracting and segmenting features using a specific shape-scale structuring element [16]. It basically suppresses all kinds of background noise and interference, realizing exact segmentation of small features from an area up to 10 square pixels. For slightly larger features, the algorithm destroys the original feature boundaries and shape, thus forfeiting utility.
The circle, or ellipse, is the most widely used basic feature in machine vision. Because extant image segmentation and enhancement methods are deficient in complex industrial environments, this paper presents an improved morphological band-pass filtering algorithm via the introduction of a morphological reconstruction operation [17, 18]. Our new method is capable of accurate segmentation of specific scale circle features through a velocity-optimized morphological operation, featuring good real-time currency and high accuracy in complex industrial environments. This algorithm also has the potential of accurate segmentation of other features of varying shapes and sizes.
2. Analysis of Basic Morphological Band-Pass Filtering
Morphological band-pass filtering is an image-processing method that, in principle, achieves exact segmentation of features of specific shapes and scales. It is carried out by the steps as follows.
Step 1. A morphological opening operation is employed to obtain the background image, , from the binarized test image, . The structuring element used in the opening operation is , which is similar to the measured feature of the shape but slightly larger in size. The formula iswhere represents the morphological opening operation; and represent the morphological erosion and expansion operations, respectively. In , the measured feature and the noise, whose size is smaller than the measured feature, are excluded.
Step 2. The morphological opening operation is employed to get the image, , from the binarized test image, . The structuring element used in the opening operation is , which is similar to the measured feature in shape, but slightly smaller in size. The formula isIn , the measured feature is preserved, but the noise with smaller size than the target feature is excluded.
Step 3. The results of morphology band-pass filtering, , is obtained via image differential operations, and . The formula is
This method is not widely used in engineering, because the opening operation destroys the original border and shape of the feature, resulting in algorithm failure when the feature is slightly larger.
By this method the test image originating from a computer vision-based testing system for the swing center of the flexible joint [19] used in rocket is segmented, as shown in Figure 1. Figure 1(a) is an original test image. Figure 1(b) is the binarization result of Figure 1(a) via the Otsu method. To facilitate the morphology operation, Figure 1(b) is color-reversed. The circles used in the testing system are elliptical after imaging. The long and short axes’ radiuses are all within an interval of 39 to 43 pixels. Figure 1(c) is the morphological opening result of Figure 1(b). The structuring element used in the operation is a circle with a radius of 44 pixels, which is similar to the measured feature in shape but slightly larger in size. Figure 1(d) is also the morphological opening result of Figure 1(b), but the structuring element used in the operation is a circle with a radius of 37 pixels, which is similar to the measured feature in shape but slightly smaller in size. Figure 1(e) is the result of the differential operation of Figures 1(d) and 1(c). Figure 1 indicates that the opening operation of the binary image via two large structuring elements at different scales will lead to a boundary distortion of the image foreground at different levels. Thus, the difference image, Figure 1(e), still has some noise, and the feature boundary has been changed significantly.
(a)
(b)
(c)
(d)
(e)
3. Proposed Algorithm
3.1. Basic Idea
To avoid serious distortion caused by opening operations using large structuring elements and to effectively segment large-scale features, the morphological reconstruction algorithm is introduced to improve morphological band-pass filtering.
First, the structuring element, , which is slightly larger in size and similar in shape, is used to erode the binarized test image, , obtaining mark image 1. The formula is
Mark image 1 is represented by , here. In , the feature and other foreground areas of smaller size are corroded to an empty set.
Afterwards, the structuring element, , which is slightly smaller in size and similar in shape, is used to erode the binarized test image, , to obtain mark image 2. The formula is
Mark image 2 is represented by , here. In, foreground areas of a smaller size than the feature are also corroded to an empty set. However, features and other foreground areas with a larger size still have some pixel residue.
Then, under the constraint of , the morphological reconstruction of and is carried out, respectively. The iterative formula iswhere the operator, , implies that the two images take the minimum value, point by point. and are, respectively, the th iteration result. is a morphological structuring element. To obtain a high computational speed and to make the boundary details of the reconstructed image coincide with , is usually a 3 × 3 rectangular structuring element:
The termination condition of (6) and (7) are (9) and (10), respectively:
That is, when the expansion result of or is unchanged in the constraint condition of taking the minimum value, point by point, with , the iteration is terminated.
Because the foreground regions, which are corroded into the empty set in the mark image, could not be recovered by morphological reconstruction, is the background image from which the feature, as well as its smaller noise in , is excluded. In , the feature is preserved, but smaller noise is excluded. The difference between and in the improved algorithm and and in the standard algorithm is that morphological reconstruction can ensure that the recovered foreground area, in principle, does not exceed the foreground area in the original binary image. Thus, the foreground area in and is highly consistent with the foreground area in the original binary image. However, and are obtained by the opening operations, in which the foreground area will be distorted. The larger the structuring element, the more significant the distortion.
Finally, the band-pass filtering result is obtained by the difference operation of and :
3.2. Large Structuring Elements of the Improved Algorithm
To apply the improved algorithm for engineering practice, it is necessary to find two large structuring elements, and , similar to the feature in shape, but slightly larger or smaller in size. This step can also be achieved via morphological operations.
First, it is necessary to obtain an ideal binarized feature image as the reference, in which all background with interference has been artificially removed, and the shape of the feature after imaging is almost undistorted. Thus, the erosion of the reference by a -square structuring element, whose origin is at the center and is an odd number greater than 1, is employed to obtain . Finally, the dilation of the reference by an -square structuring element, whose origin is at the center and is an odd number greater than 1, is employed to obtain . This process is shown in Figure 2, where and are both 3. By the properties of dilation and erosion operations, we can see that and meet the requirements of the improved algorithm for large structural elements.
After several studies, we found that the algorithm could segment the testing images accurately in a complex background with good timeliness when the intervals of and are all in the interval of three and 11 pixels.
3.3. Computational Cost Reduction
The theory of mathematical morphology shows that the time complexity of the dilation, , where is a binarized image of pixels and is a structuring element of pixels, is . The duration of the erosion operation increases rapidly with the size of the structuring element.
From Section 3.1, we can see that the once-proposed algorithm requires two erosion operations using large structuring elements, two morphological reconstruction operations, and one image differential operation. During algorithm operation, the time complexity of image difference operations is low. Thus, it is easy to control the duration of the morphological reconstruction, as there are many fast algorithms [20]. Therefore, the erosion operation using a large structuring element is the real-time bottleneck.
To meet the requirements of online processing, this paper adopts a large structuring element decomposition method to improve the computing speed of the algorithm. From the mathematical morphology theory, we can see that the erosion operation using large structuring element, , can be decomposed into a group of erosion operations using a small structuring element. That is, if thenThe expansion operation is irreversible, however, ifwhere the number of in (14) and (15) is equal. Thus, we approximate that is equal to in this engineering application. Therefore, we can use the iterative dilation method to decompose large structural elements. The iterative formula isThe termination condition is
If, when the iteration is terminated, equals , the large structuring element, , can be decomposed into where the number of is . To further improve the speed of operation, the structuring element, , can also be decomposed into a 3D row vector and a 3D column vector:where and , respectively, are
Per (12)–(19), the proposed algorithm for the structuring element decomposition formula iswhere the numbers of and are both . The optimal decomposition of the structuring element is a complex mathematical problem. The decomposition method of this paper is not the best, in theory, but it is easy to work in engineering application.
4. Experimental Results and Engineering Application
In the following section, the effectiveness and practicability of the algorithm applied to the segmentation and feature extraction of circular and laser spot in a complex environment are validated experimentally. In the experiment, image-processing algorithms run on a PC (3.2 GHz quad-core processor, 4G memory) via Visual Studio 2013.
4.1. Algorithm Validity Test
First, the validity of the proposed algorithm is tested with test images originating from a computer vision-based testing system for the swing center of the flexible joint used in rocket, mentioned in Section 2. One example of the proposed algorithm segmenting test image is shown in Figure 3. The original image and its binarization result via the Otsu method are shown in Figures 1(a) and 1(b). Figure 3(a) is mark image 1, which is the erosion result from using structuring element, , as described in Section 3.1. Figure 3(b) is mark image 2, which is the erosion result from using structuring element . The method to find and is described in Section 3.2, where the values of the parameters, and , are both seven pixels. As seen in Figures 3(a) and 3(b), the circle in has only a few pixels remaining, and the circle in has completely disappeared. This result is consistent with the expected algorithmic effect. Figures 3(c) and 3(d) are the morphological reconstruction result of and , respectively, under the constraint, . Because the circle in has completely disappeared, the corresponding reconstructed image of Figure 3(c) includes only the background information of the test image, whereas, in Figure 3(d), the circle features can be fully restored and the noise can be suppressed. Figure 3(e) is the difference image of Figures 3(c) and 3(d). All backgrounds and interferences are completely suppressed in Figure 3(e). Thus, the algorithm affects the exact segmentation of multiple circles in a complex background.
(a)
(b)
(c)
(d)
(e)
The proposed algorithm is further tested with test images from the 125 MN extruder moving parts center online monitoring system [21], shown in Figure 4. The resolution of the test image from this system is 320 × 240 pixels. The algorithm succeeds in laser spot accurate segmentation for the test set of 20 images, acquired under the interference condition of sawdust, dust, and stray light. The processing of one image is shown in Figure 5. This algorithm effectively segments the laser spot under strong interference conditions.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
After that, this algorithm is applied to segment the (left-most) coin in a specific size from the coin picture (423 × 354 pixels) extracted from [22], as shown in Figure 6. In this process, the binary image is obtained by applying the Otsu method to the red component of the original true color image. The result shows that the method could produce accurate segmentation of the circles of a specific size from differently sized circles.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
4.2. Algorithm Performance Test
Because the image segmentation accuracy of the industrial test image is not easy to directly evaluate, we indirectly evaluate the image segmentation accuracy of the algorithm via the laser spot center-positioning accuracy. The experimental device is shown in Figure 7. The device consists of a displacement platform, laser launcher, and laser-receiving device. The wavelength and the power of the laser launcher are 650 nm and 20 MW, respectively. The resolution of the imaging sensor in the laser-receiving device is 320 × 240 pixels. The lens focal length is 8 mm. Single pixel size is 0.342 × 0.342 mm.
In this experiment, the center of the spot for the first acquisition is set as the reference point. The displacement platform moves the laser-receiving device by 1 mm in both horizontal and vertical directions and collects the image once. Every image has artificially introduced disturbances, as shown in Figure 5(a). For algorithm comparison, we use four methods for central location calculation. Method 1 combines our algorithm with the curve-fitting method. Method 2 also combines our algorithm with the curve-fitting method. However, the large structuring element decomposition method is not employed for computational cost reduction. Method 3 is based on standard HT. Method 4 is based on RHT. The image binarization of methods 3 and 4 uses the Canny algorithm. The measurement error and time-consuming data are shown in Figures 8 and 9, respectively.
Figure 8 shows that methods 1 and 2 have the same measurement accuracy, which is significantly higher than that of methods 3 and 4. The root mean square error (RMSE) of methods 1 and 2 is 0.005 mm. The RMSE of methods 3 and 4 are 0.177 mm and 0.2 mm, respectively. The proposed method, combined with the curve-fitting method, has obviously higher displacement measurement accuracy. Results indirectly show that the proposed method has high image segmentation accuracy.
Figure 9 shows that the average time consumption of the four methods is 40.1 ms, 106.3 ms, 168.5 ms, and 107.9 ms, respectively. The computational cost reduction method can increase the calculating speed by 2.6 times. Our proposed method has advantages of high precision and fast calculating speed in a complex industrial environment. Thus, it can meet the requirements of industrial online monitoring.
4.3. Engineering Application
Presently, our algorithm combining the curve-fitting method has been successfully applied as an optional feature extraction model to the 125 MN extruder moving parts’ center online monitoring system in the Southwest Aluminum Group of China. Engineering application shows that this algorithm improves the real-time performance of the system, while ensuring high precision and high reliability in harsh industrial environments. Therefore, it has clear practical value.
5. Conclusion and Future Work
Because extant image segmentation and enhancement methods have their drawbacks in complex industrial environments, we introduced morphological reconstruction to improve the morphological band-pass filtering algorithm, which overcomes algorithm suitability limitations for small target segmentation. For achieving engineering application, we discussed the selection of large structuring elements in the algorithm and computational cost reduction. The experimental results on circle detection-based systems show that the proposed algorithm features good real-time performance and high accuracy in complex industrial environments, meeting the requirements of industrial online monitoring.
Our algorithm realized the accurate segmentation of features by measuring the foreground area of an image through the structuring elements of specific shapes and scales. Thus, it is theoretically applicable to more basic features. The practical value of this method of segmenting other shape features needs to be further studied and verified with application objects.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This research was supported by the National Natural Science Foundation of China (no. 51405154 and no. 51275169), the Hunan Provincial Natural Science Foundation of China (no. 2018JJ3167 and no. 2015JJ5009), and the National Key Research and Development Program of China (no. 2016YFF 0203400). Many thanks are due to the Southwest Aluminum Group of China for their assistance in the engineering testing of the algorithm.