Abstract
Because the mine is damp and dark, it is not easy to detect the rigid tank channel’s structural failure directly. Therefore, we judged the tank channel’s surface condition by detecting the magnitude of the vibration displacement of the lifting container. In our study, we used a laser vision system to measure the structural vibration displacement. In order to accurately segment the laser spot information from the vibration image, we proposed an approach that links the relationship between the gray value of the area adjacent to the threshold point and the background’s gray value to the target in the image. We used MCE to evaluate the segmentation effect of threshold segmentation and verified the improved algorithm’s accuracy by detecting the pixel centroid of laser spots. Results show that the improved algorithm in our study has the best threshold segmentation effect, the error classification can be close to 0.0003, and the minimum deviation of the obtained vibration displacement is close to 0.1 pixels, which can realize the accurate extraction of the vibration signal of the vertical shaft tank. The novelty of this method lies in the accurate threshold segmentation and noise reduction processing of the laser speck vibration image under various interference environments in the operation of the mine hoisting system and the accurate acquisition of vibration signals. The research work provides a basis for the accurate evaluation of mechanical faults of automation technology.
1. Introduction
China’s coal resources are vibrant, and it is an essential part of the energy field [1]. The mine hoist system undertakes transportation tasks such as coal, equipment, and personnel, and its operation status will directly affect the safe and efficient production of the coal mine. When the lifting container is disturbed during operation, structural vibration will be generated. By analyzing its vibration characteristics, the health of the mine lifting system can be indirectly obtained. There are many existing vibration measurement methods, and the contact acceleration sensor method is generally used. However, such methods are prone to signal distortion, delay, and other consequences [2]. In contrast, the noncontact optical measurement technology of machine vision has received an increasing attention due to its advantages, such as high accuracy and intuitiveness [3]. The visual measurement method of laser spot vibration displacement refers to combining the laser and CCD camera, marking the laser spot on the vibration, using the CCD camera to collect the marking laser spot, and performing the corresponding image processing to obtain the vibration displacement of the measured position [4], as shown in Figure 1.
Image segmentation is the technique and process of dividing an image into several specific areas and proposing feature targets. The threshold segmentation method has attracted much attention for its simplicity and adaptability [5]. The more widely used threshold algorithm is the Otsu method based on the maximum variance between classes; it is an image binarization algorithm proposed by the Japanese scholar Otsu. However, the Otsu method has certain limitations. For some interfering images and grayscale histograms that do not have prominent bimodal characteristics, the segmentation accuracy cannot be guaranteed. Many scholars have improved Otsu’s method to adapt the algorithm to complex images in more situations. Ng has proposed an interclass variance method that emphasizes troughs, uses the probability of gray value as the threshold value as the weight, and adds it to the target formula of interclass variance, so that the optimal threshold of image segmentation tends to the gray level of the trough position value [6]. On this basis, Fan and Lei [7] proposed an interclass variance method emphasizing the trough neighborhood, considering the gray value distribution of the target and the background, and adding the distribution probability weights of all pixels in the neighborhood of the threshold point. Shen et al. [8] proposed a recursive method of multithreshold segmentation. Based on emphasizing the neighborhood of troughs, factors related to the relative relationship between channels and crests in the histogram were added, which effectively improved the segmentation results’ boundary details. In order to consider the spatial relationship between the central pixel and its neighborhood, Yu proposed a new region-based active contour model, which effectively segments the image by measuring the similarity of local patches [9]. In the second year, a new level-set method for medical image segmentation based on adaptive perturbation was proposed, which can effectively segment medical images with intensity inhomogeneity [10].
The above algorithms have improved the Otsu method to a certain extent and have also been proved to be useful for similarly labeled target segmentation. However, the segmentation effect of such improved algorithms on nonuniformly illuminated images and noisy images is limited. The coal mine’s underground environment is complex, and it is challenging to ensure that the image received by CCD is not affected by noise, dust, and intense light. Moreover, the underground mine environment is humid, and the laser is prone to produce abnormal transmission light during the transmission process, that is, stray light. Ghost imaging manifests stray light caused by reimaging a few reflected lights in the optical transmission system [11, 12]. In the visual system, the ghost images gathered on the image surface received by the CCD camera will increase the image’s noise. The center of mass of the laser spot may be erroneously detected.
When the hardware measures cannot eliminate the noise, the image preprocessing can be optimized [13]. This research analyzes the related improved algorithm’s problems in marking laser speckle segmentation under the interference of pulse noise, intense light, and ghost images. Considering the influence of different interference on image gray value distribution, an improved algorithm is proposed. Combined with the gray value pixel of the neighborhood of the threshold point, the difference between the gray value of the background and the target in the image histogram, the range size of the neighborhood and the ratio of the gray value of the threshold point to the background, and the gray value of the threshold point to the target area, the optimal threshold is selected to be closer to the ideal threshold of segmentation. Mathematical morphology algorithm has good robustness to noise, combined with mathematical morphology to process the segmented image, avoiding under- or oversegmentation generated during threshold segmentation, effectively solving the impact of environmental interference and improving segmentation target quality [14].
2. Research Basis
To collect the fault vibration image of the vertical shaft hoisting tank, the vertical shaft hoisting system shown in Figure 2 was built. This experiment can simulate the three failure modes of step bump, joint misalignment, and joint gap, by acting different tank faults and collecting and hoisting the vibration image of the container. The lifting speed is 0.18 m/s, and the impact force caused by the high-frequency vibration caused by the impact step defect of the tank channel causes the displacement of the laser spot centroid pixel. The CCD camera sensor is attached to the top center of the lifting container. A fluorescent screen is placed H above the camera lens, and the fluorescent screen is relatively fixed to the CCD camera. Two laser transmitters are symmetrically installed in the same plane above the mine tank passage, and the direction of emission is downward. In this experiment, the laser spot vibration image of the vertical shaft hoisting system test bench was collected.
The experimental hardware conditions are Core i5-3470 CPU 3.20 GHZ, and the programming environment uses MATLAB R2015a. As shown in Figure 3, our research’s images are collected by the DMK33G618 CCD industrial camera provided by the Imaging Source, with a resolution of 640 × 480 pixels, a maximum frame rate of 120fps, and a lens focal length of 16 mm. The two sets of laser emitters are ultrasmall red dots, emitting pure red lasers with a wavelength of 650 nm. The size of the red spot can be adjusted manually from 0.4 mm to several centimeters. It also adopts an anticorrosion DC plug cable, which is suitable for all-weather and all-environment use, as shown in Figure 4.
The laser vision vibration measurement system was built and completed, and the program design for the camera to collect vibration images was completed. Then, a series of algorithm processing is performed on the obtained vibration image, the laser spot in the vibration image is detected, the vibration displacement information of the centroid of the laser spot is extracted, and its pixel coordinates are obtained in real time. The basic idea is to do image preprocessing analysis and threshold segmentation on the vibration images collected by the camera and store the static centroid coordinate information of laser spots at the beginning. In the next frame image, the centroid coordinates of laser spots in real time are obtained, the initial centroid coordinates are calculated, and the relative vibration displacement information of laser spots is obtained; the process is shown in Figure 5.
3. The Traditional Otsu Threshold Segmentation Algorithm
The Otsu method is an automatic threshold method to maximize the variance between classes. The algorithm is derived from the gray histogram using the principle of least squares. It has the characteristics of fast processing speed and simple operation. It is a commonly used method. It is characterized by fast processing speed and simple operation and is a commonly used threshold selection method [15]. The basic idea is as follows:
A digital image F = [f (x, y)]M×N with a size of M N, f (x, y) represents the gray value at the pixel point (x, y), in which there are L-level gray values, namely f (x, y) H, H = {0, 1, 2, …, L − 1}. The gray value l (0 l L − 1) is set as the abscissa and the number of pixels with the gray value of l in the image h(l) as the ordinate and the gray histogram of the image is drawn. The pixels with the gray value of l appear in the image’s probability is as follows:
where l = 0, 1, …, L − 1. Also,
The threshold T is set to divide the pixels in the image corresponding to the grayscale histogram into two types: background region C0 and target region C1, where C0 is the total number of pixels in the gray value [0, T] and C1 is the gray value in [T + 1, L − 1] the total number of pixels ; the proportion of the pixel area are as follows:
The average values of C0 and C1 are as follows:
From (4), we can seewhere represents the mean value of background region C0, represents the mean value of target region C1, and represents the global mean value of the image.
Then, the between-class variance is defined as follows:
Given that T ∈ [0, L − 1], the T value when reaches the maximum value is the optimal threshold which is
To study the threshold segmentation effect of the traditional Otsu algorithm, a noninterference laser spot image was selected for verification.
As shown in Figure 6, it can be seen that the traditional Otsu method segmented the noninterference laser spot image. Although the target can be extracted, the segmented target part cannot be matched entirely with the original laser spot, leading to incorrect detection of the spot center. The black area is the background part less than the threshold gray value, and the white area is the target part greater than the threshold gray value. Because the traditional Otsu method does not entirely separate the target and the background from the interference-free laser spot image, it does not explore the image segmentation’s effect under interference.
(a)
(b)
(c)
4. Other Improved Algorithms
In recent years, domestic and foreign scholars have made many optimization improvements based on Otsu method to make Otsu method widely and effectively perform threshold segmentation. Ng [6] proposed the probability of occurrence when the gray value is a threshold in the gray histogram as a weighting factor. The optimal threshold of image segmentation tends to be the gray value of the trough position. The improved specific expression is as follows:
Fan et al. proposed a smooth histogram weighted Otsu method based on Ng [7]. This weighting includes the threshold point’s gray value and considers the distribution of all pixels in the neighborhood of the threshold point. Probability information makes weighting more effective and threshold positioning more accurate. The improved specific expression is as follows:
Among them,where represents the sum of the gray value pixels’ distribution probabilities in the threshold T neighborhood in the gray histogram with a range of (2k+1), strengthening the likelihood of the valley’s gray value in the histogram.
Based on the smooth histogram’s weighting, Shen [8] further considered the probability of the trough’s gray value in the histogram and the relative height of the gray value of the canal and the neighboring peaks. They used it as the weight item when selecting the threshold. The improved specific expression is as follows:
Among them,
where hL(T) and hR(T) are the probability of the nearest peaks’ gray value pixels in the histogram. Combining the threshold neighborhood value pixels and the relative height of the threshold and the neighboring wave peaks, the influence of the target distribution ratio is reduced, and the optimal solution of the formula tends to be the valley gray value. If the gray value T is the adjacent left and right peaks, T is used to replace this peak’s gray value.
To explore the improved Otsu algorithm’s threshold segmentation effect, different specifications of noninterference laser speckle image i and laser speckle image in the case of high-intensity light ii, pulse noise iii, and ghost image iv are selected. These threshold segmentation algorithms are processed separately [12].
As can be seen from Figures 7–10, Ng, Fan, and Shen algorithms are all successful in segmenting the laser spot image without interference. Since the intense light’s gray value is close to the target point’s gray value, the Ng algorithm uses the gray neighborhood as the weight to increase the proportion of the trough. Under the interference of the intense light, the final threshold is almost entirely in the background area. Shen algorithm considers the relationship between the valley’s gray value and its left and right peak gray values, making the threshold more towards the target. The final segmentation effect is improved compared with the Fan algorithm, but the segmentation fails. Ng algorithm only considers the gray value of the valley in the image; under the interference of intense light, the final segmentation effect is higher than the Fan and Shen algorithm, but the segmentation fails. Including the Otsu threshold and 70% and 80% of the pixel range, the laser spot cannot be wholly separated from the intense light background. Ghost imaging is also a kind of stray light. Ng, Fan, and Shen algorithms fail to segment, which cannot guarantee the target spot’s integrity. It can be seen from the figure that the Ng, Fan, and Shen algorithms do not affect impulse noise.
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
5. Improved Algorithms
The specific improvement plan is: from the image histogram, filter out the gray value lm at the background peak in the histogram and the gray value ln at the target peak. The traversal range of the threshold T is lm < T < ln. The gray value at the background peak is lm. The total pixel point at the gray value ln at the target wave peak is NA; the threshold value T in the histogram and the gray value T in the neighborhood are (2k+1) the total number of pixels Na of all gray values, occupying the background gray. The full pixel point ratio from the degree value to the gray target value is Pl(T); the distances from the threshold to the background and the gray target value are Xm and Xn, respectively. The gray value T in the threshold neighborhood accounts for the threshold to the background, and the ratio of the target gray value difference is ; the above parameters are introduced as weighting factors into the Otsu method, and the improved target formula for the between-class variance is as follows:
It can be seen from the improved interclass variance target formula that Pl(T) and PX(T) are used as weighting coefficients, and the weighting factors of reference [6] are combined. The background target gray value and pixel information are added to make the target gray. The interclass variance of the degree value is relatively increased. The optimal threshold range is further approached to the target gray value, which reduces the influence on the optimal threshold when the background proportion is large.
Mathematical morphology is composed of a set of morphological algebraic operators, which can solve image processing problems such as noise suppression, feature extraction, threshold segmentation, and shape recognition [16]. In 2019, Yan proposed a novel early fault detection strategy based on enhanced scale morphological cap product filtering (ESMHPF), which realized the effects of noise suppression and feature enhancement [17]. At the same time, a multiscale morphological analysis algorithm based on a feature selection framework is proposed to build multidomain features, highlight fault symptoms, suppress noise, and improve fault detection ability [18].
After the laser spot’s target segmentation is completed, a binary image containing the laser spot is obtained. However, there will still be a certain amount of noise, which still interferes with the subsequent detection. In this regard, the morphology is selected to filter impurity interference. Since the laser spots in the image are composed of several small laser spots, combined with the characteristics of environmental noise, the radius is defined as 2-pixel structural elements S, and S is used to perform open and close operations on the segmented image, which are defined as follows:where denotes the dilation operator, denotes the erosion operator, is the opening operator, and is the closing operator.
6. Discussion of Experimental Results
To test the segmentation effect of the improved threshold segmentation algorithm on the vertical shaft tank’s vibration image, the segmentation performance is evaluated by two evaluation indicators: the misclassification error (MCE) and the pixel centroid deviation. MCE represents the result of threshold segmentation of a single image, which indicates the proportion of the wrongly divided background pixel to the target area when the image’s target pixel is classified into the background area. The pixel centroid deviation value represents the result of a comprehensive evaluation of the machine vision vibration displacement detection in actual operation. It represents the deviation value of the laser spot pixel centroid’s displacement in the segmented image and the cage’s displacement running on the tank in the vertical shaft lifting system.
7. MCE Evaluation Threshold Segmentation Results
MCE is used to evaluate the effects of threshold segmentation [19]. MCE ∈ [0, 1] represents the probability that the image is misclassified after threshold segmentation, where 0 represents no error classification of image pixels, 1 means complete error classification of image pixels, and the greater the MCE value, the more misclassified pixels. MCE is defined as follows:
where Bo and Ao, respectively, are the background area and target area pixels in the standard segmentation state (no standard segmentation image, manually adjusted to the best segmentation effect as the standard segmentation image) and BT and AT, respectively, are the background area and target area pixels in the test segmentation state. The number of points, ║, represents the number of pixels in the area.
As shown in Table 1, the segmentation effect of a single image can be seen through the MCE value. No matter if it is without interference or other interference, the improved algorithm’s segmentation effect in this research is the best. In the visual inspection failure, the MCE value is not enough to bear the overall image segmentation effect’s burden. The most important thing is whether the obtained vibration displacement is accurate and useful. Therefore, it is necessary to calculate the position deviation of the pixel centroid after segmentation.
8. Algorithm Validity Test
To test the segmentation algorithm in this research, under the same experimental environment, the improved algorithm is presented in this research using no interference image I, impulse noise under the strong light interference image II, and ghost imaging under impulse noise interference III image threshold segmentation, to explore its segmentation effect. The test images and the corresponding threshold segmentation results are shown in Figures 11–13.
As can be seen from the segmentation results in Figures 11–13, the threshold value determined by the algorithm in this research can effectively distinguish the target from the background edge region [20]. Although some noise processing is left, the target point’s feature information in the test image can be retained under feature matching.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
Centroid detection is carried out on the above segmentation images, and the standard mean square error is used to detect the original centroid coordinates and the actual centroid coordinates [21], which is defined as follows:
where (x1, y1) is the actual centroid coordinate of the laser spots in the segmented image and is the original centroid coordinate, and the unit is pixel.
As shown in Table 2, it can be seen from the deviation value of the laser spot centroid after segmentation of each algorithm that the method in this research is slightly higher than other ways for interference-free images. However, for intense light interference images under impulse noise and ghost imaging interference images under impulse noise, this algorithm’s improved threshold is far better than the other three threshold segmentation algorithms. Therefore, according to the laser speckle characteristics, this research improves a threshold segmentation algorithm based on the Otsu method and combines the mathematical morphology to segment the laser speckle vibration image.
9. Conclusion
This research proposes a laser vision detection method for vertical shaft rigid tank road surface failure. Aiming at the obtained vibration spot image, an improved Otsu segmentation algorithm is proposed. Based on the relative height of the trough’s gray value and the nearest peak in the histogram, the background is introduced. The relative relationship between the target gray value and the pixel information is used as the weight, combined with mathematical morphology processing, effectively segmenting the spot’s shape and data under impulse noise, high-intensity light interference, and ghost imaging interference. Thus, the vibration displacement of the lifting vessel obtained is more accurate. Comparing the Ng algorithm, Fan algorithm, Shen algorithm, and the improved threshold algorithm in this research, the results show that the segmentation effect of the algorithm in this research is significantly enhanced compared with other algorithms, the error classification can be close to 0.0003, and the segmentation effect is the best. And the contrast experiment of the pixel centroid detection of the spot image and the deviation value was carried out. The minimum error value was close to 0.1 pixels, which verified the spot information’s accuracy and completeness after the algorithm segmentation.
This research mainly uses a laser vision system, a fusion of mathematical morphology, and an improved Otsu threshold segmentation algorithm to detect the vertical shaft lifting container’s vibration image. The follow-up will further discuss the intelligent fault diagnosis method to diagnose vertical shaft rigid tank channel fault accurately.
Data Availability
The video data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the Natural Science Research Foundation of Anhui Universities (Grant no. KJ2020A0281), the Natural Science Foundation of Anhui Province (Grant no. 2008085ME178), the State Key Laboratories (Grant no. SKLMRDPC20ZZ01), the Anhui University Top-Notch Talents Program (Grant no. gxbjZD202020063), the Collaborative Innovation Project of Anhui Universities (Grant no. GXXT-2019-048), and the Key Research and Development Program of Anhui Province (Grant no. 202104a07020005).