Abstract
A robust particle filter (PF) and its application to fault/defect detection of nonlinear system are investigated in this paper. First, an adaptive parametric model is exploited as the observation model for a nonlinear system. Second, by incorporating the parametric model, particle filter is employed to estimate more accurate hidden states for the nonlinear stochastic system. Third, by formulating the problem of defect detection within the hypothesis testing framework, the statistical properties of the proposed testing are established. Finally, experimental results demonstrate the effectiveness and robustness of the proposed detector on real defect detection and localization in images.
1. Introduction
Fault detection, as a subfield of control engineering, is to monitor a stochastic system and identify when a fault occurred and the information of fault such as the type and its location. Fault detection approaches play a fundamental role to improve manufacturing process and application process in a variety of industry contexts [1–7], for example, quality control in process monitoring [1–3], product manufacturing [4], medium restoration [5], and facilities maintenance [8]. Advantages of low costs, high automation, and high quality of defect detection techniques resulted in growing interests in recent years. Fault/defect detection for quality inspection includes the use of cameras, eddy current, ultrasonic, X-ray sensors and other sensors, which offer the measurements to be analyzed for extraction of the information about the fault. With the high demands for quality control using cameras in industry, visual inspection systems attract more attention in recent years. In a similar vein to fault detection of linear or nonlinear systems, defect detection in visual inspection systems can be taken as monitoring the variation of the measurements from cameras. In this paper, we devote ourselves to solving the visual defect detection problems from an unusual perspective of fault detection.
Currently, two rough categories of approaches are available for defect detection, that is, signal processing-based and model-based methods. The signal processing algorithms perform mathematical and statistical analysis tools on the measurements to extract faults [8–13]. The developed methods like Gabor filter and wavelet transform have been proven to be effective solutions to locating the defects with less prior information in nonlinear systems. In recent years, defect detectors on the basis of Gabor filter [10] and wavelet transform [11] are efficient to find defects for web fabrics with stable repetition of textures, since it is easier to find defects in frequency domain. Techniques of the signal processing also cover a number of computational intelligence approaches like fuzzy logic and neural network [13], which provide some effective solutions for fault detection in various industrial problems. However, the data-driven methods suffer from large computational load and storage space.
The second category is an evolving collection of methodologies aiming to exploit the models of the system in temporal and spatial space to decide the occurrence of fault/detection [3, 5–7, 14]. Taking advantages of temporal or spatial models with the prior information, the model-based method can achieve robustness even in a scene with heavy amount of noises. This paper focuses on techniques of the defect detection of this category.
Various model-based approaches for defect/fault detection have been proposed in the past decades [3, 5, 6, 15]. Since the system state completely represents the system’s internal (hidden) status and condition containing the fault information, the intrinsic problem in the model-based methodologies is the estimation of the system states from measurements of sensors. One of the most popular components of model-based algorithms is the Bayesian method [16], which yields the posterior distribution of the system state containing the hidden state such as the occurrence and the fault type. Bayesian filter and its variations have been effective solutions to visual inspection of surface defects of materials and fault detection in process control [3], for example, chemical process. For the linear Gaussian system, Kalman filter [15–17] has been exploited to detect surface defects in visual inspection systems. Particle filter, as an implementation algorithm of Bayesian filter for nonlinear and non-Gaussian systems, has been used for fault detection of chemical process [3, 18]. However, state estimation and fault detection still remain a challenge due to the absence of suitable models for the actual application context. The linear Gaussian models in [15] cannot meet the requirements of the defect detection and localization in the actual visual inspection systems, due to the complicated intensity distributions of background pixels.
In this paper, a new intelligent defect detection algorithm without prior knowledge based on PF is presented. We formulate the visual defect detection problem to estimate the hidden states using PF and decide the occurrence or locations of defects in 2D images by chi-square test. In our method, intensity of the 2D image (the same vein as the control system) along each row or column line is assumed as a time series or a random process . To detect faults/defects in the image (the system), reasonable state and measurement models are proposed firstly for particle filter. Then, with the proposed models, particle filters along rows and columns are implemented to estimate states and the measurement innovations (residuals). Finally, abrupt changes in the measurement innovations are used to locate defects in the inspection images by chi-square test. Tests on the real database demonstrate the effectiveness of the proposed algorithm.
2. Problem Formulation of Defect Detection
The defect detection in a visual inspection system is to identify the occurrence of the defect and its location from the measurements of camera sensors, that is, the digital image of the inspected object. A measurement in a digital image is the gray level or intensity of the pixel. Then the pixel intensity along each row or column line in an image of pixels is assumed as a time series or a random process or , where is one-dimensional coordinate of the relevant pixel on the scan line (i.e., the time step of the random process).
When the inspected target is free from defects, the intensity of pixels along each scan line takes on small variations. Therefore, the gray level along a scan line in a defect-free image can be defined as a white Gaussian state series or . Due to subtle intensity changes for defect-free areas, the state sequence can be modeled by the following discrete time linear Gaussian dynamical model; that is, where is the state in the time series, is the state transition model, and is the zero-mean white Gaussian noise sequences with variance . For defect inspection problem in this paper, the dynamical mode uses the random walk model with . Each state in a Gaussian series can also be written aswhere denotes a Gaussian distribution with expectation and variance .
However, due to the influence of defects, uneven illumination, and the geometrical structure of the inspected object, the actual measurements usually follow nonlinear model, which can be formulated by a transition function approximating the Markov transition relation between the state estimate of the defect-free and the actual measurement of a pixel (defect-free or defective). Therefore, the measurement model for an arbitrary pixel can be defined as where denotes the transition function from the state estimate to the measurement and is zero-mean white Gaussian noise sequences with variance .
In terms of the established models, Bayesian filter can be exploited to yield the state estimate of for each pixel along a scan line of an image by Bayes recursion below [17]:where is the transition density defined by (1) or (2) and is the likelihood function defined by (3). The posterior density includes all information of the state at time and the state estimate of the th pixel in a scan line can be obtained by maximum a posteriori (MAP) criterion.
Once the state estimate at time step (i.e., intensity of the th pixel along the scan line) is obtained, the difference between the predicted measurement and the actual measurement is defined as residual , which can be calculated byHere, can be derived by (1) and (3) with the formwhere is the predicted state using (1). It can be seen from (6) and (7) that a residual reveals the difference between the actual measurement of a pixel and the defect-free pixel, which can be used to decide the occurrence of a fault/defect. When the measurement is a Gaussian variable, the statistic is chi-square distributed. Then a chi-square test [17] will declare detection of a defect ifwhere is the level of confidence, indicates the degrees of freedom, and is the detection threshold.
3. Particle Filter Based on Adaptive Parametric Model
3.1. Adaptive Parametric Measurement Model
As explained in Section 2, it is important to warrant that the defect information is revealed by residual of the state and the measurement. To attain this purpose, the state model about the defect-free object based on random walk model is presented in (1) and (2). Furthermore, it is also necessary to design an accurate measurement model which can closely approximate the real measurement contaminated by noise, defects, and other disturbances. As shown in Figure 1, the measurements along a scan line in a complex visual inspection system follow multimodel distribution.
(a)
(b)
In order to obtain an accurate model, the Gaussian mixture model is exploited to approximate the multimodel distribution of measurements in the image of the inspected object. Assume that at time (i.e., the th pixel in a scan line), the probability density of the measurement can be approximated by a Gaussian mixture model (GMM) aswhere is the weight of the th Gaussian component in the mixture with , is the mean, is the variance, and is the Gaussian component number with 3~5. Then, the measurement likelihood function can be designed as
All the parameters of Gaussian items in (10) can be automatically updated by the model (9) at time using -means approximation [19] in Algorithm 1. The number of Gaussian distributions is manually set by making a tradeoff among the complexity of the pixel intensity model, the available memory, and the computational power. On the one hand, due to more ideal illumination and regular geometrical structure of the inspection condition, subtle changes of illumination of the desk base surface and only repetitive changes of scene elements may occur in the real inspected images (see Figure 1(a)). Thus, it is reasonable to simply model the values of a particular pixel along a scan line as a mixture of a fixed number of Gaussian distributions. On the other hand, less Gaussian distributions can alleviate the computation load. Considering the above elements, the value of should be as small as possible, on the premise that the number of distributions is able to represent the value of each pixel along each scan line of the inspected image. For example, in this paper, is set to be 3 in the application of desk base inspection.
| ||||||||||||||||||||||
In addition, , , and can be computed by the -means approximation in Algorithm 1. It should be noticed that the weight of a Gaussian distribution is updated according towhere is the learning rate and is 1 for the model matched by (i.e., ) and 0 for the remaining models otherwise. If none of the distributions matches , the least probable distribution is replaced with a distribution with as its mean, a high variance, and a low weight.
Once parameters at time are obtained, the likelihood function (10) can be updated and be used to implement PF or Bayes recursion instead of . Due to the better ability of fitting the complex distribution of pixel values, the proposed adaptive measurement model can help to represent the system more correctly.
3.2. Particle Filter with the Adaptive Model
As analyzed in Section 2, state is required to be estimated for defect detection. Since the proposed measurement model in Section 3.1 is Gaussian and nonlinear, Bayesian filter in (4) and (5) can be solved by particle filter [18]. Particle filter is based on Monte Carlo approximation, which is able to effectively deal with nonlinear and non-Gaussian problems [18]. In terms of the proposed state model in (1) and the measurement likelihood in (10), particle filter with adaptive model is summarized below.
Step 1 (initializing). Draw particles from prior density of states . Assign weights with particles for filtering.
Step 2 (important sampling). Firstly, the particles of in (4) are propagated by the state model , which can be obtained using (1) or (2) with the form
Secondly, given a new observation , each sampled particle from (12) is weighted by the proposed adaptive likelihood in (10) with the form
Finally, the weights are normalized by
Step 3 (resampling). Resample new particles from the particles in Step 1 according to roulette method, where the probability of choosing a particle is proportional to its weight. After resampling, the resampled particles are with equal weights, which can approximate the posterior distribution in (5) at time .
Step 4 (state extraction). The state estimate can be extracted by using either the minimum mean square error criterion:or the maximum a posteriori (MAP) criterion:
4. Defect Detection and Localization Based on PF
When the state estimates or of pixels along each row or column scan line are generated by the proposed PF in Section 3, the state estimates can be employed to establish the statistic for detecting and locating defects. To yield the statistic for chi-square test, the residual for each pixel should be computed firstly using (6). Under the Gaussian assumption, since the residual sequences and are Gaussian-distributed, the normalized residual squared statistic (NRSS) is approximately chi-square distributed with the formwhere denotes the covariance of . In terms of the principle of a check of the goodness of fit, is employed as the statistic and the fault/defect can be located along the scan lines by the chi-square test: where denotes the level of confidence, is the degrees of freedom with , and the detection threshold is . Here, can be referred to in the chi-square distribution table in terms of and . If the statistic for a pixel satisfies (18), an alarm indicating a defect point is given.
When the chi-square tests along all row lines are completed, the potential defect points are stored in the detection mask , which is an matrix with value 1 for an edge point and with value 0 for a defect-free point. Similarly, the mask along columns can be obtained. Considering different sensitivity of both tests along row and column directions, the final mask (see Figure 3(b)) can be calculated bywhere denotes detection result of the th row th pixel in the image and “” denotes logical OR operator. Based on all of the above, the proposed defect detector using PF with adaptive model is described in Figure 2. In addition, in order to locate accurately the occurrence of a defect with lower false alarms, the cumulative sum of the statistic is calculated to distinguish the defect from the image (see Figure 3(a)).
(a)
(b)
5. Experimental Results
5.1. Experimental Setup
The VACE protocol [20] is introduced here to evaluate the detection performance of the proposed algorithm, which uses two scores, namely, multiple object detection accuracy (MODA) and multiple object detection precision (MODP). The MODP and MODA measures are directly proportional to the area overlap between the system output and the ground truth. Higher scores of MODP and MODA indicate better performance in defect detection precision and accuracy, respectively. Denote and as the th ground truth object and its associated detected object (defect) in the th frame, respectively. MODP and MODA are defined aswhere quantifies the spatial overlap ratio between the ground truth and the detected object in a sequence of frames, is the number of mapped objects over the entire sequence, and refers to the number of mapped objects in the th frame. Considerwhere is the number of missed detects, is the total number of false positives, and are the cost functions for and , and is the number of ground truth objects in the th frame. In our experiments, the ground truth detects are annotated manually according to annotation guidelines for VACE [15]. The best possible match between ground truth and detected objects in a global sense can be solved by Hungarian algorithm.
5.2. Experimental Results and Discussion
5.2.1. Defect Detection Experiment
The proposed detector is tested by the defect image database, which contains multiple kinds of defects on the surface of the desk base of hard disks.
A sample result using the proposed defect detector is shown in Figure 3. Figure 3(a) plots the accumulative sum of NSSRs for row and column scan lines and marks the defect location with red circles, whose centers are calculated by abrupt changes in the response curve of NSSRs. Figure 3(b) shows another defect location result of the same inspected part by a mask image in (19) directly using NSSR of each pixel. Figure 3 indicates that both methods on the NSSRs and their accumulative sum are effective to locate defects in an image. In the following experiments, each of the location methods is chosen arbitrarily.
Figure 4 shows the detection results of various types of defects, for example, corrosion, hard blister, pin holes, and crack. It is shown that the proposed detector is able to accurately and automatically identify and locate defective areas from complex background with much less false alarms. As well known, uneven illumination (see Figures 4(a) and 4(b)) and geometrical structure (see Figure 4(a)) lead to scattered discontinuities in the defect-free areas, which make defect detection more difficult. To alleviate it, a more accurate adaptive measurement model is proposed for PF, which can inhibit the influence of discontinuities in the defect-free pixels.
(a)
(b)
(c)
(d)
5.2.2. Comparison Experiment
To evaluate the performance of the proposed detector further, we compare the proposed detector (DET_PF) with the detector in [15] (DET_KAL) and Otsu thresholding (DET_THR). Figure 5 shows the comparison result of DET_PF and DET_KAL. Since DET_KAL employs a single Gaussian model as the measurement model, it cannot accurately approximate the measurements of multimodel distribution. It can be seen from Figure 5(b) that DET_KAL is not able to locate defects in a complicated background (i.e., complicated intensity distribution of pixels in defect-free areas). Figure 5 indicates that the proposed detector based on PF with a robust model outperforms DET_KAL in fault detection for nonlinear system.
(a)
(b)
To quantify the performance difference of the three methods, we compute their performance scores according to evaluation method presented in Section 5.1. Figure 6 shows the accuracy and the precision scores (i.e., MODA and MODP scores) of the three defect detectors applied to the test database. It can be observed that the proposed detector achieves higher performance scores than common thresholding DET_THR method and DET_KAL.
5.2.3. Sensitivity Analysis of Parameters
The parameters involved in the proposed detector, such as Gaussian model noise sequences and in (1), (2), and (3) and the decision threshold in (17), can be preset manually according to the statistical characteristics of the gray level or the intensity of the inspected image. As mentioned in Section 2 of this paper, and are zero-mean white Gaussian sequences and are proportional to their standard deviations and .
Figure 7 illustrates the accuracy and precision transitions of defect detection performance when the parameters , , and of the detector are changed. When the value of one parameter with scale at a time are changed and the other parameter values are fixed, then the average values of the corresponding MODP and MODA measures are obtained and plotted in Figure 7. As shown in Figures 7(a) and 7(b), the detector is sensitive to and . Both and represent the model errors between the chosen system models (i.e., for dynamical models and for the measurement model) and the truth. Smaller and larger generally leads to divergence of the particle filter and a degraded detection precision, namely, decreasing values of MODA and MODP, respectively. Similarly, choices of depend on the measurement model error, which can be determined practically in terms of image quality, and a smaller would be chosen for an image of higher quality. The theoretical value of the decision threshold can be computed according to [17]. As shown in Figure 7(c), a small variation of the detection performance scores is related to theoretical value of the decision threshold . It can be seen that, in a real application of the proposed detector, quantities of , , and can be manually heuristically chosen from several tests.
(a)
(b)
(c)
5.2.4. Computational Complexity Analysis and Comparison
The computational complexity of the proposed method depends on two main steps: the -means approximation based adaptive measurement model learning and particle filter. Indeed, at time , the -means approximation step requires calculation of cost:where is the number of Gaussian items and is the number of pixels in the inspected image.
The particle filter requireswhere denotes the state dimension in the PF and denotes the number of particles. In the application to the desk base of hard disk, is 1 and is 200. Thus, in the experiment, the computational load is .
Furthermore, in our experiments, the proposed defect detection algorithm was carried out by an Intel 2.2 GHz CPU PC. Detection was done at an average rate of 20 frames per second (i.e., 50 ms per image) for images with size of without code optimization. In comparison with the similar method (MR) in [15], MR is able to process the a test image in about 60 ms. MR suffers more intensively computational load, which may be induced by the matrix inversion operation in Kalman filter with constant velocity state model and multidimensional vector of the state. From the above analysis, with code optimization, the proposed method will be a promising solution to defect detection in real time.
6. Conclusions
This paper presents a novel methodology for automatic fault/defect detection based on PF for nonlinear systems. The defect/fault detecting framework in a stochastic dynamical system based on PF and chi-square test is proposed first. To alleviate the scattered discontinuities from the uneven illumination and geometrical structure of the inspected objects, an adaptive measurement model is designed and estimation method of the hidden states for nonlinear system based on PF is presented in detail. Third, the statistical properties of the hidden states are employed to establish an effective statistic for defect/fault defection by chi-square test. Finally, the improved PF with adaptive parametric model and the fault detection framework are applied to identify the defects in visual inspection system. Experiments demonstrate that this proposed methodology can detect most defects efficiently with high quality in nonlinear system, which can be exploited to deal with fault detection problems in other industry fields.
Conflict of Interests
The authors declare no conflict of interests.
Acknowledgments
This paper is jointly supported by the National Natural Science Foundation of China (61305016), Fundamental Research Funds for the Central Universities (Grant no. JUSRP51316B), and Fundamental Research Funds for the Central Universities (Grant no. JUSRP1059).