Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2013, Article ID 987345, 6 pages
Research Article

Fault Diagnosis of Complex Industrial Process Using KICA and Sparse SVM

1School of Electrical and Automation Engineering, Nanjing Normal University, Nanjing, Jiangsu 210042, China
2College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 210016, China

Received 11 January 2013; Accepted 11 March 2013

Academic Editor: Engang Tian

Copyright © 2013 Jie Xu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


New approaches are proposed for complex industrial process monitoring and fault diagnosis based on kernel independent component analysis (KICA) and sparse support vector machine (SVM). The KICA method is a two-phase algorithm: whitened kernel principal component analysis (KPCA). The data are firstly mapped into high-dimensional feature subspace. Then, the ICA algorithm seeks the projection directions in the KPCA whitened space. Performance monitoring is implemented through constructing the statistical index and control limit in the feature space. If the statistical indexes exceed the predefined control limit, a fault may have occurred. Then, the nonlinear score vectors are calculated and fed into the sparse SVM to identify the faults. The proposed method is applied to the simulation of Tennessee Eastman (TE) chemical process. The simulation results show that the proposed method can identify various types of faults accurately and rapidly.

1. Introduction

The demand for safe operation in the industry has propelled research into fault detection and diagnosis methods. Based on principal component analysis (PCA), independent component analysis (ICA), partial least squares (PLSs), and canonical correlation analysis (CVA), the statistical method can represent high-dimensional process data in a reduced dimension, which contains most of the variance of the original data. These methods have been used in various applications [14].

ICA is a statistical approach that has the potential ability for blind source separation (BSS) without the prior information about the mixtures under the source signals that are statistically independent [5]. However, ICA fails to separate the nonlinearly mixed source due to its intrinsic linearity. One new nonlinear ICA technique for tackling the nonlinear problem has been in development, which is called KICA [6]. KICA can efficiently compute ICs in high-dimensional feature spaces using the kernel matrix . However, KICA algorithm is difficult to diagnose faults since the mapping function is unknown.

Sparse SVM is a learning strategy based on the Newton method and conjugate gradient method. It is proposed in this paper to diagnose faults, which is able to accelerate not only the model selection process but also the training speed.

This paper is organized as follows. In Section 2, the kernel ICA algorithm is introduced. In Section 3, sparse SVM is proposed. The experimental results of this methods proposed in this paper are also presented in Section 4. Finally, our work of this paper is summarized in the last section.

2. KICA Algorithm

Let be the training samples for kernel PCA learning. By the nonlinear mapping , the measure inputs are extended into the hyperdimensional feature space. The mapping of is simply noted as . The sample covariance in the feature space is given by

Let , then can be expressed by . Let us form the Gram matrix . The matrix is an matrix, and its elements are as follows:

We can center the Gram matrix by

Normalized using the following formula:

Calculate the eigenvectors of corresponding to largest positive eigenvalues . Then, the largest positive eigenvalues of are , and the associated orthonormal eigenvectors can be expressed by Denote , and its matrix form is .

The mapped data in feature space can be whitened by the following transformation: Specifically where , .

Centering and whitening are carried out in the KPCA feature space. We designate the data after preprocessing as , , as well as are obtained from the fast ICA algorithm in the feature space. Data dimension can be reduced by selecting a few rows of based on the assumption that the rows with the largest sun of squares coefficient have the largest effect on the variation of . The detailed algorithm for fast ICA can be easily found in the literatures [7]. The selected row of constitutes a matrix (dominant part of ), and the remaining rows of constitute a matrix .

For the new sampling data , independent vector that corresponds to the principal component space and residual space is , , respectively [8].

We defined three process monitoring statistics [8]: And their confidence limits can be obtained by kernel density estimation. For more details about kernel density estimation, it can be found in the literature [9].

Specific implementation steps are shown in Figure 1.

Figure 1: Fault detection algorithm flowchart.

3. Sparse SVM

Consider the second-order norm soft margin constraints; thus obtain the following optimization problem:

Consider (9) of the dual problem:

The optimization problem can be written as an unconstrained one [10, 11]:

Let us now consider nonlinear SVMs with a kernel function and an associated the reproducing kernel Hilbert space . The optimization problem (11) becomes

We have made a change of variable by introducing the regularization parameter . We have also dropped the offset for the sake of simplicity.

Suppose now that the loss function is differentiable with respect to its second argument. Using the reproducing property, (12) becomes

For a given value of the vector , we say that a point is a support vector if and if the loss on this point is nonzero. Let us reorder the training points such that the first points are support vectors. Finally, let be the diagonal matrix with the first entries being 1 and the others 0. The gradient of (13) with respect to is And the Hessian is .

Each Newton step consists of the following update: , where the step size is found by one-dimensional search method.

Combining (14) and as , we find that after the update:

Using the fact that the lower left block is 0, the update (15) turns out to be .

Therefore, the algorithm of SVM based on the modified Newton optimization is performed as shown in Algorithm 1.

Algorithm 1: SVM based on the modified Newton optimization.

We now consider the optimization parameter is [. The augmented Hessian is where 1 should be understood as a vector of all 1.

We still use the former method of choice to choose the right “basis functions” and realize the purpose of building sparse SVM. Assume that the choice of “basis functions” collection is ; then the selection of the next “basis function” is equivalent to ensuring that is constant and optimizing the following questions:

Through the introduction of the Cholesky decomposition and the corresponding rank 1 update strategy, based on unconstrained optimization SpSVM with the following steps to achieve the following:(1)update matrix using the Cholesky decomposition;(2)optimize using Newton’s method (Algorithm 1). If changes, update matrix using rank 1 update strategy;(3)if the “basis function” collection is equal to the size of preset value, end of the program, otherwise, turn to step 4;(4)select the next “basis function” , transferred to step 1.

4. Experimental Results

The proposed method is applied to the simulation of the Tennessee Eastman chemical process to achieve fault detection and fault diagnosis. SVM training involved in the QP and the QCQP problem can be by optimizing the package MOSEK method based on the interior point, and the SVM training data are preprocessing.

The process has 12 manipulated variables, 22 continuous process measurements, and 19 composition measurements sampled less frequently. The process contains the Gaussian noise. TE process includes 20 predefined failure modes, representing a step, random changes, slow drift, and other fault type, its process flow diagram and detailed process, see the text [12]. We use the control strategy in paper [13] to conduct closed-loop control. Simulation data for a matrix are obtained as follows: 52 represent the number of process variables, and 960 represent the sampling points of each variable. The simulation is running 48 hours. The failure was in the introduction at the first 8 hours.

In order to verify the effectiveness of the KICA monitoring algorithms, KPCA and KICA were applied to detect fault 5. The monitoring charts of KPCA for fault 5 are shown in Figure 2. The figures indicate that and SPE statistics can also detect the happening of faults, but there will be false positives and false negatives in some sampling points. The monitoring charts of KICA for fault 5 are shown in Figure 3. In contrast to KPCA, it can be seen that the , , and SPE statistics all respond strongly to the faults.

Figure 2: KPCA monitoring for fault 5.
Figure 3: KICA monitoring for fault 5.

If the statistics exceed the control limits, it indicates a failure occurrence. We select 52 variables for the condition attributes and select 15 groups for the decision attribute, respectively, on behalf of 15 different types of failures. The training set is composed of 600 data, 40 samples data of each category. Test set is composed of 1200 data, 80 samples data of each category. We use 1-a.-r-mix strategy to achieve the multiclass classification SVM. The factor is then set to 1000. The size of the alternative set in SpSVM is 20. Specific diagnostic results are shown in Table 1.

Table 1: Results of fault identification.

5. Conclusions

In this paper, KICA and sparse SVM are proposed for the complex industrial process monitoring. KICA can efficiently compute ICs in high-dimensional feature spaces by means of integral operations. The method using sparse SVM can reduce the number of support vectors without reducing the classification accuracy. The application results show the effectiveness of proposed method.


This work was supported in part by the National Natural Science Foundation of China under Grant no. 60234010, the National Natural Science Youth Foundation under Grant no. 61203092, and Jiangsu Province Higher Education Natural Science Basic Research surface project under Grant no. 11KJB510007.


  1. J. E. Jackson and G. S. Mudholkar, “Control Procedures for residuals associated with principal component analysis,” Technometrics, vol. 21, no. 3, pp. 341–349, 1979. View at Google Scholar · View at Scopus
  2. M. Kano, S. Tanaka, S. Hasebe, I. Hashimoto, and H. Ohno, “Monitoring independent components for fault detection,” AIChE Journal, vol. 49, no. 4, pp. 969–976, 2003. View at Publisher · View at Google Scholar · View at Scopus
  3. S. J. Zhao, J. Zhang, and Y. M. Xu, “Performance monitoring of processes with multiple operating modes through multiple PLS models,” Journal of Process Control, vol. 16, no. 7, pp. 763–772, 2006. View at Publisher · View at Google Scholar · View at Scopus
  4. N. Mehranbod, M. Soroush, and C. Panjapornpon, “A method of sensor fault detection and identification,” Journal of Process Control, vol. 15, no. 3, pp. 321–339, 2005. View at Publisher · View at Google Scholar · View at Scopus
  5. P. Comon, “Independent component analysis, A new concept?” Signal Processing, vol. 36, no. 3, pp. 287–314, 1994. View at Google Scholar · View at Scopus
  6. J. Yang, X. Gao, D. Zhang, and J. Y. Yang, “Kernel ICA: an alternative formulation and its application to face recognition,” Pattern Recognition, vol. 38, no. 10, pp. 1784–1787, 2005. View at Publisher · View at Google Scholar · View at Scopus
  7. A. Hyvärinen, “Survey on independent component analysis,” Neural Computing Surveys, vol. 2, pp. 94–128, 1999. View at Google Scholar
  8. J. M. Lee, C. Yoo, and I. B. Lee, “Statistical process monitoring with independent component analysis,” Journal of Process Control, vol. 14, no. 5, pp. 467–485, 2004. View at Publisher · View at Google Scholar · View at Scopus
  9. M. P. Wand and M. C. Jones, Kernel Smoothing, Chapman and Hall, London, UK, 1995. View at Zentralblatt MATH · View at MathSciNet
  10. S. S. Keerthi, O. Chapelle, and D. DeCoste, “Building support vector machines with reduced classifier complexity,” Journal of Machine Learning Research, vol. 7, pp. 1493–1515, 2006. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  11. O. Chapelle, “Training a support vector machine in the primal,” Neural Computation, vol. 19, no. 5, pp. 1155–1178, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  12. L. H. Chiang, E. L. Russell, and R. D. Braatz, Fault Detection and Diagnosis in Industrial Systems, Spring, London, UK, 2001.
  13. J. H. Chen and C. M. Liao, “Dynamic process fault monitoring based on neural network and PCA,” Journal of Process Control, vol. 12, no. 2, pp. 277–289, 2002. View at Publisher · View at Google Scholar · View at Scopus