Fault Diagnosis of Complex Industrial Process Using KICA and Sparse SVM

Abstract

New approaches are proposed for complex industrial process monitoring and fault diagnosis based on kernel independent component analysis (KICA) and sparse support vector machine (SVM). The KICA method is a two-phase algorithm: whitened kernel principal component analysis (KPCA). The data are firstly mapped into high-dimensional feature subspace. Then, the ICA algorithm seeks the projection directions in the KPCA whitened space. Performance monitoring is implemented through constructing the statistical index and control limit in the feature space. If the statistical indexes exceed the predefined control limit, a fault may have occurred. Then, the nonlinear score vectors are calculated and fed into the sparse SVM to identify the faults. The proposed method is applied to the simulation of Tennessee Eastman (TE) chemical process. The simulation results show that the proposed method can identify various types of faults accurately and rapidly.

1. Introduction

The demand for safe operation in the industry has propelled research into fault detection and diagnosis methods. Based on principal component analysis (PCA), independent component analysis (ICA), partial least squares (PLSs), and canonical correlation analysis (CVA), the statistical method can represent high-dimensional process data in a reduced dimension, which contains most of the variance of the original data. These methods have been used in various applications [1–4].

ICA is a statistical approach that has the potential ability for blind source separation (BSS) without the prior information about the mixtures under the source signals that are statistically independent [5]. However, ICA fails to separate the nonlinearly mixed source due to its intrinsic linearity. One new nonlinear ICA technique for tackling the nonlinear problem has been in development, which is called KICA [6]. KICA can efficiently compute ICs in high-dimensional feature spaces using the kernel matrix 𝐾. However, KICA algorithm is difficult to diagnose faults since the mapping function is unknown.

Sparse SVM is a learning strategy based on the Newton method and conjugate gradient method. It is proposed in this paper to diagnose faults, which is able to accelerate not only the model selection process but also the training speed.

This paper is organized as follows. In Section 2, the kernel ICA algorithm is introduced. In Section 3, sparse SVM is proposed. The experimental results of this methods proposed in this paper are also presented in Section 4. Finally, our work of this paper is summarized in the last section.

2. KICA Algorithm

Let 𝑥₁,𝑥₂,…𝑥_𝑚∈𝑅^𝑚 be the 𝑛 training samples for kernel PCA learning. By the nonlinear mapping Φ∶𝑥∈𝑅^𝑚→𝑧∈𝑅^𝑛, the measure inputs are extended into the hyperdimensional feature space. The mapping of 𝑥_𝑖 is simply noted as Φ(𝑥_𝑖). The sample covariance in the feature space is given by 𝑆^𝐹=1𝑁^𝑁_𝑖=1Φ𝑥_𝑖Φ𝑥_𝑖^𝑇. (1)

Let 𝑄=[Φ(𝑥₁),…,Φ(𝑥_𝑛)], then 𝑆^𝐹 can be expressed by 𝑆^𝐹=(1/𝑁)𝑄𝑄^𝑇. Let us form the Gram matrix 𝐾=𝑄𝑄^𝑇. The matrix is an 𝑁×𝑁 matrix, and its elements are as follows: [𝐾]_𝑖𝑗=𝐾_𝑖𝑗=Φ𝑥_𝑖⋅Φ𝑥_𝑗=𝐾𝑥_𝑖,𝑥_𝑗. (2)

We can center the Gram matrix 𝐾 by 𝐾^∼=𝐾−𝐸_𝑁𝐾−𝐾𝐸_𝑁+𝐸_𝑁𝐾𝐸_𝑁,𝐸_𝑁=1𝑁1⋯1⋮⋮1⋯1∈𝑅^𝑁×𝑁. (3)

Normalized 𝐾^∼ using the following formula: 𝐾=^∼𝐾trace𝐾^∼./𝑁 (4)

Calculate the eigenvectors 𝛼₁≥𝛼₂≥⋯≥𝛼_𝑑 of 𝐾 corresponding to 𝑑 largest positive eigenvalues 𝜆₁≥𝜆₂≥⋯≥𝜆_𝑑. Then, the 𝑑 largest positive eigenvalues of 𝑆^𝐹 are 𝜆₁/𝑁,𝜆₂/𝑁,…,𝜆_𝑑/𝑁, and the associated orthonormal eigenvectors 𝑣₁,𝑣₂,…,𝑣_𝑑 can be expressed by 𝑣_𝑖=1𝜆_𝑖𝑄𝛼_𝑖(𝑖=1,2,…,𝑑). (5) Denote 𝐻=[𝛼₁,𝛼₂,…,𝛼_𝑑], Λ=diag(𝜆₁,𝜆₂,…,𝜆_𝑑) and its matrix form is 𝑉=𝑄𝐻Λ^−1/2.

The mapped data in feature space can be whitened by the following transformation: 𝑦=𝐺^𝑇Φ(𝑥). (6) Specifically 𝑦=𝐺^𝑇Φ(𝑥)=√𝑁Λ⁻¹𝐻^𝑇𝑄^𝑇Φ(𝑥)=√𝑁Λ⁻¹𝐻^𝑇𝑄^𝑇Φ𝑥₁,…,Φ𝑥_𝑁^𝑇Φ(𝑥)=√𝑁Λ⁻¹𝐻𝐾^𝑇∼𝑥₁,𝑥,…,𝐾^∼𝑥_𝑁,𝑥^𝑇=√𝑁Λ⁻¹𝐻𝐾^𝑇∼, (7) where 𝐻=[𝛼₁,𝛼₂,…,𝛼_𝑑], Λ=[𝜆₁,𝜆₂,…,𝜆_𝑑].

Centering and whitening are carried out in the KPCA feature space. We designate the data after preprocessing as 𝑋_normal, 𝑊, as well as 𝑆_normal are obtained from the fast ICA algorithm in the feature space. Data dimension can be reduced by selecting a few rows of 𝑊 based on the assumption that the rows with the largest sun of squares coefficient have the largest effect on the variation of 𝑆. The detailed algorithm for fast ICA can be easily found in the literatures [7]. The selected 𝑎 row of 𝑊 constitutes a matrix 𝑊_𝑑 (dominant part of 𝑊), and the remaining rows of 𝑊 constitute a matrix 𝑊_𝑒.

For the new sampling data 𝑋_new(𝑘), independent vector that corresponds to the principal component space and residual space is 𝑆_new𝑑(𝑘)=𝑊_𝑑𝑋_new(𝑘), 𝑆_new𝑒(𝑘)=𝑊_𝑒𝑋_new(𝑘), respectively [8].

We defined three process monitoring statistics [8]: 𝑈²(𝑘)=𝑆_new𝑑(𝑘)^𝑇𝑆_new𝑑(𝑘),SPE(𝑘)=𝑒(𝑘)^𝑇𝑒(𝑘)=(𝑥(𝑘)−𝑥(𝑘))^𝑇(𝑥(𝑘)−𝑥(𝑘,𝑈))_2𝑒(𝑘)=𝑆_new𝑒(𝑘)^𝑇𝑆_new𝑒(𝑘). (8) And their confidence limits can be obtained by kernel density estimation. For more details about kernel density estimation, it can be found in the literature [9].

Specific implementation steps are shown in Figure 1.

Figure 1 

Fault detection algorithm flowchart.

3. Sparse SVM

Consider the second-order norm soft margin constraints; thus obtain the following optimization problem: min_𝑤,𝜉⟨𝑤,𝑤⟩+𝐶^𝑛_𝑖=1𝜉²_𝑖s.t.𝑦_𝑖𝑤,Φ𝑥_𝑖+𝑏≥1−𝜉_𝑖,𝜉_𝑖≥0. (9)

Consider (9) of the dual problem: max_𝛼,𝛼∗−^𝑛_𝑖,𝑗=1𝑦_𝑖𝑦_𝑗𝛼_𝑖𝛼_𝑗𝑘𝑥_𝑖,𝑥_𝑗+𝛿_𝑖,𝑗𝐶+2^𝑛_𝑖=1𝛼_𝑖s.t.^𝑛_𝑖=1𝑦_𝑖𝛼_𝑖=0,𝛼_𝑖≥0,𝑖=1,2,…,𝑛. (10)

The optimization problem can be written as an unconstrained one [10, 11]: min_𝑤𝑤^𝑇𝑤+𝐶^𝐿_𝑖=1𝐿𝑦_𝑖,𝑤^𝑇Φ𝑥_𝑖+𝑏. (11)

Let us now consider nonlinear SVMs with a kernel function 𝑘 and an associated the reproducing kernel Hilbert space 𝐻. The optimization problem (11) becomes min_𝑓∈𝐻𝜆𝑓²_𝐻+^𝑛_𝑖=1𝐿𝑦_𝑖,𝑓𝑥_𝑖. (12)

We have made a change of variable by introducing the regularization parameter 𝜆=1/𝐶. We have also dropped the offset 𝑏 for the sake of simplicity.

Suppose now that the loss function 𝐿 is differentiable with respect to its second argument. Using the reproducing property, (12) becomes min_𝛽𝜆𝛽^𝑇𝐾𝛽+^𝑛_𝑖=1𝐿𝑦_𝑖,𝐾^𝑇_𝑖𝛽. (13)

For a given value of the vector 𝛽, we say that a point 𝑥_𝑖 is a support vector if 𝑦_𝑖𝑓(𝑥_𝑖)<1 and if the loss on this point is nonzero. Let us reorder the training points such that the first 𝑛_sv points are support vectors. Finally, let 𝐼_sv be the 𝑛×𝑛 diagonal matrix with the first 𝑛_sv entries being 1 and the others 0. The gradient of (13) with respect to 𝛽 is ∇=2𝜆𝐾𝛽+2^𝑛sv_𝑖=1𝐾_𝑖𝑦_𝑖𝑦_𝑖𝐾^𝑇_𝑖𝛽−1=2𝜆𝛽+𝐾𝐼^sv𝐾𝛽−𝑦. (14) And the Hessian is 𝐻=2(𝜆𝐾+𝐾𝐼^sv𝐾).

Each Newton step consists of the following update: 𝛽←𝛽−𝛾𝐻⁻¹∇, where the step size is found by one-dimensional search method.

Combining (14) and 𝐻 as ∇=𝐻𝛽−2𝐾𝐼^sv𝑦, we find that after the update: 𝛽=𝜆𝐾+𝐾𝐼^sv𝐾⁻¹⋅𝐾𝐼^sv𝑦=𝜆𝐼_𝑛+𝐼^sv𝐾⁻¹𝐼^sv𝑦. (15)

Using the fact that the lower left block 𝜆𝐼_𝑛+𝐼^sv𝐾 is 0, the update (15) turns out to be 𝛽_sv=(𝜆𝐼_𝑛sv+𝐾_sv)⁻¹𝑦_sv.

Therefore, the algorithm of SVM based on the modified Newton optimization is performed as shown in Algorithm 1.

Function: 𝛽=PrimalSVMENM(𝐾,𝑦,𝜆) sv←{1,…,𝑛}; 𝛽_old=0 Repeat 𝛽=0; 𝛽sv=(𝜆𝐼𝑛sv+𝐾sv,sv)−1𝑦sv; step=𝛽−𝛽_old; 𝛾=1;  While Ω(𝛽+𝛾⋅step)>Ω(𝛽)+𝜇𝛾⋅(∇𝑇⋅step)   𝛾←𝜌⋅𝛾;  End  𝛽←𝛽+𝛾⋅step;  𝛽_old←𝛽;    sv←𝑖,𝑖 satisfy 𝑦𝑖(𝐾𝑇𝑖𝛽)<1; Until sv has not changed.   Where 𝜇∈(0,0.5), 𝜌∈(0,1).

Algorithm 1 

SVM based on the modified Newton optimization.

We now consider the optimization parameter is [𝑏,𝛽^𝑇]. The augmented Hessian is 1𝐻=𝑆^𝑇𝐼^sv11^𝑇𝐾𝐼^sv𝐾𝐼^sv1𝜆𝐾+𝐾𝐼^sv𝐾, (16) where 1 should be understood as a vector of all 1.

We still use the former method of choice to choose the right “basis functions” and realize the purpose of building sparse SVM. Assume that the choice of “basis functions” collection is 𝐽; then the selection of the next “basis function” 𝑗 is equivalent to ensuring that 𝛽_𝐽 is constant and optimizing the following questions: min𝜆𝛽^𝑇_𝐽𝛽_𝐽𝐾_𝐽𝐽𝐾_𝐽𝑗𝐾_𝑗𝐽𝐾_𝑗𝑗𝛽_𝐽𝛽_𝑗+_𝑖∈sv1−𝑦_𝑖𝐾_𝑖𝐽𝛽_𝐽+𝐾_𝑖𝑗𝛽_𝑗². (17)

Through the introduction of the Cholesky decomposition and the corresponding rank 1 update strategy, based on unconstrained optimization SpSVM with the following steps to achieve the following:(1)update matrix 𝜆𝐼_𝑛sv+𝐾_sv using the Cholesky decomposition;(2)optimize 𝛽_𝐽 using Newton’s method (Algorithm 1). If sv changes, update matrix 𝜆𝐼_𝑛sv+𝐾_sv using rank 1 update strategy;(3)if the “basis function” collection 𝐽 is equal to the size of preset value, end of the program, otherwise, turn to step 4;(4)select the next “basis function” 𝑗∶𝐽←𝐽+𝑗, transferred to step 1.

4. Experimental Results

The proposed method is applied to the simulation of the Tennessee Eastman chemical process to achieve fault detection and fault diagnosis. SVM training involved in the QP and the QCQP problem can be by optimizing the package MOSEK method based on the interior point, and the SVM training data are preprocessing.

The process has 12 manipulated variables, 22 continuous process measurements, and 19 composition measurements sampled less frequently. The process contains the Gaussian noise. TE process includes 20 predefined failure modes, representing a step, random changes, slow drift, and other fault type, its process flow diagram and detailed process, see the text [12]. We use the control strategy in paper [13] to conduct closed-loop control. Simulation data for a 960×52 matrix are obtained as follows: 52 represent the number of process variables, and 960 represent the sampling points of each variable. The simulation is running 48 hours. The failure was in the introduction at the first 8 hours.

In order to verify the effectiveness of the KICA monitoring algorithms, KPCA and KICA were applied to detect fault 5. The monitoring charts of KPCA for fault 5 are shown in Figure 2. The figures indicate that 𝑇² and SPE statistics can also detect the happening of faults, but there will be false positives and false negatives in some sampling points. The monitoring charts of KICA for fault 5 are shown in Figure 3. In contrast to KPCA, it can be seen that the 𝑈², 𝑈²_𝑒, and SPE statistics all respond strongly to the faults.

(a)

(b)

Figure 2 

KPCA monitoring for fault 5.

(a)

(b)

(c)

Figure 3 

KICA monitoring for fault 5.

If the statistics exceed the control limits, it indicates a failure occurrence. We select 52 variables for the condition attributes and select 15 groups for the decision attribute, respectively, on behalf of 15 different types of failures. The training set is composed of 600 data, 40 samples data of each category. Test set is composed of 1200 data, 80 samples data of each category. We use 1-a.-r-mix strategy to achieve the multiclass classification SVM. The factor 𝐶 is then set to 1000. The size of the alternative set 𝜅 in SpSVM is 20. Specific diagnostic results are shown in Table 1.

Table 1 

Results of fault identification.

5. Conclusions

In this paper, KICA and sparse SVM are proposed for the complex industrial process monitoring. KICA can efficiently compute ICs in high-dimensional feature spaces by means of integral operations. The method using sparse SVM can reduce the number of support vectors without reducing the classification accuracy. The application results show the effectiveness of proposed method.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant no. 60234010, the National Natural Science Youth Foundation under Grant no. 61203092, and Jiangsu Province Higher Education Natural Science Basic Research surface project under Grant no. 11KJB510007.