Statistical Analysis of Nonlinear Processes Based on Penalty Factor

Zhang, Yingwei; Zhang, Chuanfang; Zhang, Wei

doi:https://doi.org/10.1155/2014/945948

Mathematical Problems in Engineering

On this page

Abstract Introduction Discussion Conclusion References Copyright Related Articles

Special Issue

New Developments in Sliding Mode Control and Its Applications 2014

View this Special Issue

Research Article | Open Access

Volume 2014 | Article ID 945948 | https://doi.org/10.1155/2014/945948

Statistical Analysis of Nonlinear Processes Based on Penalty Factor

Yingwei Zhang,¹Chuanfang Zhang,¹and Wei Zhang¹

Academic Editor: Ligang Wu

Received25 Jul 2014

Accepted17 Aug 2014

Published30 Sept 2014

Abstract

A new process monitoring approach is proposed for handling the nonlinear monitoring problem in the electrofused magnesia furnace (EFMF). Compared to conventional method, the contributions are as follows: (1) a new kernel principal component analysis is proposed based on loss function in the feature space; (2) the model of kernel principal component analysis based on forgetting factor is updated; (3) a new iterative kernel principal component analysis algorithm is proposed based on penalty factor.

1. Introduction

In consideration of ensuring the safety of the equipment and quality of product, the monitoring of the process performance has become an indispensable issue. In order to enforce the rationality and effectiveness of monitoring, in the last few decades, multivariate statistical process monitoring (MSPM) has been intensively researched. Particularly, principal component analysis (PCA) and partial least squares (PLS) which are widely applied in the industrial processes have been important approaches for monitoring of the process performance and some improved methods, such as kernel principal component analysis (KPCA) and kernel partial least squares (KPLS), have achieved great success in process monitoring and fault diagnosis [1–5].

As the scale of modern industrial processes is expanding and the complexity of process is increasing, how to ensure the safety of process operation and improve product quality are two issues need to be solved in industrial production enterprises [6, 7]. Process monitoring technology is an effective way to solve these two issues. Since the complexity and fluctuation of industrial processes, accurate process models are difficult to build and apply [8]. Therefore, application of traditional process monitoring methods based on qualitative or quantitative models is subject to certain limitation.

Because of the developments of intelligent instrumentations and computer technology in industrial process applications, a large number of high dimensional and strongly correlated process data is collected and stored [9–11]. It is difficult to remove redundancy and interference to extract useful information. It is an efficient monitoring technology to deal with the correlation of multivariate statistical process [12, 13].

In this paper, the following work focused on the process change caused by aging equipment, process drift, and sensor measurement errors in nonlinear industrial process [14–16].

In practical industrial process, outliers are contained in the collected data, while traditional kernel principal component analysis method is based on the assumption that there are no outliers in the sample data [17, 18]. Outliers still exist even after mapping them into feature space. Even if the sample data contains only a small amount of outliers, great negative effect will be applied on process model [19, 20]. Therefore, an advanced kernel principal component analysis method is proposed in this paper, which defines a loss function in feature space in the sense of minimum reconstruction error [21]. Then iteration with penalty will be carried to obtain the principal components, which can eliminate the adverse effects of outliers. Whenever a new sample is available, reconstruct it with the previous transfer matrix and calculate the reconstruct error [22–25]. If the new sample is an outlier, then update model with the reconstructed sample, otherwise update model with the original sample. Simulation results show that the advanced KPCA method can reduce the impact of outliers and improve the accuracy of the process monitoring model as well [26–28].

The rest of this paper is organized as follows. Kernel principal component analysis based on loss function in the feature space is proposed in Section 2. The model updating of kernel principal component analysis based on forgetting factor is proposed in Section 3. Improved kernel principal component analysis algorithm based on penalty factor is proposed in Section 4. Fault monitoring method is proposed in Section 5. The experiment results are given to show the effectiveness of the proposed method in Section 6. Finally, conclusions are summarized in Section 7.

2. Kernel Principal Component Analysis Based on Loss Function in the Feature Space

In practical industrial process, outliers are contained in the collected data, while traditional kernel principal component analysis method is based on the assumption that there are no outliers in the sample data [29–32]. The so-called outliers usually refer to the samples whose reconstruction error is much larger than average values and proportions are very small. Outliers still exist even after mapping them into feature space. Even if the sample data contains only a small amount of outliers, great negative effect will be applied on process model. Therefore, an improved kernel principal component analysis method is proposed in this paper, which defines a loss function in feature space in the sense of minimum reconstruction error.

2.1. Kernel Principal Component Analysis

In 1909, Mercer demonstrated the concept of positive definite kernel function and regeneration kernel Hilbert space in terms of the math and listed the necessary and sufficient condition of existence and determination of the positive definite kernel function, which is called “Mercer kernel permit conditions.” Kernel method not only has been widely used in differential geometry, differential equation, group theory, and many other mathematical disciplines and in signal processing, machine learning, the Gaussian process analysis, and many other applications but also had the very big breakthrough. Kernel methods of theoretical research and practical application attract more and more attention of scholars and experts. Scholkopf combined the kernel method with principal component analysis and formed the theory of kernel principal component analysis method.

Kernel principal component analysis as the extension of principal component analysis maintains the various mathematical and statistical properties of linear principal component analysis. KPCA uses the nonlinear mapping of data to a high dimensional eigenspace to achieve kernel matrix diagonalization and then carries on the principal component analysis. There is no need to calculate inner product of the sample data of nonlinear transformation and we can easily get the nonlinear principal component of the mapping data by the kernel function value between two data points.

2.2. Loss Function in the Feature Space

The sample number is . is mapped into high dimensional space , where . is supposed to have been centralized processing. is the transformation matrix, where . is reconstruction vector of ; then the reconstruction error of in the feature space is defined as follows:

In order to minimize the reconstruction error, here the loss function in feature space is defined as follows:

2.3. Kernel Principal Component Analysis Based on Loss Function

Formula (2) is expanded as follows: where is constant. Therefore , the loss function in the feature space, is the minimum, when is the maximum. This is equivalent to solving the following optimization problem:

By Lagrange multiplier method, it is obtained as follows:

Then

By eliminating and , it is obtained as follows:

It is defined as and then obtained as follows: where . The result of is the eigenvector of in the above formula.

Because satisfies the normalization condition , then it is obtained as follows:

In this way, the projection of in the th principal component is as follows:

3. The Model Updating of Kernel Principal Component Analysis Based on Forgetting Factor

When the running state of the process changes in multivariate statistical process monitoring, regardless of the system changing slowly or fast, mean and covariance matrix of the model will change. Therefore, when the system changes, it needs to update mean and covariance of the sample data set.

Firstly, the method of updating PCA model based on forgetting factor is introduced. Then KPCA model updating method based on forgetting factor will be got by kernel method. When new samples are collected in the process, mean and covariance will change. These changes depend on the change degree of model structure, namely, the size of the forgetting factor. For time-varying Gaussian process, therefore, mean and covariance of forgetting factor in estimate time can be used. Its formula is as follows: where , are two forgetting factors, and are mean vector and covariance matrix of . Mean vector and covariance matrix of the sample data can be in a more convenient form, namely, the weighted sum of mean and covariance matrix in moment and sample data in moment. Its formula is as follows: where is the new centralized sample of moment. As increases gradually, formula (12) can be further simplified as

It can be seen from formula (14) that if only considering the sample covariance matrix, then it is obtained as where is a diagonal matrix, whose diagonal elements are the same as diagonal elements of . The correlation coefficient matrix can also be estimated as follows:

As increases gradually, formula (16) can be further simplified as

As shown in formula (11), the updating of sample date set’s mean and covariance matrix needs determining two weighting coefficients, which are called the forgetting factors. If both forgetting factors are 1, its mean vector and covariance matrix will have the highest similarity degree with mean vector and covariance matrix calculated by all the sample data. If forgetting factors less than 1, as the process running, the weight of adding on the old data will be smaller and smaller, until being eliminated automatically, without discarding the old data by human. Then old data will gradually reduce the influence on process model and even disappear, which ensures that the model is adaptive to the time-varying system. When forgetting factors are closer to 1, there will be more number of sample data taking effect on the current process model.

So far, most of the model updating method is based on the constant value forgetting factor which is acquired by experience. However, the optimal value of forgetting factor depends on the degree of process change. Because process changes at different levels, the optimal value of forgetting factor will have a significant difference. When the process changes rapidly, the update rate of mode should be very large, namely, a few new data have the main influence on process model. When process changes slowly, update rate of the model should be smaller, namely, most of sample data has influence on process model. The basic process information will be in a long time to maintain its effectiveness. But the degree of process changes is changing with time in the actual industrial process, and then forgetting factor should be determined according to the actual situation of process changes. In order to deal with process which has nonconstant change degree, constant forgetting factor will not be used. Forgetting factor which is adjusted to different degree of process changes is being used. Here, Fortescue’s method is used to adjust the forgetting factor. The method adopts the model updating based on previous factor. It has two features: forgetting factor can take different values, which bring a degree of flexibility to the model. Forgetting factor value is decided by the change of mean and covariance matrix directly and is not dependent on and SPE statistic values. Apply the same concept to recursive principal component analysis method. Then the calculation method of the sample data set’s mean and covariance update is obtained as where and are, respectively, the forgetting factor’s maximum value and minimum value. and are parameters of the function. is the Euclidean norm of the difference between two consecutive mean vectors. is mean based on historical data . In the same way, forgetting factor which is used to update the covariance matrix can be calculated according to the formula as follows: where and are, respectively, the forgetting factor’s maximum value and minimum value. is the Euclidean norm of the difference between two consecutive correlation coefficient matrices. It can be seen that four parameters ( (or ), (or ), , and ) in formulae (18) and (19) need to be determined. The default values are , , , and .

This method is introduced to the model updating of KPCA, combining with exponential weighting KPCA method. Then the update KPCA method based on forgetting factor is obtained. Let kernel matrix be at moment. According to exponential weighting KPCA method, recursive update formula of kernel matrix at moment is as follows: where is the weighting factor; it can be calculated according to the formula as follows: where , , and is the Euclidean norm of the difference between two consecutive correlation coefficient matrices. The sensitivity of model is controlled by , whose default value is .

4. Improved Kernel Principal Component Analysis Algorithm Based on Penalty Factor

KPCA algorithm based on eigenvalue decomposition is a batch-mode algorithm, which needs to know all the sample points before modeling. It is not suitable for online monitoring or samples increased gradually. And KPCA is often based on the assumption that sample is not contaminated by outliers. There are outliers in the actual samples, in this paper, an iterative kernel principal component analysis method based on penalty factor is presented to solve the problem of outliers in the sample.

4.1. Iterative Kernel Principal Component Analysis Algorithm

For the loss function defined by formula (3), the stochastic gradient descent method is used to solve the optimization problem as follows:

Then iterative formula is as follows: where is the iteration step length, , and is convergence to the first nonlinear principal component.

Because the nonlinear principal components are orthogonal to each other, Schmidt orthogonal method is used to calculate the th principal component :

Steps of iterative KPCA algorithm can be summarized as follows.(1)Input: sample data set , maximum iterations , the initial iteration step , and principal component .(2)Calculate the kernel matrix , where . Then carry on the centralized processing , where (3)Calculate the th principal component .(4) and and return to step . output: .(5) Return to step and calculate next principal component.(6). Terminate the iteration and output .

4.2. Iterative Kernel Principal Component Analysis Algorithm Based on Penalty Factor

Although there are little outliers in sample data in KPCA algorithm, it also has great influence on KPCA model. The calculated principal components are towards the direction of outliers in order to reduce the overall square errors in the process of calculating principal components. In order to reduce the influence of the outliers on KPCA model, the penalty factor is added to the square error formula in the feature space.

Penalty factor is added to formula (3): where is the predefined threshold and . is defined as follows:

By the above formula, after adding penalty factor, points which exceeded predefined threshold are seen as outliers. After setting of outliers as , the influence on KPCA model is reduced. Noticing that is discrete, in order to use the proposed iterative KPCA to calculate principal components, continuous Sigmoid function is adopted to approximate discrete variable .

Minimum of error function in formula (25) is calculated, and iterative formula is obtained as follows: where is iteration step length, and is continuous Sigmoid function, which can adjust parameters according to the current input values and eliminate the influence of the outliers on KPCA model. So the smaller the threshold value is, the more the sample points will be treated as outliers.

Because the nonlinear principal components are orthogonal to each other, Schmidt orthogonal method is used to calculate the th principal component :

Steps of iterative KPCA algorithm based on penalty factor can be summarized as follows.(1)Input: sample data set , maximum iterations , the initial iteration step , and principal component .(2)Calculate the kernel matrix , where . Then carry on the centralized processing , where (3)Calculate the th principal component . If , formula (29) is used to calculate the first principal component . From the second iteration, sample reconstruction error is calculated in each time. Take points with ratio of as outliers, determine the threshold , and calculate penalty factor for iteration. Then the first principal component is obtained.(4)If , formula (30) is used to calculate the rest of principal components .(5) and and return to step . output: .(6) Return to step and calculate next principal component.(7). Terminate the iteration and output .

5. Fault Monitoring Method

This section provides fault monitoring method using the proposed iterative kernel principal component analysis algorithm based on penalty factor. It can be broadly divided into offline modeling phase and online monitoring phase.

5.1. Offline Modeling Phase

(1)KPCA model is established based on historical data, and the initial standardization of kernel matrix is obtained.(2)Set the ratio of outliers , determine the threshold , and calculate penalty factor and principal components .(3)Calculate and statistics and the corresponding control limits.

5.2. Online Monitoring Phase

(1)Calculate the mean , covariance matrix , , and at moment.(2)Collect new samples data of moment; calculate forgetting factors , , the mean , and covariance matrix .(3)Obtain kernel matrix of new samples and standardize it.(4)Use the method of iteration kernel principal component to update KPCA model and calculate and statistics and the corresponding control limits , .(5)Collect new sample data and return to step .

The flow chart of improved KPCA algorithm is shown in Figure 1.

6. Experiment and Discussion

With the development of technology of melting, electrofused magnesia furnace has already gotten extensive application in the industry. Electrofused magnesia furnace refining technology can enhance the quality and increase the production variety. The working conditions of the electrofused magnesia furnace are changed frequently and have complex characteristics such as strong nonlinearity and multiple modes. Electrofused magnesia furnace production process is used for fault diagnosis to verify the effectiveness of the proposed statistical analysis of nonlinear processes based on penalty factor. The improved KPCA method is used to monitor the normal and failure condition, respectively. Process fault is introduced from the 700th sample. It is caused by abnormal electrode actuators. Current of electrofused magnesia furnace plunges sharply. Temperatures become abnormal.

800 pieces of sample data in normal working condition is used to test the improved KPCA process monitoring method proposed in this paper. Then and statistics of improved KPCA method are obtained in normal working condition, as shown in Figure 2. It can be seen from Figure 2 that the changes of and statistics in improved KPCA method are reduced. This is because penalty factors are used to punish deviation larger samples in the iterative calculation process of principal components. The distance between sample points and original points of principal components is reduced. Therefore, and statistic fluctuation decrease. But only iterative KPCA is used to model without updating the control limits; fault alarms still exist in the process. If and control limits are updated at the same time, the statistics will not overrun the limits obviously. Compared with traditional methods, the proposed method has better accuracy and lower fault alarm rate. Simulation results verify the feasibility of this method to eliminate outliers.

(a)

(b)

In order to monitor the process of fault condition, faults are added to sample data in the normal working condition. Process faults are introduced from the 700th sample. The parameters start to drift and change faster at this time. Improved KPCA and conventional KPCA are used to monitor the process of fault condition. And process monitoring charts are shown in Figure 3. Figure 3 is statistics of and process monitoring using improved KPCA in fault condition. From Figure 3 you can see under the condition of process parameter drift, when faults do not occur in the process, the improved KPCA method can eliminate the influence of outliers and better describe the process of change. When faults occur in the process, the improved KPCA method can accurately and timely find them. Compared with the traditional method, the improved KPCA method has better accuracy and lower fault alarm rate.

(a)

(b)

7. Conclusion

In order to solve the problem of outliers in the sample data, an improved KPCA method is proposed in this paper. The method is based on loss function in feature space. And forgetting factor is introduced into recursive update of kernel matrix. Then penalty factor is added to calculate the process monitoring model. Compared with conventional KPCA method, improved KPCA method proposed in this paper does further research on eliminating outliers. Iterative KPCA method is more suitable for online monitoring of the process. Adding the penalty factor has good effect in eliminating outliers. In this paper, MATLAB software is used to do simulation experiments, and the simulation results verify the feasibility of the method. The improvement of KPCA method is more useful in the process of monitoring contained outliers.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

S. Bersimis, S. Psarakis, and J. Panaretos, “Multivariate statistical process control charts: an overview,” Quality and Reliability Engineering International, vol. 23, no. 5, pp. 517–543, 2007.
View at: Publisher Site | Google Scholar
T. Chen and J. Zhang, “On-line multivariate statistical monitoring of batch processes using Gaussian mixture model,” Computers & Chemical Engineering, vol. 34, no. 4, pp. 500–507, 2010.
View at: Publisher Site | Google Scholar
J. Ding, T. Chai, H. Wang, and X. Chen, “Knowledge-based global operation of mineral processing under uncertainty,” IEEE Transactions on Industrial Informatics, vol. 8, no. 4, pp. 849–859, 2012.
View at: Publisher Site | Google Scholar
J. Duarte, L. R. Lima, L. Oliveira, M. Mezaroba, L. Michels, and C. Rech, “Modeling and digital control of a single-stage step-up/down isolated PFC rectifier,” IEEE Transactions on Industrial Informatics, vol. 9, no. 2, pp. 1017–1028, 2013.
View at: Publisher Site | Google Scholar
Z. Ge, Z. Song, and F. Gao, “Incorporating setting information for maintenance-free quality modeling of batch processes,” AIChE Journal, vol. 59, no. 3, pp. 772–779, 2013.
View at: Publisher Site | Google Scholar
K. Helland, H. E. Bernstsen, O. S. Borgen, and H. Martens, “Recursive algorithm for partial least squares regression,” Chemometrics and Intelligent Laboratory Systems, vol. 14, no. 1–3, pp. 129–137, 1992.
View at: Publisher Site | Google Scholar
C.-C. Hsu and C.-T. Su, “An adaptive forecast-based chart for non-Gaussian processes monitoring: with application to equipment malfunctions detection in a thermal power plant,” IEEE Transactions on Control Systems Technology, vol. 19, no. 5, pp. 1245–1250, 2011.
View at: Publisher Site | Google Scholar
W. H. Woodall and D. C. Montgomery, “Research issues and ideas in statistical process control,” Journal of Quality Technology, vol. 31, no. 4, pp. 376–386, 1999.
View at: Google Scholar
M. Kano and Y. Nakagawa, “Data-based process monitoring, process control, and quality improvement: recent developments and applications in steel industry,” Computers and Chemical Engineering, vol. 32, no. 1-2, pp. 12–24, 2008.
View at: Publisher Site | Google Scholar
J. V. Kresta, J. F. MacGregor, and T. E. Marlin, “Multivariate statistical monitoring of process operating performance,” The Canadian Journal of Chemical Engineering, vol. 69, no. 1, pp. 35–47, 1991.
View at: Publisher Site | Google Scholar
J. F. MacGregor, C. Jaeckle, C. Kiparissides, and M. Koutoudi, “Process monitoring and diagnosis by multiblock PLS methods,” AIChE Journal, vol. 40, no. 5, pp. 826–838, 1994.
View at: Publisher Site | Google Scholar
E. C. Malthouse, A. C. Tamhane, and R. S. H. Mah, “Nonlinear partial least squares,” Computers and Chemical Engineering, vol. 21, no. 8, pp. 875–890, 1997.
View at: Publisher Site | Google Scholar
Y. Zhang, S. Li, and Z. Hu, “Improved multi-scale kernel principal component analysis and its application for fault detection,” Chemical. Engineering Research and Design, vol. 90, no. 9, pp. 1271–1280, 2012.
View at: Google Scholar
S. J. Qin, “Statistical process monitoring: basics and beyond,” Journal of Chemometrics, vol. 17, no. 8-9, pp. 480–502, 2003.
View at: Publisher Site | Google Scholar
S. J. Qin, S. Valle, and M. J. Piovoso, “On unifying multiblock analysis with application to decentralized process monitoring,” Journal of Chemometrics, vol. 15, no. 9, pp. 715–742, 2001.
View at: Publisher Site | Google Scholar
Y. Zhang and Z. Hu, “Multivariate process monitoring and analysis based on multi-scale KPLS,” Chemical Engineering Research and Design, vol. 89, no. 12, pp. 2667–2678, 2011.
View at: Google Scholar
C. M. Mastrangelo and D. C. Montgomery, “SPC with correlated observations for the chemical and process industries,” Quality and Reliability Engineering International, vol. 11, no. 2, pp. 79–89, 1995.
View at: Publisher Site | Google Scholar
Y.-S. Qi, P. Wang, and X.-J. Gao, “Fault detection and diagnosis of multiphase batch process based on kernel principal component analysis-principal component analysis,” Control Theory and Applications, vol. 29, no. 6, pp. 754–764, 2012.
View at: Google Scholar
S. J. Qin and Y. Zheng, “Quality-relevant and process-relevant fault monitoring with concurrent projection to latent structures,” AIChE Journal, vol. 59, no. 2, pp. 496–504, 2013.
View at: Publisher Site | Google Scholar
Y. Zhang and S. Li, “Modeling and monitoring between-mode transition of multimodes processes,” IEEE Transactions on Industrial Informatics, vol. 9, no. 4, pp. 2248–2255, 2013.
View at: Google Scholar
G. C. Runger and T. R. Willemain, “Model-based and model-free control of autocorrelated processes,” Journal of Quality Technology, vol. 14, no. 2, pp. 283–288, 1995.
View at: Google Scholar
D.-M. Tsai, S.-C. Wu, and W.-Y. Chiu, “Defect detection in solar modules using ICA basis images,” IEEE Transactions on Industrial Informatics, vol. 9, no. 1, pp. 122–131, 2013.
View at: Publisher Site | Google Scholar
V. Venkatasubramanian, R. Rengaswamy, S. N. Kavuri, and K. Yin, “A review of process fault detection and diagnosis part III: process history based methods,” Computers and Chemical Engineering, vol. 27, no. 3, pp. 327–346, 2003.
View at: Publisher Site | Google Scholar
V. Venkatasubramanian, R. Rengaswamy, K. Yin, and S. N. Kavuri, “A review of process fault detection and diagnosis part I: quantitative model-based methods,” Computers and Chemical Engineering, vol. 27, no. 3, pp. 293–311, 2003.
View at: Publisher Site | Google Scholar
S. Wold, “Cross-validatory estimation of the number of components in factor and principal components models,” Technometrics, vol. 20, no. 4, pp. 397–405, 1978.
View at: Google Scholar
Y. Zhang and Y. Teng, “Process data modeling using modified kernel partial least squares,” Chemical Engineering Science, vol. 65, no. 24, pp. 6353–6361, 2010.
View at: Google Scholar
Y. Zhang, H. Zhou, T. Chai, and S. J. Qin, “Decentralized fault diagnosis of large-scale processes using multiblock kernel partial least squares,” IEEE Transactions on Industrial Informatics, vol. 6, no. 1, pp. 3–10, 2010.
View at: Google Scholar
D. Zhou, G. Li, and S. J. Qin, “Total projection to latent structures for process monitoring,” AIChE Journal, vol. 56, no. 1, pp. 168–178, 2010.
View at: Google Scholar
S. Wold, “Nonlinear partial least squares modelling. II. Spline inner relation,” Chemometrics and Intelligent Laboratory Systems, vol. 14, no. 1–3, pp. 71–84, 1992.
View at: Publisher Site | Google Scholar
W. Zeng and M. Chow, “Modeling and optimizing the performance-security tradeoff on D-NCS using the co-evolutionary paradigm,” IEEE Transactions on Industrial Electronics, vol. 9, no. 1, pp. 394–402, 2013.
View at: Google Scholar
Y. Zhang, J. An, and C. Ma, “Fault detection of non-gaussian processes based on model migration,” IEEE Transactions on Control Systems Technology, vol. 21, no. 5, pp. 1517–1526, 2013.
View at: Google Scholar
E. E. Tarifa and N. J. Scenna, “Fault diagnosis, direct graphs, and fuzzy logic,” Computers and Chemical Engineering, vol. 21, no. 1, pp. 649–654, 1997.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2014 Yingwei Zhang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1181

Downloads

970

Citations