Research Article  Open Access
MultiInnovation Stochastic Gradient Identification Algorithm for Hammerstein Controlled Autoregressive Autoregressive Systems Based on the Key Term Separation Principle and on the Model Decomposition
Abstract
An input nonlinear system is decomposed into two subsystems, one including the parameters of the system model and the other including the parameters of the noise model, and a multiinnovation stochastic gradient algorithm is presented for Hammerstein controlled autoregressive autoregressive (HCARAR) systems based on the key term separation principle and on the model decomposition, in order to improve the convergence speed of the stochastic gradient algorithm. The key term separation principle can simplify the identification model of the input nonlinear system, and the decomposition technique can enhance computational efficiencies of identification algorithms. The simulation results show that the proposed algorithm is effective for estimating the parameters of INCARAR systems.
1. Introduction
There exist many nonlinear systems in process control [1–3]. A nonlinear system can be modeled by input nonlinear systems [4] and output nonlinear systems [5], inputoutput nonlinear systems [6], feedback nonlinear systems [7], and so on. Input nonlinear systems, which are called Hammerstein systems [8], include input nonlinear equation error type systems and input nonlinear output error type systems. Recently, many identification algorithms have been developed for input nonlinear systems, such as the iterative methods [9–11], the separable least squares methods [12, 13], the blind methods [14], the subspace methods [15], and the overparameterization methods [16, 17]. Some methods require paying much extra computation.
The stochastic gradient (SG) algorithm is widely applied to parameter estimation. For example, Wang and Ding presented an extended SG identification algorithm for HammersteinWiener ARMAX systems [18], but it is well known that the SG algorithm has slower convergence rates. In order to improve the convergence rate of the SG algorithm, Xiao et al. presented a multiinnovation stochastic gradient parameter estimation algorithm for input nonlinear controlled autoregressive (INCAR) models using the overparameterization method [19]; Chen et al. proposed a modified stochastic gradient algorithm by introducing a convergence index in order to improve the convergence rate of the parameter estimation [20]; Han and Ding developed a multiinnovation stochastic gradient algorithm for multiinput singleoutput systems [21]; Liu et al. studied the performance of the stochastic gradient algorithm for multivariable systems [22].
The decomposition identification techniques include matrix decomposition and model decomposition. Hu and Ding presented a least squares based iterative identification algorithm for controlled moving average systems using the matrix decomposition [23]; Ding derived an iterative least squares algorithm to estimate the parameters of output error systems, and the matrix decomposition can enhance computational efficiencies [24]. Ding also divided a Hammerstein nonlinear system into two subsystems based on the model decomposition and presented a hierarchical multiinnovation stochastic gradient algorithm for Hammerstein nonlinear systems [25].
This paper discusses identification problems of input nonlinear controlled autoregressive autoregressive (INCARAR) systems or Hammerstein controlled autoregressive autoregressive (HCARAR) systems, which is one kind of input nonlinear equation error type systems. The basic idea is using the key term separation principle [26] and the decomposition technique [24] to derive a multiinnovation stochastic gradient identification algorithm, which is different from the work in [19, 21, 25].
The rest of this paper is organized as follows. Section 2 gives the identification model for the INCARAR systems. Section 3 introduces the SG algorithm for the INCARAR system. Section 4 deduces a multiinnovation SG algorithm for INCARAR system using the decomposition technique. Section 5 provides a numerical example to show the effectiveness of the proposed algorithm. Finally, Section 6 offers some concluding remarks.
2. The System Identification Model
The paper focuses on the parameter estimation of a Hammerstein nonlinear controlled autoregressive autoregressive (HCARAR) system, that is, an input nonlinear controlled autoregressive autoregressive (INCARAR) system, which consists of a nonlinear block and a linear dynamic subsystem. It is worth noting that Xiao and Yue discussed a data filtering based recursive least squares algorithm for HCARAR systems [27] and a multiinnovation stochastic gradient parameter estimation algorithm for input nonlinear controlled autoregressive (INCAR) systems.
An INCARAR system shown as Figure 1 is expressed as [27] where and are the system input and output, is the output of the nonlinear part, and is an uncorrelated stochastic noise with zero mean. Here, is expressed as [28] where and is the parameters vector of the nonlinear part.
In (1), , , and are the polynomials, of known orders , , and , in the unit backward shift operator [], defined by Assume , , and for . , , and are the parameters to be estimated from measured input–output data .
Define the parameter vectors: and the information vectors: Equation (2) can be written as Using the key term separation principle [26], (1) can be written as This is the identification model of the INCARAR system.
3. The Stochastic Gradient Algorithm
According to [1] and based on the identification model in (11), we can obtain the stochastic gradient (SG) algorithm: where represents the estimate of at time ; for example, is the estimate of at time .
4. The MultiInnovation Stochastic Gradient Algorithm
This section deduces the multiinnovation stochastic gradient identification algorithm for the INCARAR system using the decomposition technique [1].
Define two intermediate variables, From (10), we have These two subsystems include the parameter vectors and , respectively. contains the parameters od the system model and contains the parameters od the noise model.
Define the stacked information matrices and the stacked output vectors: According to the multiinnovation identification theory [29–41], we expand the scalar innovations: to the innovation vectors, Define two criterion functions, The gradients of and with respect to and , respectively, are Minimizing and using the negative gradient search, we can obtain the multiinnovation stochastic gradient algorithm (MISG) for the INCARAR system: The initial values can be taken to be , , , and .
5. Numerical Examples
Consider the following INCARAR system: In this example, the input is taken as a persistent excitation signal sequence with zero mean and unit variance and as a white noise sequence with zero mean and variance . Applying the SG algorithm and the MISG algorithm to estimate the parameters of this INCARAR system, the parameter estimates and their estimation errors are shown in Tables 1 and 2 with the data length and the estimation error versus being shown in Figures 2 and 3.


From Tables 1 and 2 and Figures 2 and 3, we can draw the conclusions. The parameters estimation errors become smaller with the data length increasing. The estimation errors given by the MISG algorithm are much smaller than that of the SG algorithm. The convergence speed of the multiinnovation SG algorithm is faster than those of the SG algorithm. These indicate that the MISG algorithm has better performance than the SG algorithm.
6. Conclusions
The gradient and least squares algorithms are two different kinds of important identification methods. It is well known that the gradient algorithm has poor convergence rates. This paper studies the multiinnovation SG identification methods for INCARAR systems. The numerical examples show that the proposed MISG algorithm can estimate effectively the parameters of input nonlinear systems and indicate that increasing the innovation length can improve parameter estimation accuracy of the multiinnovation identification algorithm because the algorithm uses more information in each recursion for a large innovation length. The proposed method can be applied to nonlinear output error systems. Although the algorithm is presented for the INCARAR systems, the basic idea can be extended to other linear or nonlinear systems with colored noises [42–61].
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (nos. 61273194, 61203111), the University Graduate Scientific Research Innovation Program of Jiangsu Province (CXLX13738), the Fundamental Research Funds for the Central Universities (JUDCF13035), the PAPD of Jiangsu Higher Education Institutions.
References
 F. Ding, System Identification: New Theory and Methods, Science Press, Beijing, China, 2013.
 Y. Shi and H. Fang, “Kalman filterbased identification for systems with randomly missing measurements in a network environment,” International Journal of Control, vol. 83, no. 3, pp. 538–551, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 B. Yu, Y. Shi, and H. Huang, “l2 and linfnity filtering for multirate systems using lifted models,” Circuits, Systems, and Signal Processing, vol. 27, no. 5, pp. 699–711, 2008. View at: Publisher Site  Google Scholar  MathSciNet
 D. Wang, F. Ding, and Y. Chu, “Data filtering based recursive least squares algorithm for Hammerstein systems using the keyterm separation principle,” Information Sciences, vol. 222, pp. 203–212, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 D. Q. Wang and F. Ding, “Least squares based and gradient based iterative identification for Wiener nonlinear systems,” Signal Processing, vol. 91, no. 5, pp. 1182–1189, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 D. Q. Wang and F. Ding, “Hierarchical least squares estimation algorithm for HammersteinWiener systems,” IEEE Signal Processing Letters, vol. 19, no. 12, pp. 825–828, 2012. View at: Publisher Site  Google Scholar
 P. P. Hu and F. Ding, “Multistage least squares based iterative estimation for feedback nonlinear systems with moving average noises using the hierarchical identification principle,” Nonlinear Dynamics, vol. 73, no. 12, pp. 583–592, 2013. View at: Publisher Site  Google Scholar
 F. Ding, X. P. Liu, and G. Liu, “Identification methods for Hammerstein nonlinear systems,” Digital Signal Processing, vol. 21, no. 2, pp. 215–238, 2011. View at: Publisher Site  Google Scholar
 M. Dehghan and M. Hajarian, “An iterative method for solving the generalized coupled Sylvester matrix equations over generalized bisymmetric matrices,” Applied Mathematical Modelling, vol. 34, no. 3, pp. 639–654, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. Dehghan and M. Hajarian, “Two algorithms for finding the Hermitian reflexive and skewHermitian solutions of Sylvester matrix equations,” Applied Mathematics Letters, vol. 24, no. 4, pp. 444–449, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. M. Zhang and F. Ding, “On the Kronecker products and their applications,” Journal of Applied Mathematics, vol. 2013, Article ID 296185, 8 pages, 2013. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 E.W. Bai, “Identification of linear systems with hard input nonlinearities of known structure,” Automatica, vol. 38, no. 5, pp. 853–860, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, X. Liu, and J. Chu, “Gradientbased and leastsquaresbased iterative algorithms for Hammerstein systems using the hierarchical identification principle,” IET Control Theory & Applications, vol. 7, no. 2, pp. 176–184, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 E.W. Bai, “A blind approach to the HammersteinWiener model identification,” Automatica, vol. 38, no. 6, pp. 967–979, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. Verhaegen and D. Westwick, “Identifying MIMO Hammerstein systems in the context of subspace model identification methods,” International Journal of Control, vol. 63, no. 2, pp. 331–349, 1996. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, Y. Shi, and T. Chen, “Auxiliary modelbased leastsquares identification methods for Hammerstein outputerror systems,” Systems & Control Letters, vol. 56, no. 5, pp. 373–380, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, Y. Shi, and T. Chen, “Gradientbased identification methods for Hammerstein nonlinear ARMAX models,” Nonlinear Dynamics, vol. 45, no. 12, pp. 31–43, 2006. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 D. Wang and F. Ding, “Extended stochastic gradient identification algorithms for HammersteinWiener ARMAX systems,” Computers & Mathematics with Applications, vol. 56, no. 12, pp. 3157–3164, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Y. S. Xiao, G. L. Song, Y. W. Liao, and R. F. Ding, “Multiinnovation stochastic gradient parameter estimation for input nonlinear controlled autoregressive models,” International Journal of Control, Automation, and Systems, vol. 10, no. 3, pp. 639–643, 2012. View at: Publisher Site  Google Scholar
 J. Chen, L. Lv, and R. Ding, “Multiinnovation stochastic gradient algorithms for dualrate sampled systems with preload nonlinearity,” Applied Mathematics Letters, vol. 26, no. 1, pp. 124–129, 2013. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 L. L. Han and F. Ding, “Multiinnovation stochastic gradient algorithms for multiinput multioutput systems,” Digital Signal Processing, vol. 19, no. 4, pp. 545–554, 2009. View at: Publisher Site  Google Scholar
 Y. Liu, J. Sheng, and R. Ding, “Convergence of stochastic gradient estimation algorithm for multivariable ARXlike systems,” Computers & Mathematics with Applications, vol. 59, no. 8, pp. 2615–2627, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. Hu and F. Ding, “An iterative least squares estimation algorithm for controlled moving average systems based on matrix decomposition,” Applied Mathematics Letters, vol. 25, no. 12, pp. 2332–2338, 2012. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, “Decomposition based fast least squares algorithm for output error systems,” Signal Processing, vol. 93, no. 5, pp. 1235–1242, 2013. View at: Publisher Site  Google Scholar
 F. Ding, “Hierarchical multiinnovation stochastic gradient algorithm for Hammerstein nonlinear system modeling,” Applied Mathematical Modelling, vol. 37, no. 4, pp. 1694–1704, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 J. Vörös, “Iterative algorithm for parameter identification of Hammerstein systems with twosegment nonlinearities,” IEEE Transactions on Automatic Control, vol. 44, no. 11, pp. 2145–2149, 1999. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Y. Xiao and N. Yue, “Parameter estimation for nonlinear dynamical adjustment models,” Mathematical and Computer Modelling, vol. 54, no. 56, pp. 1561–1568, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding and T. Chen, “Identification of Hammerstein nonlinear ARMAX systems,” Automatica, vol. 41, no. 9, pp. 1479–1489, 2005. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding and T. Chen, “Performance analysis of multiinnovation gradient type identification methods,” Automatica, vol. 43, no. 1, pp. 1–14, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, X. P. Liu, and G. Liu, “Multiinnovation least squares identification for system modeling,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, no. 3, pp. 767–778, 2010. View at: Google Scholar
 F. Ding, X. P. Liu, and G. Liu, “Auxiliary model based multiinnovation extended stochastic gradient parameter estimation with colored measurement noises,” Signal Processing, vol. 89, no. 10, pp. 1883–1890, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, “Several multiinnovation identification methods,” Digital Signal Processing, vol. 20, no. 4, pp. 1027–1039, 2010. View at: Publisher Site  Google Scholar
 F. Ding, H. Chen, and M. Li, “Multiinnovation least squares identification methods based on the auxiliary model for MISO systems,” Applied Mathematics and Computation, vol. 187, no. 2, pp. 658–668, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 D. Q. Wang and F. Ding, “Performance analysis of the auxiliary models based multiinnovation stochastic gradient estimation algorithm for output error systems,” Digital Signal Processing, vol. 20, no. 3, pp. 750–762, 2010. View at: Publisher Site  Google Scholar
 L. Xie, Y. J. Liu, H. Z. Yang, and F. Ding, “Modelling and identification for nonuniformly periodically sampleddata systems,” IET Control Theory & Applications, vol. 4, no. 5, pp. 784–794, 2010. View at: Publisher Site  Google Scholar  MathSciNet
 Y. Liu, L. Yu, and F. Ding, “Multiinnovation extended stochastic gradient algorithm and its performance analysis,” Circuits, Systems, and Signal Processing, vol. 29, no. 4, pp. 649–667, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Y. Liu, Y. Xiao, and X. Zhao, “Multiinnovation stochastic gradient algorithm for multipleinput singleoutput systems using the auxiliary model,” Applied Mathematics and Computation, vol. 215, no. 4, pp. 1477–1483, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 L. L. Han and F. Ding, “Parameter estimation for multirate multiinput systems using auxiliary model and multiinnovation,” Journal of Systems Engineering and Electronics, vol. 21, no. 6, pp. 1079–1083, 2010. View at: Google Scholar
 L. Han and F. Ding, “Identification for multirate multiinput systems using the multiinnovation identification theory,” Computers & Mathematics with Applications, vol. 57, no. 9, pp. 1438–1449, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 J. Zhang, F. Ding, and Y. Shi, “Selftuning control based on multiinnovation stochastic gradient parameter estimation,” Systems & Control Letters, vol. 58, no. 1, pp. 69–75, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, G. Liu, and X. P. Liu, “Parameter estimation with scarce measurements,” Automatica, vol. 47, no. 8, pp. 1646–1655, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 W. Xiong, W. Fan, and R. Ding, “Leastsquares parameter estimation algorithm for a class of input nonlinear systems,” Journal of Applied Mathematics, vol. 2012, Article ID 684074, 14 pages, 2012. View at: Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, “Coupledleastsquares identification for multivariable systems,” IET Control Theory & Applications, vol. 7, no. 1, pp. 68–79, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 F. Ding, “Twostage least squares based iterative estimation algorithm for CARARMA system modeling,” Applied Mathematical Modelling, vol. 37, no. 7, pp. 4798–4808, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 J. Ding, F. Ding, X. P. Liu, and G. Liu, “Hierarchical least squares identification for linear SISO systems with dualrate sampleddata,” IEEE Transactions on Automatic Control, vol. 56, no. 11, pp. 2677–2683, 2011. View at: Publisher Site  Google Scholar  MathSciNet
 J. Ding and F. Ding, “Bias compensationbased parameter estimation for output error moving average systems,” International Journal of Adaptive Control and Signal Processing, vol. 25, no. 12, pp. 1100–1111, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 J. Ding, L. Han, and X. Chen, “Time series AR modeling with missing observations based on the polynomial transformation,” Mathematical and Computer Modelling, vol. 51, no. 56, pp. 527–536, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, Y. J. Liu, and B. Bao, “Gradient based and least squares based iterative estimation algorithms for multiinput multioutput systems,” Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, vol. 226, no. 1, pp. 43–55, 2012. View at: Google Scholar
 F. Ding and Y. Gu, “Performance analysis of the auxiliary modelbased leastsquares identification algorithm for onestep statedelay systems,” International Journal of Computer Mathematics, vol. 89, no. 15, pp. 2019–2028, 2012. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding and Y. Gu, “Performance analysis of the auxiliary modelbased stochastic gradient parameter estimation algorithm for statespace systems with onestep state delay,” Circuits, Systems, and Signal Processing, vol. 32, no. 2, pp. 585–599, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 W. Wang, F. Ding, and J. Dai, “Maximum likelihood least squares identification for systems with autoregressive moving average noise,” Applied Mathematical Modelling, vol. 36, no. 5, pp. 1842–1853, 2012. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 J. Li, F. Ding, and G. Yang, “Maximum likelihood least squares identification method for input nonlinear finite impulse response moving average systems,” Mathematical and Computer Modelling, vol. 55, no. 34, pp. 442–450, 2012. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding, G. Liu, and X. P. Liu, “Partially coupled stochastic gradient identification methods for nonuniformly sampled systems,” IEEE Transactions on Automatic Control, vol. 55, no. 8, pp. 1976–1981, 2010. View at: Publisher Site  Google Scholar  MathSciNet
 F. Ding and T. Chen, “Hierarchical gradientbased identification of multivariable discretetime systems,” Automatica, vol. 41, no. 2, pp. 315–325, 2005. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Ding and T. Chen, “Hierarchical least squares identification methods for multivariable systems,” IEEE Transactions on Automatic Control, vol. 50, no. 3, pp. 397–402, 2005. View at: Publisher Site  Google Scholar  MathSciNet
 F. Ding and T. Chen, “Hierarchical identification of lifted statespace models for general dualrate systems,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 52, no. 6, pp. 1179–1187, 2005. View at: Publisher Site  Google Scholar  MathSciNet
 F. Ding, L. Qiu, and T. Chen, “Reconstruction of continuoustime systems from their nonuniformly sampled discretetime systems,” Automatica, vol. 45, no. 2, pp. 324–332, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. Q. Han, L. Xie, F. Ding, and X. G. Liu, “Hierarchical least squares based iterative identification for multivariable systems with moving average noises,” Mathematical and Computer Modelling, vol. 51, no. 910, pp. 1213–1220, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 Y. Liu, F. Ding, and Y. Shi, “Least squares estimation for a class of nonuniformly sampled systems based on the hierarchical identification principle,” Circuits, Systems, and Signal Processing, vol. 31, no. 6, pp. 1985–2000, 2012. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Z. Zhang, F. Ding, and X. Liu, “Hierarchical gradient based iterative parameter estimation algorithm for multivariable output error moving average systems,” Computers & Mathematics with Applications, vol. 61, no. 3, pp. 672–682, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 D. Wang, R. Ding, and X. Dong, “Iterative parameter estimation for a class of multivariable systems based on the hierarchical identification principle and the gradient search,” Circuits, Systems, and Signal Processing, vol. 31, no. 6, pp. 2167–2177, 2012. View at: Publisher Site  Google Scholar  MathSciNet
Copyright
Copyright © 2013 Huiyi Hu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.