Research Article  Open Access
Weili Xiong, Wei Fan, Rui Ding, "LeastSquares Parameter Estimation Algorithm for a Class of Input Nonlinear Systems", Journal of Applied Mathematics, vol. 2012, Article ID 684074, 14 pages, 2012. https://doi.org/10.1155/2012/684074
LeastSquares Parameter Estimation Algorithm for a Class of Input Nonlinear Systems
Abstract
This paper studies leastsquares parameter estimation algorithms for input nonlinear systems, including the input nonlinear controlled autoregressive (INCAR) model and the input nonlinear controlled autoregressive autoregressive moving average (INCARARMA) model. The basic idea is to obtain linearinparameters models by overparameterizing such nonlinear systems and to use the leastsquares algorithm to estimate the unknown parameter vectors. It is proved that the parameter estimates consistently converge to their true values under the persistent excitation condition. A simulation example is provided.
1. Introduction
Parameter estimation has received much attention in many areas such as linear and nonlinear system identification and signal processing [1–9]. Nonlinear systems can be simply divided into the input nonlinear systems, the output nonlinear systems, the feedback nonlinear systems, and the input and output nonlinear systems, and so forth. The Hammerstein models can describe a class of input nonlinear systems which consist of static nonlinear blocks followed by linear dynamical subsystems [10, 11].
Nonlinear systems are common in industrial processes, for example, the deadzone nonlinearities and the valve saturation nonlinearities. Many estimation methods have been developed to identify the parameters of nonlinear systems, especially for Hammerstein nonlinear systems [12, 13]. For example, Ding et al. presented a leastsquaresbased iterative algorithm and a recursive extended least squares algorithm for Hammerstein ARMAX systems [14] and an auxiliary modelbased recursive least squares algorithm for Hammerstein output error systems [15]. Wang and Ding proposed an extended stochastic gradient identification algorithm for HammersteinWiener ARMAX Systems [16].
Recently, Wang et al. derived an auxiliary modelbased recursive generalized leastsquares parameter estimation algorithm for Hammerstein output error autoregressive systems and auxiliary modelbased RELS and MIELS algorithms for Hammerstein output error moving average systems using the key term separation principle [17, 18]. Ding et al. presented a projection estimation algorithm and a stochastic gradient (SG) estimation algorithm for Hammerstein nonlinear systems by using the gradient search and further derived a Newton recursive estimation algorithm and a Newton iterative estimation algorithm by using the Newton method (NewtonRaphson method) [19]. Wang and Ding studied leastsquaresbased and gradientbased iterative identification methods for Wiener nonlinear systems [20].
Fan et al. discussed the parameter estimation problem for Hammerstein nonlinear ARX models [21]. On the basis of the work in [14, 15, 21], this paper studies the identification problems and their convergence for input nonlinear controlled autoregressive (INCAR) models using the martingale convergence theorem and gives the recursive generalized extended leastsquares algorithm for input nonlinear controlled autoregressive autoregressive moving average (INCARARMA) models.
Briefly, the paper is organized as follows. Section 2 derives a linearinparameters identification model and gives a recursive least squares identification algorithm for input nonlinear CAR systems and analyzes the properties of the proposed algorithm. Section 4 gives the recursive generalized extended least squares algorithm for input nonlinear CARARMA systems. Section 5 provides an illustrative example to show the effectiveness of the proposed algorithms. Finally, we offer some concluding remarks in Section 6.
2. The Input Nonlinear CAR Model and Estimation Algorithm
Let us introduce some notations first. The symbol stands for an identity matrix of appropriate sizes (); the superscript denotes the matrix transpose; represents an dimensional column vector whose elements are 1; represents the determinant of the matrix ; the norm of a matrix is defined by ; and represent the maximum and minimum eigenvalues of the square matrix , respectively; represents as ; for , we write if there exists a positive constant such that .
2.1. The Input Nonlinear CAR Model
Consider the following input nonlinear controlled autoregressive (INCAR) systems [14, 21]: where is the system output, is a disturbance noise, the output of the nonlinear block is a nonlinear function of a known basis of the system input [19], and are polynomials in the unit backward shift operator [], defined as In order to obtain the identifiability of parameters and , without loss of generality, we suppose that or [14, 21].
Define the parameter vector and information vector as From (2.1), we have An alternative way is to define the parameter vector and information vector as Then (2.5) can be written as Equations (2.6) and (2.8) are both linearinparameters identification model for Hammerstein CAR systems by using parametrization.
2.2. The Recursive Least Squares Algorithm
Minimizing the cost function gives the following recursive least squares algorithm for computing the estimate of in (2.8): Applying the matrix inversion formula [22] to (2.11) and defining the gain vector , the algorithm in (2.10)(2.11) can be equivalently expressed as To initialize the algorithm, we take to be a large positive real number, for example, , and to be some small real vector, for example, .
3. The Main Convergence Theorem
The following lemmas are required to establish the main convergence results.
Lemma 3.1 (Martingale convergence theorem: Lemma D.5.3 in [23, 24]). If , , are nonnegative random variables, measurable with respect to a nondecreasing sequence of algebra , and satisfy then when , one has and . (.: almost surely) a finite nonnegative random variable.
Lemma 3.2 (see [14, 21, 25]). For the algorithm in (2.10)(2.11), for any , the covariance matrix in (2.11) satisfies the following inequality:
Theorem 3.3. For the system in (2.8) and the algorithm in (2.10)(2.11), assume that is a martingale difference sequence defined on a probability space , where is the algebra sequence generated by the observations and the noise sequence satisfies and [23], and , . Then the parameter estimation error converges to zero.
Proof. Define the parameter estimation error vector and the stochastic Lyapunov function . Let . According to the definitions of and and using (2.10) and (2.11), we have Here, we have used the inequality . Because and are uncorrelated with and are measurable, taking the conditional expectation with respect to , we have Since is nondecreasing, letting we have Using Lemma 3.2, the sum of the last term in the righthand side for from 1 to is finite. Applying Lemma 3.1 to the previous inequality, we conclude that converges a.s. to a finite random variable, say , that is: Thus, according to the definition of , we have This completes the proof of Theorem 3.3.
According to the definition of and the assumption , the estimates and , , , of and can be read from the first and second entries of , respectively. Let be the th element of . Referring to the definition of , the estimates of , , may be computed by Notice that there is a large amount of redundancy about for each . Since we do not need such estimates , one way is to take their average as the estimate of [14], that is:
4. The Input Nonlinear CARARMA System and Estimation Algorithm
Consider the following input nonlinear controlled autoregressive autoregressive moving average (INCARARMA) systems: Let or Define the parameter vector and information vector as Then (4.1) can be written as This is a linearinparameter identification model for INCARARMA systems.
The unknown and in the information vector are replaced with their estimates and , and then we can obtain the following recursive generalized extended least squares algorithm for estimating in (4.6):
This paper presents a recursive least squares algorithm for INCAR systems and a recursive generalized extended least squares algorithm for INCARARMA systems with ARMA noise disturbances, which differ not only from the input nonlinear controlled autoregressive moving average (INCARMA) systems in [14] but also from the input nonlinear output error systems in [15].
5. Example
Consider the following INCAR system: In simulation, the input is taken as a persistent excitation signal sequence with zero mean and unit variance and as a white noise sequence with zero mean and constant variance . Applying the proposed algorithm in (2.10)(2.11) to estimate the parameters of this system, the parameter estimates and and their errors with different noise variances are shown in Tables 1, 2, 3, and 4, and the parameter estimation errors and versus are shown in Figures 1 and 2. When and , the corresponding noisetosignal ratios are and , respectively.




From Tables 1–4 and Figures 1 and 2, we can draw the following conclusions.(i)The larger the data length is, the smaller the parameter estimation errors become. (ii)A lower noise level leads to smaller parameter estimation errors for the same data length. (iii)The estimation errors and become smaller (in general) as increases. This confirms the proposed theorem.
6. Conclusions
The recursive leastsquares identification is used to estimate the unknown parameters for input nonlinear CAR and CARARMA systems. The analysis using the martingale convergence theorem indicates that the proposed recursive least squares algorithm can give consistent parameter estimation. It is worth pointing out that the multiinnovation identification theory [26–33], the gradientbased or leastsquaresbased identification methods [34–41], and other identification methods [42–49] can be used to study identification problem of this class of nonlinear systems with colored noises.
Acknowledgment
This work was supported by the 111 Project (B12018).
References
 M. R. Zakerzadeh, M. Firouzi, H. Sayyaadi, and S. B. Shouraki, “Hysteresis nonlinearity identification using new Preisach modelbased artificial neural network approach,” Journal of Applied Mathematics, Article ID 458768, 22 pages, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 X.X. Li, H. Z. Guo, S. M. Wan, and F. Yang, “Inverse source identification by the modified regularization method on poisson equation,” Journal of Applied Mathematics, vol. 2012, Article ID 971952, 13 pages, 2012. View at: Publisher Site  Google Scholar
 Y. Shi and H. Fang, “Kalman filterbased identification for systems with randomly missing measurements in a network environment,” International Journal of Control, vol. 83, no. 3, pp. 538–551, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 Y. Liu, J. Sheng, and R. Ding, “Convergence of stochastic gradient estimation algorithm for multivariable ARXlike systems,” Computers & Mathematics with Applications, vol. 59, no. 8, pp. 2615–2627, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, G. Liu, and X. P. Liu, “Parameter estimation with scarce measurements,” Automatica, vol. 47, no. 8, pp. 1646–1655, 2011. View at: Google Scholar
 J. Ding, F. Ding, X. P. Liu, and G. Liu, “Hierarchical least squares identification for linear SISO systems with dualrate sampleddata,” IEEE Transactions on Automatic Control, vol. 56, no. 11, pp. 2677–2683, 2011. View at: Publisher Site  Google Scholar
 Y. Liu, Y. Xiao, and X. Zhao, “Multiinnovation stochastic gradient algorithm for multipleinput singleoutput systems using the auxiliary model,” Applied Mathematics and Computation, vol. 215, no. 4, pp. 1477–1483, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 J. Ding and F. Ding, “The residual based extended least squares identification method for dualrate systems,” Computers & Mathematics with Applications, vol. 56, no. 6, pp. 1479–1487, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 L. Han and F. Ding, “Identification for multirate multiinput systems using the multiinnovation identification theory,” Computers & Mathematics with Applications, vol. 57, no. 9, pp. 1438–1449, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, Y. Shi, and T. Chen, “Gradientbased identification methods for Hammerstein nonlinear ARMAX models,” Nonlinear Dynamics, vol. 45, no. 12, pp. 31–43, 2006. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, T. Chen, and Z. Iwai, “Adaptive digital control of Hammerstein nonlinear systems with limited output sampling,” SIAM Journal on Control and Optimization, vol. 45, no. 6, pp. 2257–2276, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 J. Li and F. Ding, “Maximum likelihood stochastic gradient estimation for Hammerstein systems with colored noise based on the key term separation technique,” Computers & Mathematics with Applications, vol. 62, no. 11, pp. 4170–4177, 2011. View at: Publisher Site  Google Scholar
 J. Li, F. Ding, and G. Yang, “Maximum likelihood least squares identification method for input nonlinear finite impulse response moving average systems,” Mathematical and Computer Modelling, vol. 55, no. 34, pp. 442–450, 2012. View at: Publisher Site  Google Scholar
 F. Ding and T. Chen, “Identification of Hammerstein nonlinear ARMAX systems,” Automatica, vol. 41, no. 9, pp. 1479–1489, 2005. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, Y. Shi, and T. Chen, “Auxiliary modelbased leastsquares identification methods for Hammerstein outputerror systems,” Systems & Control Letters, vol. 56, no. 5, pp. 373–380, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 D. Wang and F. Ding, “Extended stochastic gradient identification algorithms for HammersteinWiener ARMAX systems,” Computers & Mathematics with Applications, vol. 56, no. 12, pp. 3157–3164, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 D. Wang, Y. Chu, G. Yang, and F. Ding, “Auxiliary model based recursive generalized least squares parameter estimation for Hammerstein OEAR systems,” Mathematical and Computer Modelling, vol. 52, no. 12, pp. 309–317, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 D. Wang, Y. Chu, and F. Ding, “Auxiliary modelbased RELS and MIELS algorithm for Hammerstein OEMA systems,” Computers & Mathematics with Applications, vol. 59, no. 9, pp. 3092–3098, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, X. P. Liu, and G. Liu, “Identification methods for Hammerstein nonlinear systems,” Digital Signal Processing, vol. 21, no. 2, pp. 215–238, 2011. View at: Publisher Site  Google Scholar
 D. Wang and F. Ding, “Least squares based and gradient based iterative identification for Wiener nonlinear systems,” Signal Processing, vol. 91, no. 5, pp. 1182–1189, 2011. View at: Publisher Site  Google Scholar
 W. Fan, F. Ding, and Y. Shi, “Parameter estimation for Hammerstein nonlinear controlled autoregression models,” in Proceedings of the IEEE International Conference on Automation and Logistics, pp. 1007–1012, Jinan, China, August 2007. View at: Google Scholar
 L. Wang, F. Ding, and P. X. Liu, “Convergence of HLS estimation algorithms for multivariable ARXlike systems,” Applied Mathematics and Computation, vol. 190, no. 2, pp. 1081–1093, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 G. C. Goodwin and K. S. Sin, Adaptive Filtering, Prediction and Control, PrenticeHall, Englewood Cliffs, NJ, USA, 1984.
 Y. Liu, L. Yu, and F. Ding, “Multiinnovation extended stochastic gradient algorithm and its performance analysis,” Circuits, Systems, and Signal Processing, vol. 29, no. 4, pp. 649–667, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding and T. Chen, “Combined parameter and output estimation of dualrate systems using an auxiliary model,” Automatica, vol. 40, no. 10, p. 1739, 2004. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding and T. Chen, “Performance analysis of multiinnovation gradient type identification methods,” Automatica, vol. 43, no. 1, pp. 1–14, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 L. Han and F. Ding, “Multiinnovation stochastic gradient algorithms for multiinput multioutput systems,” Digital Signal Processing, vol. 19, no. 4, pp. 545–554, 2009. View at: Publisher Site  Google Scholar
 F. Ding, “Several multiinnovation identification methods,” Digital Signal Processing, vol. 20, no. 4, pp. 1027–1039, 2010. View at: Google Scholar
 D. Wang and F. Ding, “Performance analysis of the auxiliary models based multiinnovation stochastic gradient estimation algorithm for output error systems,” Digital Signal Processing, vol. 20, no. 3, pp. 750–762, 2010. View at: Publisher Site  Google Scholar
 J. Zhang, F. Ding, and Y. Shi, “Selftuning control based on multiinnovation stochastic gradient parameter estimation,” Systems & Control Letters, vol. 58, no. 1, pp. 69–75, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, H. Chen, and M. Li, “Multiinnovation least squares identification methods based on the auxiliary model for MISO systems,” Applied Mathematics and Computation, vol. 187, no. 2, pp. 658–668, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 L. Xie, Y. J. Liu, H. Z. Yang, and F. Ding, “Modelling and identification for nonuniformly periodically sampleddata systems,” IET Control Theory & Applications, vol. 4, no. 5, pp. 784–794, 2010. View at: Publisher Site  Google Scholar
 F. Ding, P. X. Liu, and G. Liu, “Multiinnovation leastsquares identification for system modeling,” IEEE Transactions on Systems, Man, and Cybernetics B, vol. 40, no. 3, Article ID 5299173, pp. 767–778, 2010. View at: Publisher Site  Google Scholar
 J. Ding, Y. Shi, H. Wang, and F. Ding, “A modified stochastic gradient based parameter estimation algorithm for dualrate sampleddata systems,” Digital Signal Processing, vol. 20, no. 4, pp. 1238–1247, 2010. View at: Publisher Site  Google Scholar
 F. Ding, P. X. Liu, and H. Yang, “Parameter identification and intersample output estimation for dualrate systems,” IEEE Transactions on Systems, Man, and Cybernetics A, vol. 38, no. 4, pp. 966–975, 2008. View at: Publisher Site  Google Scholar
 Y. Liu, D. Wang, and F. Ding, “Least squares based iterative algorithms for identifying BoxJenkins models with finite measurement data,” Digital Signal Processing, vol. 20, no. 5, pp. 1458–1467, 2010. View at: Publisher Site  Google Scholar
 D. Wang and F. Ding, “Inputoutput data filtering based recursive least squares identification for CARARMA systems,” Digital Signal Processing, vol. 20, no. 4, pp. 991–999, 2010. View at: Publisher Site  Google Scholar
 F. Ding, P. X. Liu, and G. Liu, “Gradient based and leastsquares based iterative identification methods for OE and OEMA systems,” Digital Signal Processing, vol. 20, no. 3, pp. 664–677, 2010. View at: Publisher Site  Google Scholar
 D. Wang, G. Yang, and R. Ding, “Gradientbased iterative parameter estimation for BoxJenkins systems,” Computers & Mathematics with Applications, vol. 60, no. 5, pp. 1200–1208, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 L. Xie, H. Yang, and F. Ding, “Recursive least squares parameter estimation for nonuniformly sampled systems based on the data filtering,” Mathematical and Computer Modelling, vol. 54, no. 12, pp. 315–324, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, Y. Liu, and B. Bao, “Gradientbased and leastsquaresbased iterative estimation algorithms for multiinput multioutput systems,” Proceedings of the Institution of Mechanical Engineers. Part I: Journal of Systems and Control Engineering, vol. 226, no. 1, pp. 43–55, 2012. View at: Publisher Site  Google Scholar
 F. Ding, “Hierarchical multiinnovation stochastic gradient algorithm for Hammerstein nonlinear system modeling,” Applied Mathematical Modelling. In press. View at: Publisher Site  Google Scholar
 F. Ding and J. Ding, “Leastsquares parameter estimation for systems with irregularly missing data,” International Journal of Adaptive Control and Signal Processing, vol. 24, no. 7, pp. 540–553, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 Y. Liu, L. Xie, and F. Ding, “An auxiliary model based on a recursive leastsquares parameter estimation algorithm for nonuniformly sampled multirate systems,” Proceedings of the Institution of Mechanical Engineers. Part I: Journal of Systems and Control Engineering, vol. 223, no. 4, pp. 445–454, 2009. View at: Publisher Site  Google Scholar
 F. Ding, L. Qiu, and T. Chen, “Reconstruction of continuoustime systems from their nonuniformly sampled discretetime systems,” Automatica, vol. 45, no. 2, pp. 324–332, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 F. Ding, G. Liu, and X. P. Liu, “Partially coupled stochastic gradient identification methods for nonuniformly sampled systems,” IEEE Transactions on Automatic Control, vol. 55, no. 8, pp. 1976–1981, 2010. View at: Publisher Site  Google Scholar
 J. Ding and F. Ding, “Bias compensationbased parameter estimation for output error moving average systems,” International Journal of Adaptive Control and Signal Processing, vol. 25, no. 12, pp. 1100–1111, 2011. View at: Publisher Site  Google Scholar
 F. Ding and T. Chen, “Performance bounds of forgetting factor leastsquares algorithms for timevarying systems with finite meaurement data,” IEEE Transactions on Circuits and Systems. I. Regular Papers, vol. 52, no. 3, pp. 555–566, 2005. View at: Publisher Site  Google Scholar
 F. Ding and T. Chen, “Hierarchical identification of lifted statespace models for general dualrate systems,” IEEE Transactions on Circuits and Systems. I. Regular Papers, vol. 52, no. 6, pp. 1179–1187, 2005. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2012 Weili Xiong et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.