Mathematical Modeling and Analysis of Soft ComputingView this Special Issue
Research Article | Open Access
Zhenwei Shi, Zhicheng Ji, "Least Squares Based and Two-Stage Least Squares Based Iterative Estimation Algorithms for H-FIR-MA Systems", Mathematical Problems in Engineering, vol. 2015, Article ID 516374, 8 pages, 2015. https://doi.org/10.1155/2015/516374
Least Squares Based and Two-Stage Least Squares Based Iterative Estimation Algorithms for H-FIR-MA Systems
This paper studies the identification of Hammerstein finite impulse response moving average (H-FIR-MA for short) systems. A new two-stage least squares iterative algorithm is developed to identify the parameters of the H-FIR-MA systems. The simulation cases indicate the efficiency of the proposed algorithms.
System modeling [1–5] and parameter estimation [6–10] are basic for controller design [11, 12]. Nonlinear Hammerstein model identification has received much attention due to its ability to describe a wide class of nonlinear systems and has extensive applications in many engineering problems [13, 14]. The Hammerstein models are special class of nonlinear systems; the nonlinear block is usually static nonlinearity and is followed by a linear system . For example, Wang et al. discussed the identification problem for a Hammerstein nonlinear system with a dynamic subspace state space ; Greblicki investigated a class of continuous time Hammerstein system identification .
There are a lot of research topics about linear or nonlinear system identification [18, 19] and control [20, 21]. For example, Ding et al. derived the gradient search based and the Newton based identification methods for Hammerstein systems ; Wang and Ding proposed a hierarchical least squares identification method for Hammerstein-Wiener systems by using the hierarchical identification principle and the auxiliary model identification idea ; Based on the data filtering technique and the key-term separation principle, Wang et al. investigated a filtering based recursive least squares identification algorithm for Hammerstein output error moving average systems . The proposed algorithm can identify not only the system model parameters but also the noise model parameters and the internal variables.
The iterative algorithm is one of the basic methods for system analysis and synthesis, and nonlinear optimization [25–28]. In , Wang and Ding presented a gradient based and least squares based iterative identification algorithms for Wiener systems through the use of the hierarchical identification principle. In , Ding et al. discussed the Newton iterative identification algorithm of a class of Wiener nonlinear systems with moving average noises from input-output measurement data. Li et al. derived iterative parameter identification methods for nonlinear functions . Pan et al. proposed a digital image correlation using iterative least squares and pointwise least squares for displacement field and strain field measurements . In the field of control, Zhang et al. applied the iterative algorithm to the predictive control field [33, 34].
Recently, the multistage identification strategy is widely applied to the system identification field [35, 36]. For example, Ding and Duan studied a new-type two-stage least squares based iterative algorithm for identifying the system model parameters and the noise model parameters .
The main concern of this paper is to investigate the parameter identification problem of Hammerstein finite impulse response moving average (H-FIR-MA) systems. The memoryless polynomial input nonlinearity is followed by a linear dynamical system, as is explained in Figure 1. Both the least squares iterative and the two-stage least squares iterative algorithms are proposed to estimate the parameters of the H-FIR-MA systems.
The layout of this paper is organized as follows. Section 2 describes the identification model of H-FIR-MA systems. Section 3 provides the least squares iterative algorithm for the H-FIR-MA systems. Section 4 introduces the two-stage least squares iterative algorithm for the H-FIR-MA systems. In Section 5, we apply the proposed algorithms to an example to illustrate their implementation. Finally, concluding remarks are offered in Section 6.
2. System Description and Identification Model
Some notation is given. stands for the norm of a matrix ; “” or “” expresses that “ is defined as ”; represents the identity matrix of appropriate sizes and is defined as an -dimensional identity column vector.
Consider an H-FIR-MA system, which is described by where and are the input and output sequences of the systems, is an uncorrelated stochastic noise sequence with zero mean and variance , and , , and are unmeasurable. The output of the nonlinear block is a linear combination, with unknown coefficients , of a known basis in the system input , and can be expressed as
Assume that the orders , , and are known in (1) and (4) and , , , and for . In order to get unique parameter estimates, here we let . The item in (1) is chosen as the key term; substituting (4) into (1) gets
Define the parameter vector and the information vector as follows:
From (5), we obtain the following identification model:
Define the cost function:
In what follows, we derive the algorithms for identifying the H-FIR-MA system using the least squares and two-stage least squares iterative estimation algorithms.
3. The Least Squares Iterative Estimation Algorithm
In this section, referring to the method in , we give simply the least squares iterative (LSI) estimation algorithm for the H-FIR-MA system for comparison.
Consider the data from to and define the stacked information matrices , the stacked output vector , and the stacked white noise vector as Hence, (7) can be rewritten asAccording to the estimation model in (12), the cost function in (8) can be written asTo minimize , letting its partial derivative of with respect to be zero, we have
It is impossible to obtain the estimate , because the information matrix (i.e., in (6)) contains the unmeasurable inner variables and the noise terms . Here we adopt the auxiliary model idea and the hierarchical identification principle: let … be iteration variable, let be the iterative estimate of at iterative and , and let be the iterative estimates of and . We replace and in (6) with their estimates and obtain the estimates and as follows:Replacing in (14) with and combining (10) and (15), we can obtain the LSI estimation algorithm of identifying for the H-FIR-MA system as follows :The computation procedures of the LSI algorithm in (16)–(22) are summarized as follows.
Step 1. Let and set the initial values , , ; is a large number (i.e., ).
Step 3. Update the parameter identification by (16).
Step 5. Increase by 1 and jump to Step 2.
4. The Two-Stage Least Squares Iterative Estimation Algorithm
Here, we derive a two-stage least squares iterative (TS-LSI) estimation algorithm for the H-FIR-MA system. From (5) and (6), we can obtain the following identification model:Define the information vector and asDefine two intermediate variables and ; then the system in (23) can be decomposed into two “suppositional” subsystems:The estimates of two “suppositional” subsystems in (25) can be obtained by minimizing the cost function:
Consider the data from to and in (25) define the stacked information matrices and and the stacked vector and asTwo intermediate variables can be rewritten as From (25), we haveAccording to the estimation model in (31), the cost function in (26) can be written as To minimize , let its partial derivative of with respect to be zero:From (34), the least squares estimate of the parameter vector can be expressed asHere, put (29) into (35) and (35) gives In accordance with the same derivation process of , we can easily get the estimation formula ofHowever, (36) and (37) contain the unknown parameter and , respectively, it is impossible to and . According to the method in Section 3, we can summarize the two-stage least squares iterative estimation algorithm for estimating and of the H-FIR-MA systems as follows:The computation procedures of the TS-LSI algorithm in (38)–(46) are summarized as follows.
Step 1. Let , and set the initial values , , , , (i.e., ).
Step 6. Increase by 1 and jump to Step 2.
5. Simulations and Case Study
In this section, we consider an H-FIR-MA system, where the static nonlinearity is chosen as polynomials. More precisely
In simulation, the input is taken as an uncorrelated measured stochastic signal sequence with zero mean, the noise is a white noise sequence with zero mean and variances and , respectively, and the corresponding noise-to-signal ratios are and . The noise-to-signal ratios can be calculated by the following formula:where and are, respectively, expressed as the variances of and in (1).
Take two different data lengths and . The parameter values are estimated using the two different methods described in the paper, namely, the LSI and the TS-LSI methods in Sections 3 and 4. We apply the LSI method to estimate the parameters of this case; the parameter estimation with different data length and noise variances are shown in Tables 1 and 2, and the estimation errors versus iteration are shown in Figures 1 and 2, where .
From the simulation results in Tables 1–4 and Figures 1–4, we can draw the following conclusions.(1)The parameter estimation errors given by the LSI and TS-LSI algorithms become small as iterations increase.(2)The parameter estimation errors given by the LSI and TS-LSI algorithms become closer to their true values with the data length increasing.(3)It is easy to see that a high noise level results in a low consistence rate of the parameter estimates to the true parameters for both of the proposed algorithms.(4)When the data length goes to infinity, the estimation errors converge to zero. The simulations of results in Tables 1–4 and Figures 1–4 indicate that the proposed algorithms based iterative algorithm should stop for about . The fluctuation of the estimation errors is caused for large due to the stationary of noise.
All in all, this shows that the proposed algorithms are effective.
The LSI and the TS-LSI identification algorithms are developed for H-FIR-MA systems. The simulation results indicate that the proposed algorithms can obtain highly accurate parameter estimates and fast convergence rate and illustrate the proposed algorithms’ performance. Compared with other methods, the LSI and TS-LSI methods must compute the matrix inversion. The proposed methods are simple in principle and the basic idea can be applied to other fields [38–41].
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was supported in part by the National High Technology Research and Development Program of China (China 863 Program) (no. 2013AA040405), the Guangdong-Hong Kong Breakthrough Bidding Project in Key Areas (no. 2012A080107015), the fundamental research funds for the central universities (JUSRP51310A), and the Chinese State Grain Administration commonweal research project (no. 201313012).
- F. Ding, System Identification—New Theory and Methods, Science Press, Beijing, China, 2013.
- F. Ding, System Identification: Performances Analysis for Identification Methods, Science Press, Beijing, China, 2014.
- Y. B. Hu, “Iterative and recursive least squares estimation algorithms for moving average systems,” Simulation Modelling Practice and Theory, vol. 34, pp. 12–19, 2013.
- Y. Hu, B. Liu, Q. Zhou, and C. Yang, “Recursive extended least squares parameter estimation for Wiener nonlinear systems with moving average noises,” Circuits, Systems, and Signal Processing, vol. 33, no. 2, pp. 655–664, 2014.
- F. Ding, “Hierarchical estimation algorithms for multivariable systems using measurement information,” Information Sciences, vol. 277, pp. 396–405, 2014.
- F. Ding, “State filtering and parameter estimation for state space systems with scarce measurements,” Signal Processing, vol. 104, pp. 369–380, 2014.
- C. Wang and T. Tang, “Recursive least squares estimation algorithm applied to a class of linear-in-parameters output error moving average systems,” Applied Mathematics Letters, vol. 29, pp. 36–41, 2014.
- C. Wang and T. Tang, “Several gradient-based iterative estimation algorithms for a class of nonlinear systems using the filtering technique,” Nonlinear Dynamics, vol. 77, no. 3, pp. 769–780, 2014.
- Y. Shi and H. Fang, “Kalman filter-based identification for systems with randomly missing measurements in a network environment,” International Journal of Control, vol. 83, no. 3, pp. 538–551, 2010.
- Y. Liu, F. Ding, and Y. Shi, “An efficient hierarchical identification method for general dual-rate sampled-data systems,” Automatica, vol. 50, no. 3, pp. 962–970, 2014.
- Y. Shi and B. Yu, “Output feedback stabilization of networked control systems with random delays modeled by Markov chains,” IEEE Transactions on Automatic Control, vol. 54, no. 7, pp. 1668–1674, 2009.
- Y. Shi and B. Yu, “Robust mixed / control of networked control systems with random time delays in both forward and backward communication links,” Automatica, vol. 47, no. 4, pp. 754–760, 2011.
- J. Vörös, “Recursive identification of systems with noninvertible output nonlinearities,” Informatica, vol. 21, no. 1, pp. 139–148, 2010.
- R. Zhang, A. Xue, and S. Wang, “Dynamic modeling and nonlinear predictive control based on partitioned model and nonlinear optimization,” Industrial and Engineering Chemistry Research, vol. 50, no. 13, pp. 8110–8121, 2011.
- B. Yu, H. Fang, Y. Lin, and Y. Shi, “Identification of Hammerstein output-error systems with two-segment nonlinearities: algorithm and applications,” Control and Intelligent Systems, vol. 38, no. 4, pp. 194–201, 2010.
- D. Wang, F. Ding, and L. Ximei, “Least squares algorithm for an input nonlinear system with a dynamic subspace state space model,” Nonlinear Dynamics, vol. 75, no. 1-2, pp. 49–61, 2014.
- W. Greblicki, “Continuous-time Hammerstein system identification,” IEEE Transactions on Automatic Control, vol. 45, no. 6, pp. 1232–1236, 2000.
- F. Ding, X. Liu, and J. Chu, “Gradient-based and least-squares-based iterative algorithms for Hammerstein systems using the hierarchical identification principle,” IET Control Theory & Applications, vol. 7, no. 2, pp. 176–184, 2013.
- F. Ding, “Hierarchical multi-innovation stochastic gradient algorithm for Hammerstein nonlinear system modeling,” Applied Mathematical Modelling, vol. 37, no. 4, pp. 1694–1704, 2013.
- X. Luan, P. Shi, and F. Liu, “Stabilization of networked control systems with random delays,” IEEE Transactions on Industrial Electronics, vol. 58, no. 9, pp. 4323–4330, 2011.
- X. Luan, S. Zhao, and F. Liu, “-infinity control for discrete-time markov jump systems with uncertain transition probabilities,” IEEE Transactions on Automatic Control, vol. 58, no. 6, pp. 1566–1572, 2013.
- F. Ding, X. P. Liu, and G. Liu, “Identification methods for Hammerstein nonlinear systems,” Digital Signal Processing, vol. 21, no. 2, pp. 215–238, 2011.
- D. Q. Wang and F. Ding, “Hierarchical least squares estimation algorithm for hammerstein-wiener systems,” IEEE Signal Processing Letters, vol. 19, no. 12, pp. 825–828, 2012.
- D. Wang, F. Ding, and Y. Chu, “Data filtering based recursive least squares algorithm for Hammerstein systems using the key-term separation principle,” Information Sciences, vol. 222, pp. 203–212, 2013.
- J. Ding, C. Fan, and J. Lin, “Auxiliary model based parameter estimation for dual-rate output error systems with colored noise,” Applied Mathematical Modelling, vol. 37, no. 6, pp. 4051–4058, 2013.
- J. Ding and J. Lin, “Modified subspace identification for periodically non-uniformly sampled systems by using the lifting technique,” Circuits, Systems, and Signal Processing, vol. 33, no. 5, pp. 1439–1449, 2014.
- K. P. Deng and F. Ding, “Newton iterative identification method for an input nonlinear finite impulse response system with moving average noise using the key variables separation technique,” Nonlinear Dynamics, vol. 76, no. 2, pp. 1195–1202, 2014.
- L. Xie and H. Yang, “Gradient-based iterative identification for nonuniform sampling output error systems,” Journal of Vibration and Control, vol. 17, no. 3, pp. 471–478, 2011.
- D. Wang and F. Ding, “Least squares based and gradient based iterative identification for Wiener nonlinear systems,” Signal Processing, vol. 91, no. 5, pp. 1182–1189, 2011.
- F. Ding, J. Ma, and Y. Xiao, “Newton iterative identification for a class of output nonlinear systems with moving average noises,” Nonlinear Dynamics, vol. 74, no. 1-2, pp. 21–30, 2013.
- J. Li, R. Ding, and Y. Yang, “Iterative parameter identification methods for nonlinear functions,” Applied Mathematical Modelling, vol. 36, no. 6, pp. 2739–2750, 2012.
- B. Pan, A. Asundi, H. Xie, and J. Gao, “Digital image correlation using iterative least squares and pointwise least squares for displacement field and strain field measurements,” Optics and Lasers in Engineering, vol. 47, no. 7-8, pp. 865–874, 2009.
- R. Zhang, P. Li, A. Xue, A. Jiang, and S. Wang, “A simplified linear iterative predictive functional control approach for chamber pressure of industrial coke furnace,” Journal of Process Control, vol. 20, no. 4, pp. 464–471, 2010.
- R. Zhang, A. Xue, J. Wang, S. Wang, and Z. Ren, “Neural network based iterative learning predictive control design for mechatronic systems with isolated nonlinearity,” Journal of Process Control, vol. 19, no. 1, pp. 68–74, 2009.
- F. Ding, “Two-stage least squares based iterative estimation algorithm for CARARMA system modeling,” Applied Mathematical Modelling, vol. 37, no. 7, pp. 4798–4808, 2013.
- Y. Gu, F. Ding, and J. H. Li, “State filtering and parameter estimation for linear systems with d-step state-delay,” IET Signal Processing, vol. 8, no. 6, pp. 639–646, 2014.
- F. Ding and H. Duan, “Two-stage parameter estimation algorithms for Box-Jenkins systems,” IET Signal Processing, vol. 7, no. 8, pp. 646–654, 2013.
- D. Q. Zhu and M. Kong, “Adaptive fault-tolerant control of nonlinear system: An improved CMAC based fault learning approach,” International Journal of Control, vol. 80, no. 10, pp. 1576–1594, 2007.
- D. Zhu, J. Liu, and S. X. Yang, “Particle swarm optimization approach to thruster fault-tolerant control of unmanned underwater vehicles,” International Journal of Robotics and Automation, vol. 26, no. 3, pp. 426–432, 2011.
- D. Q. Zhu, Q. Liu, and Z. Hu, “Fault-tolerant control algorithm of the manned submarine with multi-thruster based on quantum-behaved particle swarm optimisation,” International Journal of Control, vol. 84, no. 11, pp. 1817–1829, 2011.
- D. Zhu, H. Huang, and S. X. Yang, “Dynamic task assignment and path planning of multi-AUV system based on an improved self-organizing map and velocity synthesis method in three-dimensional underwater workspace,” IEEE Transactions on Cybernetics, vol. 43, no. 2, pp. 504–514, 2013.
Copyright © 2015 Zhenwei Shi and Zhicheng Ji. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.