Complexity

Complexity / 2017 / Article
Special Issue

Neural Network for Complex Systems: Theory and Applications

View this Special Issue

Research Article | Open Access

Volume 2017 |Article ID 5292894 | https://doi.org/10.1155/2017/5292894

Jiling Ding, "The Hierarchical Iterative Identification Algorithm for Multi-Input-Output-Error Systems with Autoregressive Noise", Complexity, vol. 2017, Article ID 5292894, 11 pages, 2017. https://doi.org/10.1155/2017/5292894

The Hierarchical Iterative Identification Algorithm for Multi-Input-Output-Error Systems with Autoregressive Noise

Academic Editor: Guang Li
Received15 Jul 2017
Accepted18 Sep 2017
Published29 Oct 2017

Abstract

This paper considers the identification problem of multi-input-output-error autoregressive systems. A hierarchical gradient based iterative (H-GI) algorithm and a hierarchical least squares based iterative (H-LSI) algorithm are presented by using the hierarchical identification principle. A gradient based iterative (GI) algorithm and a least squares based iterative (LSI) algorithm are presented for comparison. The simulation results indicate that the H-LSI algorithm can obtain more accurate parameter estimates than the LSI algorithm, and the H-GI algorithm converges faster than the GI algorithm.

1. Introduction

System identification studies mathematical models of dynamic systems by fitting experimental data to a suitable model structure [1, 2]. Many practical systems have multiple inputs and multiple outputs such as chemical processes [3, 4], automation devices [57], and network communication engineering [810]. For decades, much research has been performed on the multivariable systems [11, 12], and some typical approaches for the parameter estimation of the multivariable systems have been reported [13], such as the canonical approach [14], the iterative methods [15, 16], and the least squares methods [17]. Recently, Panda and Vijayaraghavan adopted the sequential relay feedback test to estimate the parameter of the linear multivariable systems [18]. Jafari et al. presented an iterative least squares algorithm to identify the multivariable nonlinear systems with colored noises [19].

The multivariable systems contain both parameter vectors and parameter matrices, and the systems inputs and system outputs are relevant and coupled [2022]. For the sake of reducing the computational complexity, the hierarchical identification principle is utilized to transform a complex system into several subsystems and then to estimate the parameter vector of each subsystem [23, 24], respectively. In this literature, Schranz et al. proposed a feasible hierarchical identification process for identifying the viscoelastic model of respiratory mechanics [25]. Xu et al. developed the parameter estimation for dynamical response signals [26, 27].

The iterative methods have been widely applied in identifying the parameters of linear or nonlinear systems [2830]. Many iterative algorithms for system identification are based on the gradient method [31]and the least squares method [3235]. The basic idea of iterative methods is to update the parameter estimates using batch data.

This paper focuses on the parameter estimation for output-error autoregressive (OEAR) systems using the hierarchical identification principle and the iterative identification principle and presents a hierarchical gradient based iterative (H-GI) algorithm and a hierarchical least squares based iterative (H-LSI) algorithm. The key is to decompose a multi-input OEAR system into two subsystems and then to identify each subsystem. The work in [36, 37] discussed the single-input single-output systems, but many practical systems have multiple inputs and multiple outputs with the development of industrial technology. Compared with the work in [36, 37], this paper discusses the parameter estimation for multi-input OEAR systems and the presented H-LSI algorithm can achieve higher estimation accuracy than the LSI algorithm, and the H-GI algorithm also can achieve higher estimation accuracy than the GI algorithm.

The rest of this paper is organized as follows. Section 2 gives some definitions and the identification model of multi-input OEAR systems. Section 3 presents a gradient based iterative algorithm and a least squares based iterative algorithm for multi-input OEAR systems. Section 4 derives a hierarchical gradient based iterative algorithm. Section 5 derives a hierarchical least squares based iterative algorithm. Section 6 provides two illustrative examples to demonstrate the effectiveness of the proposed algorithms. Finally, concluding remarks are given in Section 7.

2. The Problem Formulation

Let us define some notation.

Symbols: meaning: an dimensional column vector whose entries are all 1: a large positive constant, for example, : the transpose of the vector or matrix : : defined as : the maximum eigenvalue of the symmetric real matrix .

Consider the following multi-input-output-error type models: where is the system output, , , are the system inputs, and is the colored noise with zero mean. and are polynomials in the unit backward shift operator , and Assume that the orders are known, , , and as . The colored noise can be fitted by a moving average process or an autoregressive process or an autoregressive moving average process where is the white noise with zero mean and , are polynomials in the unit backward shift operator : This paper considers the colored noise to be an autoregressive process, so the models in (1) can be taken as the multi-input OEAR systems.

Define the intermediate variables: From (4) and (7), we have The output in (1) can be written as Define the parameter vectors as and the information vectors as According to the above definitions, (8) and (9) can be written as Equation (14) is the identification model of the multi-input OEAR system.

3. The Gradient Based and Least Squares Based Iterative Algorithm

Consider the data from to and define quadratic criterion function as Let be an iteration variable and be the estimate of at iteration . Minimizing by using the negative gradient search, we can obtain where is an iterative step-size. Because the information vector contains the unknown variables and , we use the estimates and at iteration to replace the unknown variables and ; we can obtain the gradient based iterative (GI) algorithm for estimating the parameter vector of the multi-input OEAR systems:The convergence rate of the GI algorithm is slow. To improve the convergence speed, we derive a least squares based iterative (LSI) identification algorithm. Minimizing and letting the derivative of with respect to be zero give the LSI identification algorithm for the multi-input OEAR systems:

4. The Hierarchical Gradient Based Iterative Algorithm

Define intermediate variables:Using the hierarchical identification principle, the multi-input OEAR system in (14) can be decomposed into two fictitious subsystems:Next, we identify the parameters and of each subsystem in (21) and (22), respectively. Define quadratic criterion functions asLet and be the estimates of and at iteration . Using the negative gradient search and minimizing and , we can obtainHere, and are the iterative step-sizes or convergence factors. Substituting (19) into (24) and (20) into (25), we can obtainThe parameter estimates and cannot be computed by (26) and (27), because the information vectors and contain unknown variables and , and the parameter vectors and in (26) and (27) are unknown. We solve this problem by replacing the unknown variables and with their corresponding estimates and at iteration and define the estimates and at iteration asFrom (12) and (14), we haveSubstituting and with their estimates and , we can get the estimates and at iteration :Replacing , in (26) and (27) with their estimates and , replacing in (26) with its estimate , and replacing in (27) with its estimate , we haveIn order to guarantee the convergence of and , a conservative choice isAt last, we can summarize the hierarchical gradient based iterative parameter estimation (H-GI) algorithm for estimating and of the multi-input OEAR systems:The steps of computing the parameter estimates and for the multi-input OEAR systems are as follows.(1)Set the data length , let , and collect the input-output data .(2)Collect the input-output data and .(3)To initialize, let , , , , for .(4)Form and by (36) and (37), and form by (35).(5)Choose a satisfying (40) and update the estimate using (33) and using (34).(6)Read and using (41) and (42) and compute using (38) and using (39).(7)Give a small positive . If , increase by 1 and go to Step (); otherwise, obtain the parameters and and increase by 1 and go to Step ().

5. The Hierarchical Least Squares Based Iterative Algorithm

The H-GI algorithm can produce higher parameter estimation accuracy compared with the GI algorithm, but it converges slowly. In order to solve this short board, we derive a hierarchical least squares based iterative algorithm for the multi-input OEAR systems.

Minimizing and letting the partial derivative of with respect to be zero and minimizing and letting the partial derivative of with respect to be zero, respectively, we can obtain the least squares estimate :Inserting (19) into (44) and (20) into (45) givesThe above estimates and are impossible to compute, since the right-hand side of (46) contains the unknown parameter vector and the unknown information vectors and and the right-hand side of (47) also contains the unknown parameter vector and the unknown information vectors and . We solve this difficulty by replacing , with their estimates , and replacing in (46) and in (47) with their estimates and . Then, we can summarize the hierarchical least squares based iterative (LSI) algorithm of estimating the parameter vectors and as follows:

The procedure for computing the parameter estimation and is as follows.(1)Give the data length , let , collect the input-output data , and give a small positive .(2)Collect the input-output data and .(3)To initialize, let , , for .(4)Form , , and by (51), (52), and (50), respectively.(5)Update the estimates and by (48) and (49) and read by (55).(6)Compute by (53) and by (54).(7)If , increase by 1 and go to Step (); otherwise, obtain the parameters and and increase by 1 and go to Step ().

6. Example

Example 1. Consider the following two-input OEAR system:The inputs are taken as two persistent excitation signal sequences with zero mean and unit variance and as a white noise sequence with zero mean and variance .
Take the data length , applying the LSI algorithm and the H-LSI algorithm to estimate the parameters of this example system. The parameter estimates and their errors of the LSI algorithm are shown in Table 1, the parameter estimates and their errors of H-LSI algorithm are shown in Table 2, and the parameter estimation errors of the LSI and H-LSI algorithms versus are shown in Figure 1.
From the simulation results in Tables 1 and 2 and Figure 1, we can draw the following conclusions. (i)The estimation errors given by the LSI algorithm and H-LSI algorithm become smaller and smaller as iteration variable increases.(ii)Under the same noise variance, the estimation errors given by the H-LSI algorithm are lower than that given by the LSI algorithm.(iii)The estimation accuracy of the H-LSI algorithm is close to their true values; this indicates that the proposed algorithm can effectively identify the multi-input OEAR systems.



1−0.001740.000530.78866−0.686310.00895−0.008180.570270.61302−0.0005055.79849
20.353690.271840.78727−0.40443−0.176290.156940.557860.510590.2717412.15009
30.350560.267460.78457−0.40524−0.193970.175070.560990.504950.383793.57156
40.350330.268200.78447−0.40537−0.186220.172550.561230.507240.386773.52725
50.350510.268380.78449−0.40522−0.186160.169820.561170.507180.386443.58654
60.350540.268430.78450−0.40520−0.188070.171620.561140.506180.386163.53048
70.350540.268390.78449−0.40520−0.187760.171560.561170.506340.386413.52131
80.350530.268380.78449−0.40521−0.187650.171410.561160.506390.386393.52730
90.350530.268390.78449−0.40520−0.187710.171450.561160.506360.386373.52652
100.350530.268390.78449−0.40520−0.187710.171460.561160.506360.386383.52588
True values0.350000.270000.78000−0.40000−0.200000.180000.560000.500000.43000



1−0.001650.000450.78903−0.686490.000000.000000.570430.613680.0485453.42619
20.337870.257740.78598−0.41664−0.177400.162010.560290.509720.2229115.77773
30.352020.269290.78458−0.40359−0.189730.172310.562830.504760.393522.96102
40.350650.268370.78458−0.40503−0.188920.171750.562970.504640.444931.66301
50.350540.268350.78458−0.40512−0.189230.171830.562960.504440.446601.73488
60.350520.268320.78458−0.40514−0.189260.171880.562960.504420.445671.68306
70.350530.268330.78457−0.40514−0.189260.171870.562960.504430.445781.68902
80.350520.268330.78457−0.40514−0.189260.171870.562960.504430.445781.68914
90.350520.268330.78457−0.40514−0.189260.171870.562960.504430.445781.68894
100.350520.268330.78457−0.40514−0.189260.171870.562960.504430.445781.68898
True values
0.350000.270000.78000−0.40000−0.200000.180000.560000.500000.43000

Example 2. Consider the following another two-input OEAR system:The simulation conditions are the same as that of Example 1, and the noise variance . Take the data length . Applying the GI algorithm and the H-GI algorithm to estimate the parameters of this example system, the simulation results are shown in Tables 35 and Figure 2.
From the simulation results in Tables 35 and Figure 2, we can draw the following conclusions. (i)Under the same noise variance and data length, the H-GI algorithm has less estimation errors than the GI algorithm. This shows that the H-GI estimation algorithm can obtain more accurate estimates than the GI algorithm.(ii)As the iteration variable increases, the H-GI parameter estimates are very close to their true values.(iii)The proposed H-GI algorithm requires more iterations than the H-LSI algorithm to achieve almost same estimation accuracy.



10.000000.000000.55541−0.786410.05229−0.028110.59789−0.925880.0126934.42173
20.14148−0.306600.55273−0.703770.18346−0.141420.60426−0.822560.5839521.10705
30.19132−0.232720.55125−0.678660.21589−0.095340.60251−0.801420.0198519.82911
40.18954−0.241970.55162−0.679790.21649−0.097300.60299−0.801910.374165.32335
50.19059−0.238970.55155−0.679150.21666−0.096550.60291−0.801670.278101.93353
60.19032−0.239770.55157−0.679310.21662−0.096700.60293−0.801720.300981.11341
70.19039−0.239560.55157−0.679270.21663−0.096670.60293−0.801710.295141.17583
80.19037−0.239610.55157−0.679280.21662−0.096680.60293−0.801710.296641.14596
90.19038−0.239600.55157−0.679280.21662−0.096670.60293−0.801710.296251.15289
100.19038−0.239600.55157−0.679280.21662−0.096680.60293−0.801710.296351.15101

True values0.18000−0.250000.55000−0.680000.22000−0.100000.60000−0.800000.30000



1 0.06363 −0.03646 0.51672 −0.72852 0.01182 −0.03937 0.54579 −0.85581 0.00627 31.71160
2 0.17357 −0.19236 0.53839 −0.74274 0.14494 −0.16525 0.57604 −0.89230 0.09049 18.71188
5 0.18370 −0.23560 0.55059 −0.69456 0.15382 −0.14339 0.59693 −0.85356 0.09666 15.92395
10 0.18597 −0.24230 0.55243 −0.68240 0.17824 −0.12609 0.60144 −0.83140 0.10937 14.08831
20 0.18821 −0.24117 0.55225 −0.68049 0.20204 −0.10795 0.60193 −0.81289 0.13405 11.85943
50 0.18994 −0.23981 0.55225 −0.67953 0.21588 −0.09733 0.60184 −0.80231 0.18766 8.00236
True values 0.18000 −0.25000 0.55000 −0.68000 0.22000 −0.10000 0.60000 −0.80000 0.30000



1 0.02264 −0.01297 0.18381 −0.25916 0.02264 −0.01297 0.19416 −0.30444 0.00295 68.31458
2 0.09666 −0.06071 0.34760 −0.48848 0.10932 −0.05447 0.36923 −0.57459 0.28748 34.49578
5 0.24231 −0.19630 0.53350 −0.64647 0.22618 −0.09991 0.57665 −0.77609 0.33914 7.35051
10 0.23004 −0.20855 0.55156 −0.65412 0.22289 −0.09268 0.59940 −0.79531 0.33141 5.44993
20 0.21210 −0.22233 0.55260 −0.66565 0.21857 −0.09528 0.60089 −0.80002 0.31890 3.45097
50 0.19366 −0.23682 0.55262 −0.67729 0.21663 −0.09677 0.60117 −0.80156 0.29411 1.47162

True values 0.18000 −0.25000 0.55000 −0.68000 0.22000 −0.10000 0.60000 −0.80000 0.30000

7. Conclusions

Combining the iterative technique and the hierarchical identification principle, a H-GI algorithm and a H-LSI algorithm are derived for identifying the multi-input OEAR systems. Compared with the GI algorithm, the H-GI algorithm can generate more accurate parameter estimates. Compared with the H-GI algorithm, the H-LSI algorithm has faster convergence speed. The proposed methods can be extended to discuss the parameter estimation of the multi-input-output systems with colored noise [3842] and time-delay systems [43, 44], such as network and signal processing [4552].

Conflicts of Interest

The author declares that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

The author is grateful to her supervisor Professor Feng Ding at the Jiangnan University for his helpful suggestions and the main idea of this work comes from him and his book Multi-Innovation Identification Theory and Methods, Beijing: Science Press, 2016. This work was supported by the Natural Science Foundation of Shandong Province (China, ZR2016FL08) and the Science Foundation of Jining University (China, 2016QNKJ01).

References

  1. L. Xu, “A proportional differential control method for a time-delay system using the Taylor expansion approximation,” Applied Mathematics and Computation, vol. 236, pp. 391–399, 2014. View at: Publisher Site | Google Scholar | MathSciNet
  2. L. Xu, L. Chen, and W. Xiong, “Parameter estimation and controller design for dynamic systems from the step responses based on the Newton iteration,” Nonlinear Dynamics, vol. 79, no. 3, pp. 2155–2163, 2015. View at: Publisher Site | Google Scholar | MathSciNet
  3. K. Rajarathinam, J. B. Gomm, D.-L. Yu, and A. S. Abdelhadi, “PID controller tuning for a multivariable glass furnace process by genetic algorithm,” International Journal of Automation and Computing, vol. 13, no. 1, pp. 64–72, 2016. View at: Publisher Site | Google Scholar
  4. D. Zumoffen, M. Gonzalo, and M. Basualdo, “Improvements on multivariable control strategies tested on the Petlyuk distillation column,” Chemical Engineering Science, vol. 93, pp. 292–306, 2013. View at: Publisher Site | Google Scholar
  5. J. Na, M. N. Mahyuddin, G. Herrmann, X. Ren, and P. Barber, “Robust adaptive finite-time parameter estimation and control for robotic systems,” International Journal of Robust and Nonlinear Control, vol. 25, no. 16, pp. 3045–3071, 2015. View at: Publisher Site | Google Scholar | MathSciNet
  6. H. Li, Y. Shi, and W. Yan, “On Neighbor Information Utilization in Distributed Receding Horizon Control for Consensus-Seeking,” IEEE Transactions on Cybernetics, vol. 46, no. 9, pp. 2019–2027, 2016. View at: Publisher Site | Google Scholar
  7. L. Feng, M. Wu, Q. Li et al., “Array Factor Forming for Image Reconstruction of One-Dimensional Nonuniform Aperture Synthesis Radiometers,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 2, pp. 237–241, 2016. View at: Publisher Site | Google Scholar
  8. G. Gu, S. Wan, and L. Qiu, “Networked stabilization for multi-input systems over quantized fading channels,” Automatica, vol. 61, pp. 1–8, 2015. View at: Publisher Site | Google Scholar | MathSciNet
  9. A. Cristofaro and S. Pettinari, “Fault accommodation for multi-input linear sampled-data systems,” International Journal of Adaptive Control and Signal Processing, vol. 29, no. 7, pp. 835–854, 2015. View at: Publisher Site | Google Scholar | MathSciNet
  10. Y. Ji and X. Liu, “Unified synchronization criteria for hybrid switching-impulsive dynamical networks,” Circuits, Systems and Signal Processing, vol. 34, no. 5, pp. 1499–1517, 2015. View at: Publisher Site | Google Scholar | MathSciNet
  11. L. Jia, X. Li, and M.-S. Chiu, “Correlation analysis based MIMO neuro-fuzzy Hammerstein model with noises,” Journal of Process Control, vol. 41, pp. 76–91, 2016. View at: Publisher Site | Google Scholar
  12. F. Ding, F. Wang, L. Xu, and M. Wu, “Decomposition based least squares iterative identification algorithm for multivariate pseudo-linear {ARMA} systems using the data filtering,” Journal of The Franklin Institute, vol. 354, no. 3, pp. 1321–1339, 2017. View at: Publisher Site | Google Scholar | MathSciNet
  13. C. Yu and M. Verhaegen, “Blind multivariable {ARMA} subspace identification,” Automatica, vol. 66, pp. 3–14, 2016. View at: Publisher Site | Google Scholar | MathSciNet
  14. G. Mercère and L. Bako, “Parameterization and identification of multivariable state-space systems: a canonical approach,” <