Instrumental Variable-Based OMP Identification Algorithm for Hammerstein Systems

Zhang, Shuo; Wang, Dongqing; Yan, Yaru

doi:https://doi.org/10.1155/2018/8420426

Complexity

On this page

Abstract Introduction Examples Conclusions Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Bio-Inspired Learning and Adaptation for Optimization and Control of Complex Systems

View this Special Issue

Research Article | Open Access

Volume 2018 | Article ID 8420426 | https://doi.org/10.1155/2018/8420426

Instrumental Variable-Based OMP Identification Algorithm for Hammerstein Systems

Shuo Zhang,¹Dongqing Wang,^1,2and Yaru Yan¹

Academic Editor: Jing Na

Received31 Mar 2018

Accepted06 May 2018

Published22 Jul 2018

Abstract

Hammerstein systems are formed by a static nonlinear block followed by a dynamic linear block. To solve the parameterizing difficulty caused by parameter coupling between the nonlinear part and the linear part in a Hammerstein system, an instrumental variable method is studied to parameterize the Hammerstein system. To achieve in simultaneously identifying parameters and orders of the Hammerstein system and to promote the computational efficiency of the identification algorithm, a sparsity-seeking orthogonal matching pursuit (OMP) optimization method of compressive sensing is extended to identify parameters and orders of the Hammerstein system. The idea is, by the filtering technique and the instrumental variable method, to transform the Hammerstein system into a simple form with a separated nonlinear expression and to parameterize the system into an autoregressive model, then to perform an instrumental variable-based orthogonal matching pursuit (IV-OMP) identification method for the Hammerstein system. Simulation results illustrate that the investigated method is effective and has advantages of simplicity and efficiency.

1. Introduction

Nonlinear system modeling and identification are very important in theory and application [1–6]; block-oriented nonlinear systems, which combine nonlinear and linear blocks in various styles, are the typical representation of nonlinear systems. Hammerstein systems, a static nonlinear block plus a dynamic linear block, are a type of commonly used block-oriented nonlinear systems, and a lot of work has contributed in identification methods for Hammerstein systems. The traditional identification methods for Hammerstein systems mainly include the overparameterization model-based methods [7–10] and the recursive/iterative identification methods [11–14]. The highly efficient identification methods include the key-term separation principle-based identification methods [15–17], the hierarchical identification methods [18, 19], the filtering technique based-identification methods [20–22], the maximum likelihood estimation methods [23–25], and the evolution optimization methods [26–28]. Recently, Chen et al. investigated a particle swarm optimization method [29], Krishnanathan et al. discussed a continuous-time nonlinear systems using approximate Bayesian computation [30], and Wang et al. studied a model recovery for Hammerstein systems using the auxiliary model-based orthogonal matching pursuit method [31] and so on.

It is difficult to parameterize Hammerstein systems due to existing parameter coupling between the nonlinear part and the linear part of Hammerstein systems. In order to solve this problem, an instrumental variable-based method is studied to parameterize the Hammerstein systems. Further, to achieve in simultaneously identifying parameters and orders and to promote the computational efficiency of the estimated method, a sparsity-seeking orthogonal matching pursuit optimization method of compressive sensing is extended to identify parameters and orders of the Hammerstein systems. The idea is, by the filtering technique and the instrumental variable method, to transform the Hammerstein system into a simple form with a separated nonlinear expression, so as to easily parameterize the system into autoregressive form, and to perform the instrumental variable-based orthogonal matching pursuit (IV-OMP) identification method for the Hammerstein systems.

In the compressive sensing theory [32–34], there mainly exist two sparse signal recovery methods: the orthogonal matching pursuit algorithms and the basis pursuit algorithms. The orthogonal matching pursuit (OMP) algorithm is a kind of greedy parameter recovering method [35–37], it selects the best fitting column of the measurement matrix and the corresponding sparse signal in each selected step. Due to the selection being orthogonal, the OMP algorithm has a lower computational complexity compared with the basis pursuit algorithms [38–40].

Recently, Mao et al. investigated parameter estimation algorithms for Hammerstein time-delay systems based on the OMP scheme to estimate the system parameters and the time delay [41] by means of the compressed sensing recovery theory and the auxiliary model identification idea. In this paper, an instrumental variable-based orthogonal matching pursuit (IV-OMP) algorithm is investigated to simultaneously estimate the orders and parameters of a Hammerstein system. The contributions lie in four aspects: (i)To solve the parameterizing difficulty caused by the coupling between the nonlinear part and the linear part in Hammerstein systems, a filtering technique-based instrumental variable method is studied to parameterize the Hammerstein systems.(ii)To achieve in simultaneously identifying parameters and orders and to promote the computational efficiency of the system, an instrumental variable-based orthogonal matching pursuit (IV-OMP) optimization method of compressive sensing is extended to identify parameters and orders of the systems.(iii)The advantage of the proposed IV-OMP method over the traditional methods is that it is not necessary to collect a lot of data and invest a lot of power into the parameter identification based on the sparse principle.(iv)Simulation results illustrate that the investigated method is effective and has advantages of simplicity and efficiency.

The rest of the paper is organized as follows. Section 2 demonstrates the identification problem of a Hammerstein system. Section 3 studies the parameterizing method of the Hammerstein system. Section 4 presents the IV-OMP identification algorithm by using the instrumental variables. Section 5 provides a numerical example for the proposed method. Finally, the concluding remarks are involved in Section 6.

2. The Problem Formulation

For the narrative convenience, we define some notation. “” represents “ is defined as ”; represents a unit forward shift operator: and stand for the estimate of at time . The input nonlinear and output linear functions of a Hammerstein CARMA system in Figure 1 are expressed as where and are the system input and output, is an internal variable, and is stochastic white noise with zero mean; the input nonlinearity is modeled as a linear combination of basis functions ; is the number of the basis functions; the linear block is a CARMA model; , , and are polynomials in the unit backward shift operator and defined by

If the order is known and the orders , , and are unknown, then we face two parameterizing problems for the Hammerstein system: (i)The coupling between the nonlinear part and the linear part in a Hammerstein system makes the parametrization of the Hammerstein system more difficult.(ii)Simultaneous and efficient identification of system parameters and orders is difficult to be achieved, due to existing unknown orders.

3. The Parametrization of The Hammerstein CARMA System

Divide both sides of (2) by the filtering function , we get

Define the instrumental variables and as

Then, (4) can be written as

Replacing the expressions of and and and into (5) and (6), respectively, we can get

Substituting and in (8) and (9) and in (1) into (7) gives

Since the orders , , and are unknown, we assume a sufficient length of order for , , and (). Then, (10) can be rewritten as

Define the information vectors , , and as and the parameter vectors , and , as

Then, (11) can be written as

Sampling sets of data and substituting them into (14) get

Define the accumulated information vectors and the matrix, then (15) can be described as

If there are enough measurements, that is, reaches several thousands, we can get the least squares estimate of , as follows:

But with an assumed big order length (≥5) and , the above least squares algorithm leads a big computational burden and is not suitable for solving system parameters and orders on line. To avoid this problem and get an efficient identification algorithm, we choose a more advantageous method over the traditional least squares method; the aim is to identify the parameter vector with less observations () by using the OMP theory based on the mentioned instrumental variable method.

According to the CS theory, we can regard the parameter vector as a sparse signal. Let be the number of the nonzero entries in , that is, the sparsity level of , then the identification problem can be described as an orthogonal matching pursuit (OMP) method: where is the estimate of and is the error tolerance.

4. The IV-OMP Identification Algorithm

For the sparse parameter vector , if , there will be nonzero scalar components in . What we should do in this identification method is to pick out and recover valid data in from by using the OMP theory.

Define as the th column vector of (a is also called an atom), and as the th element of , that is,

Then, the vector in (17) can be written as

Obviously, the output vector contains a linear combination of all atoms . The main idea is to find the nonzero items and corresponding valid atoms at the right-hand side of (21).

To describe the IV-OMP recovery algorithm, we need to give some notations. Let be the iterative number and be the index of the solution support in at the th iteration; is a set composed of , ; is the submeasurement matrix composed by the columns of indexed by ; denotes the residual at the th iteration; and is the parameter estimation at the th iteration. The algorithm is initialized as and .

Define a cost function at the th iteration

Remark 1. Minimizing with respect to means that the derivation of with respect to is equal to 0, that is, and we get Substitute the above into (22) to get a minimized as

Remark 2. The result says that minimizing is equivalent to the quest for the largest inner product between the residual and the normalized column vector of .

The next step is to find the index and the column corresponding to the largest inner product between the residual and the normalized column vectors and assign them into and at the th step:

Remark 3. Minimizing the least squares cost function is equal to taking the derivation of with respect to It means that the residual is orthogonal to the subinformation matrix . Therefore, the index corresponding to the largest inner product between the residual and the normalized column vector is computed by

Update the support set and the subinformation matrix by

With the obtained index set and the corresponding subinformation matrix , the least squares estimation for the nonzero parameters at the th iteration can be achieved. Define a cost function:

Minimizing should get the least squares estimates of the nonzero parameters. But there exist the unknown instrumental variables and and the unmeasurable noise term in the information vector of ; the parameter vector cannot be estimated by using the standard OMP methods in the CS theory. The method is to use their estimates , , and to replace them; the computation of the estimates , , and is as follows: (i)Expand the orders , , and in (1), (2), (8), and (9).(ii) is computed by replacing and with their estimates ^and in (8).(iii) is computed by replacing , , , and with their estimates , , , and in (2) with (1) being inserted.(iv) is computed by replacing and with their estimates and in (9). The results are and the estimates of , , and at iteration can be written as

With replacing the above unknown variables with their estimates, the least squares estimate for the parameter vector at the step is given:

Remark 4. At the beginning, the IV-OMP algorithm with the estimates , , and in the subinformation matrix causes an inaccurate support atom selecting. With the iteration increasing, these estimated variables become more accurate, the misselected support atoms certainly unmeet the threshold requirement, and the corresponding element in the estimated parameter support set will be a small nonzero value. Thus, we set an appropriate small threshold to filter the parameter estimate . If (where is the th element of ), eliminate from and the corresponding from Because is the th element in and is the th column in , the expressions of eliminating from and from in MATLAB are Then we have

Briefly, the proposed IV-OMP algorithm can be implemented as in Algorithm 1.

IV-OMP for CS Recovery
Measurement matrix: .
output vector: , sparsity level: K,
parameter vector
Initialization: repeat

Form by Equation (32);

= index of the highest amplitude component of


If let to get .
and let to get

let to get







untilk = a fixed positive integer s,.t. .
Estimated parameter vector: and with

5. Examples

Consider the following Hammerstein CARMA system

In the simulation, the input is taken as an uncorrelated persistently excited signal vector sequence with zero mean and unit variance, and is taken as a white noise sequence with zero mean and variance . Applying the proposed IV-OMP algorithm to estimate the parameters and the orders of this system, the parameter estimates and their errors are shown in Table 1 and Figures 2 and 3. The relative error of the parameters is

From Table 1 and Figures 2–3, we can get the following: (1)The parameter estimation errors become (generally) smaller and smaller with the iteration increasing.(2)There exist 3 wrongly selected atoms corresponding to parameters , , and (black-colored lines in Figure 3). With the iteration increasing, the wrongly chosen atoms are deleted, and the parameter estimation errors become smaller.

6. Conclusions

This paper is aimed at exploring the identification of both the parameters and orders of the Hammerstein CARMA system with limited sampling data. To solve the parameterizing difficulty caused by parameter coupling between the nonlinear part and the linear part in a Hammerstein system, firstly, by filtering the equation of the linear block with the coefficient function of the controlled term, we separate the parameter coupling between the linear block and the nonlinear block. Moreover, by using two instrumental variables, the Hammerstein system is parameterized into an autoregressive form. To achieve in simultaneously identifying parameters and orders and to promote the computational efficiency of the identification algorithm, an instrumental variable-based orthogonal matching pursuit (IV-OMP) optimization method of compressive sensing is extended to identify parameters and orders of the Hammerstein system. Simulation results illustrate that the investigated method is effective and has advantages of simplicity and efficiency. The proposed IV-OMP optimization method can be extended to the colored noise systems, the networked dynamic systems [42–46], and so on.

Data Availability

The data that support the findings of this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (no. 61573205) and the Shandong Provincial Natural Science Foundation of China (no. ZR2015FM017).

References

G.-y. Chen, M. Gan, and G.-l. Chen, “Generalized exponential autoregressive models for nonlinear time series: stationarity, estimation and applications,” Information Sciences, vol. 438, pp. 46–57, 2018.
View at: Publisher Site | Google Scholar
J. Na, J. Yang, X. Ren, and Y. Guo, “Robust adaptive estimation of nonlinear system with time-varying parameters,” International Journal of Adaptive Control and Signal Processing, vol. 29, no. 8, pp. 1055–1072, 2015.
View at: Publisher Site | Google Scholar
A. Cordero, E. Gómez, and J. R. Torregrosa, “Efficient high-order iterative methods for solving nonlinear systems and their application on heat conduction problems,” Complexity, vol. 2017, Article ID 6457532, 11 pages, 2017.
View at: Publisher Site | Google Scholar
M. Gan, C. L. P. Chen, G.-Y. Chen, and L. Chen, “On some separated algorithms for separable nonlinear least squares problems,” IEEE Transactions on Cybernetics, vol. PP, pp. 1–9, 2017.
View at: Publisher Site | Google Scholar
J. Zambrano, J. Sanchis, J. M. Herrero, and M. Martínez, “WH-EA: an evolutionary algorithm for Wiener-Hammerstein system identification,” Complexity, vol. 2018, Article ID 1753262, 17 pages, 2018.
View at: Publisher Site | Google Scholar
J. Na, J. Yang, X. Wu, and Y. Guo, “Robust adaptive parameter estimation of sinusoidal signals,” Automatica, vol. 53, pp. 376–384, 2015.
View at: Publisher Site | Google Scholar
Y. Tang, Z. Han, Y. Wang, L. Zhang, and Q. Lian, “A changing forgetting factor RLS for online identification of nonlinear systems based on ELM-Hammerstein model,” Neural Computing and Applications, vol. 28, Supplement 1, pp. 813–827, 2017.
View at: Publisher Site | Google Scholar
H. Salhi and S. Kamoun, “A recursive parametric estimation algorithm of multivariable nonlinear systems described by Hammerstein mathematical models,” Applied Mathematical Modelling, vol. 39, no. 16, pp. 4951–4962, 2015.
View at: Publisher Site | Google Scholar
M. Jafari, M. Salimifard, and M. Dehghani, “Identification of multivariable nonlinear systems in the presence of colored noises using iterative hierarchical least squares algorithm,” ISA Transactions, vol. 53, no. 4, pp. 1243–1252, 2014.
View at: Publisher Site | Google Scholar
J. Chen, Y. Liu, and X. Wang, “Recursive least squares algorithm for nonlinear dual-rate systems using missing-output estimation model,” Circuits, Systems, and Signal Processing, vol. 36, no. 4, pp. 1406–1425, 2017.
View at: Publisher Site | Google Scholar
J. Zhang, F. Yu, and K. S. Chin, “Recursive identification of Hammerstein systems with dead-zone nonlinearity in the presence of bounded noise,” International Journal of Systems Science, vol. 48, no. 11, pp. 2394–2404, 2017.
View at: Publisher Site | Google Scholar
P. Mattsson and T. Wigren, “Convergence analysis for recursive Hammerstein identification,” Automatica, vol. 71, pp. 179–186, 2016.
View at: Publisher Site | Google Scholar
E. W. Bai and K. Li, “Convergence of the iterative algorithm for a general Hammerstein system identification,” Automatica, vol. 46, no. 11, pp. 1891–1896, 2010.
View at: Publisher Site | Google Scholar
G. Li, C. Wen, W. X. Zheng, and G. Zhao, “Iterative identification of block-oriented nonlinear systems based on biconvex optimization,” Systems & Control Letters, vol. 79, pp. 68–75, 2015.
View at: Publisher Site | Google Scholar
J. Vörös, “Identification of nonlinear cascade systems with output hysteresis based on the key term separation principle,” Applied Mathematical Modelling, vol. 39, no. 18, pp. 5531–5539, 2015.
View at: Publisher Site | Google Scholar
J. Vörös, “Iterative identification of nonlinear dynamic systems with output backlash using three-block cascade models,” Nonlinear Dynamics, vol. 79, no. 3, pp. 2187–2195, 2015.
View at: Publisher Site | Google Scholar
Y. Tan, R. Dong, and R. Li, “Recursive identification of sandwich systems with dead zone and application,” IEEE Transactions on Control Systems Technology, vol. 17, no. 4, pp. 945–951, 2009.
View at: Publisher Site | Google Scholar
Z. Wang, Y. Wang, and Z. Ji, “Hierarchical recursive least squares algorithm for Hammerstein systems using the filtering method,” Nonlinear Dynamics, vol. 77, no. 4, pp. 1773–1781, 2014.
View at: Publisher Site | Google Scholar
D. Wang, “Hierarchical parameter estimation for a class of MIMO Hammerstein systems based on the reframed models,” Applied Mathematics Letters, vol. 57, no. 3, pp. 13–19, 2016.
View at: Publisher Site | Google Scholar
F. Ding, L. Xu, F. E. Alsaadi, and T. Hayat, “Iterative parameter identification for pseudo-linear systems with ARMA noise using the filtering technique,” IET Control Theory and Applications, vol. 12, no. 7, pp. 892–899, 2018.
View at: Publisher Site | Google Scholar
C. Wang and T. Tang, “Several gradient-based iterative estimation algorithms for a class of nonlinear systems using the filtering technique,” Nonlinear Dynamics, vol. 77, no. 3, pp. 769–780, 2014.
View at: Publisher Site | Google Scholar
D. Wang, F. Ding, and Y. Chu, “Data filtering based recursive least squares algorithm for Hammerstein systems using the key-term separation principle,” Information Sciences, vol. 222, no. 4, pp. 203–212, 2013.
View at: Publisher Site | Google Scholar
J. Li, W. X. Zheng, J. Gu, and L. Hua, “Parameter estimation algorithms for Hammerstein output error systems using Levenberg–Marquardt optimization method with varying interval measurements,” Journal of the Franklin Institute, vol. 354, no. 1, pp. 316–331, 2017.
View at: Publisher Site | Google Scholar
D. Q. Wang, Z. Zhang, and J. Y. Yuan, “Maximum likelihood estimation method for dual-rate Hammerstein systems,” International Journal of Control, Automation and Systems, vol. 15, no. 2, pp. 698–705, 2017.
View at: Publisher Site | Google Scholar
L. Ma and X. Liu, “Recursive maximum likelihood method for the identification of Hammerstein ARMAX system,” Applied Mathematical Modelling, vol. 40, no. 13-14, pp. 6523–6535, 2016.
View at: Publisher Site | Google Scholar
S. Mete, S. Ozer, and H. Zorlu, “System identification using Hammerstein model optimized with differential evolution algorithm,” AEU - International Journal of Electronics and Communications, vol. 70, no. 12, pp. 1667–1675, 2016.
View at: Publisher Site | Google Scholar
E. Cuevas, P. Díaz, O. Avalos, D. Zaldívar, and M. Pérez-Cisneros, “Nonlinear system identification based on ANFIS-Hammerstein model using gravitational search algorithm,” Applied Intelligence, vol. 48, no. 1, pp. 182–203, 2018.
View at: Publisher Site | Google Scholar
M. Schoukens and K. Worden, “Evolutionary identification of block-structured systems,” Dynamics of Coupled Structures, vol. 4, pp. 359–366, 2017.
View at: Publisher Site | Google Scholar
J. Chen, J. Li, and Y. Liu, “Gradient iterative algorithm for dual-rate nonlinear systems based on a novel particle filter,” Journal of the Franklin Institute, vol. 354, no. 11, pp. 4425–4437, 2017.
View at: Publisher Site | Google Scholar
K. Krishnanathan, S. R. Anderson, S. A. Billings, and V. Kadirkamanathan, “Computational system identification of continuous-time nonlinear systems using approximate Bayesian computation,” International Journal of Systems Science, vol. 47, no. 15, pp. 3537–3544, 2015.
View at: Publisher Site | Google Scholar
D. Wang, L. Li, Y. Ji, and Y. Yan, “Model recovery for Hammerstein systems using the auxiliary model based orthogonal matching pursuit method,” Applied Mathematical Modelling, vol. 54, pp. 537–550, 2018.
View at: Publisher Site | Google Scholar
Y. Zhou, S. Kwong, H. Guo, X. Zhang, and Q. Zhang, “A two-phase evolutionary approach for compressive sensing reconstruction,” IEEE Transactions on Cybernetics, vol. 47, no. 9, pp. 2651–2663, 2017.
View at: Publisher Site | Google Scholar
Q. Lin, B. Hu, Y. Tang et al., “A local search enhanced differential evolutionary algorithm for sparse recovery,” Applied Soft Computing, vol. 57, pp. 144–163, 2017.
View at: Publisher Site | Google Scholar
M. Nasiri, B. Minaei, and Z. Sharifi, “Adjusting data sparsity problem using linear algebra and machine learning algorithm,” Applied Soft Computing, vol. 61, pp. 1153–1159, 2017.
View at: Publisher Site | Google Scholar
J. A. Tropp and A. C. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” IEEE Transactions on Information Theory, vol. 53, no. 12, pp. 4655–4666, 2007.
View at: Publisher Site | Google Scholar
H. Rauhut, K. Schnass, and P. Vandergheynst, “Compressed sensing and redundant dictionaries,” IEEE Transactions on Information Theory, vol. 54, no. 5, pp. 2210–2219, 2008.
View at: Publisher Site | Google Scholar
D. Q. Wang, Y. R. Yan, Y. J. Liu, and J. H. Ding, “Model recovery for Hammerstein systems using the hierarchical orthogonal matching pursuit method,” Journal of Computational and Applied Mathematics, vol. 345, pp. 135–145, 2019.
View at: Publisher Site | Google Scholar
S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM Journal on Scientific Computing, vol. 20, no. 1, pp. 33–61, 1998.
View at: Publisher Site | Google Scholar
J. F. C. Mota, J. M. F. Xavier, P. M. Q. Aguiar, and M. Puschel, “Distributed basis pursuit,” IEEE Transactions on Signal Processing, vol. 60, no. 4, pp. 1942–1956, 2012.
View at: Publisher Site | Google Scholar
D. L. Donoho, M. Elad, and V. N. Temlyakov, “Stable recovery of sparse overcomplete representations in the presence of noise,” IEEE Transactions on Information Theory, vol. 52, no. 1, pp. 6–18, 2006.
View at: Publisher Site | Google Scholar
Y. Mao, F. Ding, and Y. Liu, “Parameter estimation algorithms for Hammerstein time-delay systems based on the orthogonal matching pursuit scheme,” IET Signal Processing, vol. 11, no. 3, pp. 265–274, 2017.
View at: Publisher Site | Google Scholar
X. Wang, F. Ding, A. Alsaedi, and T. Hayat, “Filtering based parameter estimation for observer canonical state space systems with colored noise,” Journal of the Franklin Institute, vol. 354, no. 1, pp. 593–609, 2017.
View at: Publisher Site | Google Scholar
J. Ding, “The hierarchical iterative identification algorithm for multi-input-output-error systems with autoregressive noise,” Complexity, vol. 2017, Article ID 5292894, 11 pages, 2017.
View at: Publisher Site | Google Scholar
J. Na, G. Herrmann, and K. Zhang, “Improving transient performance of adaptive control via a modified reference model and novel adaptation,” International Journal of Robust and Nonlinear Control, vol. 27, no. 8, pp. 1351–1372, 2017.
View at: Publisher Site | Google Scholar
H. Liu and D. Wang, “Robust state estimation for wireless sensor networks with data-driven communication,” International Journal of Robust and Nonlinear Control, vol. 27, no. 18, pp. 4622–4632, 2017.
View at: Publisher Site | Google Scholar
H. Liu and H. Yu, “Decentralized state estimation for a large-scale spatially interconnected system,” ISA Transactions, vol. 74, pp. 67–76, 2018.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2018 Shuo Zhang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

887

Downloads

1087

Citations