Least-Squares Parameter Estimation Algorithm for a Class of Input Nonlinear Systems

Abstract

This paper studies least-squares parameter estimation algorithms for input nonlinear systems, including the input nonlinear controlled autoregressive (IN-CAR) model and the input nonlinear controlled autoregressive autoregressive moving average (IN-CARARMA) model. The basic idea is to obtain linear-in-parameters models by overparameterizing such nonlinear systems and to use the least-squares algorithm to estimate the unknown parameter vectors. It is proved that the parameter estimates consistently converge to their true values under the persistent excitation condition. A simulation example is provided.

1. Introduction

Parameter estimation has received much attention in many areas such as linear and nonlinear system identification and signal processing [1–9]. Nonlinear systems can be simply divided into the input nonlinear systems, the output nonlinear systems, the feedback nonlinear systems, and the input and output nonlinear systems, and so forth. The Hammerstein models can describe a class of input nonlinear systems which consist of static nonlinear blocks followed by linear dynamical subsystems [10, 11].

Nonlinear systems are common in industrial processes, for example, the dead-zone nonlinearities and the valve saturation nonlinearities. Many estimation methods have been developed to identify the parameters of nonlinear systems, especially for Hammerstein nonlinear systems [12, 13]. For example, Ding et al. presented a least-squares-based iterative algorithm and a recursive extended least squares algorithm for Hammerstein ARMAX systems [14] and an auxiliary model-based recursive least squares algorithm for Hammerstein output error systems [15]. Wang and Ding proposed an extended stochastic gradient identification algorithm for Hammerstein-Wiener ARMAX Systems [16].

Recently, Wang et al. derived an auxiliary model-based recursive generalized least-squares parameter estimation algorithm for Hammerstein output error autoregressive systems and auxiliary model-based RELS and MI-ELS algorithms for Hammerstein output error moving average systems using the key term separation principle [17, 18]. Ding et al. presented a projection estimation algorithm and a stochastic gradient (SG) estimation algorithm for Hammerstein nonlinear systems by using the gradient search and further derived a Newton recursive estimation algorithm and a Newton iterative estimation algorithm by using the Newton method (Newton-Raphson method) [19]. Wang and Ding studied least-squares-based and gradient-based iterative identification methods for Wiener nonlinear systems [20].

Fan et al. discussed the parameter estimation problem for Hammerstein nonlinear ARX models [21]. On the basis of the work in [14, 15, 21], this paper studies the identification problems and their convergence for input nonlinear controlled autoregressive (IN-CAR) models using the martingale convergence theorem and gives the recursive generalized extended least-squares algorithm for input nonlinear controlled autoregressive autoregressive moving average (IN-CARARMA) models.

Briefly, the paper is organized as follows. Section 2 derives a linear-in-parameters identification model and gives a recursive least squares identification algorithm for input nonlinear CAR systems and analyzes the properties of the proposed algorithm. Section 4 gives the recursive generalized extended least squares algorithm for input nonlinear CARARMA systems. Section 5 provides an illustrative example to show the effectiveness of the proposed algorithms. Finally, we offer some concluding remarks in Section 6.

2. The Input Nonlinear CAR Model and Estimation Algorithm

Let us introduce some notations first. The symbol 𝐈(𝐈_𝑛) stands for an identity matrix of appropriate sizes (𝑛×𝑛); the superscript 𝑇 denotes the matrix transpose; 𝟏_𝑛 represents an 𝑛-dimensional column vector whose elements are 1; |𝐗|=det[𝐗] represents the determinant of the matrix 𝐗; the norm of a matrix 𝐗 is defined by ‖𝐗‖²=tr[𝐗𝐗^𝑇]; 𝜆_max[𝐗] and 𝜆_min[𝐗] represent the maximum and minimum eigenvalues of the square matrix 𝐗, respectively; 𝑓(𝑡)=𝑜(𝑔(𝑡)) represents 𝑓(𝑡)/𝑔(𝑡)→0 as 𝑡→∞; for 𝑔(𝑡)⩾0, we write 𝑓(𝑡)=𝑂(𝑔(𝑡)) if there exists a positive constant 𝛿₁ such that |𝑓(𝑡)|⩽𝛿₁𝑔(𝑡).

2.1. The Input Nonlinear CAR Model

Consider the following input nonlinear controlled autoregressive (IN-CAR) systems [14, 21]: 𝐴(𝑧)𝑦(𝑡)=𝐵(𝑧)𝑢(𝑡)+𝑣(𝑡),(2.1) where 𝑦(𝑡) is the system output, 𝑣(𝑡) is a disturbance noise, the output of the nonlinear block 𝑢(𝑡) is a nonlinear function of a known basis (𝑓₁,𝑓₂,…,𝑓_𝑚) of the system input 𝑢(𝑡) [19], 𝑢(𝑡)=𝑓(𝑢(𝑡))=𝑐₁𝑓₁(𝑢(𝑡))+𝑐₂𝑓₂(𝑢(𝑡))+⋯+𝑐_𝑚𝑓_𝑚(𝑢(𝑡)),(2.2)𝐴(𝑧) and 𝐵(𝑧) are polynomials in the unit backward shift operator 𝑧⁻¹ [𝑧⁻¹𝑦(𝑡)=𝑦(𝑡−1)], defined as 𝐵𝐴(𝑧()𝑧∶)=∶=𝑏₁1+𝑎₁𝑧⁻¹+𝑎₂𝑧⁻²+⋯+𝑎_𝑛𝑧^−𝑛,𝑧⁻¹+𝑏₂𝑧⁻²+𝑏₃𝑧⁻³+⋯+𝑏_𝑛𝑧^−𝑛.(2.3) In order to obtain the identifiability of parameters 𝑏_𝑖 and 𝑐_𝑖, without loss of generality, we suppose that 𝑐₁=1 or 𝑏₁=1 [14, 21].

Define the parameter vector 𝝑 and information vector 𝝍(𝑡) as 𝝑∶=[𝐚^𝑇,𝑐₁𝐛^𝑇𝐚,𝑐∶₂=𝐛^𝑇[,𝑎…₁,𝑐_𝑚𝐛^𝑇]^𝑇∈ℝ^𝑛0,𝑛₀∶=𝑛+𝑚𝑛,𝐛∶=[𝑏₁,𝑎₂,…,𝑎_𝑛]^𝑇∈ℝ^𝑛,𝝍(𝑡)∶=[𝝍_𝑇0,𝑏₂,…,𝑏_𝑛]^𝑇∈ℝ^𝑛,𝝍₀(𝑡),𝝍^𝑇1(𝑡),𝝍^𝑇2(𝑡),…,𝝍^𝑇𝑚(𝑡)]^𝑇∈ℝ^𝑛0,𝝍_𝑗(𝑡)∶=[−𝑦(𝑡−1),−𝑦(𝑡−2),…,−𝑦(𝑡−𝑛)]^𝑇∈ℝ^𝑛,(𝑡)∶=𝑓_𝑗(𝑢(𝑡−1)),𝑓_𝑗(𝑢(𝑡−2)),…,𝑓_𝑗(𝑢(𝑡−𝑛))^𝑇∈ℝ^𝑛,𝑗=1,2,…,𝑚.(2.4) From (2.1), we have 𝑦(𝑡)==−[1−_𝑛𝐴(𝑧)]𝑦(𝑡)+𝐵(𝑧)𝑢(𝑡)+𝑣(𝑡)_𝑖=1𝑎_𝑖𝑦(𝑡−𝑖)+^𝑛_𝑖=1𝑏_𝑖𝑚_𝑗=1=−_𝑛𝑐_𝑗𝑓_𝑗(𝑢(𝑡−𝑖))+𝑣(𝑡)_𝑖=1𝑎_𝑖𝑦(𝑡−𝑖)+^𝑚_𝑛𝑗=1_𝑖=1=−_𝑛𝑐_𝑗𝑏_𝑖𝑓_𝑗(𝑢(𝑡−𝑖))+𝑣(𝑡)(2.5)_𝑖=1+𝑐₂𝑎_𝑖𝑦(𝑡−𝑖)+𝑐₁𝑏₁𝑓₁(𝑢(𝑡−1))+𝑐₁𝑏₂𝑓₁(𝑢(𝑡−2))+⋯+𝑐₁𝑏_𝑛𝑓₁(𝑢(𝑡−𝑛))+𝑐_𝑚𝑏₁𝑓₂(𝑢(𝑡−1))+𝑐₂𝑏₂𝑓₂(𝑢(𝑡−2))+⋯+𝑐₂𝑏_𝑛𝑓₂(𝑢(𝑡−𝑛))+⋯=𝝍_𝑇𝑏₁𝑓_𝑚(𝑢(𝑡−1))+𝑐_𝑚𝑏₂𝑓_𝑚(𝑢(𝑡−2))+⋯+𝑐_𝑚𝑏_𝑛𝑓_𝑚(𝑢(𝑡−𝑛))+𝑣(𝑡)(𝑡)𝝑+𝑣(𝑡).(2.6) An alternative way is to define the parameter vector 𝜽 and information vector 𝝋(𝑡) as 𝜽∶=[𝐚𝐚^𝑇∶,=𝑏₁[𝐜𝑎^𝑇₁,𝑏₂𝐜^𝑇,…^𝑇,𝑏_𝑛𝐜^𝑇]^𝑇∈ℝ^𝑛0,𝐜∶=[𝑐₁,𝑎₂,…,𝑎_𝑛]^𝑇∈ℝ^𝑛,𝝋(𝑡)∶=[𝝋_𝑇0,𝑐₂,…,𝑐_𝑚]^𝑇∈ℝ^𝑚,𝝋₀(𝑡),𝝋^𝑇1(𝑡),𝝋^𝑇2(𝑡),…,𝝋^𝑇𝑛(𝑡)]^𝑇∈ℝ^𝑛0,𝝋_𝑗(𝑡)∶=[−𝑦(𝑡−1),−𝑦(𝑡−2),…,−𝑦(𝑡−𝑛)]^𝑇∈ℝ^𝑛,(𝑡)∶=𝑓₁(𝑢(𝑡−𝑗)),𝑓₂(𝑢(𝑡−𝑗)),…,𝑓_𝑚(𝑢(𝑡−𝑗))^𝑇∈ℝ^𝑚,𝑗=1,2,…,𝑛.(2.7) Then (2.5) can be written as 𝑦(𝑡)=−^𝑛_𝑖=1𝑎_𝑖𝑦(𝑡−𝑖)+^𝑛_𝑚𝑖=1_𝑗=1=−_𝑛𝑏_𝑖𝑐_𝑗𝑓_𝑗(𝑢(𝑡−𝑖))_𝑖=1+𝑏₂𝑎_𝑖𝑦(𝑡−𝑖)+𝑏₁𝑐₁𝑓₁𝑢(𝑡−1)+𝑏₁𝑐₂𝑓₂𝑢(𝑡−1)+⋯+𝑏₁𝑐_𝑚𝑓_𝑚𝑢(𝑡−1)+𝑏_𝑛𝑐₁𝑓₁𝑢(𝑡−2)+𝑏₂𝑐₂𝑓₂𝑢(𝑡−2)+⋯+𝑏₂𝑐_𝑚𝑓_𝑚𝑢(𝑡−2)+⋯=𝝋_𝑇𝑐₁𝑓₁𝑢(𝑡−𝑛)+𝑏_𝑛𝑐₂𝑓₂𝑢(𝑡−𝑛)+⋯+𝑏_𝑛𝑐_𝑚𝑓_𝑚𝑢(𝑡−𝑛)+𝑣(𝑡)(𝑡)𝜽+𝑣(𝑡).(2.8) Equations (2.6) and (2.8) are both linear-in-parameters identification model for Hammerstein CAR systems by using parametrization.

2.2. The Recursive Least Squares Algorithm

Minimizing the cost function 𝐽(𝜽)∶=^𝑡_𝑗=1𝑦(𝑗)−𝝋^𝑇(𝑗)𝜽²(2.9) gives the following recursive least squares algorithm for computing the estimate 𝜽(𝑡) of 𝜽 in (2.8): 𝜽𝐏(𝑡)=𝜽(𝑡−1)+𝐏(𝑡)𝝋(𝑡)𝑦(𝑡)−𝝋(𝑡)𝜽(𝑡−1),(2.10)₋₁(𝑡)=𝐏⁻¹(𝑡−1)+𝝋(𝑡)𝝋^𝑇(𝑡),𝐏(0)=𝑝₀𝐈.(2.11) Applying the matrix inversion formula [22] (𝐀+𝐁𝐂)⁻¹=𝐀⁻¹−𝐀⁻¹𝐁𝐈+𝐂𝐀⁻¹𝐁⁻¹𝐂𝐀⁻¹(2.12) to (2.11) and defining the gain vector 𝐋(𝑡)=𝐏(𝑡)𝝋(𝑡)∈ℝ^𝑛0, the algorithm in (2.10)-(2.11) can be equivalently expressed as 𝜽𝐋(𝑡()𝑡)==𝜽𝐏(𝑡(−𝑡)𝝋1)(+𝑡)𝐋=(𝑡)𝑦(𝐏𝑡)(−𝑡−𝝋1()𝑡)𝝋𝜽((𝑡)𝑡−1),𝐏(𝑡)=𝐏(𝑡−1)−𝐏(1𝑡)+𝝋𝝋(𝑡^T)𝝋(𝑡)_𝑇𝐏(𝑡−1)𝝋(𝑡),(𝑡)𝐏(𝑡)=𝐈−𝐋(𝑡)𝝋_𝑇1+𝝋^𝑇(𝑡)𝐏(𝑡−1)𝝋(𝑡)(𝑡)𝐏(𝑡−1),𝐏(0)=𝑝₀𝐈.(2.13) To initialize the algorithm, we take 𝑝₀ to be a large positive real number, for example, 𝑝₀=10⁶, and 𝜽(0) to be some small real vector, for example, 𝜽(0)=10⁻⁶𝟏_𝑛0.

3. The Main Convergence Theorem

The following lemmas are required to establish the main convergence results.

Lemma 3.1 (Martingale convergence theorem: Lemma D.5.3 in [23, 24]). If 𝑇_𝑡, 𝛼_𝑡, 𝛽_𝑡 are nonnegative random variables, measurable with respect to a nondecreasing sequence of 𝜎 algebra ℱ_𝑡−1, and satisfy 𝐸𝑇_𝑡∣ℱ_𝑡−1⩽𝑇_𝑡−1+𝛼_𝑡−𝛽_𝑡,a.s.,(3.1) then when ∑^∞𝑡=1𝛼_𝑡<∞, one has ∑^∞𝑡=1𝛽_𝑡<∞,a.s.and 𝑇_𝑡→𝑇,a.s. (a.s.: almost surely) a finite nonnegative random variable.

Lemma 3.2 (see [14, 21, 25]). For the algorithm in (2.10)-(2.11), for any 𝛾>1, the covariance matrix 𝐏(𝑡) in (2.11) satisfies the following inequality: ^∞_𝑡=1𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)ln||𝐏⁻¹(𝑡)||^𝛾<∞,a.s.(3.2)

Theorem 3.3. For the system in (2.8) and the algorithm in (2.10)-(2.11), assume that {𝑣(𝑡),ℱ_𝑡} is a martingale difference sequence defined on a probability space {Ω,ℱ,𝑃}, where {ℱ_𝑡} is the 𝜎 algebra sequence generated by the observations {𝑦(𝑡),𝑦(𝑡−1),…,𝑢(𝑡),𝑢(𝑡−1),…} and the noise sequence {𝑣(𝑡)} satisfies E[𝑣(𝑡)∣ℱ_𝑡−1]=0,and E[𝑣²(𝑡)∣ℱ_𝑡−1]⩽𝜎²<∞,a.s [23], and [ln|𝐏⁻¹(𝑡)|]^𝛾=𝑜(𝜆_min[𝐏⁻¹(𝑡)]), 𝛾>1. Then the parameter estimation error 𝜽(𝑡) converges to zero.

Proof. Define the parameter estimation error vector 𝜽(𝑡)∶=𝜽(𝑡)−𝜽 and the stochastic Lyapunov function 𝑇(𝑡)∶=𝜽^𝑇(𝑡)𝐏⁻¹(𝑡)𝜽(𝑡). Let 𝑦(̃𝑡)∶=𝝋^𝑇(𝑡)𝜽(𝑡−1)−𝝋^𝑇(𝑡)𝜽=𝝋^𝑇(𝑡)𝜽(𝑡−1). According to the definitions of 𝜽(𝑡) and 𝑇(𝑡) and using (2.10) and (2.11), we have 𝜽(𝑡𝑇)=(𝑡)𝜽=(𝑡𝑇−(𝑡1−)+1)𝐏−(𝑡)1𝝋−(𝑡𝝋)[_𝑇−𝑦̃(𝑡)+𝑣(𝑡)],+21−𝝋_𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)𝑦²̃(𝑡)+𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)𝑣²(𝑡)⩽𝑇(𝑡−1)+𝝋(𝑡_𝑇)𝐏(𝑡)𝝋(𝑡)𝑦̃(𝑡)𝑣(𝑡)(𝑡)𝐏(𝑡)𝝋(𝑡)𝑣²(𝑡)+21−𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)𝑦̃(𝑡)𝑣(𝑡).(3.3) Here, we have used the inequality 1−𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)=[1+𝝋^𝑇(𝑡)𝐏(𝑡−1)𝝋(𝑡)]⁻¹⩾0. Because 𝑦(̃𝑡) and 𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡) are uncorrelated with 𝑣(𝑡) and are ℱ_𝑡−1 measurable, taking the conditional expectation with respect to ℱ_𝑡−1, we have 𝐸𝑇(𝑡)∣𝐹_𝑡−1⩽𝑇(𝑡−1)+2𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)𝜎².(3.4) Since ln|𝐏⁻¹(𝑡)| is nondecreasing, letting 𝑉(𝑡)∶=𝑇(𝑡)ln||𝐏⁻¹(𝑡)||^𝛾,𝛾>1,(3.5) we have 𝐸𝑉(𝑡)∣𝐹_𝑡−1⩽𝑇(𝑡−1)ln||𝐏⁻¹(𝑡)||^𝛾+2𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)ln||𝐏⁻¹(𝑡)||^𝛾𝜎²⩽𝑉(𝑡−1)+2𝝋^𝑇(𝑡)𝐏(𝑡)𝝋(𝑡)ln||𝐏⁻¹(𝑡)||^𝛾𝜎²,a.s.(3.6) Using Lemma 3.2, the sum of the last term in the right-hand side for 𝑡 from 1 to ∞ is finite. Applying Lemma 3.1 to the previous inequality, we conclude that 𝑉(𝑡) converges a.s. to a finite random variable, say 𝑉₀, that is: 𝑉(𝑡)=𝑇(𝑡)ln||𝐏⁻¹(𝑡)||^𝛾⟶𝑉₀<∞,a.s.,or𝑇(𝑡)=𝑂ln||𝐏⁻¹(𝑡)||^𝛾,a.s.(3.7) Thus, according to the definition of 𝑇(𝑡), we have ‖‖𝜽(𝑡)‖‖²⩽tr𝜽^𝑇(𝑡)𝐏⁻¹(𝑡)𝜽(𝑡)𝜆_min𝐏⁻¹(𝑡)=𝑂ln||𝐏⁻¹(𝑡)||^𝛾𝜆_min𝐏⁻¹(𝑡)=𝑂𝑜𝜆^min𝐏⁻¹(𝑡)𝜆_min𝐏⁻¹(𝑡)⟶0,a.s.(3.8) This completes the proof of Theorem 3.3.

According to the definition of 𝜽 and the assumption 𝑏₁=1, the estimates 𝐚̂(𝑡)=[𝑎₁̂(𝑡),𝑎₂̂(𝑡),…,𝑎_𝑛̂(𝑡)]^𝑇 and 𝐜(̂𝑡)=[𝑐̂₁(𝑡), 𝑐̂₂(𝑡), …, 𝑐̂_𝑚(𝑡)]^𝑇 of 𝐚 and 𝐜 can be read from the first 𝑛 and second 𝑚 entries of 𝜽, respectively. Let 𝜃̂_𝑖 be the 𝑖th element of 𝜽. Referring to the definition of 𝜽, the estimates 𝑏̂_𝑗(𝑡) of 𝑏_𝑗, 𝑗=2,3,…,𝑛, may be computed by 𝑏̂_𝑗(𝑡)=𝜃̂^{𝑛+(𝑗−1)𝑚+𝑖}(𝑡)𝑐̂_𝑖(𝑡),𝑗=2,3,…,𝑛;𝑖=1,2,…,𝑚.(3.9) Notice that there is a large amount of redundancy about 𝑏̂_𝑗(𝑡) for each 𝑖=1,2,…,𝑚. Since we do not need such 𝑚 estimates 𝑏̂_𝑗(𝑡), one way is to take their average as the estimate of 𝑏_𝑗 [14], that is: 𝑏̂_𝑗(𝑡)=1𝑚^𝑚_𝑖=1𝜃̂_{𝑛+(𝑗−1)𝑚+𝑖}(𝑡)𝑐̂_𝑖(𝑡),𝑗=2,3,…,𝑛.(3.10)

4. The Input Nonlinear CARARMA System and Estimation Algorithm

Consider the following input nonlinear controlled autoregressive autoregressive moving average (IN-CARARMA) systems: 𝐴(𝑧)𝑦(𝑡)=𝐵(𝑧)𝑢(𝑡)+𝐷(𝑧)𝛾(𝑧)𝑣(𝑡),(4.1)𝑢(𝑡)=𝑓(𝛾𝑢((𝑧𝑡)))∶==𝑐1₁𝑓+₁(𝛾𝑢₁(𝑡))+𝑐₂𝑓₂(𝑢(𝑡))+⋯+𝑐_𝑚𝑓_𝑚(𝑢(𝑡)),𝐷(𝑧)∶=1+𝑑₁𝑧⁻¹+𝛾₂𝑧⁻²+⋯+𝛾_𝑛𝛾𝑧^−𝑛𝛾,𝑧⁻¹+𝑑₂𝑧⁻²+⋯+𝑑_𝑛𝑑𝑧^−𝑛𝑑.(4.2) Let 𝑤(𝑡)∶=𝐷(𝑧)𝛾(𝑧)𝑣(𝑡),(4.3) or 𝑤(𝑡)==[−1−_𝑛𝛾𝛾(𝑧)]𝑤(𝑡)+𝐷(𝑧)𝑣(𝑡)_𝑖=1𝛾_𝑖𝑤(𝑡−𝑖)+^𝑛𝑑_𝑖=1𝑑_𝑖𝑣(𝑡−𝑖)+𝑣(𝑡).(4.4) Define the parameter vector 𝜽 and information vector 𝝋(𝑡) as 𝝋(𝑡)∶=[𝝋_𝑇1𝜽∶=[𝜽^𝑇1,𝛾₁,𝛾₂,…,𝛾_𝑛𝛾,𝑑₁,𝑑₂,…,𝑑_𝑛𝑑]^𝑇∈ℝ^{𝑛+𝑚𝑛+𝑛𝛾+𝑛𝑑},(𝑡),−𝑤(𝑡−1),−𝑤(𝑡−2)…,−𝑤𝑡−𝑛_𝛾,𝑣(𝑡−1),𝑣(𝑡−2),…,𝑣𝑡−𝑛_𝑑]^𝑇∈ℝ^{𝑛+𝑚𝑛+𝑛𝛾+𝑛𝑑},𝜽₁∶=⎡⎢⎢⎢⎢⎢⎢⎢⎢⎣𝑏𝐚¹𝑏₂𝐜𝑏⋮_𝑛𝐜𝐜⎤⎥⎥⎥⎥⎥⎥⎥⎥⎦∈ℝ^{𝑛+𝑛𝑚},𝝋₁(𝑡)∶=⎡⎢⎢⎢⎢⎢⎢⎢⎢⎣𝝋⁰𝝋(𝑡)₁𝝋(𝑡)₂𝝋⋮(𝑡)_𝑛(𝑡)⎤⎥⎥⎥⎥⎥⎥⎥⎥⎦∈ℝ^{𝑛+𝑛𝑚}𝐚∶=⎡⎢⎢⎢⎢⎢⎢⎣𝑎₁,𝑎₂𝑎⋮_𝑛⎤⎥⎥⎥⎥⎥⎥⎦∈ℝ_𝑛,𝐜∶=⎡⎢⎢⎢⎢⎢⎢⎣𝑐¹𝑐₂𝑐⋮_𝑚⎤⎥⎥⎥⎥⎥⎥⎦∈ℝ_𝑚𝝋₀,(𝑡)∶=⎡⎢⎢⎢⎢⎢⎢⎣−−−𝑦𝑦𝑦(((⋮𝑡𝑡𝑡−−−12𝑛)))⎤⎥⎥⎥⎥⎥⎥⎦∈ℝ^𝑛,𝝋_𝑗(𝑡)∶=⎡⎢⎢⎢⎢⎢⎢⎣𝑓¹𝑓(𝑢(𝑡−𝑗))₂𝑓(𝑢(⋮𝑡−𝑗))_𝑚(𝑢(𝑡−𝑗))⎤⎥⎥⎥⎥⎥⎥⎦∈ℝ^𝑚,𝑗=1,2,…,𝑛.(4.5) Then (4.1) can be written as 𝑦(𝑡)==[−1−_𝑛𝐴(𝑧)]𝑦(𝑡)+𝐵(𝑧)𝑢(𝑡)+𝑤(𝑡)_𝑖=1𝑎_𝑖𝑦(𝑡−𝑖)+^𝑛_𝑖=1𝑏_𝑖𝑚_𝑗=1=−_𝑛𝑐_𝑗𝑓_𝑗(𝑢(𝑡−𝑖))+𝑤(𝑡)_𝑖=1𝑎_𝑖𝑦(𝑡−𝑖)+^𝑛_𝑚𝑖=1_𝑗=1=𝝋_𝑇1𝑏_𝑖𝑐_𝑗𝑓_𝑗(𝑢(𝑡−𝑖))+𝑤(𝑡)=𝝋_𝑇1(𝑡)𝜽₁+𝑤(𝑡)(𝑡)𝜽₁−^𝑛𝛾_𝑖=1𝛾_𝑖𝑤(𝑡−𝑖)+^𝑛𝑑_𝑖=1=𝝋_𝑇𝑑_𝑖𝑣(𝑡−𝑖)+𝑣(𝑡)(𝑡)𝜽+𝑣(𝑡).(4.6) This is a linear-in-parameter identification model for IN-CARARMA systems.

The unknown 𝑤(𝑡−𝑖) and 𝑣(𝑡−𝑖) in the information vector 𝝋(𝑡) are replaced with their estimates 𝑤(𝑡−𝑖) and 𝑣(̂𝑡−𝑖), and then we can obtain the following recursive generalized extended least squares algorithm for estimating 𝜽 in (4.6): 𝐋𝜽((𝑡)𝑡)==𝐏𝜽((𝑡𝑡−−11))𝝋+(𝑡𝐋)(𝑡1)+𝑦𝝋(𝑡_𝑇)−𝝋^𝑇(𝑡)𝜽(𝑡−1),𝐏(𝑡)=𝐈−𝐋(𝑡)𝝋_𝑇(𝑡)𝐏(𝑡−1)𝝋(𝑡)⁻¹,𝝋(𝑡)=[𝝋_𝑇1(𝑡)𝐏(𝑡−1),𝐏(0)=𝑝₀𝐈,𝝋(₁𝑡),−𝑤(𝑡−1),−𝑤(𝑡−2),…,−𝑤𝑡−𝑛_𝛾,𝑣̂(𝑡−1),𝑣̂(𝑡−2),…,𝑣̂𝑡−𝑛_𝑑]^𝑇,(𝑡)=⎡⎢⎢⎢⎢⎢⎢⎢⎢⎣𝝋⁰𝝋(𝑡)₁𝝋(𝑡)₂𝝋⋮(𝑡)_𝑛(𝑡)⎤⎥⎥⎥⎥⎥⎥⎥⎥⎦,𝝋⁰(𝑡)=⎡⎢⎢⎢⎢⎢⎢⎣−−−𝑦𝑦𝑦(((⋮𝑡𝑡𝑡−−−12𝑛)))⎤⎥⎥⎥⎥⎥⎥⎦,𝝋_𝑗(𝑡)=⎡⎢⎢⎢⎢⎢⎢⎣𝑓¹𝑓(𝑢(𝑡−𝑗))₂𝑓(𝑢(⋮𝑡−𝑗))_𝑚𝑤(𝑡)=𝑦(𝑡)−𝝋_𝑇1(𝑢(𝑡−𝑗))⎤⎥⎥⎥⎥⎥⎥⎦,𝑣̂(𝑡)=𝑦(𝑡)−𝝋_𝑇(𝑡)𝜽₁(𝑡),𝜽(𝑡)=[𝜽_𝑇1(𝑡)𝜽(𝑡),(𝑡),𝛾̂₁(𝑡),𝛾̂₂(𝑡),…,𝛾̂_𝑛𝛾(𝑡),𝑑₁(𝑡),𝑑₂(𝑡),𝑑_𝑛𝑑(𝑡)]^𝑇.(4.7)

This paper presents a recursive least squares algorithm for IN-CAR systems and a recursive generalized extended least squares algorithm for IN-CARARMA systems with ARMA noise disturbances, which differ not only from the input nonlinear controlled autoregressive moving average (IN-CARMA) systems in [14] but also from the input nonlinear output error systems in [15].

5. Example

Consider the following IN-CAR system: 𝐴(𝑧)=1+𝐴𝑎(₁𝑧)𝑦(𝑡)=𝐵(𝑧)𝑢(𝑡)+𝑣(𝑡),𝐵(𝑧)=𝑧𝑏⁻¹₁+𝑎₂𝑧⁻²=1−1.35𝑧⁻¹+0.75𝑧⁻²,𝑧⁻¹+𝑏₂𝑧⁻²=𝑧⁻¹+1.68𝑧⁻²,𝑢(𝑡)==𝑢𝑓((𝑡𝑢)(+𝑡)0).=50𝑐𝑢₁₂𝑢(𝑡)+𝑐₂𝑢²(𝑡)+𝑐₃𝑢³(𝑡)𝜽=𝜃₁(𝑡)+0.20𝑢³(𝑡),,𝜃₂,𝜃₃,𝜃₄,𝜃₅,𝜃₆,𝜃₇,𝜃₈^𝑇=𝑎₁,𝑎₂,𝑐₁,𝑐₂,𝑐₃,𝑏₂𝑐₁,𝑏₂𝑐₂,𝑏₂𝑐₃^𝑇𝜽_𝑠=[−1.350,0.75,1.00,0.50,0.20,1.68,0.84,0.336]^𝑇,=𝑎₁,𝑎₂,𝑏₂,𝑐₁,𝑐₂,𝑐₃^𝑇=[−1.35,0.75,1.68,1.00,0.50,0.20]^𝑇.(5.1) In simulation, the input {𝑢(𝑡)} is taken as a persistent excitation signal sequence with zero mean and unit variance and {𝑣(𝑡)} as a white noise sequence with zero mean and constant variance 𝜎². Applying the proposed algorithm in (2.10)-(2.11) to estimate the parameters of this system, the parameter estimates 𝜽 and 𝜽_𝑠 and their errors with different noise variances are shown in Tables 1, 2, 3, and 4, and the parameter estimation errors 𝛿∶=‖𝜽(𝑡)−𝜽‖/‖𝜽‖ and 𝛿_𝑠∶=‖𝜽_𝑠(𝑡)−𝜽‖/‖𝜽_𝑠‖ versus 𝑡 are shown in Figures 1 and 2. When 𝜎²=0.50² and 𝜎²=1.50², the corresponding noise-to-signal ratios are 𝛿_ns=10.96% and 𝛿_ns=32.87%, respectively.

Table 1 

The parameter estimates (𝜽) (𝜎²=0.50², 𝛿_ns=10.96%).

Table 2 

The parameter estimates (𝜽_𝑠) (𝜎² = 0.50², 𝛿_ns = 10.96%).

Table 3 

The parameter estimates (𝜽) (𝜎² = 1.50², 𝛿_ns = 32.87%).

Table 4 

The parameter estimates (𝜽_𝑠) (𝜎² = 1.50², 𝛿_ns = 32.87%).

Figure 1 

The parameter estimation errors 𝛿 versus 𝑡.

Figure 2 

The parameter estimation errors 𝛿𝑠 versus 𝑡.

From Tables 1–4 and Figures 1 and 2, we can draw the following conclusions.(i)The larger the data length is, the smaller the parameter estimation errors become. (ii)A lower noise level leads to smaller parameter estimation errors for the same data length. (iii)The estimation errors 𝛿 and 𝛿_𝑠 become smaller (in general) as 𝑡 increases. This confirms the proposed theorem.

6. Conclusions

The recursive least-squares identification is used to estimate the unknown parameters for input nonlinear CAR and CARARMA systems. The analysis using the martingale convergence theorem indicates that the proposed recursive least squares algorithm can give consistent parameter estimation. It is worth pointing out that the multi-innovation identification theory [26–33], the gradient-based or least-squares-based identification methods [34–41], and other identification methods [42–49] can be used to study identification problem of this class of nonlinear systems with colored noises.

Acknowledgment

This work was supported by the 111 Project (B12018).