Abstract
The identification of a class of linear-in-parameters multiple-input single-output systems is considered. By using the iterative search, a least-squares based iterative algorithm and a gradient based iterative algorithm are proposed. A nonlinear example is used to verify the effectiveness of the algorithms, and the simulation results show that the least-squares based iterative algorithm can produce more accurate parameter estimates than the gradient based iterative algorithm.
1. Introduction
Parameter estimation plays an important role in adaptive forecasting [1], system modeling [2–6], system control [7–9], and adaptive control [10–15]. For decades, many identification methods have been developed [16–20], for example, the bias compensation based least-squares methods [21–23] and the iterative identification methods [24–26]. These methods can be used for identifying linear systems and nonlinear systems. In the literature, Ding presented a decomposition based fast least-squares algorithm for output error systems [27]. Recursive algorithms and iterative algorithms are two types of parameter estimation algorithms. The recursive algorithms use the data as it becomes available [28], whereas the iterative algorithms tend to exploit the advantage of processing a complete batch of available data, which can provide highly accurate parameter estimation. Iterative methods can also be used for solving matrix equations [29–31]. In the literature, Ding proposed a two-stage least-squares based iterative parameter estimation algorithm for CARARMA systems using the decomposition technique [32].
As a basic class of multivariable systems, multiple-input single-output (MISO) systems have lots of applications in industrial processes. Several works on MISO system identification have been reported [33]. For example, in order to improve the convergence rate, Liu et al. developed a stochastic gradient algorithm for MISO systems using the multi-innovation theory [34]. The least-squares methods can also be found in the literature.
Recently, Wang and Tang studied the identification algorithms for a class of linear-in-parameters single-input single-output (SISO) systems with colored noises using the recursive least-squares method [35]. In this work, we extend these results from SISO systems into a class of linear-in-parameters MISO systems with the colored noises shown in Figure 1 [36, 37]. Consider where is the system output, are the system inputs, and is the stochastic white noise with zero mean. and are polynomials, of known orders , in the unit backward shift operator , and defined by , , and are the unknown parameters to be estimated. The superscript denotes the matrix/vector transpose. It is worth noting that the models in (1) include but are not limited to linear MISO systems; that is, when and are defined by system (1) denotes an MISO output error moving average system. When is a nonlinear function of , for example, system (1) denotes a nonlinear MISO system.
On the basis of the iterative algorithms for linear-in-parameters SISO systems [37, 38], this paper develops the least-squares based and gradient based iterative identification algorithms to improve the parameter estimation accuracy for a class of linear-in-parameters MISO output error moving average systems. Compared with the gradient based iterative algorithm, the least-squares based iterative algorithm can provide more accurate parameter estimates.
The remainder of this paper is organized as follows. Section 2 introduces the identification model. Section 3 derives the least-squares based iterative algorithm. Section 4 proposes a gradient based iterative algorithm. Section 5 presents an illustrative example to show the effectiveness of the algorithms. Finally, concluding remarks are offered in Section 6.
2. The Identification Model
Let us define some symbols. The symbol denotes an identity matrix of order ; denotes an -dimensional column vector whose elements are 1; and represent the maximum eigenvalue and the inverse of the square matrix .
To further develop new identification algorithms for estimating the parameter vector and the parameters of and by utilizing the input-output measured data , we derive an identification model for system (1). Without loss of generality, assume that , , and for .
Define the intermediate variables as follows:
Define the parameter vectors as follows:
and define the information vectors as follows:
Then we can express (5) as
and system (1) can be rewritten as Equation (9) is the identification model of system (1), and parameter vector contains all the parameters of the system.
3. The Least-Squares Based Iterative Algorithm
Consider the newest data from to and define the quadratic criterion function as follows: By minimizing and letting the derivative of with respect to be zero, we can obtain the least-squares estimate of as The above estimate is impossible to implement due to the unknown noise-free outputs and unmeasurable noise items in . Here, the difficulties are solved by using the iterative identification technique [38]: let be the iterative variable, and let and be the iterative estimates of and at iteration , replace the unknown items and with their iterative estimates and at iteration , and define the estimated vectors as follows: Let and be the estimates of and at iteration , let and be the estimates of and at iteration . Replacing in (11) with its corresponding estimate , we can obtain the following least-squares based iterative algorithm for MISO systems in (1) (the MISO-LSI algorithm for short) into: The steps of computing involved in the algorithm are summarized as follows. (1)Given , let and collect the input-output data .(2)Collect the present input-output data and .(3)To initialize, let , = random number, and = random number, .(4)Form using (15) and using (17), and form using (16) and using (14).(5)Update the parameter estimate using (13).(6)Compute using (18) and using (19).(7)If (a given small number), obtain the iterative time and the parameter estimate ; let , increase by 1, and go to Step 2; otherwise, increase by 1 and go to Step 4.
4. The Gradient Based Iterative Algorithm
By minimizing through the negative gradient search, we obtain the following recursive relation of computing the estimate of at iteration : where is the step-size or the convergence factor to be given later. The same difficulties arise in that the noise-free outputs in and the noise items in of on the right-hand side of (24) are unknown. Here we apply the same scheme used in the previous section, replacing the unknown vectors with their corresponding iterative estimates. Referring to the method in [38], replacing in (24) with , we can summarize the following gradient based iterative algorithm for MISO systems in (1) (the MISO-GI algorithm for short): The steps of computing involved in the algorithm are summarized as follows. (1)Given , let and collect the input-output data set , .(2)Collect the present input-output data and .(3)To initialize, let , , and , .(4)Form using (27) and using (29), and form using (28).(5)Choose an appropriate step-size using (32) and update the parameter estimate using (25).(6)Compute using (30) and using (31).(7)If (a given small number), obtain the iterative time and the parameter estimate ; let , increase by 1, and go to Step 2; otherwise, increase by 1 and go to Step 4.
5. Example
Consider the following nonlinear multiple-input single-output simulation system: Here, the inputs and are taken as uncorrelated persistent excitation signal sequences with zero means and unit variances and as a white noise sequence with zero mean.
Using data and applying the MISO-GI algorithm in (25)–(32) and the MISO-LSI algorithm in (13)–(19) to estimate the parameters of this nonlinear system, the parameter estimates of each algorithm and their errors with noise variance are shown in Table 1; the parameter estimation errors versus of each algorithm are illustrated in Figure 2. We also investigate the performance of two algorithms under a relatively high noise level with noise variance , and the corresponding simulation results are illustrated in Table 2 and Figure 3.
From the simulation results in Tables 1 and 2 and Figures 2 and 3, we can draw the following conclusions.(i)The parameter estimation errors are getting smaller as the iterative variable increases.(ii)Both algorithms can produce highly accurate parameter estimates under different noise variances.(iii)The MISO-LSI algorithm converges faster than the MISO-GI algorithm does; however, due to the use of a batch of data, the MISO-LSI algorithm involves many matrix computations, resulting in the high computational complexity. One possible solution for reducing the computational load of the MISO-LSI algorithm with large is using the decomposition technique [27], which is widely adopted in the least-squares based iterative algorithms.
6. Conclusions
In this work, we have presented two iterative identification algorithms, a least-squares based iterative algorithm and a gradient based iterative algorithm, for a class of linear-in-parameters multiple-input single-output output error moving average systems. The illustrative example shows that both algorithms can provide more accurate parameter estimates. The proposed methods can be extended to study the identification problems of linear multivariable systems [39, 40] or multirate or nonuniformly sampled systems [41, 42]. The methods in this paper can combine the multi-innovation identification methods [43–50], the iterative identification methods [51, 52], and other identification methods [53–56] to present new identification algorithms for nonlinear systems [57–59] and can also be applied in other fields [60–67].
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the “Beijing Laboratory For Mass Transit” Program (I13H100010) and the Project of Beijing Municipal Science and Technology (D141100000814002).