Mathematical Problems in Engineering

Volume 2013, Article ID 293456, 7 pages

http://dx.doi.org/10.1155/2013/293456

## State Estimation for Nonlinear Discrete-Time Systems with Markov Jumps and Nonhomogeneous Transition Probabilities

Key Laboratory of Advanced Process Control for Light Industry, Ministry of Education, Institute of Automation, Jiangnan University, Wuxi 214122, China

Received 3 October 2013; Accepted 23 November 2013

Academic Editor: Shuping He

Copyright © 2013 Shunyi Zhao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

State estimation problem is addressed for a class of nonlinear discrete-time systems with Markov parameters and nonhomogeneous transition probabilities (TPs). In this paper, the optimal estimation mechanism of transition probability matrix is proposed in the minimum mean square error sense to show some critical points. Based on this mechanism, the extended Kalman filters are employed as the subfilters to obtain the subestimates with corresponding models. A novel operator which fuses the prior knowledge and the posterior information embedded in observations is developed to modify the posterior mode probabilities. A meaningful example is presented to illustrate the effectiveness of our method.

#### 1. Introduction

Markov jump systems (MJSs) are special systems which involve both time-evolving and event-driven mechanisms and have been used to model a wide variety of dynamic systems with unpredictable system structures. For MJSs, the state of the systems has two components: the finite valued state (denoted as mode) and the state in a finite dimensional Euclidean space. Motivated by a wide spectrum of applications, state estimation for MJSs has received considerable attention in the literatures [1–7] over the last three decades. It is well known that the optimal estimator for linear MJSs without the information about the underlying mode sequence is obtained by a bank of Kalman filters which requires exponentially increasing memory and computation with time. In order to limit the resource requirement, some suboptimal but executable algorithms have been proposed, including the generalized pseudo-Bayesian (GPB) [2], the interacting multiple model (IMM) [3], the improved IMM method [4, 5], and the particle filters (PF) [6, 7]. Moreover, methods based on guaranteed-cost approach which attempt to guarantee that the covariance of state is upper bounded by a certain constant value are also derived in quantity [8–14].

It is worth noting that the all the Markov chains considered in the aforementioned estimators are assumed to be homogeneous. More specifically, the transition probabilities (TPs) are constant and are not allowed to be time-variant during the entire estimation process [15]. However, since the external practical environment (humidity, temperature, etc.) always exhibits complex and rich dynamics, it is impossible to guarantee the TPs of all likely modes to be time-invariant. In other words, nonhomogeneous Markov process is a common phenomenon, not the exception in practice [16, 17]. On the other hand, TPs are introduced to depict the uncertainties of transitions between possible system behavior patterns and the accuracy of TPs is critical to the estimation performance of MM-type estimators [18]. In this sense, it is wise and meaningful to consider the nonhomogeneous Markov chain in systems. By observing this, nonlinear stochastic hybrid systems with state-dependent TPs were considered in [19], where PF was embedded into IMM method to obtain the IMM-PF. With the same motivation, state-dependent TPs were modeled in terms of Gaussian-shaped functions in [20] and the Gaussian hypotheses were used to reduce the computational cost of the developed algorithms. However, the existing literatures seem to lack a complete approach of state estimation for MJSs with uncertain time-varying TPs, until the H-infinity filter for nonhomogeneous linear MJSs was proposed in [21]. In this paper, the character of nonhomogeneous TPs is reflected by a higher-level transition matrix and TPs vary within a finite set in discrete-time domain. Later, the convenience brought by the assumption that the transition probability matrix (TPM) randomly takes value in a finite set was demonstrated analytically in [22] by comparing the optimal Bayesian equation derived under two different conditions. Nevertheless, all these estimators require the information about the TPM. More specifically, all the candidate TPMs and high-level transition matrix are required to be known accurately, making the resulting methods intractable in practical applications.

In this paper, a recursive estimator is derived for nonlinear MJSs with nonhomogeneous TPs. Different from [21, 22], the information about the dynamic of TPM including candidate TPMs and high-level transition matrix is not required to be known completely. This is a particular advantage of the framework raised here, and to the best of authors’ knowledge, no similar situation has been considered. The Bayesian equation that exploits the likelihoods of TPM to update the prior PDFs is developed. For the output calculation, we are required to find both the likelihoods and the dynamic of TPM which unfortunately turn out to be impossible. Therefore, instead of considering all the possible values of TPM in an admissible region, a constant diagonally dominant TPM which is rudely determined in advance is employed in the entire estimation process. In this sense, the problems involved in the estimation of TPM are solved naturally. The extended Kalman filter (EKF) [23, 24] is utilized as the subfilter under different models to obtain the subestimates and posterior mode probabilities. The exponentially increased computational problem is solved by the IMM-type approximation. Once the posterior mode probabilities are available, a novel operator is proposed to modify the posterior mode probabilities. Specifically, the probabilities of dominant modes are modified to approach one automatically according to their dominating degree. This operation is reasonable, as if the probability of one model is much larger than the others and we can almost make sure that it is the real one.

The content of this paper is organized as follows. In Section 2, some definitions and preliminaries are given. Section 3 proposes Bayesian estimation of TPM. The derivation of the suboptimal implementation is given in Section 4, where we propose a novel operator to modify the posterior mode probabilities after time update. The objective of Section 5 is to demonstrate the effectiveness of our method with a simulation example. Finally, Section 6 concludes this paper.

#### 2. Problem Formulation and Preliminaries

We consider the discrete-time nonlinear JMSs described by the following equations:where and are nonlinear transition and measurement functions, respectively; is the continuous state in Euclidean space with known initial distribution of state ; is the measurement; and are, respectively, uncorrelated white process and measurement noises with known statistics; the sequence is a continuous-time discrete-valued Markov chain and takes values in a finite state space . In this paper, the Markov chain governing the system dynamics is nonhomogeneous, which means that the time exists as a guard condition of the probability of a jump from mode at time to mode at time as where . Then, the TPM can be given as , where notation denotes the transpose of matrix or vector, , and . Note that if is known as a prior, the existing algorithms such as IMM-PF method [2, 5, 6] can be extended to the nonhomogeneous cases easily by using corresponding TPM at each time step. Unfortunately, TPs always change randomly and it is unrealistic to obtain the dynamic trajectories of real TPM in practical applications. Hence, it is assumed in the paper [21] that for arbitrary , is unknown but takes its value in a finite set , according a Markov chain with a high-level transition matrix , where and . In this sense, the discrete distribution of , where , can be considered as a point-mass approximation of over the simplex of valid TPMs. However, the requirement that both the candidate elements in and the high-level transition matrix should be available in advance is difficult to be satisfied in applications. That is to say, the finite-valued hypothesis is also impractical. In fact, a filter designer often uses a diagonally dominant constant matrix to replace the true values in reality, where and , for any . Intuitively, if no further operator is implemented, using such an inaccurate matrix directly may lead to performance degradation, as it failed to capture the real possible system behavior patterns. Therefore, the objective of this paper is to derive a recursive, filtering algorithm that yields satisfactory estimates of states in the cases that only the approximate matrix is available.

#### 3. Bayesian Estimation

It is well known that the recursive Bayesian equation of the JMSs with accurate TPs is given as where is a normalization constant, and denote the likelihoods and system dynamic under mode , respectively, are the joint probabilities of system modes at current and previous time steps, and denote the prior PDFs of state under mode , where . To see the derivation in detail, one can refer to one of our recent papers [22]. Considering (3), show that the effect of TPs in filtering process is mainly exhibited by the terms , which govern the dynamic of modes. Note that can be decomposed in a more compact way as where are the mode probabilities at a previous time step. At this point, we can see that using incorrect TPs will lead to the deviation of mode probabilities and reduce the estimation accuracy further due to the following fact: Thus, it is highly desirable and really nice to obtain the true TPM at each time step. Unfortunately, is assumed to be unavailable, and a natural solution of this problem is to estimate the unknown TPM recursively. In the following, we thus focus our attention on the problem of finding the posterior PDF of the nonhomogeneous TPM in the Bayesian framework. In view of Bayes’ ruler, we have where is considered as a normalization constant, is the likelihood, and denotes the prior PDF of . Using the law of the total probability, the prior PDF can be equivalently written as where and denote the dynamic of itself and posterior PDF at a previous time step. Using the total probability theorem, the first term in the numerator of the right-hand side of (6) becomes Defining the likelihood vector and mode probability vector as , , where and , we obtain By the Chapman-Kolmogorov equation, the denominator of the right-hand of (6) can be rewritten as Substituting (9) and (10) into (6) yields where is given by (7). Note that formula (11) is the key of the entire estimation process, as it tells that the known prior distribution is updated using the likelihoods of TPM to obtain the posterior conditional PDF . Unfortunately, calculating (11) analytically is almost impossible as it involves the following problems: (i) the dynamic of in (7) is difficult to obtain; (ii) notice that to get the posterior conditional distribution, one has to calculate the likelihood of . However, since is unknown and is also required to be estimated at time , evaluating the likelihood directly is impossible. Due to the above problems, the implementation of (11) is intractable and an alternative approach or some necessary approximation will be introduced in the following section.

*Remark 1. *If we assume that the TPM takes in a finite set , the dynamic of TPM in (7) can be expressed by the high-level transition probabilities , where . Moreover, we are not required to consider all possible values in an admissible region and only consider the likelihood of the values in that need to be evaluated, making the problem considerable simpler.

#### 4. Suboptimal Estimator

Since the system under consideration is nonlinear, in this paper, EKFs are utilized as subfilters. It is well known that EKF is one of the most widely used algorithms in the framework of nonlinear filtering. Let , denote the estimate of state, covariance matrix computed by the EKF at time , respectively. Let denote the mode probability of mode at time . Then, the PDF of state at time conditioned on is denoted as where denotes the normal distribution. As done in IMM method, the optimal Bayesian equation is decomposed into a series of basic transitions. In this sense, the PDFs can be equivalently given as follows: Note that we have Here, the IMM approximation is employed. More specifically, -component Gaussian mixture is replaced by a single Gaussian density to solve the computational sensitive problem as where , Once and are available, EKFs are executed to obtain the posterior filtered states and covariance as follows: where is the filter gain given as , where and are the covariance matrices of the process noise and measurement noise, respectively. At this point, the posterior mode probabilities can be updated as where . Considering (14) and (22), show that only the approximate TPs are employed to compute the mode probabilities which are used as “weights” of subfilters in the filtering process. That is to say, the estimation performance will be unsatisfied, as the mode probabilities deviate from their real trajectories. On the other hand, it is known that the ideal case of estimation of MJSs is that the underlying mode sequence is known. In such case, only one subfilter is required to calculate the estimate at every time step and the multiple-model estimation algorithm will degenerate to be the Kalman filter with corresponding model. In this sense, the information of TPs is not required any more in filtering process. By observing these, a novel operator is developed to modify the dominating mode probabilities after time update to make them approach to one according to their dominating degree. Without loss of generality, we present at time as where ; the index parameter is given as follows. For the cases with and , we have else , where and are the maximum mode probability and minimum mode probability at time , respectively. Note that it is not difficult to verify that in the cases . Therefore, the denominator of the right-hand side of (23) will not be zero and can be used safely in computer simulation. For the cases with and , the index parameter can be evaluated by else . At this point, we can see that the index parameter which governs the modification degree in our method is adaptive and varies automatically according to the dominance of posterior mode probabilities. Specifically, a large parameter is evaluated when the dominance is evident, otherwise either 1 or another relatively small value is employed. It is also necessary to point out that the assumption that the real TPs are constants is not required, as the compensation parameter is updated using online information at each sampling time step. The proposed method works effectively as long as the dominance is obvious. This is an intuitively meaningful property and can be considered a particular advantage of the method introduced here. After modifying the posterior mode probabilities, the final estimate is calculated by merging the posterior estimates and covariance using the modified mode probability as The algorithm based on the above illustrations is detailed below.

*Step 1. **Initializations*: given , and , set .

*Step 2. **Interacting*: compute mixed states and according to (16).

*Step 3. **Extended Kalman filtering*: in accordance with the steps of EKF, obtain the , , and posterior mode probabilities according to (18)–(22).

*Step 4. **Modification*: modify the posterior mode probabilities according to (23) and (24) to obtain .

*Step 5. **Output*: calculate the overall estimate as (26), set , and go back to Step 2.

#### 5. Numerical Example

To illustrate the performance of our method, a nonlinear dynamic system modeled which is a slightly modified version of the one given in [25] was carried out. Consider where and . The mode-dependent parameters are aggregated into three possible modes shown as and . The nonhomogeneous TPM takes its value in a finite set , where , , and are presented as follows: The high-level transition matrix is equal to It is necessary to point out that all the candidate TPMs , where and high-level transition matrix are unknown in advance and only an approximate TPM which is chosen by designer crudely is available as Two different algorithms were performed:(i)the IMM-EKF with ;(ii)the proposed algorithms (denoted as OA) with .

To assess the performance of algorithms, the average root mean square (RMS) error based on 5000 times Monte-Carlo simulations is employed. All the filters are initiated with the prior distribution . Results from 5000 Monte Carlo runs are shown in Figures 1 and 2 and Table 1.

In Figure 1, the average RMS error of state based on 5000 Monte Carlo runs of different algorithms is shown. Together with Table 1 and Figure 1, we can clearly see that OA yields improvements in estimation performance significantly over IMM-EKF as expected. This is the result of using more reasonable mode probabilities, which are mainly caused by the operator developed. That is, the information embedded in the measurements is exploited to counteract the negative effect brought by the wrong TPs. At each time step, the posterior mode probability of dominating mode is modified to approach one according to the dominating degree. The index parameter in a typical run is given in Figure 2. We see that the index parameter varies with the time instants and constantly adjusts and realigns itself according to the dominance of the posterior mode probabilities. In this sense, the operator can be considered as an instrument that can increase the probabilities of dominating modes automatically. Another important issue in practical application is the computation time. Note that in OA, the index parameter is required to be calculated at every time step. However, the computational time of OA is almost the same as IMM-EKF. This is mainly because the computation of OA is quite simple, no circulation or inversion is required, making the proposed method more competitive in online applications.

#### 6. Conclusions

This paper proposed a novel state estimation method for a class of nonlinear JMSs with nonhomogeneous TPs. In our work, the optimal Bayesian estimation of TPM is proposed and EKF is employed to handle the nonlinear problem. Moreover, an operator is developed to modify the posterior mode probabilities to compensate the effect caused by wrong TPs used. Simulation implicates that our method yields better estimation performance than IMM-EKF and shows more robustness against the wrong TPs.

#### Acknowledgments

This project was jointly supported by National Natural Science Foundation of China (Grants nos. 61104121 and 61273087), Program for Excellent Innovative Team of Jiangsu Higher Education Institutions, and the 111 Project (Grant no. B12018).

#### References

- Y. Bar-Shalom and X.-R. Li,
*Estimation and Tracking: Principles, Techniques, and Software*, Artech House, Norwell, Mass, USA, 1993. View at MathSciNet - J. K. Tugnait, “Adaptive estimation and identification for discrete systems with Markov jump parameters,”
*IEEE Transactions on Automatic Control*, vol. 27, no. 5, pp. 1054–1065, 1982. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - H. A. P. Blom and Y. Bar-Shalom, “Interacting multiple model algorithm for systems with Markovian switching coefficients,”
*IEEE Transactions on Automatic Control*, vol. 33, no. 8, pp. 780–783, 1988. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus - L. A. Johnston and V. Krishnamurthy, “An improvement to the interacting multiple model (IMM) algorithm,”
*IEEE Transactions on Signal Processing*, vol. 49, no. 12, pp. 2909–2923, 2001. View at Publisher · View at Google Scholar · View at Scopus - H. Qu, L. Pang, and S. Li, “A novel interacting multiple model algorithm,”
*Signal Processing*, vol. 89, no. 11, pp. 2171–2177, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus - A. Doucet, N. J. Gordon, and V. Krishnamurthy, “Particle filters for state estimation of jump Markov linear systems,”
*IEEE Transactions on Signal Processing*, vol. 49, no. 3, pp. 613–624, 2001. View at Publisher · View at Google Scholar · View at Scopus - S. Zhao and F. Liu, “Risk-sensitive filtering for nonlinear Markov jump systems on the basis of particle approximation,”
*International Journal of Adaptive Control and Signal Processing*, vol. 26, no. 2, pp. 158–170, 2012. View at Publisher · View at Google Scholar - H. Liu, D. W. C. Ho, and F. Sun, “Design of ${H}_{\infty}$ filter for Markov jumping linear systems with non-accessible mode information,”
*Automatica*, vol. 44, no. 10, pp. 2655–2660, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - H. Shen, X. Huang, J. Zhou, and Z. Wang, “Global exponential estimates for uncertain Markovian jump neural networks with reaction-diffusion terms,”
*Nonlinear Dynamics*, vol. 69, no. 1-2, pp. 473–486, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - Z.-G. Wu, P. Shi, H. Su, and J. Chu, “Reliable ${H}_{\infty}$ control for discrete-time fuzzy systems with infinite-distributed delay,”
*IEEE Transactions on Fuzzy Systems*, vol. 20, no. 1, pp. 22–31, 2012. View at Publisher · View at Google Scholar · View at Scopus - S. He and F. Liu, “Robust ${L}_{2}\text{-}{L}_{\infty}$ filtering of time-delay jump systems with respect to the finite-time interval,”
*Mathematical Problems in Engineering*, vol. 2011, Article ID 839648, 17 pages, 2011. View at Publisher · View at Google Scholar · View at MathSciNet - S. P. He and F. Liu, “Finite-time ${H}_{\infty}$ control of nonlinear jump systems with time-delays via dynamic observer-based state feedback,”
*IEEE Transactions on Fuzzy Systems*, vol. 20, no. 4, pp. 605–614, 2012. View at Publisher · View at Google Scholar - H. Zhang, J. Wang, and Y. Shi, “Robust ${H}_{\infty}$
sliding-mode control for Markovian jump systems subject to intermittent observations and partially known transition probabilities,”
*Systems & Control Letters*, vol. 62, no. 12, pp. 1114–1124, 2013. View at Publisher · View at Google Scholar · View at MathSciNet - H. Zhang, Y. Shi, and M. Liu, “${H}_{\infty}$ step tracking control for networked discrete-time nonlinear systems with integral and predictive actions,”
*IEEE Transactions on Industrial Informatics*, vol. 9, no. 1, pp. 337–345, 2013. View at Publisher · View at Google Scholar - Y. Y. Yin, P. Shi, F. Liu, and K. Teo, “Fuzzy model-based robust ${H}_{\infty}$
filtering for a class of nonlinear nonhomogeneous Markov jump systems,”
*Signal Processing*, vol. 93, pp. 2381–2391, 2013. View at Publisher · View at Google Scholar - M. Iosifescu,
*Finite Markov Processes and Their Applications*, John Wiley & Sons, Bucharest, Romania, 1980. View at MathSciNet - V. Morio, F. Cazaurang, and P. Vernis, “Flatness-based hypersonic reentry guidance of a lifting-body vehicle,”
*Control Engineering Practice*, vol. 17, no. 5, pp. 588–596, 2009. View at Publisher · View at Google Scholar · View at Scopus - X.-R. Li and Y. Bar-Shalom, “Multiple-model estimation with variable structure,”
*IEEE Transactions on Automatic Control*, vol. 41, no. 4, pp. 478–493, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - H. A. P. Blom and E. A. Bloem, “Exact Bayesian and particle filtering of stochastic hybrid systems,”
*IEEE Transactions on Aerospace and Electronic Systems*, vol. 43, no. 1, pp. 55–70, 2007. View at Publisher · View at Google Scholar · View at Scopus - C. E. Seah and I. Hwang, “State estimation for stochastic linear hybrid systems with continuous-state-dependent transitions: an IMM approach,”
*IEEE Transactions on Aerospace and Electronic Systems*, vol. 45, no. 1, pp. 376–392, 2009. View at Publisher · View at Google Scholar · View at Scopus - L. Zhang, “${H}_{\infty}$ estimation for discrete-time piecewise homogeneous Markov jump linear systems,”
*Automatica*, vol. 45, no. 11, pp. 2570–2576, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. Zhao and F. Liu, “Bayesian estimation for jump Markov linear systems with non-homogeneous transition probabilities,”
*Journal of the Franklin Institute*, vol. 350, no. 10, pp. 3029–3044, 2013. View at Publisher · View at Google Scholar · View at MathSciNet - L. Ljung, “Asymptotic behavior of the extended Kalman filter as a parameter estimator for linear systems,”
*IEEE Transactions on Automatic Control*, vol. 24, no. 1, pp. 36–50, 1979. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - A. Germani, C. Manes, and P. Palumbo, “Polynomial extended Kalman filter,”
*IEEE Transactions on Automatic Control*, vol. 50, no. 12, pp. 2059–2064, 2005. View at Publisher · View at Google Scholar · View at MathSciNet - M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking,”
*IEEE Transactions on Signal Processing*, vol. 50, no. 2, pp. 174–188, 2002. View at Publisher · View at Google Scholar · View at Scopus