Research Article | Open Access

Marco H. Terra, Gildson Jesus, João Y. Ishihara, " Estimates for Discrete-Time Markovian Jump Linear Systems", *Mathematical Problems in Engineering*, vol. 2013, Article ID 945342, 7 pages, 2013. https://doi.org/10.1155/2013/945342

# Estimates for Discrete-Time Markovian Jump Linear Systems

**Academic Editor:**Weihai Zhang

#### Abstract

This paper deals with the problem of filtering for discrete-time Markovian jump linear systems. Predicted and filtered estimates are obtained based on the game theory. Both filters are solved through recursive algorithms. The Markovian system considered assumes that the jump parameters are not accessible. Necessary and sufficient conditions are provided to the existence of the filters. A numerical example is provided in order to show the effectiveness of the approach proposed.

#### 1. Introduction

Filtering of Markovian jump linear systems (MJLSs) has been subject of intensive study in the last years. Different and creative approaches to deal with this class of filter have been considered in the literature. One of the main alternatives to solve this kind of problem is based on Kalman filter algorithms, see for instance [1, 2]. Filters proposed in these references are based on linear minimum mean square error estimates of discrete-time MJLS. They present an interesting feature related with recursiveness; however, they are not robust in nature.

When the robustness is relevant to the filtering process and demands an extra performance of the filtering approach, techniques are always considered as one of the best solutions to be adopted. In this way, a closed-loop transfer function from the unknown disturbances to the estimation error is designed in order to satisfy a prescribed -norm constraint. In general, algorithms developed to deduce Markovian filters are based on linear matrix inequalities (LMIs), see for instance [3–12]. In particular, [4, 6] assume that the jump parameter of the Markov chain is not available.

In this paper, we propose filters for discrete-time MJLS which are calculated in terms of recursive algorithms based on Riccati equations. Following the approach considered in [13], we develop predicted and filtered estimates based on the two-players game theory. The idea of game theory was also adopted in sthochastic control, see for instance [14].

We define the status of the players in the filtering problems considered in this paper in order to reach an equilibrium between two contradictory objectives. The first player can be interpreted as the maximizer of the estimation cost, whereas the second player tries to find an estimate that brings the quadratic cost to a minimum. A solution exists for a specified -level if the resulting cost is positive. We assume in these Markovian filtering problems that the jump parameter is not accessible. The recursiveness of the approach we are proposing is the main advantage of these Markovian filters if compared with the filters aforementioned. As a by-product of this approach, we provide necessary conditions for the existence of them based on only known parameters of the Markovian system.

This paper is organized as follows: in Section 2, we present the problem statement; in Section 3, filters for discrete-time MJLS are presented; and in Section 4, a comparative study, based on numerical example, between the approach we are proposing and the filter developed in [4] is performed.

#### 2. Problem Definition

The recursive filters developed in this paper are based on the following discrete-time MJLS: where is the valued state, is the valued output sequence, is the valued signal to be estimated, , , and are random disturbances; is a discrete-time Markov chain with finite state space and transition probability matrix . We set , , , , , , and , for . The random disturbances , , and are assumed to be null mean with finite second order moments, independent wide sense stationary sequences mutually independent with covariance matrices equal to the , , and , respectively. are random vectors with (where denotes Dirac measure) and ; and are independent of , and . A scalar and a sequence of sets of observations are defined to find at each instant a filtered estimate of such that is satisfied for , and is satisfied for . For the predicted estimate, given and a sequence of observations (2), the problem is to find at each instant a prediction of such that is satisfied for and is satisfied for . The sequence of solutions and is outputs of the respective filters. Due to the hybrid nature of this class of system, at each instant of time a new model of a countable set of models is used to calculate the functionals (4) and (6). In general, to synthesize recursive filtering algorithms for this class of problems is not an easy task. Thanks to the augmented model of (1) proposed in [1] we can redefine these functionals in order to develop recursive filters similar to those we find in the literature for standard state-space systems, without jumps. The augmented model of (1) can be written as follows: where the parameter matrices are given by whose dimensions are defined by , , and ; and the state variable is defined as We define for and , where is given by the following recursive equation:

We define also in the next lemma, weighting matrices , , and as variances of random disturbances given by , , and , respectively.

Lemma 1. *Let , , and defined by
**
for and , where
**
The variances of , , and can be obtained by the following equations:
**
respectively, where
*

*Proof. *The proof follows the arguments proposed in [15], Chapter 3, and in [1].

#### 3. Estimates for DMJLS

In this section, we propose recursive estimates for discrete-time MJLS (DMJLS). It is known that for standard state-space systems, this kind of filtering approach is difficult to be implemented *online* due to the fact that we do not know the minimum for each step of the recursion. In general, it depends on the estimate error variance matrix which should be calculated at the same instant of time. In this sense, the existence condition of this filter is not known *a priori*. To ameliorate this limitation we provide, for this Markovian problem, necessary conditions to tune this parameter depending on the variances of the random disturbances , , and and on the known parameter matrices of the augmented model of (1) given in (7).

##### 3.1. Filter

Consider (3) and (4) for the augmented model (7): for , and for . We can rewrite this filtering problem in terms of the following optimization problem: where

Notice that it is not necessary to maximize over in the optimization problem (19). The solution of the filter is guaranteed if, and only if, there exists a for which has a minimum . Therefore, in order to deal with only the minimization problem (19), for each it is easy to show that (21) can be rewritten as where For all , we can conclude that the following recurrent relations are valid:

According to [16], there exists a minimum for (22) if and only if the positiveness of is guaranteed.

Lemma 2. *Consider matrices and and column vectors and of appropriate dimensions with symmetric. For any we have
**
if and only if and . If the minimum is attained, it is unique if and only if and the optimal solution is given by . *

With this fundamental lemma in mind, we can obtain an existence condition for the filter we are proposing in the following, based on the known parameter matrices of the Markovian model. For , the term of can be written as and a necessary condition for is the positiveness of , which in terms of the augmented Markovian model of (1) can be established as follows:

*Remark 3. *A lower bound for can be calculated through (27) based on the augmented model (7) and Lemma 1. Notice that (27) is a natural and interesting extension of the filtering of standard state-space systems [13], without jumps, to the DMJLS we are dealing with in this paper.

Now we are in a position to deduce the recursive filtered estimate for DMJLS. The next theorem provides the solution for this problem based on the augmented model (7) and on the sequence of measurements (2) of the original Markovian system (1).

Theorem 4. *Consider the augmented Markovian model (7). There exists a recursive filter for this system, defined by the following recursive equations:
**
if and only if , , and the attenuation level is positive. *

*Proof. *This proof follows standard arguments used to deduce Kalman filters and basically consists in checking the positiveness of (26) and in finding the equation of the filter through the Lemma 2. In order to have , we can rewrite this term as (26). The sub block of (26) must be positive definite. Assuming that the positiveness of is guaranteed and
the term of (26) is positive definite and if and only if the Schur complement of the block,
is positive definite. For , is the block of . With defined in (24), we obtain
Considering in terms of the original data (23), we obtain (28). From Lemma 2 we have the solution for the minimization of (22). Based on the recurrent relations (23) and (24), considering for , and introducing
holds (29) taking into account that .

##### 3.2. Predictor

In this subsection, we present the recursive predictor filter for DMJLS. The original problem (5)-(6) can be rewritten in terms of the following optimization problem: where

The predictor filter exists at the instant if and only if there exists a sequence such that has a minimum for which

The prediction optimization problem is equivalent in nature to the filter problem, aforementioned. In the next theorem we present the predictor filter whose proof follows the line of the filtered version.

Theorem 5. *The Markovian prediction problem (35) is solvable if, and only if, , where the sequence and the predictor filter are calculated by the recursions
*

In order to recover the original predicted and filtered state estimates of System (1), we calculate the following summations: where and . is defined in the same way.

*Remark 6. *To establish the stability of the stationary prediction filter, it is assumed that all matrices of (1) and the transition probabilities are time invariant; System (1) is mean square stable (MSS) and its Markov chain is ergodic. There exists a unique positive-definite solution (with ) to the algebraic Riccati equation
providing that , for any fixed, which guarantees that and thus the above inverse is well defined. Following the guidelines defined in [1], the stability of the predictor filter is assured with
where denotes spectral radius of the dynamic matrix of the filter with stationary gain. The asymptotic stability of the filtered version can also be assured following this line of argumentation.

*Remark 7. *Similar to the filtered case, we can also find a necessary existence condition to the Markovian prediction filter given by

*Remark 8. *If we consider , the filter proposed in this paper reduces to the Kalman filter for MJLS proposed in [1]. For the case with no jumps , the predicted and the filtered estimates reduce to the filters for state-space systems given in [17].

#### 4. Numerical Example

In this section, we compare the filtering approach proposed with the filter developed in [4]. With two Markovian states, the model (1) is defined based on the following parameters: We compare both filters in the predicted form, written in the following way: where The filter of [4] is considered in the strictly proper form as whose solution is given in terms of linear matrix inequalities. We show in Figure 1 the root-mean-square errors (rms) of both filters. We performed 1000 Monte Carlo simulations from with the values of generated randomly. The initial condition is considered Gaussian with mean and variance ; , , , and are independent sequences of noises with , , and identity matrices with appropriate dimensions, , and . We obtained for the filter proposed in this paper and for the filter proposed in [4]. The parameter matrices of our filter were computed as and for the filter of [4] were computed as

In spite of the smaller provided by the filter of [4], the rms of both filters are equivalent. This difference in is due to the fact that the functionals of both approaches are different in nature. However, if we consider the same numerical value of all parameters but with matrices and multiplied by , we obtain for the proposed recursive filter . On the other hand, we obtain an infeasible solution with the LMI proposed in Section IV of [4] (computed through Matlab 7.4.0). The mode-independent filter provided in [4] deals with only sufficient conditions. Considering *offline* computations, our approach provides necessary and sufficient conditions for the existence of the filter.

It is important to recall that the proposed approach enables all parameter matrices of (1) to be time varying. It also provides a practical estimate for the minimum admissible through the necessary condition (27), which depends on only known parameters of the Markovian system.

#### 5. Conclusion

This paper developed predicted and filtered estimates for DMJLS. They were deduced based on the assumption that jump parameter is not accessible. The numerical example showed the effectiveness of this approach. Through an augmented model of the standard Markovian system, these filters were deduced based on the game theory. It allows us to deal with recursive algorithms to solve this kind of filtering problem where the parameter matrices can be time varying for each operation mode of the Markovian system. As future works, we intend to solve nonlinear filtering and control problems for DMJLS based on the problems raised by [12, 18].

#### References

- O. L. V. Costa and S. Guerra, “Stationary filter for linear minimum mean square error estimator of discrete-time Markovian jump systems,”
*IEEE Transactions on Automatic Control*, vol. 47, no. 8, pp. 1351–1356, 2002. View at: Publisher Site | Google Scholar | MathSciNet - M. H. Terra, J. Y. Ishihara, and G. Jesus, “Information filtering and array algorithms for discrete-time Markovian jump linear systems,”
*IEEE Transactions on Automatic Control*, vol. 54, no. 1, pp. 158–162, 2009. View at: Publisher Site | Google Scholar | MathSciNet - Y. Dong, J. Sun, and Q. Wu, “${\mathscr{H}}_{\infty}$ filtering for a class of stochastic Markovian jump systems with impulsive effects,”
*International Journal of Robust and Nonlinear Control*, vol. 18, no. 1, pp. 1–13, 2008. View at: Publisher Site | Google Scholar | MathSciNet - A. P. C. Gonçalves, A. R. Fioravanti, and J. C. Geromel, “${\mathscr{H}}_{\infty}$ filtering of discrete-time Markov jump linear systems through linear matrix inequalities,”
*IEEE Transactions on Automatic Control*, vol. 54, no. 6, pp. 1347–1351, 2009. View at: Publisher Site | Google Scholar | MathSciNet - H. Liu, F. Sun, K. He, and Z. Sun, “Design of ${\mathscr{H}}_{\infty}$ reduced-order filter for Markovian jumping systems with time delay,”
*IEEE Transactions Circuits and Systems*, vol. 51, no. 11, pp. 607–612, 2004. View at: Google Scholar - C. E. Souza, “A mode-independent ${\mathscr{H}}_{\infty}$ filter design for discrete-time Markovian jump linear systems,” in
*Proceedings of the 42nd Conference on Decision and Control*, Maui, Hawaii, USA, December 2003. View at: Google Scholar - C. E. de Souza and M. D. Fragoso, “${\mathscr{H}}_{\infty}$ filtering for Markovian jump linear systems,”
*International Journal of Systems Science*, vol. 33, no. 11, pp. 909–915, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - C. E. de Souza and M. D. Fragoso, “${\mathscr{H}}_{\infty}$ filtering for discrete-time linear systems with Markovian jumping parameters,”
*International Journal of Robust and Nonlinear Control*, vol. 13, no. 14, pp. 1299–1316, 2003. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - C. E. de Souza, A. Trofino, and K. A. Barbosa, “Mode-independent ${\mathscr{H}}_{\infty}$ filters for Markovian jump linear systems,”
*IEEE Transactions on Automatic Control*, vol. 51, no. 11, pp. 1837–1841, 2006. View at: Publisher Site | Google Scholar | MathSciNet - F. Sun, H. Liu, K. He, and Z. Sun, “Reduced-order ${\mathscr{H}}_{\infty}$ filtering for linear systems with Markovian jump parameters,”
*Systems & Control Letters*, vol. 54, no. 8, pp. 739–746, 2005. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - L. Wu, P. Shi, H. Gao, and C. Wang, “${\mathscr{H}}_{\infty}$ filtering for 2D Markovian jump systems,”
*Automatica*, vol. 44, no. 7, pp. 1849–1858, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - Z. Lin, Y. Lin, and W. Zhang, “A unified design for state and output feedback ${\mathscr{H}}_{\infty}$ control of nonlinear stochastic Markovian jump systems with state and disturbance-dependent noise,”
*Automatica*, vol. 45, no. 12, pp. 2955–2962, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - J. Y. Ishihara, M. H. Terra, and B. M. Espinoza, “${\mathscr{H}}_{\infty}$ filtering for rectangular discrete-time descriptor systems,”
*Automatica*, vol. 45, no. 7, pp. 1743–1748, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - B.-S. Chen and W. Zhang, “Stochastic ${\mathscr{H}}_{2}/{\mathscr{H}}_{\infty}$ control with state-dependent noise,”
*IEEE Transactions on Automatic Control*, vol. 49, no. 1, pp. 45–57, 2004. View at: Publisher Site | Google Scholar | MathSciNet - O. L. V. Costa, M. D. Fragoso, and R. P. Marques,
*Discrete-Time Markov Jump Linear Systems: Probability and Its Applications*, Springer, London, UK, 2005. View at: MathSciNet - S. Haykin,
*Adaptive Filter Theory*, Prentice Hall, Upper Saddle River, NJ, USA, 2001. - B. Hassibi, A. H. Sayed, and T. Kailath,
*Indefinite-Quadratic Estimation and Control: A Unified Approach to*, SIAM Studies in Applied Mathematics, SIAM, Philadelphia, Pa, USA, 1999. View at: Publisher Site | MathSciNet*ℋ*_{2}and*ℋ*_{∞}Theories - W. Zhang, B.-S. Chen, and C.-S. Tseng, “Robust ${\mathscr{H}}_{\infty}$ filtering for nonlinear stochastic systems,”
*IEEE Transactions on Signal Processing*, vol. 53, no. 2, part 1, pp. 589–598, 2005. View at: Publisher Site | Google Scholar | MathSciNet

#### Copyright

Copyright © 2013 Marco H. Terra et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.