Robust Control, Optimization, and Applications to Markovian Jumping SystemsView this Special Issue
Research Article | Open Access
Optimal State Estimation for Discrete-Time Markov Jump Systems with Missing Observations
This paper is concerned with the optimal linear estimation for a class of direct-time Markov jump systems with missing observations. An observer-based approach of fault detection and isolation (FDI) is investigated as a detection mechanic of fault case. For systems with known information, a conditional prediction of observations is applied and fault observations are replaced and isolated; then, an FDI linear minimum mean square error estimation (LMMSE) can be developed by comprehensive utilizing of the correct information offered by systems. A recursive equation of filtering based on the geometric arguments can be obtained. Meanwhile, a stability of the state estimator will be guaranteed under appropriate assumption.
Discrete-time Markov jump linear systems (MJLSs) are basically linear discrete-time systems with discretional parameters evolving with a finite-state Markov chain. It can be used in modeling systems with abrupt structures, for example, those which may be found in signal processing, fault detection [1, 2], and subsystem switching. One classical application is maneuvering target tracking, in which signals of interest are modeled by using MJLSs . In these fields, the problems of state estimation for MJLSs play an essential role in recovering some desired variables from given noisy observations for output variables. However, many approaches of achieving the state estimation of MJLSs include the generalized pseudo-Bayesian (GPB) algorithm [4, 5], the interacting multiple model (IMM) filtering , stochastic sampling based methods [7, 8], and LMMSE filter. Those methods are different from each other in their estimation criteria and means [2, 9–12]. Among them, LMMSE filter has been well studied for MJLSs in many of literary works .
On the other hand, since applications of sensors networks are becoming ubiquitous in practical systems, wireless or wireline communication channels are essential for data communication. Examples are offered ranging from advanced aircraft, spacecraft to manufacturing process. As communication channels are time varying and unreliable, the phenomena of random time delays and random packet dropout usually occur in these networked systems. Hence, more and more attention has been paid to systems with observer-based fault during the past years. For example, studies on optimal recursive filter for systems with intermittent observations can be traced back to Nahi , whose work assumed that uncertainty of observations is independent and identically distributed. Afterwards, by using linear matrix inequalities (LMIs) techniques, the performance, performance, finite-time performance, and finite-time performance have been well studied for solving filtering and control problems occurring in stochastic systems with uncertain elements [14–21]. In [22–25], the stability analysis of random Riccati equation arising from Kalman filtering with intermittent observations was investigated elaborately. filtering algorithm [26–28] has been developed for discrete systems with random packet losses in [29, 30]. In , a robust filtering algorithm was developed for state estimation of MJLSs with random missing observation by applying basic IMM approach and technique. Reference  dealt with the fault detection filtering (FDF) design within stochastic filtering frame for a class of discrete-time nonlinear Markov jump systems with lost measurements.
Although the aforementioned references give efficient and practical tools to deal with the filtering problems for systems with package dropout, the results given by the methods constructed based on LMIs techniques are sometimes too conservative. What is more, IMM approach mentioned a priori requires online calculations. Inspired by the effectiveness of LMMSE mechanic used in solving state estimation problem of MJLSs with random time delays in , the problem of state estimation of MJLSs with random missing observations is formulated into LMMSE filtering frame. This frame can lead to a time-varying linear filter easy to implement. At the same time, most calculations can be performed off-line.
Aiming at solving the issue of uncertain observations in MJLSs, this paper provides a heuristic method for detecting the fault in process of transmitting observation. An approach of fault detection and isolation (FDI) [32, 34] for a class of MJLSs with missing observations will be investigated. The key point of FDI is to construct the residual generator and determine the residual evaluation function and the threshold. Then, by comparing the value of the evaluation function with the prescribed threshold, we will make judgment whether an alarm of fault is generated. The situation of uncertainties of observation can be naturally and conveniently reflected. With knowing the information of the faulty case, a conditional prediction of observation will be obtained, which can be used as replacement of the faulty one. At this time, we can utilize the optimal state estimator of pervious instant and parameters for constructing observer of system to estimate the observation at current time. By this way, we can skip and avoid the fault observation.
Accordingly, by applying the basic FDI approach and basic LMMSE algorithm, an FDI-LMMSE filtering algorithm is developed for state estimation of MJLSs with random missing observation. In order to solve the optimal estimation problem, the measurements’ loss process is modeled as a Bernoulli distributed white sequence taking values from 0 to 1 randomly. The estimation problem is then reformulated as an optimal linear filtering of a class of MJLSs, which have random missing observation and necessary model compensation, via state augmentation [35–38]. A recursive filtering is formulated in terms of Riccati difference equations. At the same time, we will show that estimator is stable under necessary assumptions in this paper.
This paper is organized as follows. Section 2 gives the problem formulation. A recursive optimal solution is given in Section 3. Its stability is discussed in Section 4. In Section 5, a numerical example is shown to explain the effectiveness of approach proposed in our paper. At last, the conclusions are drawn in Section 6.
2. Problem Formulation
On the stochastic basis , considering the following jump Markov linear system model: where is continuous-valued based-state sequence with known initial distribution . is assumed to be known time-varying constant to each value of . is the noisy observation sequence. is the noisy observation sequence with distribution . is a white measurement noise sequence independent of the process noise with distribution .
Remark 1. is a compensation between practical systems and models applied in this paper.
is the unknown discrete-valued Markov chain with a finite-state space . The transition probability matrix is , where . We set . The basic variables , , and the modal-state sequence are assumed to be mutually independent for all . , , , are assumed to be known time-varying system matrices to each value of . For notational simplicity, the following notations and definitions hold in the rest of the paper:
In this paper, consider that the observations are sent to the estimator via a Gilbert-Elliot channel, where the packet arrival is modeled using a binary random variable , with probability , and with independent of if . Let be independent of ; that is, according to this model, the measurement equation consists of noise alone or noise plus signal, depending on whether is 0 or 1.
Notation 1. Some notations which we will use throughout the paper should be presented first. We will denote by the space of real matrices and by the space of -dimensional real vectors. The superscript indicates transpose of a matrix. For a collection of matrices , with , represents the diagonal matrix formed by in the diagonal.
Notation 2. Define and . For , , if for each , we have .
3. Recursive Optimal Solution
In this section, a solution to the optimal estimator will be presented via the projection theory and the state augmentation in the Hilbert space.
First, we denote by the linear space spanned by the observation . If for some , , the random variable .
Let represent an indicator of Markov process, which is defined as follows: And call . Define also as the projection of onto the linear space and Then, we first define the following second-moment matrices associated with the aforementioned variables. They play key roles in deriving the covariance matrices of the estimator errors and optimal estimator: Considering the following augment matrices: then system can be described as follows: Note that .
Assumption 2. For all , , where convergency value of , which will be given in Section 3.
3.2. Optimal Estimator
From geometric arguments in , the LMMSE filter for MJLSs with uncertain observations can be derived in this section. The following lemmas present necessary and sufficient conditions on derivation of FDI-LMMSE filtering.
Lemma 3. For any given time instant , one has where .
Proof. For any given instant , we have from (8) that
Recalling that , initial covariance matrix .
To derive the optimal filter, we first define the innovation sequence as where conditional prediction is the projection of onto the linear space of . Consider Then, according to (4) and (8), the generated residual will be obtained as
In the following, an FDI scheme will be constructed, which can detect whether observation at instant is lost. In this paper, we choose the following mean square of the residual as the residual evaluation function to measure the energy of the residual: From (12)-(13) we get that
Suppose that is convergent to at the instant , from Assumption 2, we have that If , we have that If , From (8), we have that The FDI scheme in the following lemma will play a key role in deriving the main results of this paper.
Lemma 4. With above derivation, we can decide whether the observations of system were lost and detect the lost information at instant according to the following rule: where
With the fault being detected, the missing information can be taken into consideration when designing the FDI-LMMSE filter. The fault observation can be replaced and isolated by . By the above approach, we can skip the error information at the instant and use the correct information of pervious instant to estimate the value of state at instant directly.
Theorem 5. Consider the system represented by (8). Then the LMMSE is given by where satisfies the recursive equation where .
Proof. Recall that observation estimator is given by (12).
Now, can be rewritten as the following equation:
Considering the geometric argument as in , the estimator satisfies the following equations:
From (26), we get that
Because is independent of , we have that
showing that is orthogonal to . Similar reasoning shows the orthogonality between and . Recalling that and are orthogonal to , we can obtain that is orthogonal to . Then, from (27), the result can be obtained as follows: From (11), (28) and (26), (29), we get that
The positive-semidefinite matrices are obtained from And the recursive equation about is given as follows: where .
can be derived directly as a recursive Riccati equation in the following derivation. In the following, we denote the linear operator by , in which is
Theorem 6. satisfies the following recursive Riccati equation:
where is given by the recursive equation (9) from Lemma 3.
Unlike the classical case, the sequence is now random, which result from its dependence on the random sequence .
Proof. Rewrite state equation in (8) as follows:
From (32), we define
From (25) and (32), we have that
Then from (41) and (38), we get that
Therefore, at this point, we obtain the recursive equation for as follows:
By a series of algebraic manipulations, we have
Substituting (44) into (43) yields the recursive equation for as
4. Stability of the State Estimator
As we all see, the intermittent observations are the source of potential instability. From Theorem 6, however, the error covariance matrix obtained from the LMMSE can be rewritten in terms of a recursive Riccati equation of . In this section, based on that following assumptions hold, we show that the proposed estimator is stable as provided in our paper.
Assumption 7. is assumed to be ergodic Markov chain.
First, (37) describes a recursive Riccati equation for . We should establish now its convergence when . It follows from Assumption 2 that exists and it is independent of . We define We redefine the matrix as follows:
Then, we give the following facts and lemmas for system, which will be used in the proof of stability of the covariance matrix of estimation error.
Then we have holding for all (since , we have as ). Defining , then we get At the same time, exponentially fast.
From (37), as , we obtain the mean state covariance as follows:
An operator is introduced for any positive-semidefinite matrix as follows:
Define now with , and
Lemma 9 (see ). and for each , one can get that
Now, one defines where , .
From the definition of and condition of , one notices that the inverse of exists.
Lemma 10. For each , one gets that
Theorem 11. Suppose that Assumptions 7 and 8 hold. Consider that the algebraic Riccati equation satisfies (48), where . Then, there exists a unique nonnegative definite solution to (60). , and for any , , , and , one has given by (50) satisfying .
Proof. Due to MSS of 5.38 , we have from Proposition 3.6 in chapter 3  that . According to the standard results for algebraic Riccati equation there is a unique positive-semidefinite solution to (60). And moreover .
From Theorem 11, we get that satisfied
Then (50) can be rewritten as
Suppose that , we have that
By definition, . Suppose that . From (64), we have that . Therefore we have shown by induction that for all . From MSS and ergodicity of the Markov chain we have that , , and exponentially fast. From and same reasoning as in the proof of proposition 3.36 in  we have that as , where satisfies
And is the unique solution to (65). Recalling that satisfies (62), we get that is also a solution to (65) and from uniqueness, . Then, we obtain that
And . From (66) and (56) in Lemma 10 it follows that . And thus we can conclude that whenever for some . Moreover, from the fact that and , we have that satisfies (60).
From uniqueness of the positive-semidefinite solution to (60), we can conclude that . From (66) and (56), and since and as , we get that .
The upper bound for the error covariance matrix to a stationary value for linear minimum mean square error (LMMSE) estimation can be easily obtained. It is described that if the system is MSS and the missing information is detected, then the error covariance matrix will converge to the unique nonnegative definite solution of an algebraic Riccati equation associated with the problem.
5. Numerical Example
In order to evaluate the performance of our method, in this section, we are going to use a scalar MJLS described by the following equations: where , , and denote the target position, velocity, and acceleration, respectively. The initial state is normally distributed with mean 10 and variance 1. , and , are independent white noise sequences with covariance of , and . The transition probability matrix for the finite-state Markov chain is
To assess the performance of algorithms, the average root mean square (RMS) error based on times Monte-Carlo simulation is defined as where the time step is chosen as 500, .
The simulation results are obtained as follows. Figure 5 presents the real states and their estimators subject to fault-free case and faulty case, respectively, based on the given path. Figure 1 shows the observations with lost data from unreliable channel and observations from reliable channel. As the proposed algorithm can be thought of as a generalization of the well-known LMMSE filtering, we denote it by FDI-LMMSE filtering in the simulation. The RMS in the position of FDI-LMMSE filtering in the faulty case is compared with that of LMMSE filtering in the fault-free case in the Figure 4. It can be shown in Figures 2 and 3 that the residual can deliver fault alarms soon after the fault occurs. From the simulation results, we can see that the obtained linear estimator for systems with random missing data are tracking well to the real state value, which is the estimation scheme proposed in this paper produces good performance.
This paper has addressed the estimation problem for MJLSs with random missing data. Random missing data introduced by the network is modeled as Bernoulli distribution variable. By usage of an observer-based FDI as a residual generator, the design of FDI-LMMSE filter has been formulated in the framework of LMMSE filtering. Complete analytical solution has been obtained by solving the recursive Riccati equations. It has been proved from theorem derivation and a numerical example simulation that the proposed state estimator is effective.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was partially supported by the National Natural Science Foundation of China 61273087 and the Program for Excellent Innovative Team of Jiangsu Higher Education Institutions.
- N. O. Maget, G. Hetreux, J. M. L. Lann, and M. V. L. Lann, “Model-based fault diagnosis for hybrid systems: application on chemical processes,” Computers & Chemical Engineering, vol. 33, no. 10, pp. 1617–1630, 2009.
- A. Logothetis and V. Krishnamurthy, “Expectation maximization algorithms for MAP estimation of jump Markov linear systems,” IEEE Transactions on Signal Processing, vol. 47, no. 8, pp. 2139–2156, 1999.
- Y. Bar-Shalom and L. Xiao-Rong, Estimation and Tracking: Principles, Techniques, and Software, Artech House, Norwell, Mass, USA, 1993.
- G. Ackerson and K. S. Fu, “On state estimation in switching environments,” IEEE Transactions on Automatic Control, vol. 15, no. 1, pp. 10–17, 1970.
- J. K. Tugnait, “Adaptive estimation and identification for discrete systems with Markov jump parameters,” IEEE Transactions on Automatic Control, vol. 27, no. 5, pp. 1054–1065, 1982.
- H. A. P. Blom and Y. Bar-Shalom, “Interacting multiple model algorithm for systems with Markovian switching coefficients,” IEEE Transactions on Automatic Control, vol. 33, no. 8, pp. 780–783, 1988.
- A. Doucet, N. J. Gordon, and V. Krishnamurthy, “Particle filters for state estimation of jump Markov linear systems,” IEEE Transactions on Signal Processing, vol. 49, no. 3, pp. 613–624, 2001.
- A. Doucet, A. Logothetis, and V. Krishnamurthy, “Stochastic sampling algorithms for state estimation of jump Markov linear systems,” IEEE Transactions on Automatic Control, vol. 45, no. 2, pp. 188–202, 2000.
- O. L. V. Costa, “Linear minimum mean square error estimation for discrete-time Markovian jump linear systems,” IEEE Transactions on Automatic Control, vol. 39, no. 8, pp. 1685–1689, 1994.
- O. L. V. Costa and S. Guerra, “Stationary filter for linear minimum mean square error estimator of discrete-time Markovian jump systems,” IEEE Transactions on Automatic Control, vol. 47, no. 8, pp. 1351–1356, 2002.
- L. A. Johnston and V. Krishnamurthy, “An improvement to the interacting multiple model (IMM) algorithm,” IEEE Transactions on Signal Processing, vol. 49, no. 12, pp. 2909–2923, 2001.
- Q. Zhang, “Optimal filtering of discrete-time hybrid systems,” Journal of Optimization Theory and Applications, vol. 100, no. 1, pp. 123–144, 1999.
- N. E. Nahi, “Optimal recursive estimation with uncertain observation,” IEEE Transactions on Information Theory, vol. 15, no. 4, pp. 457–462, 1969.
- Z. Hui, S. Yang, and A. S. Mehr, “Robust energy-to-peak filtering for networked systems with time-varying delays and randomly missing data,” IET Control Theory & Applications, vol. 4, no. 12, pp. 2921–2936, 2010.
- S. Jun and H. Shuping, “Nonfragile robust finite-time controller design for a class of uncertain Lipschitz nonlinear systems with time-delays,” Abstract and Applied Analysis, vol. 2013, Article ID 265473, 9 pages, 2013.
- Z. Hui, W. Junmin, and S. Yang, “Robust sliding-mode control for Markovian jump systems subject to intermittent observations and partially known transition probabilities,” Systems & Control Letters, vol. 62, no. 12, pp. 1114–1124, 2013.
- Z. Hui, S. Yang, and L. Mingxi, “ step tracking control for networked discrete-time nonlinear systems with integral and predictive actions,” IEEE Transaction on Industrial Informations, vol. 9, no. 1, pp. 337–345, 2013.
- H. Shuping, S. Jun, and L. Fei, “Unbiased estimation of Markov jump systems with distributed delays,” Signal Processing, vol. 100, pp. 85–92, 2014.
- H. Shuping, “Resilient - filtering of uncertain Markovian jumping systems within the finite-time interval,” Abstract and Applied Analysis, vol. 2013, Article ID 791296, 7 pages, 2013.
- H. Shuping and L. Fei, “Robust finite-time estimation of Markovian jumping systems with bounded transition probabilities,” Applied Mathematics and Computation, vol. 222, pp. 297–306, 2013.
- H. Shuping and L. Fei, “Finite-time fuzzy control of nonlinear jump systems with time delays via dynamic observer-based state feedback,” IEEE Transactions on Fuzzy Systems, vol. 20, no. 4, pp. 605–613, 2012.
- M. Yilin and B. Sinopoli, “Kalman filtering with intermittent observations: tail distribution and critical value,” IEEE Transactions on Automatic Control, vol. 57, no. 3, pp. 677–689, 2012.
- E. Cinquemani and M. Micheli, “State estimation in stochastic hybrid systems with sparse observations,” IEEE Transactions on Automatic Control, vol. 51, no. 8, pp. 1337–1342, 2006.
- S. C. Smith and P. Seiler, “Estimation with lossy measurements: jump estimators for jump systems,” IEEE Transactions on Automatic Control, vol. 48, no. 12, pp. 2163–2171, 2003.
- B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M. I. Jordan, and S. S. Sastry, “Kalman filtering with intermittent observations,” IEEE Transactions on Automatic Control, vol. 49, no. 9, pp. 1453–1464, 2004.
- C. E. DeSouza and M. D. Fragoso, “ filtering for Markovian jump linear systems,” International Journal of Systems Science, vol. 33, no. 11, pp. 909–915, 2002.
- C. E. DeSouza and M. D. Fragoso, “Robust filtering for uncertain Markovian jump linear systems,” International Journal of Robust and Nonlinear Control, vol. 12, no. 5, pp. 435–446, 2002.
- Z. Hui, S. Yang, and S. M. Aryan, “Robust weighted filtering for networked systems with intermittent measurements of multiple sensors,” International Journal of Adaptive Control and Signal Processing, vol. 25, no. 4, pp. 313–330, 2011.
- S. M. K. Mohamed and S. Nahavandi, “Robust filtering for uncertain discrete-time systems with uncertain noise covariance and uncertain observations,” in Proceedings of the 6th IEEE International Conference on Industrial Informatics (IEEE INDIN '08), pp. 667–672, Daejeon, Republic of Korea, July 2008.
- S. M. K. Mohamed and S. Nahavandi, “Robust finite-horizon Kalman filtering for uncertain discrete-time systems,” IEEE Transactions on Automatic Control, vol. 57, no. 6, pp. 1548–1552, 2012.
- L. Wenling, J. Yingmin, D. Junping, and Z. Jun, “Robust state estimation for jump Markov linear systems with missing measurements,” Journal of the Franklin Institute, vol. 350, no. 6, pp. 1476–1487, 2013.
- L. Yueyang, Z. Maiying, and Y. Shuai, “On designing fault detection filter for discrete-time nonlinear Markovian jump systems with missing measurements,” in Proceedings of the 31st Chinese Control Conference, pp. 5384–5389, Hefei, China, 2012.
- C. Han and H. Zhang, “Optimal state estimation for discrete-time systems with random observation delays,” Acta Automatica Sinica, vol. 35, no. 11, pp. 1446–1451, 2009.
- H. Shuping and L. Fei, “Adaptive observer-based fault estimation for stochastic Markovian jumping systems,” Abstract and Applied Analysis, vol. 2012, Article ID 176419, 11 pages, 2012.
- O. L. V. Costa, M. D. Fragoso, and R. P. Marques, Discrete-Time Markov Jump Linear Systems, Springer, New York, NY, USA, 2005.
- U. Orguner and M. Demirekler, “An online sequential algorithm for the estimation of transition probabilities for jump Markov linear systems,” Automatica, vol. 42, no. 10, pp. 1735–1744, 2006.
- O. L. V. Costa and M. D. Fragoso, “Stability results for discrete-time linear systems with Markovian jumping parameters,” Journal of Mathematical Analysis and Applications, vol. 179, no. 1, pp. 154–178, 1993.
- O. L. V. Costa and S. Guerra, “Stationary filter for linear minimum mean square error estimator of discrete-time Markovian jump systems,” IEEE Transactions on Automatic Control, vol. 47, no. 8, pp. 1351–1356, 2002.
- M. H. A. Davis and R. B. Vinter, Stochastic Modelling and Control, Monographs on Statistics and Applied Probability, Chapman & Hall, New York, NY, USA, 1985.
Copyright © 2014 Qing Sun et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.