New Results on Stability and Stabilization of Markovian Jump Systems with Partly Known Transition Probabilities

Abstract

This paper investigates the problem of stability and stabilization of Markovian jump linear systems with partial information on transition probability. A new stability criterion is obtained for these systems. Comparing with the existing results, the advantage of this paper is that the proposed criterion has fewer variables, however, does not increase any conservatism, which has been proved theoretically. Furthermore, a sufficient condition for the state feedback controller design is derived in terms of linear matrix inequalities. Finally, numerical examples are given to illustrate the effectiveness of the proposed method.

1. Introduction

Over the past few decades, Markov jump systems (MJSs) have drawn much attention of researchers throughout the world. This is due to their important roles in many practical systems. That is, MJSs are quite appropriate to model the plants whose structures are subject to random abrupt changes, which may result from random component failures, abrupt environment changes, disturbance, changes in the interconnections of subsystems, and so forth [1].

Since the transition probabilities in the jumping process determine the behavior of the MJSs, the main investigation on MJSs is to assume that the information on transition probabilities is completely known (see, e.g., [2–5]). However, in most cases, the transition probabilities of MJSs are not exactly known. Whether in theory or in practice, it is necessary to further consider more general jump systems with partial information on transition probabilities. Recently, [6–9] considered the general MJSs with partly unknown transition probabilities. But in these papers, when the terms containing unknown transition probabilities were separated from others, the fixed connection weighting matrices were introduced, which may lead to the conservatism. As noticed, it, currently [10], have achieved an excellent work of reducing the conservatism. The basic idea is to introduce free-connection weighting matrices to substitute the fixed connection weighting matrices. However, this means that the method of [10] has to increase the number of decision variables. As shown in [11], more decision variables imply the augmentation of the numerical burden. Therefore, developing some new methods without introducing any additional variable meanwhile without increasing conservatism will be a valuable work, which motivates the present study.

In this paper, we are concerned with the problem of the stability and stabilization of MJSs with partly unknown transition probabilities. By fully unitizing the relationship among the transition rates of various subsystems, we obtain a new stability criterion. The proposed criterion avoids introducing any connection weighting matrix; however, do not increase any conservatism comparing to that of [10], which has been proved theoretically. More importantly, because the proposed stability criterion need not introduce any slack matrix, the relationships among Lyapunov matrices are highlighted. Therefore, it helps us to understand the effect of the unknown transition probabilities on the stability. Then, based on the proposed stability criterion, the condition for the controller design is derived in terms of LMIs. Finally, numerical examples are given to illustrate the effectiveness of the proposed method.

Notation
In this paper, ℝ^𝑛 and ℝ^𝑛×𝑚 denote the 𝑛-dimensional Euclidean space and the set of all 𝑛×𝑚 real matrices, respectively. ℤ⁺ represents the set of positive integers. The notation 𝑃>0(𝑃≥0) means that 𝑃 is a real symmetric and positive definite (semipositive-definite) matrix. For notation (Ω,ℱ,𝒫), Ω represents the sample space, ℱ is the 𝜎-algebra of subsets of the sample space, and 𝒫 is the probability measure on ℱ. 𝐸{⋅} stands for the mathematical expectation. Matrices, if their dimensions are not explicitly stated, are assumed to be compatible for algebraic operations.

2. Problem Formulation

Consider the following stochastic system with Markovian jump parameters: 𝑥̇(𝑡)=𝐴𝑟_𝑡𝑥(𝑡)+𝐵𝑟_𝑡𝑢(𝑡), (2.1) where 𝑥(𝑡)∈ℝ^𝑛 is the state vector, 𝑥₀ denotes initial condition, and {𝑟_𝑡},𝑡≥0 is a right-continuous Markov process on the probability space taking values in a finite state space 𝕊={1,2,…,𝑁} with generator Λ={𝜋_𝑖𝑗},𝑖,𝑗∈𝕊, given by Pr𝑟_𝑡+Δ=𝑗∣𝑟_𝑡=𝜋=𝑖^𝑖𝑗Δ+𝑜(Δ),𝑖≠𝑗,1+𝜋_𝑖𝑗Δ+𝑜(Δ),𝑖=𝑗, (2.2) where Δ>0, lim_Δ→0𝑜(Δ)/Δ=0, and 𝜋_𝑖𝑗≥0, for 𝑖≠𝑗, is the transition rate from mode 𝑖 at time 𝑡 to mode 𝑗 at time 𝑡+Δ, and 𝜋_𝑖𝑖=−∑^𝑁_{𝑗=1,𝑖≠𝑗}𝜋_𝑖𝑗.  𝐴(𝑟_𝑡) are known matrix functions of the Markov process.

Since the transition probability depends on the transition rates for the continuous-time MJSs, the transition rates of the jumping process are considered to be partly accessible in this paper. For instance, the transition rate matrix Λ for system (2.1) with 𝑁 operation modes may be expressed as 𝜋₁₁?𝜋₁₃⋯????⋯𝜋_2𝑁⋮⋮⋮⋱⋮?𝜋_𝑁2𝜋_𝑁3⋯𝜋_𝑁𝑁, (2.3) where “?” represents the unknown transition rate.

For notational clarity, for all 𝑖∈𝕊, we denote 𝕊=𝕊^𝑖_𝑘∪𝕊^𝑖_𝑢𝑘 with 𝕊^𝑖_𝑘≜{𝑗∶𝜋_𝑖𝑗 is known for 𝑗∈𝕊} and 𝕊^𝑖_𝑢𝑘≜{𝑗∶𝜋_𝑖𝑗 is unknown for 𝑗∈𝕊}.

Moreover, if 𝕊^𝑖_𝑘≠∅, it is further described as 𝕊^𝑖_𝑘=𝑘^𝑖₁,𝑘^𝑖₂,…,𝑘^𝑖_𝑚, (2.4) where 𝑚 is a nonnegative integer with 1≤𝑚≤𝑁 and 𝑘^𝑖_𝑗∈ℤ⁺,1≤𝑘^𝑖_𝑗≤𝑁,𝑗=1,2,…,𝑚 represent the 𝑗th known element of the set 𝕊^𝑖_𝑘 in the 𝑖th row of the transition rate matrix Λ.

For the underlying systems, the following definitions will be adopted in the rest of this paper. More details refer to [2].

Definition 2.1. The system (2.1) with 𝑢(𝑡)=0 is said to be stochastically stable if the following inequality holds 𝐸^∞₀‖𝑥(𝑡)‖²𝑑𝑡∣𝑥₀,𝑟₀<∞, (2.5) for every initial condition 𝑥₀∈ℝ^𝑛 and 𝑟₀∈𝕊.
To this end, we introduce the following result on the stability analysis of systems (2.1).

Lemma 2.2 (see [2]). The system (2.1) with 𝑢(𝑡)=0 is stochastically stable if and only if there exists a set of symmetric and positive-definite matrices 𝑃_𝑖, 𝑖∈𝕊, satisfying 𝐴^𝑇_𝑖𝑃_𝑖+𝑃_𝑖𝐴_𝑖+^𝑁_𝑗=1𝜋_𝑖𝑗𝑃_𝑗<0. (2.6)

Remark 2.3. Since the unknown transition rates may have infinitely admissible values, it is impossible to be directly used the inequalities of Lemma 2.2 to test the stability of the system.

3. Stochastic Stability Analysis

In this section, a stochastic stability criterion for MJSs is given without any additional weighting matrix.

Theorem 3.1. The system (2.1) with a partly unknown transition rate matrix (2.3) and 𝑢(𝑡)=0 is stochastically stable if there exist matrices 𝑃_𝑖>0, such that the following LMIs are feasible for 𝑖=1,2,…,𝑁.
If 𝑖∉𝕊^𝑖_𝑘, 𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘−𝜋^𝑖_𝑘𝑃_𝑖𝑃<0,_𝑖−𝑃_𝑗≥0,∀𝑗∈𝕊^𝑖_𝑢𝑘,𝑗≠𝑖. (3.1)(3.2)
If 𝑖∈𝕊^𝑖_𝑘, 𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘−𝜋^𝑖_𝑘𝑃_𝑗<0,∀𝑗∈𝕊^𝑖_𝑢𝑘, (3.3) where 𝒫^𝑖_𝑘=∑_{𝑗∈𝕊𝑖𝑘}𝜋_𝑖𝑗𝑃_𝑗 and 𝜋^𝑖_𝑘=∑_{𝑗∈𝕊𝑖𝑘}𝜋_𝑖𝑗.

Proof. Based on Lemma 2.2, we know that the system (2.1) with 𝑢(𝑡)=0 is stochastically stable if (2.6) holds. Now we prove that (3.1)–(3.3) guarantee that (2.6) holds by the following two cases.Case I (𝑖∉𝕊^𝑖_𝑘). In this case, (2.6) can be rewritten as Φ_𝑖≜𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘+𝜋_𝑖i𝑃_𝑖+∑_{𝑗∈𝕊𝑖𝑢𝑘,𝑗≠𝑖}𝜋_𝑖𝑗𝑃_𝑗<0. (3.4) Note that in this case ∑_{𝑗∈𝕊𝑖𝑢𝑘,𝑗≠𝑖}𝜋_𝑖𝑗=−𝜋_𝑖𝑖−𝜋^𝑖_𝑘 and 𝜋_𝑖𝑗≥0,𝑗∈𝕊^𝑖_𝑢𝑘,𝑗≠𝑖; then from (3.2), we have Φ_𝑖≤𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘+𝜋_𝑖𝑖𝑃_𝑖+_{𝑗∈𝕊𝑖𝑢𝑘,𝑗≠𝑖}𝜋_𝑖𝑗𝑃_𝑖=𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘+𝜋_𝑖𝑖𝑃_𝑖+−𝜋_𝑖𝑖−𝜋^𝑖_𝑘𝑃_𝑖=𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘−𝜋^𝑖_𝑘𝑃_𝑖. (3.5) Therefore, if 𝑖∉𝕊^𝑖_𝑘, inequalities (3.1) and (3.2) imply that (2.6) holds.Case II (𝑖∈𝕊^𝑖_𝑘). Note that in this case −𝜋^𝑖_𝑘=∑_{𝑗∈𝕊𝑖𝑢𝑘}𝜋_𝑖𝑗 and 𝜋_𝑖𝑗≥0,𝑗∈𝕊^𝑖_𝑢𝑘. So, if −𝜋^𝑖_𝑘=0, then we must have 𝕊^𝑖_𝑢𝑘=∅. Therefore, (2.6) becomes 𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘<0, (3.6) which is equivalent to (3.3) noticing −𝜋^𝑖_𝑘=0.
Else, if −𝜋^𝑖_𝑘≠0, then we must have −𝜋^𝑖_𝑘>0. Then, (2.6) can be rewritten as _{𝑗∈𝕊𝑖𝑢𝑘}𝜋_𝑖𝑗𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+𝒫^𝑖_𝑘−𝜋^𝑖_𝑘+𝑃_𝑗<0. (3.7) Obviously, (3.3) implies that (3.7) holds. Then, if 𝑖∉𝕊^𝑖_𝑘, (3.3) implies that (2.6) holds.
Therefore, if LMIs (3.1)–(3.3) hold, we conclude that system (2.1) is stochastically stable according to Lemma 2.2. The proof is completed.

Theorem 3.1 proposed in this paper does not introduce any free variable. It involves 𝑁𝑛(𝑛+1)/2 variables, while Theorem 3.3 in [10] involves 𝑁𝑛(𝑛+1) variables. Namely, the number of variables in this paper is only half of [10]. Generally, reducing the number of decision variables easily results in increasing conservatism of stability criteria. However, Theorem 3.1 in this paper does not increase conservatism while with less variables. To show this, we rewrite it as follows.

Theorem 3.2 (see [10]). The system (2.1) with partly unknown transition rate matrix (2.3) and 𝑢(𝑡)=0 is stochastically stable if there exist matrices 𝑃_𝑖>0, 𝑊_𝑖=𝑊^𝑇_𝑖, such that the following LMIs are feasible for 𝑖=1,2,…,𝑁, 𝑃_𝑖𝐴_𝑖+𝐴^𝑇_𝑖𝑃_𝑖+_{𝑗∈𝕊𝑖𝑘}𝜋_𝑖𝑗𝑃_𝑗−𝑊_𝑖𝑃<0,_𝑗−𝑊_𝑖≤0,𝑗∈𝕊^𝑖_𝑢𝑘𝑃,𝑗≠𝑖,_𝑗−𝑊_𝑖≥0,𝑗∈𝕊^𝑖_𝑢𝑘,𝑗=𝑖. (3.8)(3.9)(3.10)

Now we have the following conclusion.

Theorem 3.3. Suppose for system (2.1) with partly unknown transition rate matrix (2.3) there exist 𝑃_𝑖>0 and 𝑊_𝑖=𝑊^𝑇_𝑖, 𝑖=1,2,…,𝑁, such that (3.8)–(3.10) hold; then the matrices 𝑃_𝑖>0, 𝑖=1,2,…,𝑁, satisfy (3.1)–(3.3).

Proof. If 𝑖∉𝕊^𝑖_𝑘, (3.9)–(3.10) imply that (3.2) and the following inequality hold: 𝑃_𝑖⩾𝑊_𝑖. (3.11) From (3.11) and (3.8), we can obtain (3.1).
If 𝑖∈𝕊^𝑖_𝑘, (3.9) and (3.10) guarantee that 𝑃_𝑗−𝑊_𝑖≤0,𝑗∈𝕊^𝑖_𝑢𝑘. (3.12) In addition, under this circumstance, we have _{𝑗∈𝕊𝑖𝑘}𝜋_𝑖𝑗≤0. (3.13) Then, From (3.12), (3.13), and (3.8), we obtain that (3.3) holds. The proof is completed.

Remark 3.4. The stability condition in [10] and that in Theorem 3.1 are derived via different techniques. Now Theorem 3.3 proves that the former can be simplified to the latter without increasing any conservatism. More importantly, because Theorem 3.1 of this paper does not involve any slack matrix, the relationships among Lyapunov matrices are highlighted. Therefore, it is clearer how the unknown transition probabilities affect on the stability.

4. State-Feedback Stabilization

In this section, the stabilization problem of system (2.1) with control input 𝑢(𝑡) is considered. The mode-dependent controller with the following form is designed: 𝑢(𝑡)=𝐾𝑟_𝑡𝑥(𝑡), (4.1) where 𝐾(𝑟_𝑡) for all 𝑟_𝑡∈𝕊 are the controller gains to be determined. In the following, for given 𝑟_𝑡=𝑖∈𝕊,𝐾(𝑟_𝑡)=𝐾_𝑖.

Using (4.1), the system (2.1) is represented as𝑥̇(𝑡)=𝐴𝑟_𝑡+𝐵𝑟_𝑡𝐾𝑟_𝑡𝑥(𝑡). (4.2) The following theorem is proposed to design the mode-dependent stabilizing controller with the form (4.1) for system (2.1).

Theorem 4.1. The closed-loop system (4.2) with a partly unknown transition rate matrix (2.3) is stochastically stable if, there exist matrices 𝑄_𝑖>0 and 𝑌_𝑖, 𝑖=1,2,…,𝑁 such that the following LMIs are feasible for 𝑖=1,2,…,𝑁.
If 𝑖∉𝕊^𝑖_𝑘, Ξ_𝑖−𝜋^𝑖_𝑘𝑄_𝑖Π_1𝑖∗−Ψ_1𝑖<0,−𝑄_𝑖𝑄_𝑖∗−𝑄_𝑗≤0,∀𝑗∈𝕊^𝑖_𝑢𝑘&𝑗≠𝑖. (4.3)(4.4)
If 𝑖∈𝕊^𝑖_𝑘, Ξ_𝑖+𝜋_𝑖𝑖𝑄_𝑖Π_2𝑖−𝜋^𝑖_𝑘𝑄_𝑖∗−Ψ_2𝑖0∗∗−𝑄_𝑗<0,∀𝑗∈𝕊^𝑖_𝑢𝑘, (4.5) where Ξ_𝑖=𝐴_𝑖𝑄_𝑖+𝑄_𝑖𝐴^𝑇_𝑖+𝐵_𝑖𝑌_𝑖+𝑌^𝑇_𝑖𝐵^𝑇_𝑖,Π_1𝑖=𝜋_𝑖𝑘^𝑖₁𝑄_𝑖𝜋_𝑖𝑘^𝑖₂𝑄_𝑖⋯𝜋_{𝑖𝑘𝑖𝑚}𝑄_𝑖,Ψ_1𝑖=diag𝑄_𝑘^𝑖₁,𝑄_𝑘^𝑖₂,…,𝑄_𝑘𝑖𝑚,Π_2𝑖=𝜋_𝑖𝑘^𝑖₁𝑄_𝑖⋯𝜋_𝑖𝑘^𝑖_𝑙−1𝑄_𝑖𝜋_𝑖𝑘^𝑖_𝑙+1𝑄_𝑖⋯𝜋_{𝑖𝑘𝑖𝑚}𝑄_𝑖,Ψ_2𝑖=diag𝑄_𝑘^𝑖₁,…,𝑄_𝑘^𝑖_𝑙−1,𝑄_𝑘^𝑖_𝑙+1,…,𝑄_𝑘𝑖𝑚, (4.6) with 𝑘^𝑖₁,𝑘^𝑖₂,…,𝑘^𝑖_𝑚 described in (2.4) and 𝑘^𝑖_𝑙=𝑖.
Moreover, if (4.3)–(4.5) are true, the stabilization controller gains from (4.1) are given by 𝐾_𝑖=𝑌_𝑖𝑄_𝑖⁻¹. (4.7)

Proof. It is clear that the system (4.2) is stable if the following conditions are satisfied.
If 𝑖∉𝕊^𝑖_𝑘, 𝑃_𝑖𝐴_𝑖+𝐵_𝑖𝐾_𝑖+𝐴_𝑖+𝐵_𝑖𝐾_𝑖^𝑇𝑃_𝑖+𝒫^𝑖_k−𝜋^𝑖_𝑘𝑃_𝑖𝑃<0,_𝑗−𝑃_𝑖≤0,∀𝑗∈𝕊^𝑖_𝑢𝑘,𝑗≠𝑖. (4.8)(4.9)
If 𝑖∈𝕊^𝑖_𝑘, 𝑃_𝑖𝐴_𝑖+𝐵_𝑖𝐾_𝑖+𝐴_𝑖+𝐵_𝑖𝐾_𝑖^𝑇𝑃_𝑖+𝒫^𝑖_𝑘−𝜋^𝑖_𝑘𝑃_𝑗<0,∀𝑗∈𝕊^𝑖_𝑢𝑘. (4.10) Pre- and postmultiply the left sides of (4.8)–(4.10) by 𝑃_𝑖⁻¹, respectively, and introduce the following new variables: 𝑄_𝑖=𝑃_𝑖⁻¹,𝑌_𝑖=𝐾_𝑖𝑃_𝑖⁻¹. (4.11) Then, inequalities (4.8)–(4.10) are equivalent to the following matrix inequalities, respectively.
If 𝑖∉𝕊^𝑖_𝑘, Ξ_𝑖−𝜋^𝑖_𝑘𝑄_𝑖+Π_1𝑖Ψ⁻¹_1𝑖Π^𝑇_1𝑖𝑄<0,_𝑖𝑃_𝑗𝑄_𝑖−𝑄_𝑖≤0,∀𝑗∈𝕊^𝑖_𝑢𝑘,𝑗≠𝑖. (4.12)(4.13)
If 𝑖∈𝕊^𝑖_𝑘, Ξ_𝑖+𝜋_𝑖𝑖𝑄_𝑖+Π_𝑖2Ψ⁻¹_2𝑖Π^𝑇_𝑖2+−𝜋^𝑖_𝑘𝑄_𝑖𝑃_𝑗𝑄_𝑖<0,∀𝑗∈𝕊^𝑖_𝑢𝑘. (4.14) By applying Schur complement, inequalities (4.12)–(4.14) are equivalent to LMIs (4.3)–(4.5), respectively.
Therefore, if LMIs (4.3)–(4.5) hold, the closed-loop system (4.2) is stochastically stable according to Theorem 3.1. Then, system (2.1) can be stabilized with the state feedback controller (4.1), and the desired controller gains are given by (4.7). The proof is completed.

Remark 4.2. The number of variables involved in Theorem 4.1 in this paper is also 𝑁𝑛(𝑛+1)/2 less than that in the corresponding result of [10]. Furthermore, it can be seen that no conservativeness is introduced when deriving Theorem 4.1 from Theorem 3.1. Therefore, the stabilization method presented in Theorem 4.1 is not more conservative than that of [10], too.

5. Numerical Example

In this section, an example is provided to illustrate the effectiveness of our results.

Consider the following MJSs, which borrowed from [10] with small modifications,𝐴₁=32−7.301.480.81,𝐴₂=,𝐴0.89−3.111.480.21₃=−0.11−0.852.31−0.10,𝐴₄=,𝐵−0.17−1.481.59−0.27₁=0.571.23,𝐵₂=,𝐵0.78−0.49₃=1.340.39,𝐵₄=.−0.381.07 (5.1) The partly transition rate matrix Λ is considered as Λ=−1.3−𝑎0.2????0.30.30.6?−1.5?0.4???, (5.2) where the parameter 𝑎 in matrix Λ can take different values for extensive comparison purpose.

We consider the stabilization of this system corresponding to different 𝑎 by using different approaches. Considering the precision of comparison, we let 𝑎 increase starting from 0 with a very small constant increment 0.01. Using the LMI toolbox in MATLAB, both the LMIs in Theorem 5 of [10] and the ones in Theorem 4.1 of this paper are feasible for all 𝑎=0,0.01,…,1.64, and are infeasible when 𝑎 increases to 1.65. It can be seen that for this example the stabilization method in our paper is not conservative than that in [10].

Now by some simulation results, we further show the effectiveness of the stabilization method of this paper. For example, when 𝑎=1.64, in our method, the controller gains are obtained as 𝐾₁=−151435252,𝐾₂=,𝐾−873700392930₃=−1086.6−290.8,𝐾₄=440620−300480. (5.3)

Figure 1 is the state response cures in 1000 random sampling with initial condition 𝑥₀=[1−1]^𝑇 when 𝑢(𝑡)=0. In each random sampling, the transition rate matrix is randomly generated but satisfies the partly transition rate matrix Λ in (5.2). Figure 1 shows that the open-loop system is unstable.

Figure 1 

State response of the open-loop system with 1000 random samplings.

Applying the controllers in (5.3), the trajectory simulation of state response for the closed-loop system with 1000 random sampling is shown in Figure 2 under the given initial condition 𝑥₀=[1−1]^𝑇. In this case, the transition rate matrix is also randomly generated but satisfies the partly transition rate matrix Λ in (5.2). Figure 2 shows that the stabilizing controller effectively keeps the running reliability of the system.

Figure 2 

State response of the closed-loop system with 1000 random samplings.

6. Conclusions

This paper has considered the problem of stability and stabilization of a class continuous-time MJSs with unknown transition rates. A new stability criterion has been proposed. The merit of the proposed criterion is that it has less decision variables without increasing conservatism comparing those in the literature to date. Then, the mode-dependent state feedback controller designing method has been proposed, which possess the same merit as the stability criterion. Numerical examples have been given to illustrate the effectiveness of the results in this paper.

Acknowledgment

This work was supported by National Natural Science Foundation of China (61104115 and 61074009), Research Fund for the Doctoral Program of Higher Education of China (20110072120018), and Young Excellent Talents in Tongji University (2009KJ035).