Research Article | Open Access

Chunyan Zhai, Dingyu Xue, Shuchen Li, Guoliang Wang, "Stabilization of Discrete-Time Markovian Jump Systems via Controllers with Partially Mode-Dependent Characterization", *Mathematical Problems in Engineering*, vol. 2013, Article ID 936375, 9 pages, 2013. https://doi.org/10.1155/2013/936375

# Stabilization of Discrete-Time Markovian Jump Systems via Controllers with Partially Mode-Dependent Characterization

**Academic Editor:**Hao Shen

#### Abstract

A kind of stabilizing controller in terms of being partially mode-dependent is developed for discrete-time Markovian jump systems (MJSs). The property referred to be partially mode-dependent is described by the Bernoulli variable. Based on the established model, the stabilization for MJSs over unreliable networks is considered, where both network-induced delay and packet dropout take place in system modes and states. Such effects of network are taken into account in controller design. All the conditions are derived in terms of linear matrix inequalities (LMIs). Finally, illustrative examples are presented to show the effectiveness and applicability of the proposed method.

#### 1. Introduction

Markovian jump systems (MJSs) [1–9] are regarded as a special class of hybrid systems with finite operation modes. Their structures are subject to random abrupt changes, which may result from abrupt phenomena such as random component failures or repairs, changes in subsystem interconnections, and sudden environmental changes and modification of the operating point of a linearized model of a nonlinear system. Over the past decade, many important issues have been investigated for discrete-time Markovian jump systems (DMJSs), such as stability and stabilization [10, 11] and and control [12–15]. By investigating such references, it is known that system mode is assumed to be available online and plays important roles in system analysis and synthesis. This will have the application scope of mode-dependent results limited. In order to remove such ideal assumption, a mode-independent stabilization problem for DMJSs was considered in [16]. The operation mode is not necessary in the work of the designed controller and is totally ignored.

However, the feedback control systems are usually closed via real time networks. Instead of point-to-point wiring, the use of shared communication networks is mainly due to its great advantages, such as low cost, reduced weight and power requirements, simple installation and maintenance, and high reliability. Despite the advantages and potentials, the introduction of communication networks in feedback control loops complicates the networked control systems (NCSs) analysis and synthesis [17], which also proposes new interesting and challenging problems. Network-induced delays and packet dropouts are usually considered in NCSs. Because of such effects, the data transmitted through networks is usually accessible with some probability. Based on this fact, it is said that the mode-independent control method is an overdesign method, which sometimes will lead to no solution to the desired control law. Since mode-independent and mode-dependent controller designs are worst-case and best-case design methods, respectively, a question whether there is a control method to bridge the aforementioned two extreme cases is proposed naturally.

Via modeling two random delays as two independent Markov chains, sufficient and necessary condition on the existence of stabilizing controller is established in [18]. On the other hand, [19, 20] considered the stability and controller design of NCSs with packet dropouts modeled as Markov chains. It is seen that the plant of the underlying NCSs in most literature is a deterministic system. That means that there is only one operation mode in the original system. When the controlled plant is an MJS, few results are available. Then, it is natural to consider how the insertion of networks affects the stability and performance of the closed-loop MJSs. Recently, for DMJSs with time-delays in both system state and mode signal, the authors in [21] first showed that the closed-loop system is a time-varying delayed MJS with extended state space. A general case that the time delays in signal mode and system state are the same and time-varying is considered in [22]. To the best of the authors’ knowledge, the stabilization problem of MJSs via networks has not been investigated fully and still remains challenging. All the observations motivate the current research.

In this paper, a partially mode-dependent controller design method is developed to solve the stabilization problem of DMJS. Instead of a common Lyapunov function selected for all system modes, a stochastic Lyapunov function is exploited. Moreover, we utilize the proposed technique to stabilize NCSs, whose controlled plant is an MJS. Both time-induced delays and packet dropouts are taken into account in controller design. All the conditions for the solvability of the stabilizing controllers are established in terms of LMIs.

*Notation*. denotes the dimensional Euclidean space and is the set of all real matrices. denotes the Euclidean norm. is the expectation operator with respect to some probability measure. In symmetric block matrices, we use “” as an ellipsis for the terms induced by symmetry, for a block-diagonal matrix, and .

#### 2. Preliminaries

Fix the probability space and consider the following DMJS where is the state vector, is the control input, and is a discrete-time homogeneous Markov chain taking values in a finite set with generator ; that is where , , and . For notation simplification, a matrix will be denoted by .

Now, consider a partially mode-dependent, state feedback control law where and are control gains to be determined and is a Bernoulli distributed sequence with where is a constant. Furthermore, from (4), we have

*Remark 1. *It is said that the stochastic variable is used to reflect the observation probability of system mode. If , controller (3) will become a mode-dependent controller, and, when , we will have a mode-independent controller. It is worth mentioning that controller (3) is more advantageous than a totally mode-dependent or mode-independent controller. Compared with mode-independent controller in [16], one advantage of (3) avoids the overdesign problem which may lead to no solution to the desired control law. To the contrary, compared with mode-dependent control methods [11–14], another advantage is that it can reduce the burden of data transmission. That is, we can measure mode signal with some probability instead of measuring them exactly online. Both the two advantages are shown via numerical examples.

*Remark 2. *It should be remarked that the introduction of (3) applied to (1) does not destroy the Markov process in (1) with respect to . Because the Bernoulli process is a special case of Markov process, it is seen as another Markov process. In this paper, two stochastic variables and are assumed to be independent from each other.

Applying the controller in (3) to the system in (1) results in the following closed-loop system:
where for all.

*Definition 3. *The closed-loop system (6) is stochastically stable if
for every every initial condition and .

#### 3. Main Results

Firstly, we give the stabilization condition for DMJS (1) via controller (3).

Theorem 4. *Considering DMJS (1), there exists a controller (3) such that closed-loop system (6) is stochastically stable if there exist matrices , , , and , satisfying the following LMIs, for all :
**
Moreover, the corresponding control laws are given by
*

*Proof. *Choose a stochastic Lyapunov function for the system in (6) as

By condition (5), one has
where
From (8), it is concluded that is nonsingular. Pre- and postmultiplying (8) with and its transpose, respectively, and letting , we have
Since , we have
By the Schur complement and (14), it is known that (13) implies . Therefore, we have
where . From Dynkin’s formula and summing up (15) form 0 to with respect to yields, we obtain
By Definition 3, system (6) is stochastically stable. This completes the proof.

*Remark 5. *It is worth mentioning that controller with partially mode-dependent property (3) is established in terms of LMIs via a stochastic Lyapunov function instead of a common Lyapunov function, where both totally mode-dependent and mode-independent controllers are contained. It is said that controller (3) bridges such two extreme cases. It is also seen that in partially mode-dependent method plays an important role, which should be known. When it is unknown or unavailable, how to design an effective controller is necessary to be considered. Two possible ways may deal with such problem. The first one is to use a robust method to handle (6). On the other hand, this general case can be done by use an adaptive control. The unknown probability will be estimated by an adaption law.

Next, we will apply key idea of (3) to NCSs. The controlled plant is an MJS as shown in Figure 1.

As demonstrated in Figure 1, system state and mode signal are transmitted through networks with communication delays and , respectively. Considering the effect of random delays and , the control law is described as where and .

*Remark 6. *If delay of system mode is constant, controller (17) is similar to that in [21] where the authors first showed that the resulting closed-loop system is a new time-varying delayed MJS with extended state space. For the case that delay is similar to delay , the resulting closed-loop system is modeled as an MJS with two modes via augmentation technique [22]. Both of the aforesaid references constructed a mode-dependent stabilizing controller with an assumption that there is no packet missing in channel 1. However, it is a more realistic situation that not only network-induced delays but also packet dropouts exist in channel 1, when system mode is transmitted. Since both network-induced delays and packet dropouts affect system mode , it is natural to consider both of them in the controller design. Unfortunately, controller (17) will not be applicable to this case.

Based on the key idea of (3), we know that the effect of dropouts on system mode does not destroy the Markov process with respect to . So we construct the following controller:
where is function as

Thus, is a Bernoulli distributed sequence satisfying (4) and (5). From [11], we know that constant delay in system mode results in the original system being no longer traditional MJSs. However, the closed system is a time-varying delayed MJS with extended state space. In addition to Remark 2, we know that the closed-loop system (1) via controller (18) is also an MJS with respect to .

Define , and applying controller (18) to system (1) results in the following closed-loop system:
where

*Definition 7. *Let be the solution to closed-loop system (20). The closed-loop system is stochastically stable if
for every initial conditions , , and , .

Now, we will consider the stabilization condition of MJS (20).

Theorem 8. *Considering DMJS (1), there exists a controller (18) such that closed-loop system (20) is stochastically stable if there exist matrices , , , , , and , satisfying the following LMIs, for all , and :
**
where
**Moreover, the corresponding control laws are given by
*

*Proof. *Let . Choose a stochastic Lyapunov function for system in (20) as
where

From [21], we have
where is similar to that in [21], and , if ; otherwise, . Thus, we have

Then, by computation, we have

Taking into account (30), we conclude
where

From (23), we have that is nonsingular. Defining , , and , we have , , and . Similar to (14), we have

In addition, by the Schur complement, we obtain that is equivalent to the following:

Pre- and postmultiplying (34) by and its transpose, respectively, and taking into account (23), (25), and (33), we have that (34) holds, which implies (31) holds. Then, we have that the system in (20) is stochastically stable via the similar lines in [13]. This completes the proof.

*Remark 9. *It should be pointed out that it is not appreciate to model transmission delays or packet losses as Markov process, since both of them occur in channel 1 simultaneously. However, the control technique based on partially mode-pendent controller can be used to tackle this case. The scalar may reflect the jam degree of networks. That is, the lager value of denotes the more expedite of networks. Comparing to mode-independent controller with , it is less conservative in the sense that the desired controller is also constructed, when there is no solution to a totally mode-independent controller. When in terms of being a mode-dependent controller, a partially mode-dependent controller (18) is less dependent on the transmitted system modes, which could reduce the burden of communication networks. Both of the advantages can be shown via a numerical example. In addition, the solvability of controller (18) is established in terms of LMIs, which is solved directly by Matlab LMI toolbox.

#### 4. Numerical Examples

In this section, two examples are given to demonstrate the effectiveness of the proposed approaches in this paper.

*Example 1. *Consider the following inverted pendulum system [23]:
where , is the half length of the pendulum, and and are the masses of the cart and the pendulum, respectively. Here, we choose two samples; that is, , , and and , , and, respectively. Assuming the sampling period , we have two discrete-time models for the system in (35) described as

The transition probability matrix is given by

Under the initial condition , the state response of open system is shown in Figure 2. Obviously, it is not stable. By Theorem 4, we have the following controllers with :

Applying such controllers to the above system, the trajectory of the closed loop is illustrated in Figure 3.

*Example 2. *Consider the following Markovian jump system described as

The transition probability matrix is given by

Let , , and . The initial conditions are and and and, respectively. The state response of the open-loop system (with ) is shown in Figure 4, which is not stable. By Theorem 8, it is known that there is no solution, when . That means that we cannot construct a totally mode-independent controller. Instead, we have the following controllers with and , respectively.

*Case *

*Case *

The state trajectories of the closed-loop system via controllers and are shown in Figure 5, respectively, so is shown in Figure 6. Though is a partially mode-pendent controller, it can also stabilize this system. Because , it means that the system mode of controller is not necessarily transmitted through network online. In this sense, it is said that the partially mode-pendent controller is advantageous over a mode-dependent controller in terms of reducing network expedite.

#### 5. Conclusions

This paper has investigated the stabilization problem for DMJSs via a partially mode-dependent controller. Sufficient condition for such controller is developed within LMI framework. Moreover, the key idea of this control scheme has been utilized to networked control system. In this case, the controlled plant is an MJS with simultaneous consideration of network-induced delays and data packet dropouts in system mode transmission. Numerical simulations are used to illustrate the effectiveness and applicability of the developed results.

#### Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grants 61104066 and 61374043, the China Postdoctoral Science Foundation funded project under Grant 2012M521086, and the Program for Liaoning Excellent Talents in University under Grant LJQ2013040.

#### References

- G. Z. Wu, P. Shi, H. Su, and J. Chu, “Passivity analysis for discrete-time stochastic Markovian jump neural networks with mixed time delays,”
*IEEE Transactions on Neural Networks*, vol. 22, no. 10, pp. 1566–1575, 2011. View at: Google Scholar - S. P. He and F. Liu, “Adaptive observer-based fault estimation for stochastic Markovian jumping systems,”
*Abstract and Applied Analysis*, vol. 2012, Article ID 176419, 11 pages, 2012. View at: Publisher Site | Google Scholar - S. P. He and F. Liu, “Robust stabilization of stochastic Markovian jumping systems via proportional integral control,”
*Signal Processing*, vol. 91, no. 11, pp. 2478–2486, 2011. View at: Google Scholar - H. Shen, X. Huang, J. Zhou, and Z. Wang, “Global exponential estimates for uncertain Markovian jump neural networks with reaction-diffusion terms,”
*Nonlinear Dynamics*, vol. 69, no. 1-2, pp. 473–486, 2012. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - H. Shen, Z. Wang, X. Huang, and J. Wang, “Fuzzy dissipative control for nonlinear Markovian jump systems via retarded feedback,”
*Journal of the Franklin Institute*, 2103. View at: Publisher Site | Google Scholar - H. Shen, S. Xu, J. Lu, and J. Zhou, “Passivity-based control for uncertain stochastic jumping systems with mode-dependent round-trip time delays,”
*Journal of the Franklin Institute*, vol. 349, no. 5, pp. 1665–1680, 2012. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - G. L. Wang and Q. L. Zhang, “Adaptive control of stochastic nonlinear systems with Markovian switching,”
*International Journal of Adaptive Control and Signal Processing*, vol. 26, no. 9, pp. 848–860, 2012. View at: Google Scholar | Zentralblatt MATH | MathSciNet - G. Wang and Q. Zhang, “Robust control of uncertain singular stochastic systems with Markovian switching via proportional-derivative state feedback,”
*IET Control Theory & Applications*, vol. 6, no. 8, pp. 1089–1096, 2012. View at: Publisher Site | Google Scholar | MathSciNet - G. L. Wang, Q. L. Zhang, and V. Sreeram, “Design of reduced-order ${H}_{\infty}$ filtering for Markovian jump systems with mode-dependent time delays,”
*Signal Processing*, vol. 89, pp. 187–196, 2009. View at: Google Scholar - O. L. V. Costa and M. D. Fragoso, “Stability results for discrete-time linear systems with Markovian jumping parameters,”
*Journal of Mathematical Analysis and Applications*, vol. 179, no. 1, pp. 154–178, 1993. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - P. Shi, E.-K. Boukas, and R. K. Agarwal, “Control of Markovian jump discrete-time systems with norm bounded uncertainty and unknown delay,”
*IEEE Transactions on Automatic Control*, vol. 44, no. 11, pp. 2139–2144, 1999. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - O. L. V. Costa and R. P. Marques, “Robust ${H}_{2}$-control for discrete-time Markovian jump linear systems,”
*International Journal of Control*, vol. 73, no. 1, pp. 11–21, 2000. View at: Publisher Site | Google Scholar | MathSciNet - E. K. Boukas and Z. K. Liu, “Robust ${H}_{\infty}$ control of discrete-time Markovian jump linear systems with mode-dependent time-delays,”
*IEEE Transactions on Automatic Control*, vol. 46, no. 12, pp. 1918–1924, 2001. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - P. Seiler and R. Sengupta, “A bounded real lemma for jump systems,”
*IEEE Transactions on Automatic Control*, vol. 48, no. 9, pp. 1651–1654, 2003. View at: Publisher Site | Google Scholar | MathSciNet - G. Wang, Q. Zhang, and V. Sreeram, “${H}_{\infty}$ control for discrete-time singularly perturbed systems with two Markov processes,”
*Journal of the Franklin Institute*, vol. 347, no. 5, pp. 836–847, 2010. View at: Publisher Site | Google Scholar | MathSciNet - C. E. de Souza, “Robust stability and stabilization of uncertain discrete-time Markovian jump linear systems,”
*IEEE Transactions on Automatic Control*, vol. 51, no. 5, pp. 836–841, 2006. View at: Publisher Site | Google Scholar | MathSciNet - W. Zhang, M. S. Branicky, and S. M. Phillips, “Stability of networked control systems,”
*IEEE Control Systems Magazine*, vol. 21, no. 1, pp. 84–99, 2001. View at: Google Scholar - L. Zhang, Y. Shi, T. Chen, and B. Huang, “A new method for stabilization of networked control systems with random delays,”
*IEEE Transactions on Automatic Control*, vol. 50, no. 8, pp. 1177–1181, 2005. View at: Publisher Site | Google Scholar | MathSciNet - J. Wu and T. Chen, “Design of networked control systems with packet dropouts,”
*IEEE Transactions on Automatic Control*, vol. 52, no. 7, pp. 1314–1319, 2007. View at: Publisher Site | Google Scholar | MathSciNet - J. Xiong and J. Lam, “Stabilization of linear systems over networks with bounded packet loss,”
*Automatica*, vol. 43, no. 1, pp. 80–87, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - J. Xiong and J. Lam, “Stabilization of discrete-time Markovian jump linear systems via time-delayed controllers,”
*Automatica*, vol. 42, no. 5, pp. 747–753, 2006. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - M. Liu, D. W. C. Ho, and Y. Niu, “Stabilization of Markovian jump linear system over networks with random communication delay,”
*Automatica*, vol. 45, no. 2, pp. 416–421, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - H. Gao and T. Chen, “New results on stability of discrete-time systems with time-varying state delay,”
*IEEE Transactions on Automatic Control*, vol. 52, no. 2, pp. 328–334, 2007. View at: Publisher Site | Google Scholar | MathSciNet

#### Copyright

Copyright © 2013 Chunyan Zhai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.