#### Abstract

This paper focuses on the stability issue of discrete-time networked control systems with random Markovian delays and uncertain transition probabilities, wherein the random time delays exist in the sensor-to-controller and controller-to-actuator. The resulting closed-loop system is modeled as a discrete-time Markovian delays system governed by two Markov chains. Using Lyapunov stability theory, a result is established on the Markovian structure and ensured that the closed-loop system is stochastically stable. A simulation example illustrates the validity and feasibility of the results.

#### 1. Introduction

Networked control systems (NCS) find many successful applications in power grids, manufacturing plants, vehicles, aircrafts, spacecrafts, remote surgery, and so on [1]. Compared with the traditional control systems, the use of the communication networks brings many advantages such as low cost, reduced weight, and simple installation and maintenance, as well as high efficiency, flexibility, and reliability. However, inserting communication networks into feedback control loops has also resulted in several interesting and challenging issues, such as packet dropouts [2], time delays [3–10], quantization [11], time-varying transmission intervals [12], distributed synchronization [13], or some of the constraints considered simultaneously [14–17], which make the analysis and design of NCS complex. These imperfections block the way of harvesting reliable NCS by implementing existing control techniques [18]. To overcome these drawbacks, significant attention has been paid to the NCS research ranging from system identification and stability analysis to controller and filter designs. See the survey papers [1, 19, 20] and the references therein.

The network-induced time delays are known to be the major challenges in NCS, which may be potential causes for the deteriorating performance or instability of NCS. Consequently, numerous works have been conducted on the time-delay issue in the past years. For example, in [21], the mixed control issue of NCS with random time delays has been investigated based on Markovian jump linear systems method. In [9], the stability problem of NCS with uncertain time-varying delays has been investigated. The stability and stabilization of NCS with random time delay usually use Markovian jump linear systems (MJLS) approach, and, recently, many significant achievements have been obtained for MJLS in [22–26]. However, most of the approaches for NCS based on Markovian jump systems framework assumed that the Markovian transition probabilities are known a priori, which severely limit the utility of the Markov model. Furthermore, such assumption may not hold true especially in the case where networked control is applied to the remote plants. Recently, the filter problem for a class of uncertain Markovian jumping systems with bounded transition probabilities has been investigated in [27], but the well-established results cannot be directly used to NCS. To the best of the authors’ knowledge, up to now, very limited efforts have been devoted to studying the system with uncertain transition probability matrices for NCS, which motivates our investigation.

In this paper, we address the analysis and design of NCS with random time delays modeled by Markov chains in forward sensor-to-controller (S-C) and feedback controller-to-actuator (C-A) communication links and with the uncertain transition probability matrices. The main contributions of this paper are highlighted as follows. (i) A model is proposed for NCS with random Markovian delays and uncertain transition probability matrices. (ii) The system modeled will be more generalized and avoid the ideal assumption that the transition probabilities are known a priori. (iii) New criteria for stability are obtained based on a Lyapunov approach. Finally, a numerical example is provided to demonstrate the effectiveness of the proposed control scheme for NCS with random time delays and uncertain transition probability matrices.

The remainder of this paper is organized as follows. A model with Markovian delays and uncertain transition probabilities is obtained in Section 2. The main results are obtained based on a Lyapunov approach and the linear matrix inequalities technique in Section 3. Section 4 presents the simulation results. Finally, the conclusions are provided in Section 5.

*Notations.* Matrices are assumed to have appropriate dimensions. and denote the -dimensional Euclidean space and the set of all real matrices, respectively. The notations () indicate that is a real symmetric positive (negative) definite matrix. and denote the identity matrix and the zero matrix with appropriate dimensions, respectively. Superscripts “” and “” stand for the matrix transposition and the matrix inverse, respectively. stands for the mathematical expectation and stands for a block-diagonal matrix of and . and denote the identity matrix and zero matrix with appropriate dimensions, respectively. denotes the expression , and means symmetric terms in symmetric entries.

#### 2. NCS Model

The framework of networked control systems is depicted in Figure 1. The plant, sensor, controller, and actuator are spatially distributed and closed through a network. Random time delays exist in both of S-C and C-A.

The plant is described by the following discrete-time linear time-invariant plant model: where is the system state vector and is the control input. and are known real constant matrices with appropriate dimensions.

For this system, we will consider a state feedback controller as follows: where is the state feedback controller gain.

Random S-C and C-A time delays are and , respectively. and are assumed be bounded; that is, , , where , , , and . One way to model delays and is by using the finite-state Markov chains presented in [21]. The main advantages of the Markov model considering the dependence between delays are that the current time delays in real networks delays are frequently related to the previous delays. In this paper and are modeled as two homogeneous Markov chains.

By substituting controller (2) to plant (1), we obtain a closed-loop system as follows: where .

In system (3), and are two finite state discrete-time homogeneous Markov chains with values in the finite sets and with the uncertain transition probability matrices and . and denote the uncertain transition probability matrices of Markov chain and , respectively, with probabilities and , which are defined by where , and , for all and . The transition probability matrices and are unknown a priori but belong to the following bounded compact set: where () and () are known constant matrices. () and () denote the uncertainty in the transition probability matrices, where and satisfy where , , and and are the known small scalar for all and , respectively.

*Remark 1. *Closed-loop system (3) is a linear system with the Markovian delays and , which describe the behavior of the S-C and C-A random time delays, and with the uncertain transition probabilities.

*Remark 2. *The uncertain transition probabilities and contain the certain terms and , and the uncertain terms and , respectively. The uncertain terms and are bounded, and the sums of the elements in each row are zeros.

#### 3. Stability Analysis and Controller Design

By applying a Lyapunov approach and a linear matrix inequality technique, this section provides sufficient conditions for the stochastic stability and the synthesis of state feedback controller design of the system (3).

*Definition 3 (see [21]). *The closed-loop system (3) is said to be stochastically stable if, for every finite , initial mode and , there exists a finite such that

Theorem 4. *For the system (3), random but bounded scalars and . If, for each mode and and matrices , , , , , and , and exist that satisfy the following matrix inequalities:
**
where
**
and and are defined in (4) and (5).*

Then the closed-loop system (3) is stochastically stable.

*Proof. *For the closed-loop system (3), the stochastic Lyapunov functional candidate is constructed as follows:
with
where , , , , , , and .

Let , noting that . Then . For simplicity, we will use the following notations: . Then, for any weighting matrices with compatible dimensions (and let ), we have . Along the trajectory of the solution of the closed-loop system (3), we obtain
By Jensen’s inequality, we can obtain
Similarly, we have
By substituting (16) and (17) to (15) and then combining (12), (13), and (14), we have
where . By using the Schur complement, (8) guarantees that . Therefore,
where denotes the minimal eigenvalue of and . From (19), it follows that, for any ,
Furthermore
By taking as the limit, we obtain
According to Definition 3, the closed-loop system (3) exhibits stochastic stability for all uncertain transition probability matrices.

Theorem 4 gives a sufficient condition for the stochastic stability of the system (3). However, it should be noted that the controller gain cannot be obtained according to the condition in (8) because of the nonlinear terms and and the uncertain terms and . To handle this problem, the equivalent LMI conditions are given as follows.

Before proceeding further, we provide the following lemma that will play a significant role in processing the uncertainty terms and of uncertain transition probability matrices and .

Lemma 5 (see [28]). *
For any vectors of and positive matrix , the following holds:
*

Theorem 6. *For the system (3), the random but bounded scalars and . If, for each mode and , the tuning parameters and , the scalars and , and matrices , , , , , , , , , and , and exist such that
**
where
**Then the closed-loop system (3) is stochastically stable and the controller is a state feedback controller of the system (3).*

*Proof. *Let , , , and , where and are known tuning parameters. We restrict to be the same for all (namely, ) and give the notations as
Pre- and postmultiplying and to (8), respectively, we have
where
where and are defined in Theorem 4 and , , and are defined in Theorem 6.

According to the assumption on uncertain transition probabilities and and the fact that and , one has
Note that
By Lemma 5 and the fact that , we have
Similarly, we have

Note that, for any matrix , we have for and . Combining (29), (31), and (32) and by the Schur complement, (24) can be yielded easily from (27); this completes the proof of Theorem 6.

#### 4. Numerical Example

In this section, we illustrate our results through an example. We apply the results in Theorem 6 to a simple inverted pendulum system [5] shown in Figure 2, which is a two-order unstable system. The state variables are , where is the angular position of the pendulum. The parameters used are kg and m, without friction surfaces. The sampling time is s. The plant matrices are given by

We assume that the stochastic Markovian jumping S-C delay and C-A delay and their uncertain transition probability matrices are given as follows: The eigenvalues of are 1.1695 and 0.8551. Therefore, the discrete-time model is unstable.

Figures 3 and 4 show part of the simulation of the Markov chains mode. The initial conditions are as follows: , , and . By Theorem 6, when , , , and , we can obtain the gain matrix of state feedback controller (2) which is constructed as

The state trajectories of the system (3) are shown in Figure 5, where two curves represent state trajectories under the controller gains . Figure 5 also indicates that the system (3) is stochastically stable.

*Remark 7. *In this example, the uncertain transition probabilities are given as a discrete probability distribution function. When the uncertain transition probability is given as a continuous probability distribution function, we can use the norm of the continuous probability distribution function as the upper bound to simulation.

#### 5. Conclusions

The state feedback stabilization problem for a class of NCS with the S-C and C-A random time delays is investigated in this paper. The resulting closed-loop NCS is modeled as a linear system with uncertain Markovian transition probabilities. New sufficient conditions on stochastic stability and stabilization are obtained by Lyapunov stability theory and linear matrix inequalities method. An example is presented to illustrate the effectiveness of the approach. Although only the time-delay issue for NCS is addressed in this paper, the method can be extended to the NCS with the random packet dropouts, time delays, and packet dropouts and to the MJLS with the uncertain Markovian transition probabilities.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

#### Acknowledgments

This work was supported partly by the National Natural Science Foundation of China (Grant nos. 61174070 and 61104107), the Natural Science Foundation of Guangdong Province (Grant no. S2013040016183), the Foundation of Key Laboratory of Autonomous Systems and Networked Control, Ministry of Education, China, and the Natural Science Foundation of Shenzhen University (Grant no. 201207).