## Stochastic Systems: Modeling, Analysis, Synthesis, Control, and their Applications to Engineering

View this Special IssueResearch Article | Open Access

# Stability and Stabilization of Networked Control System with Forward and Backward Random Time Delays

**Academic Editor:**Xue-Jun Xie

#### Abstract

This paper deals with the problem of stabilization for a class of networked control systems (NCSs) with random time delay via the state feedback control. Both sensor-to-controller and controller-to-actuator delays are modeled as Markov processes, and the resulting closed-loop system is modeled as a Markovian jump linear system (MJLS). Based on Lyapunov stability theorem combined with Razumikhin-based technique, a new delay-dependent stochastic stability criterion in terms of bilinear matrix inequalities (BMIs) for the system is derived. A state feedback controller that makes the closed-loop system stochastically stable is designed, which can be solved by the proposed algorithm. Simulations are included to demonstrate the theoretical result.

#### 1. Introduction

Feedback control systems in which the control loops are closed through a real-time network are called networked control systems (NCSs) [1]. Recently, much attention has been paid to the study of stability analysis and controller design of NCSs [2, 3] due to their low cost, reduced weight and power requirements, simple installation and maintenance, and high reliability. Consequently, NCSs have been applied to various areas such as mobile sensor networks [4], remote surgery [5], haptics collaboration over the Internet [6–8], and automated highway systems and unmanned aerial vehicles [9, 10]. However, the sampling data and controller signals are transmitted through a network, so network-induced delays in NCSs are always inevitable [11, 12].

One of the main issues in NCSs is network-induced delays, which are usually the major causes for the deterioration of system performance and potential system instability [13]. For different scheduling protocols, the network-induced delay may be constant, or time-varying, but in most cases, it is random [14]. Hence, systems with random time delay attract considerable attention [15–18]. Based on stochastic control theory and a separation property, the effect of random delay is treated as an LQG problem in [15]. However, the network-induced random delay has to be less than one sampling interval. The results in [15] have recently been extended to the case with longer delays in [16]. It is noted that the given controller depends only on sensor-to-controller delay. In [17], a control problem for Bernoulli binary random delay is considered, and a linear matrix inequalities (LMIs) problem for the analysis of stochastic exponential mean square stability is established. The model-based NCSs with random transmission delay is studied in [18]. Sufficient conditions for almost sure stability and stochastic exponential mean square stability are presented.

On the other hand, the study of stochastic systems has attracted a great deal of attention [19–38]. Some of these results are applied to networked control systems with random time delays [39–43]. In [39, 40], the network-induced random delays are modeled as Markov chains such that the closed-loop systems are jump linear systems with one mode. It is noticed that in [39], the state feedback gain is mode independent, and in [40], the state feedback gain only depends on the delay from sensor to controller. Recently, stabilization of networked control systems with the sensor-to-controller and controller-to-actuator delays are considered in [41]. In [42, 43], a class of Markovian jump linear systems with time delays both in the system state and in the mode signal is considered. Based on Lyapunov method, a time-delayed, mode-dependent, and state feedback controller such that the closed-loop system is stochastically stable is designed. It is noticed that the time delay in the mode signal is constant in [42, 43], and the time delay in the mode signal is random. It is worth pointing out that in all of the aforementioned papers, the plant is in the discrete-time domain. To the best of the authors' knowledge, the stability and stabilization problems for NCSs with the plant being in the continuous-time domain have not been fully investigated to date. Especially for the case where both sensor-to-controller and controller-to-actuator network-induced delays are random and longer than one sampling interval, very few results related to NCSs have been available in the literature so far, which motivates the present study.

The aim of this paper is to consider a class of networked control systems with sensors and actuators connected to a controller via two communication networks in the continuous-time domain. Two Markov processes are introduced to describe sensor-to-controller transmission delay and the controller-to-actuator transmission delay. Based on Lyapunov stability theorem, a method for designing a mode-dependent state feedback controller that stabilizes this class of networked control systems is proposed. The existence of such a controller is given in terms of BMIs, which can be solved by the proposed algorithm.

This paper is organized as follows. In Section 2, the problem is stated and some useful definitions and lemmas are given, and then the main results of this paper are given in Section 3. Simulation results are presented in Section 4. Finally, the conclusions are provided in Section 5.

*Notation. * denotes the -dimensional Euclidean space, and is identity matrix. stands for the transpose of the corresponding matrix . The notation means that the matrix is a positive semidefinite (positive definite) matrix. For an arbitrary matrix and two symmetric matrices and , denotes a symmetric matrix, where denotes a block matrix entry implied by symmetry, and refers to the Euclidean norm for vectors and induced 2-norm for matrices. stands for the mathematical expectation operator, and for probability operator.

#### 2. Problem Formulation

Consider linear systems described by the differential equation where is the state vector, and is the control input. Matrices and are known matrices of appropriate dimensions.

The plant is interconnected by a controller over a communication network, see Figure 1. The sensor and controller are periodically sampled with the sampling interval . We describe the sensor-to-controller transmission delay as and the controller-to-actuator transmission delay as . The mode switching of is governed by the continuous-time discrete-state Markov process taking the values in the finite set with generator given by where is the transition rate from mode to with when and , and is such that . The mode switching of is governed by the continuous-time discrete-state Markov process taking the values in the finite set with generator given by with and .

Throughout the paper, the following assumption is needed for the considered networked control systems.

*Assumption 2.1. *The switching difference of consecutive delays is less than one sampling interval, that is,
where is the th sampling instant.

*Remark 2.2. *Although Assumption 2.1 restricts that the switching difference of consecutive delays is less than one sampling interval , this does not imply that the network delay and are less than .

According to Figure 1, for , the control law has the form:

Define the time delay as follows: which can be illustrated by Figure 2.

Then, we have The associated upper bounds of are defined as

Applying controller (2.7) to the open-loop system (2.1) results in the closed-loop networked control system where , is the initial function.

We have the following stochastic stability concept for system (2.9).

*Definition 2.3. *The system (2.9) is said to be stochastically stable if there exists a constant such that
for any initial condition .

The following lemmas will be essential for the proofs in Section 3.

Lemma 2.4 (see [44]). *Given any real matrices of appropriate dimensions and a scalar such that , Then the following inequality holds:
*

For the delay functional differential equation, where is completely continuous, , and is defined as Then we have the following Razumikhin lemma.

Lemma 2.5 (see [45]). *Suppose that are continuous, strictly monotonous increasing functions, then , and are positive for , and . If there is a continuous function such that
**
and there is a continuous nondecreasing function for , and for any ,
**
if
**
then the zero solution of (2.12) is uniformly asymptotically stable.*

#### 3. Main Results

The following theorem provides sufficient conditions for existence of a mode-dependent state feedback controller for the system (2.9).

Theorem 3.1. *Consider the closed-loop system (2.9) satisfying Assumption 2.1. If there exist symmetric matrix , matrix , and positive scalar , such that the following matrix inequalities hold for all and ,
**
where
**
with , then the system is stochastically stable with the state feedback gain:
*

*Proof. *Consider the following Lyapunov candidate:
where is the positive symmetric matrix. From (3.6), it follows that
where
Note that
Thus, the closed-loop system (2.9) can be rewritten as
Let be the weak infinitesimal generator of , then for , we haveAccording to Lemma 2.4, we have
From (3.2), (3.3), and Lemma 2.5, we can obtain
which yields
Following Lemma 2.5, for , we assume that for any , the following inequality holds:
then we have
where is given by
for some positive scalars and . before and after multiplying by and its transpose, it gives
Since
we have from (3.16) that
From (3.1) and Lemma 2.5, it follows that
which is equivalent to
Using the continuity properties of the eigenvalues of with respect to , then there exists a sufficiently small such that (3.21) still holds. Thus, for such a , we have
which yields
where
Applying Dynkin's formula, we have
Note that
Then we can obtain
This completes the proof.

*Remark 3.2. *In case of constant transmission delay, that is, , and , Theorem 3.1 can be directly applied to systems with constant delay.

It should be noted that the terms and in (3.1)–(3.3) are bilinear. Therefore, we propose the following algorithm to solve these bilinear matrix inequality problems.*Step 1. *Set , and such that the following LMI holds:
where
*Step 2. *For given in the previous step, find , and by solving the following convex optimization problem:
*Step 3. *For , , and given in the previous step, find by solving the following quasiconvex optimization problem
*Step 4. *Return to step 2 until the convergence of is attained with a desired precision.

*Remark 3.3. *For a given , the considered optimization problem consists of minimizing an eigenvalue problem which is a convex one. On the other hand, for given , and , the considered optimization problem consists of minimizing a generalized eigenvalue problem which is a quasiconvex optimization problem. Therefore, the proposed algorithm gives a suboptimal solution.

#### 4. Simulations

In this section, simulations of the position control for robotic manipulator ViSHaRD3 [46] are included to illustrate the effectiveness of the proposed method. Combining computed torque feedback approach [47] with friction compensation, the system is decoupled into three systems. The first and second joints of the ViSHaRD3 are and the third is

For simplicity, we only discuss the third joint of ViSHaRD3. Suppose that the sampling interval is s, and the Markov process that governs the mode switching of the SC delay takes values in and has the generator and the Markov process that governs the mode switching of the CA delay takes values in and has the generator

Associated with modes 1 and 2, let the system have time delay s, s and s, s, respectively. From (2.8), we have s, and the initial condition is . By the proposed algorithm and Theorem 3.1, we can obtain the controllers as follows:

The simulations of the state response and the control input for the closed-loop system are depicted in Figures 3 and 4, respectively, which shows that the system is stochastically stable.

#### 5. Conclusions

In this paper, a technique of designing a mode-dependent state feedback controller for networked control systems with random time delays has been proposed. The main contribution of this paper is that both the sensor-to-controller and controller-to-actuator delays have been taken into account. Two Markov processes have been used to model these two time delays. Based on Lyapunov stability theorem combined with Razumikhin-based technique, some new delay-dependent stability criteria in terms of BMIs for the system are derived. A state feedback controller that makes the closed-loop system stochastically stable is designed, which can be solved by the proposed algorithm. Simulations results are presented to illustrate the validity of the design methodology.

#### Acknowledgments

This work is partially supported by the Youth Fund of Anhui Province no. 2010SQRL162 and the program of science and technology of Huainan no. 2011A08005.

#### References

- W. Zhang, M. S. Branicky, and S. M. Phillips, “Stability of networked control systems,”
*IEEE Control Systems Magazine*, vol. 21, no. 1, pp. 84–99, 2001. View at: Publisher Site | Google Scholar - J. Nilsson, B. Bernhardsson, and B. Wittenmark, “Stochastic analysis and control of real-time systems with random time delays,”
*Automatica*, vol. 34, no. 1, pp. 57–64, 1998. View at: Publisher Site | Google Scholar | Zentralblatt MATH - F. C. Liu, Y. Yao, F. H. He, and S. Chen, “Stability analysis of networked control systems with time-varying sampling periods,”
*Journal of Control Theory and Applications*, vol. 6, no. 1, pp. 22–25, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH - P. Ogren, E. Fiorelli, and N. E. Leonard, “Cooperative control of mobile sensor networks: adaptive gradient climbing in a distributed environment,”
*IEEE Transactions on Automatic Control*, vol. 49, no. 8, pp. 1292–1302, 2004. View at: Publisher Site | Google Scholar - C. Meng, T. Wang, and W. Chou, “Remote surgery case: robot-assisted teleneurosurgery,” in
*Proceedings of the IEEE International Conference on Robotics and Automation*, pp. 819–823, May 2004. View at: Publisher Site | Google Scholar - J. P. Hespanha, M. L. Mclaughlin, and G. Sukhatme, “Haptic collaboration over the Internet,” in
*Proceedings of the 5th Phantom Users Group Workshop*, 2000. View at: Publisher Site | Google Scholar - K. Hikichi, H. Morino, and I. Arimoto, “The evaluation of delay jitter for haptics collaboration over the Internet,” in
*Proceedings of the IEEE Global Telecomm*, vol. 2, pp. 1492–1496, 2002. View at: Google Scholar - S. Shirmohammadi and N. H. Woo, “Evaluating decorators for haptic collaboration over internet,” in
*Proceedings of the 3rd IEEE International Workshop on Haptic, Audio and Visual Environments and their Applications, (HAVE '04)*, pp. 105–109, October 2004. View at: Google Scholar - P. Seiler and R. Sengupta, “Analysis of communication losses in vehicle control problems,” in
*Proceedings of the American Control Conference*, pp. 1491–1496, June 2001. View at: Google Scholar - P. Seiler and R. Sengupta, “An ${H}_{\infty}$ approach to networked control,”
*IEEE Transactions on Automatic Control*, vol. 50, no. 3, pp. 356–364, 2005. View at: Publisher Site | Google Scholar - J. Baillieul and P. J. Antsaklis, “Control and communication challenges in networked real-time systems,”
*Proceedings of the IEEE*, vol. 95, no. 1, Article ID 4118454, pp. 9–28, 2007. View at: Publisher Site | Google Scholar - X. M. Zhang, Y. F. Zheng, and G. P. Lu, “Stochastic stability of networked control systems with network-induced delay and data dropout,”
*Journal of Control Theory and Applications*, vol. 6, no. 4, pp. 405–409, 2008. View at: Publisher Site | Google Scholar - K. Gu and S. I. Niculescu, “Survey on recent results in the stability and control of time-delay systems,”
*Transactions of the ASME*, vol. 125, no. 2, pp. 158–165, 2003. View at: Publisher Site | Google Scholar - P. Seiler,
*Coordinated control of unmanned aerial vehicles*, Ph.D. thesis, University of California, Berkeley, Calif, USA, 2001. - J. Nilsson,
*Real-time control systems with delay*, Ph.D. thesis, Lund Institute of Technology, 1998. - S. S. Hu and Q. X. Zhu, “Stochastic optimal control and analysis of stability of networked control systems with long delay,”
*Automatica*, vol. 39, no. 11, pp. 1877–1884, 2003. View at: Publisher Site | Google Scholar | Zentralblatt MATH - L. Montestruque and P. Antsaklis, “Stability of model-based networked control systems with time-varying transmission times,”
*IEEE Transactions on Automatic Control*, vol. 49, no. 9, pp. 1562–1572, 2004. View at: Publisher Site | Google Scholar - F. Yang, Z. Wang, Y. S. Hung, and M. Gani, “${H}_{\infty}$ control for networked systems with random communication delays,”
*IEEE Transactions on Automatic Control*, vol. 51, no. 3, pp. 511–518, 2006. View at: Publisher Site | Google Scholar - Y. Ji and H. J. Chizeck, “Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control,”
*IEEE Transactions on Automatic Control*, vol. 35, no. 7, pp. 777–788, 1990. View at: Publisher Site | Google Scholar | Zentralblatt MATH - S. Xu, T. Chen, and J. Lam, “Robust ${H}_{\infty}$ filtering for uncertain Markovian jump systems with mode-dependent time delays,”
*IEEE Transactions on Automatic Control*, vol. 48, no. 5, pp. 900–907, 2003. View at: Publisher Site | Google Scholar - Y.-Y. Cao and J. Lam, “Robust ${H}_{\infty}$ control of uncertain Markovian jump systems with time-delay,”
*IEEE Transactions on Automatic Control*, vol. 45, no. 1, pp. 77–83, 2000. View at: Publisher Site | Google Scholar | Zentralblatt MATH - X. J. Xie and N. Duan, “Output tracking of high-order stochastic nonlinear systems with application to benchmark mechanical system,”
*IEEE Transactions on Automatic Control*, vol. 55, no. 5, pp. 1197–1202, 2010. View at: Publisher Site | Google Scholar - X. J. Xie, N. Duan, and X. Yu, “State-feedback control of high-order stochastic nonlinear systems with SiISS inverse dynamics,”
*IEEE Transactions on Automatic Control*, vol. 56, no. 8, pp. 1921–1926, 2011. View at: Publisher Site | Google Scholar - X. Yu and X. J. Xie, “Output feedback regulation of stochastic nonlinear systems with stochastic iISS inverse dynamics,”
*IEEE Transactions on Automatic Control*, vol. 55, no. 2, pp. 304–320, 2010. View at: Publisher Site | Google Scholar - N. Duan and X. J. Xie, “Further results on output-feedback stabilization for a class of stochastic nonlinear systems,”
*IEEE Transactions on Automatic Control*, vol. 56, no. 5, pp. 1208–1213, 2011. View at: Publisher Site | Google Scholar - N. Duan, X. Yu, and X. J. Xie, “Output feedback control using small-gain conditions for stochastic nonlinear systems with SiISS inverse dynamics,”
*International Journal of Control*, vol. 84, no. 1, pp. 47–56, 2011. View at: Publisher Site | Google Scholar | Zentralblatt MATH - X. Yu, X. J. Xie, and Y. Q. Wu, “Further results on output-feedback regulation of stochastic nonlinear systems with SiISS inverse dynamics,”
*International Journal of Control*, vol. 83, no. 10, pp. 2140–2152, 2010. View at: Publisher Site | Google Scholar | Zentralblatt MATH - X. J. Xie and W. Q. Li, “Output-feedback control of a class of high-order stochastic nonlinear systems,”
*International Journal of Control*, vol. 82, no. 9, pp. 1692–1705, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH - J. Tian and X. J. Xie, “Adaptive state-feedback stabilization for high-order stochastic non-linear systems with uncertain control coefficients,”
*International Journal of Control*, vol. 80, no. 9, pp. 1503–1516, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH - Z. J. Wu, X. J. Xie, and S. Y. Zhang, “Stochastic adaptive backstepping controller design by introducing dynamic signal and changing supply function,”
*International Journal of Control*, vol. 79, no. 12, pp. 1635–1646, 2006. View at: Publisher Site | Google Scholar | Zentralblatt MATH - W. Q. Li, X. J. Xie, and S. Y. Zhang, “Output-feedback stabilization of stochastic high-order nonlinear systems under weaker conditions,”
*SIAM Journal on Control and Optimization*, vol. 49, no. 3, pp. 1262–1282, 2011. View at: Publisher Site | Google Scholar - X. Yu, X. J. Xie, and Y. Q. Wu, “Decentralized adaptive output-feedback control for stochastic interconnected systems with stochastic unmodeled dynamic interactions,”
*International Journal of Adaptive Control and Signal Processing*, vol. 25, no. 8, pp. 740–757, 2011. View at: Publisher Site | Google Scholar - Z. J. Wu, X. J. Xie, and S. Y. Zhang, “Adaptive backstepping controller design using stochastic small-gain theorem,”
*Automatica*, vol. 43, no. 4, pp. 608–620, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH - W. Q. Li and X. J. Xie, “Inverse optimal stabilization for stochastic nonlinear systems whose linearizations are not stabilizable,”
*Automatica*, vol. 45, no. 2, pp. 498–503, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH - X. J. Xie and J. Tian, “Adaptive state-feedback stabilization of high-order stochastic systems with nonlinear parameterization,”
*Automatica*, vol. 45, no. 1, pp. 126–133, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH - X. Yu, X. J. Xie, and N. Duan, “Small-gain control method for stochastic nonlinear systems with stochastic iISS inverse dynamics,”
*Automatica*, vol. 46, no. 11, pp. 1790–1798, 2010. View at: Publisher Site | Google Scholar | Zentralblatt MATH - L. Liu and X. J. Xie, “Output-feedback stabilization for stochastic high-order nonlinear systems with time-varying delay,”
*Automatica*, vol. 47, no. 12, pp. 2772–2779, 2011. View at: Publisher Site | Google Scholar - X. J. Xie and J. Tian, “State-feedback stabilization for high-order stochastic nonlinear systems with stochastic inverse dynamics,”
*International Journal of Robust and Nonlinear Control*, vol. 17, no. 14, pp. 1343–1362, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH - R. Krtolica, U. Ozguner, and H. Chan, “Stability of linear feedback systems with random communication delays,”
*International Journal of Control*, vol. 59, no. 4, pp. 925–953, 1994. View at: Publisher Site | Google Scholar | Zentralblatt MATH - L. Xiao, A. Hassibi, and J. P. How, “Control with random communication delays via a discrete-time jump system approach,” in
*Proceedings of the American Control Conference*, pp. 2199–2204, Chicago, Ill, USA, June 2000. View at: Google Scholar - L. Zhang, Y. Shi, T. Chen, and B. Huang, “A new method for stabilization of networked control systems with random delays,”
*IEEE Transactions on Automatic Control*, vol. 50, no. 8, pp. 1177–1181, 2005. View at: Publisher Site | Google Scholar - J. Xiong and J. Lam, “Stabilization of discrete-time Markovian jump linear systems via time-delayed controllers,”
*Automatica*, vol. 42, no. 5, pp. 747–753, 2006. View at: Publisher Site | Google Scholar | Zentralblatt MATH - M. Liu, D. W. C. Ho, and Y. Niu, “Stabilization of Markovian jump linear system over networks with random communication delay,”
*Automatica*, vol. 45, no. 2, pp. 416–421, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH - E. N. Sanchez and J. P. Perez, “Input-to-state stability (ISS) analysis for dynamic neural networks,”
*IEEE Transactions on Circuits and Systems-I*, vol. 46, no. 11, pp. 1395–1398, 1999. View at: Publisher Site | Google Scholar | Zentralblatt MATH - J. K. Hale and S. M. Verduyn Lunel,
*Introduction to Functional-Differential Equations*, vol. 99 of*Applied Mathematical Sciences*, Springer, New York, NY, USA, 1993. - C. C. Chen, S. Hirche, and M. Buss, “Stability, stabilization and experiments for networked control systems with random time delay,” in
*Proceedings of the American Control Conference*, Seattle, Wash, USA, 2008. View at: Google Scholar - A. Isidori,
*Nonlinear Control Systems*, Springer, New York, NY, USA, 1995.

#### Copyright

Copyright © 2012 Ye-Guo Sun and Qing-Zheng Gao. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.