Abstract

This paper is concerned with an optimal control framework for networked control systems in which the channel access of actuators is governed by a group random access protocol. The system is modeled as a switching Markov jump system with multiple modes according to channel-access status of the actuators, and an independent identical distribution Bernoulli process is used to describe the random packet dropouts of the channel. Then, an optimal control design methodology is addressed to satisfy the quadratic cost function by applying the well-developed theory for jump linear systems and stochastic optimal control while guaranteeing the mean-square exponential stability of networked control systems. And, finally, a numerical example is exploited to demonstrate the effectiveness of the proposed method.

1. Introduction

Control systems whose components (e.g., sensors, actuators, and controllers) are connected via a real-time communication network are called networked control systems (NCSs). Using a control network has many advantages, such as reduced system wiring and cost, increased flexibility, reliability and safety, easier deployment, diagnosis, and maintenance. However, as any practical communication networks (e.g., wireless networks and underwater acoustic networks) have limited bandwidth, NCSs are subject to several issues that can lead to performance degradation and stability loss. Among these issues, network-induced delays [1–4], packet dropouts [5–8], data rate limitation, and quantization effects [9–11] have attracted the most attention in past decade. This paper is concerned with another interesting problem frequently encountered in NCSs, which is termed medium access constraint. This issue arises when an NCS has a large number of nodes (sensors/actuators or subsystems) while the network has insufficient channels to accommodate them at a time. In such a situation, for a system to meet certain performance requirements, the procedure of controller design should be accompanied by a channel access assignment or scheduling policy for the nodes.

Up to date, there have been generally three types of scheduling policies that are frequently used in addressing the problem of medium access constraint, that is, static scheduling, dynamic scheduling, and hybrid static-dynamic scheduling. Static scheduling policies are also called static access protocols, which are designed offline. One of the earliest efforts on static access scheduling of NCSs can be seen in [12], where the concept of communication sequence was introduced to assign channel access to the nodes. Germinated by this work, a series of network access scheduling methods were presented from different viewpoints; see [13, 14], to name a few. Recently, research attention has been paid to dynamic access protocols or dynamic scheduling policies; see, for example, [15, 16]. A network access protocol is dynamic (or online) when it is designed based on real-time feedback information and the system performance requirement. Generally speaking, static access protocols are quite simple and easily realizable, but they are not adaptable to situations with disturbances or uncertainties (e.g., parameter perturbation). As a result, the existing results on static scheduling, like those in [13, 14], can only handle NCSs whose dynamics are uncoupled. On the contrary, the dynamic access protocols in [15, 16], to mention some, can be used to handle systems with coupled dynamics robustly. However, such online medium access control protocols are either very conservative or computationally complicated. In addition, they require frequent monitoring of the channel for feedback information and complicated processing. One can take advantages of both static and dynamic strategies and derive a combined access control policy, which is called hybrid static-dynamic access protocol. A hybrid policy has been shown to be more promising for managing the constrained network medium access in [17, 18].

The scheduling strategy presented in [18] is of special interest because it gives an optimal control solution to address the medium access constraint problem, aside from providing a special type of hybrid static-dynamic access protocol. The problem of integrated optimal control and scheduling was first investigated in [19], where a static scheduling policy was obtained by applying dynamic programming with pruning of the search tree to avoid combinatory explosion. Similar studies along this line can be seen in [20, 21]. For example, in [20], an optimal static scheduling strategy was derived by solving a complex combinatorial optimization problem with the neighbourhood search method and more efficient search heuristics. In [21], a novel optimal dynamic scheduling framework for NCSs was established on the basis of mixed integer programming and model predictive control techniques. Since network access scheduling and system control are strongly coupled, finding a jointly optimal design approach is indeed very complicated or even unsolvable [22], unless strong assumptions are made on either the plant to be controlled or the network to be scheduled. For this reason, it would be more practical for a control engineer to seek for optimal control solutions for specific medium access protocols that are given. In fact, even for a given medium access policy, the problem of designing an optimal controller is very hard, especially for random access protocols [23].

Though stochastic access protocols are convenient in managing access to shared communication channels without feedback information, they are hardly seen in the literature for control purpose. Here we are interested in investigating optimal control of NCSs in which the network is governed by the random channel-access policy investigated by Wang and Guo [24]. We will consider systems whose actuators share a limited number of channels and the channels are governed by a group random access protocol. At each time, the so-called group random access protocol allows a subset of active actuators to have access to the network and receive the control signal from the controller. At the next time, another subset of actuators is assigned access to the channels. The switching between two successive subsets of active actuators is determined by the random access protocol, which is defined by a vector-valued Markov process [25]. This paper is concerned with an optimal control framework for NCSs in which the channel access of actuators is controlled by a Markov group random access protocol. What makes the problem more complicated is that we consider packet-dropping issue at the same time. The random packet-loss of each network channel is described by an independent identical distribution (iid) Bernoulli process. The channel-access status of the actuators is determined by a group random access protocol, which is described by a Markov process with known transition probability matrix. By transforming the closed-loop system to a Markovian jump system with random packet loss, an optimal control design methodology is presented applying the well-developed theory for jump linear systems and stochastic optimal control.

The organization of the paper is as follows. Section 2 describes the NCS framework and models of access constraints and packet-loss process. The problem of designing an optimal state feedback law is solved in Section 3. Numerical example is presented in Section 4 to show the effectiveness of our results. Finally, Section 5 gives some concluding remarks and discusses future research directions.

Remark 1. It should be pointed out that the issue of control subject to packet dropout and medium access constraint was previously investigated in [26]. An assumption made in this paper is that multiple sensors or actuators in the system share a single unreliable channel that is manipulated by a static access protocol. Compared with [26], this paper considers a more complicated and challenging situation in which the actuators share multiple but insufficient channels according to a group random access protocol. In addition, we have further considered that packet dropouts could randomly occur during packet transmission in the channel.

Notations. The following notations will be used throughout the paper. and denote the -dimensional Euclidean space and the set of real matrices, respectively. is the combinatorial number where elements are selected from a total of elements. (≥0) means that is real symmetric and positively definite (semidefinite). and represent the transpose and the inverse of matrix , and denotes the trace of the matrix . and are maximal and minimal eigenvalue of the matrix , respectively. denotes the set of the matrices . stands for the mathematical expectation of the stochastic variable , and means the occurrence probability of the event “·”. denotes the set of nonnegative integers. Define the sets and , where and are positive integer.

2. Problem Formulation

The NCSs considered in the paper are diagrammatically depicted in Figure 1, where the plant is a linear time-invariant system described by where is the state vector, is the control vector actually executed by the actuators, denotes the white noise with zero means, which satisfies , and for all . , and are constant matrices with appropriate dimensions, respectively.

Figure 1 

A basic NCS configuration.

There are actuators and sensors in the NCS. Without loss of generality, we assume that the sensors are time-driven, and the controller and actuators are event-driven. To simplify analysis, we only consider the communication constraints at the actuators side but ignored the same for the sensors. It is assumed that the state of the plant is directly measured by the sensors and sent to the controller with full precision, while the control signals are transmitted in separate packets to the actuators via multiple communication channels. Due to bandwidth limitations, assume that there are actuators in the system but only () channels so that the controller can only communicate with actuators at any time . Therefore, there are ( is a natural number) possible channel-access modes for the actuators. Furthermore, control packet dropouts can randomly occur during transmission due to node failures, network congestion, or communication noise.

For , let the binary-valued function denote channel-access status of the actuator at time instant . means that the actuator is accessing the channel, and the corresponding control signal is transmitted; that is, , where denotes the control vector generated by the controller; otherwise, means that the control input packet is not transmitted and the actuator will use zero as the control signal; that is, . Then, at time instant , the medium access status of the actuators is represented by the m-to- communication sequence . Let denote the matrix form of the communication sequence that assigns channel access to the actuators at time instant . We thus have It is assumed that could be modeled by a Markov process taking values in a finite set with the following conditional probability: where is the transition probability matrix and , , denotes the transition probability from mode to mode , satisfying . is the initial probability of mode , while is the probability of mode at time instant , and obeys the recursion . For simplicity, let be the indicator of the mode of the Markov process, , if .

For the packet dropout issue, we use a binary-valued random variable , , to denote the packet-loss phenomenon of the channels at time . When the packet is successfully transmitted to the actuator, we have . If the packet is dropped during transmission, we have . Assume that any packet is lost according to a Bernoulli process with the following probability distribution: where is a known constant denoting the th channel’s arrival probability of the control input packet. It is assumed that and are iid for and that different channels are also independent of each other as their expected values could be different; that is, , for . In consideration of the effect of packet loss, control signals actually received by the actuators can be expressed as follows: where is the probability matrix that denotes the successful transmission of control signals.

We assume that the control network uses a TCP-like protocol with a perfect acknowledgment mechanism. The actuators will send an acknowledgment signal to the controller at time instant , to denote whether the control input packets were successfully delivered to their respective actuators at the previous time instant. It is noteworthy that the actuator acknowledgment has one-step time delay. Then, we define the corresponding information set as . Apparently, .

Combining the system dynamics (1), the activation status (2) of the actuators, and the effect of packet dropouts (5) and taking into account the fact that and are both diagonal matrices, we can obtain where . It is clear that is also a Markov process.

In the forthcoming development, we will need the following definition.

Definition 2. The NCS is said to be exponentially mean-square stable if, for every initial state , there exist constants and such that
The purpose of this paper is to derive an optimal state feedback sequence , for the presented networked system with the initial conditions , such that the system in (6) is exponentially mean-square stable, and minimize the quadratic cost function where , , , , . The minimal is denoted by .

Remark 3. There are generally two different strategies to deal with the situation that the actuator does not receive an expected control input packet at a given time. The first one is zero-input control strategy, which denotes the fact that plant’s scalar input receives zero values from the designated actuator as its control signal, and the second one is the last available control strategy, which means that the latest control values stored in the actuator buffer will be sent to the plant in this case. In this paper, we adopt the former strategy when actuators could not receive effective control information; see, for example, [15, 27, 28] and references therein.

Remark 4. From the above description, we know that the system in (6) is a discrete-time Markovian jump linear system with random packet dropouts described by an iid Bernoulli process. This system representation is convenient as it allows us to explore stability analysis and controller design of NCSs by utilizing the well-developed theory for jump linear system and stochastic optimal control.

3. Control Synthesis with Markov Actuator Assignment and Packet Dropouts

In this section, we first solve the optimal control problem in finite-time horizon for the system in (6) and derive the optimal control sequence to minimize the quadratic cost function in (8). Subsequently, we will prove that the optimal control sequence can also render the system in (6) exponentially mean-square stable. Define an admissible control input set as . By exploiting the independence of the noise input , we can know that , , , and are uncorrelated; then we have From the definition of the mathematical expectation, for any measurable functions and , the following equation holds:

Theorem 5. Given the system in (6), with noise free measurement of the state vector , that is, , the control law that minimizes the quadratic cost function in (8) is given by with the optimal cost function where

Proof. Let us define the following optimal value function : Obviously, . We claim that the optimal value function can be written as follows: where , , and . We will prove the theorem by deducing that (15), (16), and (17) are equivalent. The mathematical induction is employed in the derivation process.
It is easy to verify that the claim in (17) is true for with parameters , , and . We suppose that the claim in (17) is true for : The value function at time is then Note that, the conditional expectations have the smoothing property [29] as follows: Substituting (20) into (19) leads to
Since medium access constraints and packet dropouts are two independent random processes, and are uncorrelated random variables. According to (6), we can obtain
Taking the mathematical expectation on (22) and substituting (10) into (22) yield where is the set of channels for successful transmission data. It is easy to see that is a subset of the index set and has possible values. denotes the probability of taking values in , where is a diagonal matrix and its diagonal elements are It follows from (9) that Substituting (25) and (26) into (23), we obtained Substituting (27) into (21), then one hasSince the control input is unconstrained, let which yields
From , we have that , so we can obtain the optimal control sequence in (11). Noting that and substituting (11) into the optimal value function , we obtainedLetting we can obtain From the last equation we can easily show that (17) is satisfied also at time instant for all if and only if the matrices satisfy (14). As a consequence, since , it follows that the optimal value function is The proof is thus completed.