Abstract

This problem of intelligent switched fault detection filter design is investigated in this article. Firstly, the mode-dependent average dwell time (MDADT) method is applied to generate the time-dependent switching signal for switched systems with all subsystems unstable. Afterwards, the switched fault detection filter is proposed for the generation of residual signal, which consists of dynamics-based filter and learning-based filter. The MDADT method and multiple Lyapunov function (MLF) method are employed to guarantee the stability and prescribed attenuation performance. The parameters of dynamics-based filter are given by solving a series of linear matrix inequalities. To improve the transient performance, the deep reinforcement learning is introduced to design learning-based filter in the framework of actor-critic. The output of learning-based filter can be viewed as uncertainties of dynamics-based filter. The deep deterministic policy gradient algorithm and nonfragile control are adopted to guarantee the stability of algorithm and compensate the external disturbance. Finally, simulation results are given to illustrate the effectiveness of the method in the paper.

1. Introduction

Switched systems [13] are an important component of hybrid systems, which are composed of a series of subsystems and build the connection between nonlinear systems and linear systems. As an efficient way to deal with complex nonlinear systems, switched systems have attacked considerable attentions and were applied in the areas of both military and economics [4, 5], such as industrial manufacturing, flight control, robotic control, process control, and so on. The problems of modeling, stability analysis [6], controller design [7], and filter design [8] have been investigated recently and fruitful encouraging results emerged [9]. To mention a few, the problem of switched fault tolerant controller design is investigated in [10], the event-triggered controller design method for discrete-time switched systems is proposed in [11], and the stability and l2-gain analysis are given in [12].

The basic problem for switched systems lies in stability analysis [13, 14]. The scholars have developed tools to analyze the stability in past decades, such as common Lyapunov function (CLF) method, MLF method, and persistence Lyapunov function method [15]. The CLF method is mainly applied for the switched systems with arbitrary switching, which means that the switched systems are stable if all the subsystems share a CLF. However, it is difficult to design a CLF for all subsystems. To obtain tighter bounds, the stability analysis can be developed by the aid of MLF, which is mainly applied to constrained switching. Moreover, in practical system, the switching signal always depends on state or time. Because it is difficult to obtain the state measurements, the switching signal is time-dependent in many situations. Thus, the average dwell time (ADT) method and MDADT method provide efficient solutions to deal with the stability analysis for switched systems with constrained switching. In the work of [16], the issue of stability analysis for switched positive linear systems is studied. The ADT method and multiple linear copositive Lyapunov function are combined and sufficient stability criteria for stability analysis are given. However, it is straightforward that common parameters for all subsystems with different characteristic will lead to conservativeness, which motivates the researches on MDADT. The definition of MDADT is firstly proposed in [17]. The problem of stability analysis is studied in the framework of MDADT method and tighter bounds are obtained. In [18], the problem of stability and robust controller design for switched systems with external disturbances is studied. The MDADT switching and MLF method are introduced to ensure the stability. Moreover, in practice, the error of the switched system will lead to unstability. Therefore, the researches of stability analysis for switched systems with unstable subsystems are still one of the most important topics in control areas. In [19], the piecewise Lyapunov functions and MDADT method are combined to deal with the problem of stability analysis for switched systems with unstable modes. The fast switching is applied to unstable modes and slow switching is applied to stable modes. In [20], the problem of stability for switched systems with input time delay is studied. The unstable subsystems and asynchronous switching caused by time delay are taken into consideration. The extended stability criteria are obtained by the aid of Lyapunov-Krasovskii function method and sufficient conditions for stability analysis are presented.

The fault case of system will no doubt lead to undesirable response [21, 22]. One issue in the industrial systems or aeronautic engineering is that the maintenance cannot be given immediately to ensure the reliability and safety [23]. Thus, the presence of undesirable fault and the possibility of the occurrence of faults has to be considered in the stability analysis and system design, which motivates the study on fault detection and fault tolerant control. As an efficient way to deal with the undesirable faults, fault detection technique has attracted more and more attention, which can be seen in the recent important results, such as [2426]. The fault detection system can detect the fault in time and efficiently so that we can reconstruct the control diagram to adjust to the fault environment. Among these fault detection methods, the most valuable and applicable method is the model-based fault detection filter design method. It can be inferred in existing literature that the model-based method consists of a residual generator and residual evaluator, which is applied in the areas of networked control systems, aerospace engineering, and process control. In [27], the problem of H/H-fault detection for switched systems with unstable modes is investigated. The robustness to external disturbance and the sensitivity to fault are both taken into consideration. In [28], the problem of cooperative control for multiagent systems is investigated based on adaptive control and fuzzy control. The unknown control parameters and actuator fault are taken into consideration. The backstepping control is utilized to derive the controller. In the work of [29], the problem of fault detection subject to nonlinearities and disturbance is investigated according to event-triggered scheme. Sufficient conditions to guarantee the system is stable with the prescribed performance are obtained based on ADT method and MLF method. Based on the literature mentioned above, it can be inferred that the model-based fault detection filter is popularly applied in most of engineering problems because of the design simplicity and physical execution. However, it is difficult to achieve optimal compromise between robustness and transient performance. The model-based method can guarantee the stability and robustness of the closed-loop system. But the transient performance cannot be guaranteed. It is essential and significant to improve the transient performance of fault detection filters. Moreover, due to the lack of online-learning ability, this type of fault detection filters is relatively ineffective to have optimal performance in the real-world uncertain environment.

With the development of machine learning and computer science, the intelligent control has drawn considerable attention [30]. As a powerful nonlinear approximation approach, deep learning and deep reinforcement learning has promoted the considerable performance in realistic applications, such as controller parameters tuning, decision making, and so on. In particular, the DDPG algorithm has been illustrated to perform stably and efficiently on many high-dimensional action control tasks. In the work of [31], a noninteger PID controller based on DDPG algorithm is proposed for the tracking problem. To accomplish the control methodology, a kinematic controller and a dynamic controller are established independently, in which the kinematic controller is proposed based on the model of the vehicle and the dynamic controller is realized for the supplementary kinematic controller to achieve optimal performance. In [32], the assembly task is defined as a Markov decision process and a fuzzy DDPG algorithm is given to realize the task. To improve the performance and learning efficiency, a fuzzy reward system is developed for the assembly task. In addition, in the work of [3335], the machine learning is applied in the design of controller and guidance law.

Inspired by the aforementioned discussion, an effort is conducted in the paper to design an intelligent fault detection filter for switched system with all modes unstable, which is composed of dynamics-based filter and learning-based filter. As well known, the model-based fault detection filter is designed in the existing literature, which can guarantee the stability of closed-loop system and the robustness to external disturbance. However, how to improve the transient performance of fault detection still remains an open problem. On the other hand, in most literature, the fault detection filter is proposed based on the assumption that all the subsystems are stable. But in many practical situations, the subsystems may be unstable, which motivates the study in this paper. The robust control theory is applied to establish the dynamics-based filter. The switched system with all modes unstable is considered and the fault detection filter is presented for generation of the residual signal. The MDADT method and MLF method are combined to ensure the stability and prescribed attenuation performance index of switched system. To achieve optimal performance, the learning-based filter is introduced based on DDPG algorithm in the framework of actor-critic, in which the filter parameters are optimized by online learning. The output of learning-based filter can be viewed as variation of the parameters of dynamics-based filter. Thus, the nonfragile control theory is introduced to guarantee the stability of switched systems. The main contributions of this article are stated as follows: (1) An observer-based filter is proposed to deal with the problem of fault detection for switched system with all modes unstable, in which the stability and attenuation performance index can be guaranteed by MDADT method and MLF method. Compared to the existing results, the characteristic of each subsystem is considered, and tighter bounds can be obtained, which provides more room for the improvement of flexibility. (2) The learning-based fault detection filter is presented based on DDPG algorithm to achieve optimal performance, which overcomes the undesirable response caused by external disturbance and uncertainties. The action is defined by the variation of parameters of the fault detection filter. (3) The nonfragile control theory is applied to ensure the stability of closed-loop system, for the reason that the output of actor network is viewed as the variation of parameters of dynamics-based filter.

The rest of the paper is presented as follows: the model of switched system with all modes unstable is established in Section 2, in which the fault detection filter is proposed to generate the residual signal. In Section 2, the main results of fault detection are proposed, which is composed of dynamics-based filter and learning-based filter. The numerical example is given in Section 4 to validate the effectiveness of proposed method, which is followed by the conclusion in Section 5.

2. Problem Statement

The continuous-time switched system in this paper can be described as follows: is the state vector; denotes the input signal; is the output signal; is the external disturbances; represents the fault signal to be detected; is piecewise switching signal; , , , and are system matrices with appropriate dimensions.

To improve the transient performance of fault detection filter and achieve optimal performance, it is supposed that the fault detection filter in this paper is composed of two parts: the dynamics-based filter and the learning-based filter. The dynamics-based filter is designed according to robust control theory and the learning-based filter is proposed based on deep reinforcement learning, which can be described as follows: is the parameters of dynamics-based fault detection filter, which is determined by robust control theory; is the compensation for external disturbance, which is generated by the aid of DDPG algorithm. To ensure the stability of DDPG algorithm, the compensation can be viewed as the variation of parameters. Thus, the stability of optimization algorithm is guaranteed by nonfragile control theory. The parameters obtained by DDPG algorithm can be written as and are the known matrices with appropriate dimensions; are unknown matrices satisfying .

Remark 1. The designing process can be divided into two steps: (1) regarding the design of robust control theory, it is proposed to ensure the stability of closed-loop system and prescribed attenuation performance. (2) The deep reinforcement learning is utilized to improve the performance of fault detection, where the additional compensation of fault detection filter is viewed as the action of the agent. Then the parameters of fault detection filter are optimized based on DDPG algorithm in the framework of actor-critic. Compared with the traditional methods, not only can the stability be guaranteed by robust control theory, but also the transient performance of fault detection can be improved based on DDPG algorithm.

Remark 2. The problem of fault detection for switched systems with all modes unstable is studied in this paper. The unstable modes are taken into consideration and it is more applicable. The MDADT method and MLF method are combined to analyze the stability and tighter bounds on dwell time can be obtained than the traditional ADT method. There is more room for the switched systems to dwell long enough time to decrease the system energy. Therefore, the method proposed in this paper will improve the design flexibility of fault detection system.
The dynamics-based fault detection filter is proposed to generate the residual signal, which can be described as follows:where is the state of fault detection filter; denotes the estimation of output signal ; is the residual signal and denote the parameter matrices of fault detection filter to be determined.
Define the error of state measurement as ; the error of fault estimation . Thus, we set the augmented state vector and the augmented input signal as , so we can obtain the closed-loop switched system as follows:where , , , .
Then we can conclude that the design of robust fault detection filter can be converted by the problem of H filter design. Therefore, for given prescribed attenuation performance index , the closed-loop switched system in equation (5) is globally uniformly exponentially stable (GUES) when ; the following inequality holds for all nonzero under zero-initial condition.The residual signal is generated by fault detection filter; it is necessary to design a residual evaluator, which consists of an evaluation function and a threshold. The evaluation function and threshold can be written as denotes the length of time window of evaluation function.
Based on residual evaluator and threshold, the decision logic can be expressed as follows:

3. Main Results

3.1. Robust Filter Design

The definitions and lemmas are given as follows to make the proof convenient.

Definition 1. [17]. For given switching logic and time interval , we define as the number of switching during the time interval . If there exist constants and , such thatThen, is called the MDADT of fast switching and is called the chattering bounds; we set in this paper.

Definition 2. (see [12]). For given switching logic , if there exist constant scalars , , and equation (10) holds for , then the switched system in (5) is GUES.

Lemma 1. (see [27]). For given matrices S, T and symmetric matrix Y, if there exist constant scalar , such thatthen for matrix Ei with , we have

Theorem 1. For given constant scalars , , if there exist positive definite matrices , such thatthen the switched system in equation (5) is GUES if MDADT satisfies the following equation:

Proof. We set the Lyapunov function as follows:Thus, we haveCombining equation (14) and , we can conclude thatIt is supposed that the switching instants during time interval are set to be t1, t2, …,tk with tk+1 = t; we can derive thatTogether with Definition 1, we haveIf the MDADT of switched system satisfies equation (15), we haveCombining with Definition 2, one can conclude that the switched system in equation (5) is GUES when .

Theorem 2. For given constants , , , if there exist positive definite matrices , such thatThen, the switched system with MDADT satisfying equation (15) is GUES with prescribed attenuation performance , where .

Proof. Define the Lyapunov function in equation (16); we can obtain the following equations under zero-initial condition.Based on Schur complement, we haveAccording to the statement above, we haveTogether with the condition , one can concludeMultiplying both sides of equation (26) by , we can obtainTogether with equation (21) and the condition in equation (28), we can obtain equation (29).and it can be inferred thatIntegrating both sides of equation (30) from 0 to , we haveWe can conclude that the switched system in equation (5) is GUES with prescribed attenuation performance .

Theorem 3. For given constant scalars , , and , if there exist positive definite matrices , such thatThen, the switched system with MDADT satisfying equation (16) is GUES with prescribed attenuation performance ; the parameters of fault detection filter can be given bywhere , , , , , , , , , , .

Proof. It can be inferred that equation (14) can be rewritten aswhere , , , , .
Based on Lemma 1, we haveThus, by the aid of Schur complement, we can conclude that the switched system in equation (5) is GUES with prescribed performance ; the parameters of fault detection filters can be obtained by equation (33).

Remark 3. The learning-based fault detection filter is viewed as the compensation of the robust filter. Therefore, we can obtain the scheduling interval to ensure the stability and prescribed attenuation performance by the aid of nonfragile control theory in Theorem 3. The upper bounds on action in DDPG algorithm can be obtained by the predefined scheduling interval.

3.2. Intelligent Filter Design

The stability and prescribed attenuation performance are guaranteed by the theorems aforementioned. However, the transient performance needs to be improved. To solve the problem, deep reinforcement learning is applied in the framework of actor-critic. The optimization of parameters of fault detection filter can be viewed as an infinite Markov decision process, which is a series of continuous optimization processes. Thus, the DDPG algorithm is developed in this paper to improve the performance of the filter.

The frame of reinforcement learning consists of an agent and the environment. The state at kth time instant is defined as , an action is chosen by the agent, and then a reward function and the state of next step are obtained, where is developed to evaluate the performance of state-action pair generated by the agent. The fault detection system is viewed as the environment. We define the additional compensation as the action, which is utilized to maximize the sum of the expected discounted reward function over given future steps. The action and the sum of expected discounted reward function can be described in equations (36) and (37).where is defined as the discount factor and Kf represents the terminal step.

The state of the agent is given in the following equation:

The DDPG algorithm is proposed in the framework of deep Q learning and actor-critic. There are two actor networks and two critic networks. The optimal policy is tried for realization based on policy gradient theory in continuous action spaces. The action-value is approximated by employing the critic network , whose weights are defined as . The current output of compensated parameters is generated based on the actor network , whose weights are defined as . The weights are updated according to the loss function, which is described in equation the following:where .

The weights of actor network are updated according to the policy gradient theory, which is given in equations (40) and (41).where represents the learning rate of actor network.

Moreover, in the DDPG algorithm, two networks are adopted as actor target network and critic target network, which are defined as and . The weights of actor target network are defined as , which are updated according to the following equation: denotes the learning rate of actor target network.

Similarly, the weights of critic target network are defined as , which are updated according to the following equation: represents the learning rate of critic target network.

In order to improve the robustness of the proposed algorithm, an exploration noise is introduced as a compensation of the output of actor network, which can be generated based on

Therefore, based on the statement above, the pseudocode of intelligent fault detection filters design can be presented in Algorithm 1.

(1)Construct the fault detection filters for switched systems
(2)Set the variation range of fault detection filters
(3)Stability analysis and calculate the MDADT based on Eq. (15)
(4)Design the fault detection filters based on Theorem 3
(5)Randomly generate the initial weights of the actor network and critic network
(6)Initialize the critic target network and actor target network
(7)Initialize the Replay buffer and define episode = 0
(8)for episode = 1 to M do
(9) Initialize a random noise process to improve the robustness
(10) Initialize a random state vector s1, then the initial observation can be obtained
(11)  for t = 1 to K do
(12)   Chose action based on the current state and noise in the environment
(13)   Calculate the reward function and state
(14)   Store the transition pair in the Replay buffer R
(15)   Sample a random minibatch of N transitions from the Replay buffer
(16)   Set
(17)   Update the critic network, actor network and target networks
(18)end for
(19)end for

Remark 4. The DDPG algorithm is developed in this paper to improve the transient performance of the fault detection filter. Compared with the traditional method, the stability, robustness, and dynamic performance can be guaranteed simultaneously, in which the robust control theory and nonfragile control theory are introduced to ensure the stability, and the compensation of controller can be viewed as the variation of predesigned controller. Therefore, the stability of closed-loop system can be guaranteed by nonfragile control theory.

4. Numerical Example

In this section, simulation results are given to validate the effectiveness of the proposed method. The system matrices are given as follows:

The eigenvalues of A1 and A2 can be obtained as follows:

It can be inferred that the subsystems A1 and A2 are unstable, respectively. The external disturbance is defined as follows:

The other parameters of switched systems are listed as follows:

Therefore, the MDADT of switched systems are . The prescribed attenuation performance index is set to be . Then the parameter matrices of filters can be obtained in Theorem 3.

To validate the effectiveness of the proposed method, we give two numerical examples. The stability criteria of the switched systems in Theorem 2 are demonstrated by Example 1. Moreover, the effectiveness of the fault detection filter in Theorem 3 and the DDPG algorithm is illustrated in Example 2.

Example 1. Firstly, we provide the proof that the proposed conditions can ensure the stability of switched systems. As a comparison, a randomly generated switching signal is introduced to show that the switched systems cannot stay stable if all the subsystems cannot share a CLF. The switching logic satisfying equation (20) is given in Figure 1 and the randomly generated switching logic is given in Figure 2. The state response of the proposed switching logic is given in Figure 3 and the state response of randomly generated switching logic is showed in Figure 4. We can see that the switched systems cannot ensure the stability under randomly generated switching logic. However, the stability can be guaranteed according to Theorem 1 in the paper.

Example 2. To validate the effectiveness of the fault detection filter, the traditional MDADT method and the proposed method are given. The fault signal is set to beThe results are given in Figures 512. The residual signal and J(k) of MDADT method are given in Figures 5 and 6; the residual signal and J(k) of the proposed method are showed in Figures 7 and 8. We can see that the detection time of the MDADT method is 0.1 s, whereas, the detection time of the proposed method is 0.06 s. The fault can be detected efficiently, and the transient performance of fault detection filter can be improved by the DDPG algorithm. The state responses of the MDADT method and the proposed method are depicted in Figures 912. It can be inferred that the fault detection filter proposed in this paper can track the state response. The response of episodes reward is showed in Figure 13; we can see that the reward function can converge to a neighbor of the origin, which illustrate the effectiveness of the optimization algorithm.
In summary, we can see that the tighter bounds on dwell time and less conservative results are obtained. The stability of switched systems can be guaranteed by the proposed results in this paper. The fast switching strategy is applied, and it allows the subsystems of switched systems to remain unstable. The transient performance of fault detection filters can be improved by the aid of DDPG algorithm, and the stability, robustness, and optimal policy can be guaranteed simultaneously by the method proposed in this paper.

5. Conclusions

The problem of fault detection and online scheduling for switched systems with all modes unstable is studied. The observer-based fault detection filter is proposed to generate the residual signal, which consists of two parts: the dynamics-based fault detection filter and the learning-based fault detection filter. By employing MDADT method and MLF method, the stability of the switched systems is guaranteed. The solutions of fault detection filters are given in the form of LMIs. To achieve optimal control policy and improve the transient performance, the DDPG algorithm is utilized as learning-based fault detection filter, in which the output can be viewed as the variation of robust fault detection filter. Therefore, the nonfragile control is provided to ensure the stability of the optimization algorithm. Finally, the simulation results are introduced to demonstrate the effectiveness and superiority of the proposed method.

Data Availability

The data used to support the findings of this study are included within the article.

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgments

This study was cosupported by the National Natural Science Foundation of China (nos. 61973253, 62176214, and 62101590), the Aeronautical Science Foundation of China (no. 201907053001), and the National Natural Science Foundation of Shaanxi Province (nos. 2019JQ-014 and 2020JQ-481).