Stabilizing Solution for a Discrete-Time Modified Algebraic Riccati Equation in Infinite Dimensions

Abstract

We provide necessary and sufficient conditions for the existence of stabilizing solutions for a class of modified algebraic discrete-time Riccati equations (MAREs) defined on ordered Banach spaces of sequences of linear and bounded operators. These MAREs arise in the study of linear quadratic (LQ) optimal control problems for infinite-dimensional discrete-time linear systems (DTLSs) affected simultaneously by multiplicative white noise (MN) and Markovian jumps (MJs). Unlike most of the previous works, where the detectability and observability notions are key tools for studying the global solvability of MAREs, in this paper the conditions of existence of mean-square stabilizing solutions are given directly in terms of system parameters. The methods we have used are based on the spectral theory of positive operators and the properties of trace class and compact operators. Our results generalise similar ones obtained for finite-dimensional MAREs associated with stochastic DTLSs without MJs. Also they complete and extend (in the autonomous case) former investigations concerning the existence of certain global solutions (as minimal, maximal, and stabilizing solutions) for generalized discrete-time Riccati type equations defined on infinite-dimensional ordered Banach spaces.

1. Introduction

In recent years, the study of optimal control problems associated with stochastic systems with Markovian regime switching is of particular interest to researchers due to their various applications in finance, biology, engineering, and so forth. Even in the case of linear systems, the optimization problems become considerably hard, when we are in infinite dimensions and/or the Markovian process has infinite state space (see, e.g., [1–5] and the references therein). As we know from the LQ optimal control theory of DTLSs with MN and MJs (see, e.g., [5–7]), the design of the optimal control is closely related to the existence of a stabilizing solution (SS) for an associated generalized discrete-time Riccati equation (GDTRE). In this paper, we investigate the solution properties for a class of MAREs of control associated with autonomous infinite-dimensional DTLSs with MN and infinite MJs. These MAREs are time-invariant versions of the infinite-dimensional GDTREs studied in [8]. For a detailed treatment of finite-dimensional GDTREs, the reader can consult [6, 7] and the references therein. The problem of existence of SSs for infinite-dimensional GDTREs associated with stabilizable DTLSs with infinite MJs was investigated in [3, 5, 9, 10] under either stochastic detectability or observability hypotheses. In [8], a set of necessary and sufficient conditions for the existence of SSs is expressed in terms of feasibility of some linear matrix inequalities (LMIs) system.

The general theory of GDTREs applies to our MAREs but new special results are strongly expected in the autonomous case. For example, in finite dimensions it is proved (see [11]) that necessary and sufficient conditions for the existence of stabilizing solutions for MAREs can be given directly in terms of system parameters. They consist in verifying whether the stochastic system is stabilizable and is not an unobservable eigenvalue for a pair of associated operators. Although it is not always possible to give an infinite-dimensional version of a finite-dimensional result (as an example, we recall the researches proving that Hautus Lemma does not work in infinite dimensions [12, 13]), in this paper we have tried to do this for Theorem 12 from [11].

The approach we propose is based on operator theory tools and the properties of positive operators and their adjoints. Using the results from [8], we prove that conditions similar to those in [11] are necessary for the existence of SSs for infinite-dimensional MAREs (see Theorem 10). To obtain sufficient conditions, we extend the class of eigenvalues that must not be unobservable to the set and we impose additional constraints for the coefficients of the DTLSs with MN and MJs (see Theorems 11 and 13). When being applied to finite-dimensional DTLSs with MN, our results recover the ones in [11]. However, Corollary 14 seems to be new for finite-dimensional DTLSs with finite Markovian jumps. As we have shown in Example 1, it provides necessary and sufficient conditions for the existence of SSs for MAREs associated with DTLSs with MJs which are neither stochastically observable nor stochastically detectable. In this case, the results like those from [2, 5, 10] do not work, while a LMIs approach would lead to a large number of operations for systems of large dimensions.

The paper is organized as follows. In Section 2, we introduce some basic notation and terminology and we formulate the problem. Section 3 briefly reminds us about certain properties of nuclear (trace class) and compact operators and extends to a more general framework, the well-known result: “the dual space of trace-class operators space is isomorphic to the set of bounded operators.” Also, we define the notions of maximal and stabilizing solution for a MARE and we show how to apply the results from [8] to the MAREs discussed in this paper. In Section 4, we obtain the main results. We first introduce the notion of unobservable eigenvalue for a pair of operators, by using spectral properties of positive operators. Then, we provide necessary and sufficient conditions for the existence of stabilizing solutions to MAREs (see Theorems 10, 11, and 13). To obtain the sufficient conditions from Theorems 11 and 13, we assume additional hypotheses like compactness of the coefficients or trace-class membership. Corollary 14 shows that the finite-dimensional results from [11] are direct consequences of Theorems 10, 11, and 13.

Section 5 is devoted to numerical examples. We study three examples, which cover the cases of MAREs associated with finite-dimensional DTLSs with finite MJs and infinite-dimensional DTLSs with infinite MJs. As mentioned above, Example 1 shows the advantages of Corollary 14 over previously published results [2, 5, 10].

Finally, the last section provides conclusions and further research lines.

2. Notations and Statement of the Problem

Let , , and be real separable Hilbert spaces. We denote by the real Banach space of linear and bounded operators from into and by the Banach subspace of , formed by all self-adjoint operators. As usual denotes either the adjoint of a linear and bounded operator or the dual of a Banach space. We will write for the inner product and for norms of elements and operators, unless otherwise is indicated. An operator is called nonnegative and we write , if is self-adjoint and for all .

Let be an interval of integers, which may be finite or infinite. If is a real Banach space, then is a real linear space with the usual termwise addition and (real) scalar multiplication. Moreover, is a Banach space when endowed with the norm . If is or , then will be denoted by or . Let be the set of natural numbers and . We denote by the Banach space obtained from by replacing and by and , respectively.

An element is said to be nonnegative (we write ) iff for all . The cone of all nonnegative elements of introduces the following order on :

Let be the identity operator on and let be the element of . We say that is positive and we write , if there is such that . A sequence is called nonnegative if for all and uniformly positive (we write ) if there is such that for all .

A linear and bounded operator is called positive iff . For and , we define the product by , .

Now let us introduce two special notations on spaces . If , then denotes the element of defined by , (i.e., the th component of is the adjoint operator of ). Also, if is such that exists for all , then . Note that may not belong to .

Let be a probability space. For any integrable random variable on , we write for its expectation (mean). We will denote by the Hilbert space of all valued random variables with the property . If and are integrable real valued random variables, we denote by the conditional expectation on the event . We recall that, by definition, if , and otherwise.

Let be a homogeneous Markov chain on with the state space , the transition probability matrix and the property that , , . The last condition ensures the nontrivial computation of the conditional mean , .

Throughout this paper, we assume that the following hypotheses hold.(P1)(i) is fixed, , , , and , , and are such that (ii), , are sequences of real valued, zero mean, square integrable random variables on having the properties that , , is a sequence of independent random vectors, , and , and are independent. (Here denotes the transposition.)

Let and be the -algebras generated by and . We introduce the -algebra defined by and if .

We consider the stochastic system with MNs and MJs denoted and the cost functional

An optimal control problem associated with (5) and (7) consists in minimizing the cost functional , , subject to (5), over the set of admissible controls with the property that , , are -valued, -measurable random variables such that and for all .

Let , and As we know from [5, 7], has a solution if the following modified algebraic Riccati equation (MARE): has a mean-square stabilizing solution (see Definition 8).

Our problem is to provide a set of necessary and sufficient conditions for the existence of a stabilizing solution for MARE (10).

3. Preliminary Results

3.1. Sequences of Nuclear and Compact Operators

As in [9], we denote by the Banach space of all nuclear operators from . It is known that is a Banach space when endowed with the nuclear norm . Here, is the trace operator. It is well know (see, e.g., [14, 15]) that For further properties of nuclear operators, the reader is referred to [9, 14–16]. It is known that the linear subspace of is a Banach space (see [9]) when endowed with the norm . An easy computation (see [9]) shows that In the sequel we sometimes use a special element of , defined for any and by

Here, is defined by .

Let denote the set all compact operators from . Then, will be the subset of formed by sequences of compact operators; that is, for all .

It is well known that is dense in in the uniform operator topology (see, e.g., Problem 5.69 from [15]). The following lemma, whose proof is a simple exercise for the reader, states that a similar result remains true for finite sequences of nuclear and compact operators.

Lemma 1. If is finite, then is dense in with respect to .

Remark 2. In the case when is infinite, the above result is not true as it is proved by the following counter example. Let and let . Then, and . Since is separable and is not, it follows that is not dense in . Thus, is not dense in .

Proposition 3. The dual space of the Banach space is isometrically isomorphic with . The isomorphism is defined by , where is the linear functional:

Proof. Let and be arbitrarily chosen. The following inequalities: show that and We will prove that is the isometric isomorphism that maps into . Let . For any , we define by Obviously, . Hence , . It follows that and On the other hand, a simple computation shows that and (17) and (14) imply that for all , . It is well known that every can be written as a sum , where the series is -convergent and , [15]. Hence, the element , defined by and , for , , can be written as where the series is -convergent. Then, (20) and the -continuity of and ensure that for all . Now, for an arbitrary X , we define the sequence , , X , if , and for , . It is not difficult to see that , converges to X in . From (21), it follows easily that for all . Using again the -continuity of and , we get XX. Therefore, and combining (16) and (18), it follows that . We conclude that the map is an isometric isomorphism from to . The proof is complete.

Remark 4. A direct consequence of (17) and the above lemma is the following property of the isomorphism .
is a positive functional (i.e., for any , ), if and only if .

3.2. Maximal and Stabilizing Solutions of MARE

In this section, we recall some definitions and results which we frequently use in the rest of the paper. To be consistent with previous publications (see, e.g., [7, 8] and the references therein) concerning discrete-time Riccati equations associated with DTLSs with MN and MJs, we introduce the following notation. For all , let We see that , , , and is a positive operator. Then, MARE (10) can be equivalently rewritten as where, for all ,

Following [8], we define the dissipation operator and the subsets and of

Lemma 28 from [8] (which is an operatorial version of the Schur complement Lemma) ensures that iff (if and only if) .

Definition 5. (a) We say that is a solution of MARE (10) if is invertible for all and satisfies (10).
(b) We say that a solution of MARE (10) is maximal if , , for arbitrary .

Now, for any , we define

Obviously, is a positive operator.

As in [8], let , , , be the evolution operator defined by . We say that , is exponentially stable iff there are , such that for all , or, equivalently, iff , where denotes the spectral radius of .

In view of Theorem 14 from [8], we have the following.

Definition 6. The linear and positive operator is stabilizable if there is such that the evolution operator , , , is exponentially stable.

Since (4) and (22) imply that , the following proposition is a direct consequence of Theorem 9 from [8].

Proposition 7. Assume (P1). If there is such that is exponentially stable, then MARE (10) has a maximal solution which has the property that

Theorems 9 and 13 from [8] ensure that, in the time invariant case, the notion of stabilizing solution of MARE (10) from [8] is equivalent with the one introduced by the following.

Definition 8. By a stabilizing solution of MARE (10), we mean a solution with the property that the evolution operator , , , where is exponentially stable.

Finally, we recall (see (4.8) in [8]) that mapping can be equivalently rewritten as for all . We will use later this formula for different values of .

4. Main Results

In this section we establish necessary (see Theorem 10) and sufficient conditions (see Theorems 11 and 13) for the existence of a stabilizing solution to MARE (10). They are related to the nonexistence of certain unobservable eigenvalues for a pair of operators which are defined with the coefficients of (10). For finite-dimensional MAREs, we recover the results from [17] (see also [11] for the continuous time case). Unlike [17], our results apply to stochastic systems with Markovian jumps.

Assume (P1) and let and , be defined by (28) and (29), respectively. Since is a positive operator on , we know from [18] (see Theorem 3.2.3 therein) that there is such that , , and where is the spectral radius of . Therefore, has nonnegative eigenvalues.

Definition 9. We say that is an unobservable eigenvalue for the pair , , iff there is an such that , , and Otherwise, we will say that is not an unobservable eigenvalue for the pair .

The next theorem gives necessary conditions for the existence of a stabilizing solution to MARE (10).

Theorem 10. If the algebraic Riccati equation (10) has a stabilizing solution, then(a) is stabilizable;(b) is not an unobservable eigenvalue for the pair .

Proof. Statement (a) follows from Definition 8. Let us prove (b). If (10) has a stabilizing solution , then it coincides with the maximal solution, according to Theorem 13 from [8]. Since , we deduce that (see Definition 5(b)). Using Theorem 13 from [8] again, we see that . A slight modification of the proof of Theorem 13 from [8] leads us to the conclusion that, in the time invariant case, contains a constant sequence , . For such we have and, consequently, there is a such that Assume by contradiction that (b) is not satisfied. This means that there is , , such that Using successively (36), (33) (with replacing ), and (37), we get Hence . From (36) and the positiveness of , we deduce that It follows that . Since for all , it follows easily that . We have obtained a contradiction. Therefore, (b) holds and is not an unobservable eigenvalue for the pair . The proof is complete.

Now let us introduce a new hypothesis (P2)(i) and , , , are compact operators.

In the case when is finite, we have the following converse of Theorem 10.

Theorem 11. Assume that is finite and (P1) and (P2) hold. If(a) is stabilizable and(b)the pair has not an unobservable eigenvalue , then the algebraic Riccati equation (10) has a stabilizing solution.

Proof. From Proposition 7, we know that the algebraic Riccati equation (10) has a maximal solution . Assume by contradiction that is not stabilizing; that is, is not exponentially stable. Then, and there is , , such that . Hence, for all . For the sake of simplicity, let us denote by .
Let us prove that . Indeed, if , then . Since , according to Lemma 1, we deduce that . On the other hand, hypothesis (P2) implies that for all and, therefore, for all . Hence, that is a contradiction. It follows that .
Thus, is a nonzero linear functional. Obviously, , , and . Moreover, , . By Proposition 3, we know that there is , , , such that for all . Rewriting (33) for replaced by , we see that the maximal solution of the algebraic Riccati equation (10) satisfies the following equation: Hence, . From (P1)(i) and (29), . Using the properties of , we obtain It follows that . An easy computation shows that Lemma 28 from [8] (a Schur complement result for sequences of operators) and (P1)(i) imply that . Since and , it follows easily that and .
We know that, for any and an orthonormal basis of , the increasing sequence is weakly convergent to . It is not difficult to see that this property remains valid for each component of any . So there is a nonnegative, increasing sequence , which is componentwise weakly convergent to . The positiveness of and the inequality imply that . Since we can apply (40) and we get for all . Now let us recall the following property of nonnegative operators: . From (47), we obtain Passing to the limit as in the above equality, we get Taking into account that is invertible for all , we deduce that , for all or, equivalently, Thus, for all and for all . In view of (40), we get for all . Let us prove that the above equality remains true if we replace with an arbitrary . As mentioned above, there is a sequence that converges componentwise and weakly to . That is, , weakly, for all . From hypothesis (P2), it follows that is norm convergent to for all and . This is because the multiplication to the right and to the left with compact operators promotes the weak convergence of operators to the uniform convergence.
Therefore, is -convergent to . Similarly, we deduce that is -convergent to . From (52) we have and passing to the limit for we obtain So . In view of (45), we just have obtained a contradiction of (b). Therefore, is a stabilizing solution and the conclusion follows.

Remark 12. If the maximal solution would have the property that , the proof of the above theorem could be modified such that condition is to be replaced by .

Before stating the next result, we note that for all and ; that is, is a two-sided ideal of .

In the case when is infinite, we have the following version of Theorem 11.

Theorem 13. Assume that is infinite and , . If(a) is stabilizable and(b)the pair has not an unobservable eigenvalue , then the algebraic Riccati equation (10) has a stabilizing solution.

Proof. Arguing as in the proof of the above theorem, we see that (10) has a maximal solution . Since for all and is a two-sided ideal of , we deduce easily that for all . Assuming by contradiction that is not a stabilizing solution and reasoning as in the proof of the above theorem, we deduce that and there is , , , such that . Property (56) ensures that . Otherwise, (56) implies that for all , which contradicts the assumption . As in the proof of Theorem 11, we obtain and Letting and repeating step by step the arguments in the proof of Theorem 11, we obtain successively , , and for all . Now, property (56) implies that for all and (54) holds. Therefore, and . Since , we have obtained a contradiction of statement (b). Consequently, is a stabilizing solution and the proof is complete.

Now, let us apply our results to finite-dimensional MAREs associated with stochastic systems of the forms (5) and (6). We will prove that conditions (a) and (b) from Theorem 13 are necessary and sufficient for the existence of a stabilizing solution to MARE (10).

Corollary 14. Assume that is finite and the Hilbert spaces and are finite-dimensional. The algebraic Riccati equation (10) has a stabilizing solution if and only if(a) is stabilizable,(b) is not an unobservable eigenvalue for the pair .

Proof. We first recall that, in finite dimensions, all linear and bounded operators are compact and nuclear. Also, if is stabilizable, the maximal solution of MARE (10) has the property . Indeed, from the proof of Theorem 9 from [8], we know that is the strong limit of an increasing sequence , , of solutions of certain associated Lyapunov equations. Denoting by the Lyapunov operators associated with these equations, we also know that converges to and (see [7, 8], e.g.,). Since the eigenvalues of a matrix and consequently its spectral radius depend continuously on the matrix coefficients, we pass to the limit as in the last inequality and we get . The conclusion follows.
Further, we observe that the hypotheses of Theorems 10 and 11 are fulfilled. The necessity part of the corollary follows from Theorem 10 while the sufficiency part is a direct consequence of Theorem 11 and Remark 12.

Remark 15. (a) Under the hypotheses of the above corollary, the operator coincides with the Lyapunov operator associated with the close-loop DTLS from [17, Theorem 12] for the stabilizing feedback controller .
(b) In view of (a), Corollary 14 shows that Theorems 10, 11, and 13 are infinite-dimensional extensions of Theorem 12 from [17].

5. Numerical Examples

In this section, we provide some numerical examples which show the efficiency of our theory. Even in the finite-dimensional case our results (see Corollary 14) seem to be new when applied to DTLSs with Markovian jumps. So, let us begin with a finite-dimensional example which proves that Corollary 14 is a viable alternative when stochastic detectability and stochastic observability conditions fail to hold.

Example 1. Consider (5) and (7) in the special case where the multiplicative noise is missing (, , ) and(P3), , is fixed, , , , , , , , for all , and for , , and otherwise.
Let be the output of (5). Note that, in its operation, system (5) switches between the modes according to the law of the Markov chain . For example, if the initial state is , it obeys (59) till time when the Markov chain switches to from and system (5) obeys (59) with replacing .
Let us solve the optimization problem defined in Section 1.
In this special case, problem is exactly a linear quadratic control problem associated with (5), (58). The associated MARE (10) has the following particular form: We know [7] that if MAREs (60) and (61) have a stabilizing solution , then the optimal control problem has a solution , where is defined by (32). So we only have to find a stabilizing solution of (60) and (61).
Let be the stochastic system (5) without control (i.e., , ) and let be the system defined by and the output (58). The existing literature results (see [7] and the references therein) show that (10) has a stabilizing solution if the stochastic system with control is stabilizable and if is either stochastically observable or stochastically detectable. We will establish that is neither stochastically uniformly observable nor stochastically detectable. (Note that stochastic observability does not imply stochastic detectability [7].) In this case, we will see that Corollary 14 ensures the existence of the stabilizing solution.
(I) System Is Not Stochastically Uniformly Observable. According to Theorem 8 from [10], is stochastically uniformly observable iff there are and such that where is defined by (22) with the coefficients from (P3). A direct computation shows that for all . Using (P3), we see that for all and (62) fails to hold for . Consequently, is not stochastically uniformly observable.
(II) System Is Not Stochastically Detectable. Following [5] (see Definition 2 and Proposition 16 therein), the system is detectable in conditional mean iff there is such that the nonnegative operator defined by generates an exponentially stable evolution operator. We observe that the matrix associated with is upper triangular with the main diagonals , , and . Since , we deduce that is an eigenvalue of and . So cannot generate an exponentially stable evolution operator and is not detectable in conditional mean.
Now let us show that Corollary 14 can be used to prove that MARE (10) has a stabilizing solution in the special case when (P3) holds.
(III) Stabilizing Solution for the Algebraic Riccati Equation. We will prove that conditions (a) and (b) of Corollary 14 hold and, consequently, (60) has a stabilizing solution . Setting , , we have (see (28)) for all . The matrix associated with is upper triangular. Since all the elements of the main diagonal are less than , it follows that and is exponentially stable. Hence, is stabilizable, according to Definition 8, and condition (a) of Corollary 14 is satisfied. Let us prove that (b) holds, too. From (P3), we see that and , equal to , are given by (63). The matrix associated with is also upper triangular with the main diagonal for and . It is not difficult to see that is not an eigenvalue of because . Consequently, is not an eigenvalue of and it cannot be an unobservable eigenvalue for the pair . Condition (b) of Corollary 14 is satisfied and (60) has a stabilizing solution .
Proposition 14 from [10] ensures that this stabilizing solution is nonnegative and maximal among all nonnegative solutions of (60). Then, solving (60) and choosing the maximal solution, we get the stabilizing solution .
For , the stabilizing solution is and the feedback stabilizing gain is Then, the optimal control problem has a solution , with the optimal cost being , .

In the following example, we apply Theorem 13 to prove the existence of stabilizing solutions for infinite-dimensional MAREs.

Example 2. Consider MARE (10) under the following additional hypotheses. The Hilbert space is infinite-dimensional, , , , , and , , where , . The other coefficients of MARE (10) are , , , , , , , and otherwise. Let us show that the corresponding MARE (10) has a stabilizing solution.
Obviously , , and . Assume that is an unobservable eigenvalue of and let be a positive eigenvector of . We have It follows that . From Lemma 6 from [10], we know that . Hence, and we get a contradiction. Thus, cannot be an unobservable eigenvalue for the pair . Condition (a) of Theorem 13 holds.
Further, we define , , where . Then, , , and . Therefore, and generates an exponentially stable evolution operator. Thus, is stabilizable, according to Definition 8. Conditions (a) and (b) of Theorem 13 are satisfied and (10) has a stabilizing solution. Theorem 9 from [8] provides an iterative algorithm for the computation of this stabilizing solution.

The following example is an application of Theorem 10.

Example 3. Assume that, except , , and , the other coefficients of MARE (10) satisfy (P3) (see Example 1) with replaced by . Let , , for all , and . We recall that and and is a positive operator. Since , we can apply Lemma 6 from [10] to deduce that . On the other hand, is an eigenvalue of and, consequently, . Theorem 3.2.3 from [18] ensures that there is , , , such that . Obviously, and now it is clear that is an unobservable eigenvalue for . From Theorem 10, it follows that MARE (10) has not a stabilizing solution.

6. Conclusions and Further Research

In this paper we have obtained necessary and sufficient conditions for the existence of stabilizing solutions for MARE (10) in infinite dimensions. These conditions are similar to those given in [11, 17] for finite-dimensional MAREs and do not involve other detectability and observability conditions. They are viable alternatives to existing results (see, e.g., [3, 5, 7] and the references therein) obtained under detectability or observability hypotheses.

The main difficulties in obtaining a “perfect” analog of the results from [17] (or [11]) are related to the fact that Lemma 1 is not true for infinite and we cannot ensure the existence of a nonnegative linear functional satisfying (34) and condition . To compensate these gaps, we have assumed a compactness hypothesis in Theorem 11 and the condition , , in Theorem 13. A natural question is how to relax these conditions.

Another open problem is whether the assertion , where is the maximal solution of MARE (10), remains true in infinite dimensions or for infinite . The validity of this assertion will improve Theorems 11 and 13, as indicated in Remark 12.

Also, further research is required to study the existence of stabilizing solutions for MARE (10) in the case when the state space of the Markov chain is a general Borel Space as in [1].

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.