`International Journal of Differential EquationsVolume 2011, Article ID 319375, 13 pageshttp://dx.doi.org/10.1155/2011/319375`
Research Article

## Existence of Solutions of a Riccati Differential System from a General Cumulant Control Problem

1Department of Electrical & Computer Engineering, Kettering University, Flint, MI 48504, USA
2Department of Mathematics, Bradley University, Peoria, IL 61625, USA

Received 31 May 2011; Accepted 15 November 2011

Copyright © 2011 Stanley R. Liberty and Libin Mou. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

We study a system of infinitely many Riccati equations that arise from a cumulant control problem, which is a generalization of regulator problems, risk-sensitive controls, minimal cost variance controls, and -cumulant controls. We obtain estimates for the existence intervals of solutions of the system. In particular, new existence conditions are derived for solutions on the horizon of the cumulant control problem.

#### 1. Introduction

Consider a linear control system and a quadratic cost function: where , and are continuous matrix functions for , ( is the set of symmetric matrices.), is the state with known initial state , the control, and a standard Wiener process. Because is completely determined by the first equation in (1.1) in terms of , the cost function is only a function of .

For , denote by the (general) expectation of the random variable . For , let be the th cumulant of the cost . Let be a sequence of nonnegative real numbers. Consider the following combination of :

The cumulant control problem, considered in [1], is to find a control that minimizes the combined cumulant defined in (1.3). This problem leads to the following system of (infinitely many) equations of Riccati type: where denotes the derivative to , , are as in (1.1), and . and are the unknown matrix functions, which are required to be continuously differentiable for . For convenience, the time variable is often suppressed. System (1.4) will be combined with the following equation

If is a given integer and for , then is the -cost cumulant investigated in [2, 3]. In particular, if , then and the cumulant problem is the classical regulator problem that minimizes . If , then the cumulant problem is the minimal cost variance control considered in [4, 5]. Interested readers are referred to [16] for the investigations, generalizations, and applications of cumulant controls.

Another important cumulant control occurs when for . In this case, is precisely the cumulant generating function, and the cumulant problem is the risk-sensitive control; see, for example, [7]. In this case, (1.4) and (1.5) lead an equation for the matrix function:

As shown in [1], the solution of (1.4) is related to by the equation: and the equations in (1.4) for can be obtained by differentiating (1.6) to at .

For a feedback control with a given matrix function , it was shown in [8] and [1, Theorem  2] that the th cumulant of has the following representation: where is the solution of (1.4). Consequently (1.3) can be written as

In [1] the cumulant control problem was restated as minimizing in (1.9) with as a control, as a state, and (1.4) a state equation. Furthermore, the following result is proved in [1, Theorem  3].

Theorem 1.1. If the control is the optimal feedback control of (1.9), then the solution of (1.4) and must satisfy (1.5).

By Theorem 1.1, it is necessary to solve (1.4) and (1.5) in order to find a solution of a cumulant control problem. Because of the nonlinearity of (1.4) and (1.5) in , a global solution may not exist on the whole horizon of the cumulant problem. This can be illustrated by a scalar case of (1.6). Suppose , then (1.6) becomes and . The solution is tan , which is defined on with . So (1.6) has no solution unless .

By the local existence theory of differential equations, the solutions and of (1.4) and (1.5) exist on a maximal subinterval . Our interest is to give an estimate for this interval. In particular, we will obtain conditions that guarantee . The idea of our approach is to show that the trace tr of satisfies a scalar differential inequality: with some functions on . A key of the proof is Proposition 2.1 below. It follows that is bounded by the solution of the Riccati equation: where . Consequently, the existence interval of (1.12) gives an estimate for that of system (1.4) and (1.5); see Theorems 2.4 and 3.5 below. By a similar argument, we prove that the cumulant problem is well posed under appropriate conditions; see Theorems 2.3 and 3.4 below.

In [9] the norm of a solution of a coupled matrix Riccati equation was shown to satisfy a differential inequality similar to (1.11). Consequently, specific sufficient conditions were derived for the existence of solutions of the Riccati equation in [9]. Estimates for maximal existence interval of a classical Riccati equation had been obtained in [10] in terms of upper and lower solutions. For the coupled Riccati equation associated with the minimal cost variance control, some implicit sufficient conditions had been given in [11] for the existence of a solution. In this paper, we use the trace to bound the solution of system (1.4) and (1.5), which generally leads to a better estimate for the existence interval.

#### 2. Comparison Results for Traces

We start with an assumption and some preparations. In this paper we assume that

For the sequence in (1.3), we will assume that where

Note that the assumption is not essential. The assumption that for and Proposition 2.1 below imply that the matrix in (1.4) is a positive semidefinite series. The requirement that imposes some growth condition for the sequence ; see the proofs of Theorems 2.3 and 2.4.

Also note that if , then and for all . Theorem 3.4 below shows that the cumulant control problem is well posed for any sequence with a small . Some examples of are as follows:(i); (ii);(iii).

We need the following properties of .

Proposition 2.1. Suppose is a solution of (1.4) on some interval with a given , then each for .

Proof. The formula of the th cumulant in [8] implies that is nonnegative for all . It follows from the representation (1.9) that must be positive semidefinite. This argument continues to hold with replaced by any .

Next we verify some properties related to matrix trace that are needed for our analysis. For , denote by tr the trace of , and and the smallest and largest eigenvalue of , respectively.

Proposition 2.2. (a) For all , , .
(b) For all , ,
(c) If , then
(d) If are all , then

Proof. The properties in (a) are obvious by the definitions of trace and matrix multiplication. Some of the inequalities in (b)–(d) might be known, but the authors were not able to find proofs in existing literature. So we include our proofs of (b)–(d) below for readers’ convenience.
To prove (b), let be a unitary matrix such that where is diagonal with eigenvalues of . Then where is the entry of at . Let be the th row of , which is a unit vector. Then . Since we have This implies (2.4).
To show (c), use the symmetry of , and ; we get that tr.
Inequality (2.6) follows from part (b) and the fact that since .
To show (2.7), first note that trtr by (2.4); then it remains to show that . Choose a with such that . By Schwartz inequality, Since , we have and . Therefore, .
For (2.8), using notations in the proof of (b), we first get Then (2.8) follows from the inequalities

Now we estimate the existence intervals of solutions of (1.4) and (1.5). First, let be given and be the solution of system (1.4). We have the following result, which will be used in the proof of Theorem 3.5

Theorem 2.3. Suppose and in (1.4) is given. Let , , and be functions on satisfying
(a) If is a solution of system (1.4), then satisfies the differential inequality
(b) If the equation has a solution on , then system (1.4) has a solution on such that is convergent.

Proof. (a) Denote . Multiplying the equation in (1.4) for by and sum over , we obtain
Taking traces of both sides of (2.17) gives Note that and by Proposition 2.1, , for . Proposition 2.2 (a), (b), and (c) imply that
By (1.4) and definition (2.3) of , we have
Substituting (2.19) and (2.20) into (2.18) and using the definition of in (2.14) we get (b) Suppose that (2.16) has a solution on . By local existence theory, system (1.4) has a solution on a maximal interval By (a), satisfies inequality (2.15); that is is a lower solution of (2.16). By a comparison theorem of lower-upper solutions, on . Since series (1.10) is positive semidefinite, it follows that and are all bounded and (1.10) is convergent on . Since satisfies system (1.4), each is in fact continuously differentiable on . If , then the local existence theory implies that can be extended further to the left of , a contradiction to the maximality of . Therefore and (1.4) has a solution on , which can be extended to .

Now consider system (1.4) and (1.5). We have

Theorem 2.4. Denote . Let , and be functions on satisfying (a) Suppose and are solutions of (1.4) and (1.5) on some . Then on satisfies (b) Suppose the equation has a solution on , then system (1.4) and (1.5) have solutions and on .

Proof. (a) Substituting into system (2.17) we get where . Taking traces of both sides of (2.25) gives
As in the proof of previous theorem, we have Using the fact that and (2.8), we have
By combining (2.28), (2.27), and (2.26) we obtain (2.23).
(b) Suppose that (2.24) has solution on . By local existence theory, system (1.5) and (1.4) have solutions and on a maximal interval . By part (a), tr satisfies inequality (2.23) on . It follows that tr on , which implies that and are continuously differentiable on If , then local existence theory implies that and can be extended further to the left of , a contradiction to the maximality of . Therefore , and system (1.4) and (1.5) have solutions on .

#### 3. Well-Posedness and Sufficient Existence Conditions

In this section we will derive specific conditions that ensure that the scalar equations (2.16) and (2.24) have solutions on . Consequently we will obtain sufficient conditions for the well-posedness of the cumulant control and the existence of solutions of (1.4) and (1.5).

First we consider an autonomous scalar equation where is a polynomial with degree . Assume for some that has distinct zeros . Let and . Since is locally Lipschitz, the solution of (3.1) exists and is unique for every for in a maximal interval, say . If for some , then for all . If for some , then for . This implies that for , has the same sign as In particular, as decreases, is strictly increasing if and decreasing if . Denote . Then

The following is a well-known fact in stability theory of differential equations. where . Indeed, implies that for and for . In either cases, by (3.2).

Consider (1.12) as an example. We have the following.

Proposition 3.1. Denote and if . Then
(a) for all if and ;
(b) if and ;
(c) if or and .

Proof. If , then , which has root . Since , by (3.3). This shows (a).
Next we prove (b) and (c). First assume . If , then for all . So by (3.2). Next consider the case , in which . If , then , which implies that . If , then we have either when or when . In either cases, we have by (3.2). This finishes the proof of (b) and (c) when . If , then consider , which satisfies and , and the conclusions follow from the special case just proved.

Write (3.1) as and integrate it against from to , then we get

Note that if is finite, then must be a zero of . It follows that and . If is infinite, then must converge because has a degree . In summary, we have the following.

Proposition 3.2. Suppose is a polynomial of degree . Then
(a) is finite if and only if the solution of (3.1) exists on .
(b) if and only the solution of (3.1) exists on a finite maximal interval with length .

Applying Proposition 3.2 to (1.12) we obtain the following.

Proposition 3.3. (a) If either and , or and , then the solution of (1.12) exists on .
(b) If either or and , then the solution of (1.12) exists on a finite interval with

Proof. Part (a) directly follows from Proposition 3.1(a) and Proposition 3.2 (a) (b).
For part (b), first assume that . Then and , where . So
Next assume and . Then and , where . We have
Finally when and , we have and

Now we show that the cumulant control problem is well posed by Theorem 2.3 and Proposition 3.3.

Theorem 3.4. For any number there is such that the series in (1.3) converges for each matrix and sequence with and .

Proof. Suppose that is a matrix function with . Choose , , and as follows: Note that tr, which depends only on . In addition, as . It follows that when is sufficiently small, has two real roots with and as . In particular, since , we have and so . Proposition 3.3 implies that
So as . In particular, (2.16) has a solution on when is sufficiently small. By Theorem 2.3 (b), system (1.4) has a solution such that converges.
Finally we apply Proposition 3.3 to (2.24) to give a sufficient existence condition for (1.4) and (1.5) and the cumulant control problem. Choose

Theorem 3.5. System (1.4) and (1.5) have solutions and on if where is defined as in (3.2) with . In particular, system (1.4) and (1.5) have solutions and on if one of the following holds.
(a) .
(b) , , and , where .

Proof. The general conclusion follows directly from Proposition 3.3 and Theorem 2.4. In the case (a), has two roots . Since , . In the case (b), has two solutions . So also holds. The conclusion follows from Propositions 3.1 and 3.2 and Theorem 2.4.

Note that in Theorem 3.5 condition (a) holds if has full rank (i.e., ) and is sufficiently small, while condition (b) holds if the system in (1.1) is stable (i.e., ) and the product tr is relatively small. The cumulant control problem has an optimal control under each of these conditions.

As an existence theorem, Theorem 3.5 gives one of the very few existence results for a Riccati differential system of infinitely many equations. In terms of the cumulant controls that lead to the system (3) and (4), Theorem 3.5 generalizes the corresponding results in [1, 5] for risk-sensitive controls (where ) and in [2, 4] for finite cumulant controls (where has only finite nonzero components). Numerical examples for risk-sensitive and finite cumulant controls satisfying the conditions in Theorem 3.5 may be found in [36].

#### 4. Conclusions

In general it is very difficult to determine the existence interval of a differential Riccati equation (or system). By the approach in this paper, we can at least give an estimate for the existence interval of the Riccati system. Such an estimate leads to sufficient conditions for the existence of solutions to the Riccati system and the cumulant control problem.

#### Acknowledgments

The authors wish to express their sincere thanks to the reviewers for their valuable comments that helped improve this paper. The second author wishes to acknowledge the support of a Caterpillar Fellowship from Bradley University.

#### References

1. L. Mou, S. R. Liberty, K. D. Pham, and M. K. Sain, “Linear cumulant control and its relationship to risk-sensitive control,” in Proceedings of the 38th Allerton Conference on Communication, Control, and Computing, pp. 422–430, 2000.
2. K. D. Pham, S. R. Liberty, and M. K. Sain, “Linear optimal cost cumulant control: a k-cumulant problem class,” in Proceedings of the 36th Allerton Conference on Communication, Control, and Computing, pp. 460–469, 1998.
3. K. D. Pham, M. K. Sain, and S. R. Liberty, “Cost cumulant control: state-feedback, finite-horizon paradigm with application to seismic protection,” Journal of Optimization Theory and Applications, vol. 115, no. 3, pp. 685–710, 2002.
4. M. K. Sain, C. H. Won, and B. F. Spencer Jr., “Cumulant minimization and robust control,” in Stochastic Theory and Adaptive Control, T. E. Duncan and B. Pasik-Duncan, Eds., Lecture Notes in Control and Information Sciences, pp. 411–425, Springer, Berlin, Germany, 1992.
5. M. K. Sain, C. H. Won, B. F. Spencer Jr., and S. R. Liberty, “Cumulants and risk-sensitive control: a cost mean and variance theory with application to seismic protection of structures,” in Advances in Dynamic Games and Applications, J. A. Filar, V. Gaitsgory, and K. Mizukami, Eds., vol. 5 of Annals of the International Society of Dynamic Games, pp. 427–459, Birkhäuser, Boston, Mass, USA, 2000.
6. M. J. Zyskowski, M. K. Sain, and R. W. Diersing, “State-feedback, finite-horizon, cost density-shaping control for the linear quadratic Gaussian framework,” Journal of Optimization Theory and Applications, vol. 150, no. 2, pp. 251–274, 2011.
7. P. Whittle, Risk-sensitive optimal control, John Wiley & Sons, New York, NY. USA, 1990.
8. S. R. Liberty and R. C. Hartwig, “On the essential quadratic nature of LQG control-performance measure cumulants,” Information and Computation, vol. 32, no. 3, pp. 276–305, 1976.
9. G. P. Papavassilopoulos and J. B. Cruz, Jr., “On the existence of solutions to coupled matrix Riccati differential equations in linear quadratic Nash games,” IEEE Transactions on Automatic Control, vol. 24, no. 1, pp. 127–129, 1979.
10. S. R. Liberty and L. Mou, “Estimation of maximal existence intervals for solutions to a Riccati equation via an upper-lower solution method,,” in Proceedings of the 39th Allerton Conference on Communication, Control, and Computing, pp. 281–282, 2001.
11. G. Freiling, S.-R. Lee, and G. Jank, “Coupled matrix Riccati equations in minimal cost variance control problems,” IEEE Transactions on Automatic Control, vol. 44, no. 3, pp. 556–560, 1999.