Existence of Solutions of a Riccati Differential System from a General Cumulant Control Problem

Abstract

We study a system of infinitely many Riccati equations that arise from a cumulant control problem, which is a generalization of regulator problems, risk-sensitive controls, minimal cost variance controls, and 𝑘-cumulant controls. We obtain estimates for the existence intervals of solutions of the system. In particular, new existence conditions are derived for solutions on the horizon of the cumulant control problem.

1. Introduction

Consider a linear control system and a quadratic cost function: 𝑑𝑥=(𝐴𝑥+𝐵𝑢)𝑑𝑡+𝐺𝑑𝑤,𝑡∈𝑡₀,𝑡_𝑓;𝑥𝑡₀=𝑥_0,𝐽(𝑢)=𝑥^𝑇𝑡_𝑓𝑄_𝑓𝑥𝑡_𝑓+^𝑡𝑓_𝑡0𝑥^𝑇𝑄𝑥+𝑢^𝑇𝑅𝑢𝑑𝑡, (1.1) where 𝐴(𝑡)∈ℝ^𝑛×𝑛,𝐵(𝑡)∈ℝ^𝑛×𝑚,𝑄(𝑡)∈𝕊^𝑛, and 𝑅(𝑡)∈𝕊^𝑚 are continuous matrix functions for 𝑡∈[𝑡₀,𝑡_𝑓], 𝑄_𝑓∈𝕊^𝑛 (𝕊^𝑛 is the set of 𝑛×𝑛 symmetric matrices.), 𝑥(𝑡)∈ℝ^𝑛 is the state with known initial state 𝑥₀, 𝑢(𝑡)∈ℝ^𝑚 the control, and 𝑤(𝑡)∈ℝ^𝑝 a standard Wiener process. Because 𝑥 is completely determined by the first equation in (1.1) in terms of 𝑢, the cost function 𝐽 is only a function of 𝑢.

For 𝜃∈ℝ, denote by 𝐸(𝑒^𝜃𝐽) the (general) expectation of the random variable 𝑒^𝜃𝐽. For 𝑖≥1, let 𝜅_𝑖=𝑑^𝑖𝑑𝜃^𝑖ln𝐸𝑒^𝜃𝐽|_𝜃=0 (1.2) be the 𝑖th cumulant of the cost 𝐽(𝑢). Let {𝜇_𝑖}^∞_𝑖=1 be a sequence of nonnegative real numbers. Consider the following combination of 𝜅_𝑖: 𝜅=^∞_𝑖=1𝜇_𝑖𝜅_𝑖𝑖!. (1.3)

The cumulant control problem, considered in [1], is to find a control 𝑢 that minimizes the combined cumulant 𝜅 defined in (1.3). This problem leads to the following system of (infinitely many) equations of Riccati type: 𝐻₁′+(𝐴+𝐵𝐾)^𝑇𝐻₁+𝐻₁(𝐴+𝐵𝐾)+𝑄+𝐾^𝑇𝑅𝐾=0,𝐻₁𝑡_𝑓=𝑄_𝑓,𝐻_𝑖′+(𝐴+𝐵𝐾)^𝑇𝐻_𝑖+𝐻_𝑖(𝐴+𝐵𝐾)+2^𝑖−1_𝑗=1𝐻_𝑗𝑊𝐻_𝑖−𝑗=0,𝐻_𝑖𝑡_𝑓=0,𝑖≥2, (1.4) where ^′=𝑑/𝑑𝑡 denotes the derivative to 𝑡, 𝐴(𝑡)∈ℝ^𝑛×𝑛,𝐵(𝑡)∈ℝ^𝑛×𝑚,𝑄(𝑡)∈𝕊^𝑛, 𝑅(𝑡)∈𝕊^𝑚 are as in (1.1), and 𝑊(𝑡)=𝐺(𝑡)𝐺^𝑇(𝑡)∈𝕊^𝑛. 𝐾(𝑡)∈ℝ^𝑚×𝑛 and 𝐻_𝑖(𝑡)∈𝕊^𝑛 are the unknown matrix functions, which are required to be continuously differentiable for 𝑡∈[𝑡₀,𝑡_𝑓]. For convenience, the time variable 𝑡 is often suppressed. System (1.4) will be combined with the following equation 𝐾=−𝑅⁻¹𝐵^𝑇∞_𝑖=1𝜇_𝑖𝐻_𝑖. (1.5)

If 𝑘≥1 is a given integer and 𝜇_𝑖=0 for 𝑖>𝑘, then 𝜅=∑^𝑘_𝑖=1𝜇_𝑖𝜅_𝑖/𝑖! is the 𝑘-cost cumulant investigated in [2, 3]. In particular, if 𝑘=1, then 𝜅=𝜇₁𝐸{𝐽(𝑢)} and the cumulant problem is the classical regulator problem that minimizes 𝐸(𝐽). If 𝑘=2, then the cumulant problem is the minimal cost variance control considered in [4, 5]. Interested readers are referred to [1–6] for the investigations, generalizations, and applications of cumulant controls.

Another important cumulant control occurs when 𝜇_𝑖=𝜃^𝑖−1 for 𝑖≥1. In this case, 𝜅=(1/𝜃)ln𝐸(𝑒^𝜃𝐽) is precisely the cumulant generating function, and the cumulant problem is the risk-sensitive control; see, for example, [7]. In this case, (1.4) and (1.5) lead an equation for the matrix function𝑃(𝜃;𝑡)=∑^∞_𝑖=1𝜃^𝑖−1𝐻_𝑖(𝑡): 𝑃^′+𝐴^𝑇𝑃+𝑃𝐴+𝑃2𝜃𝑊−𝐵𝑅⁻¹𝐵^′𝑃+𝑄=0;𝑃𝜃;𝑡_𝑓=𝑄_𝑓. (1.6)

As shown in [1], the solution 𝐻_𝑖(𝑡) of (1.4) is related to 𝑃(𝜃;𝑡) by the equation: 𝐻_𝑖(𝑡)=1𝜕𝑖!𝜕𝜃^𝑖𝑖[𝜃𝑃(𝜃;𝑡)]|_𝜃=0 (1.7) and the equations in (1.4) for 𝐻_𝑖(𝑡) can be obtained by differentiating (1.6) to 𝜃 at 𝜃=0.

For a feedback control 𝑢=𝐾𝑥 with a given matrix function 𝐾, it was shown in [8] and [1, Theorem  2] that the 𝑖th cumulant 𝜅_𝑖 of 𝐽(𝑢) has the following representation: 𝜅_𝑖=𝑖!𝑥^𝑇₀𝐻_𝑖𝑡₀𝑥₀+^𝑡𝑓_𝑡0tr𝐻_𝑖(𝑠)𝑊(𝑠)𝑑𝑠, (1.8) where {𝐻_𝑖}^∞_𝑖=1 is the solution of (1.4). Consequently (1.3) can be written as 𝜅=^∞_𝑖=1𝜇_𝑖𝑥^𝑇₀𝐻_𝑖𝑡₀𝑥₀+^𝑡𝑓_𝑡0tr𝐻_𝑖(𝑠)𝑊(𝑠)𝑑𝑠. (1.9)

In [1] the cumulant control problem was restated as minimizing 𝜅 in (1.9) with 𝐾 as a control, {𝐻_𝑖}^∞_𝑖=1 as a state, and (1.4) a state equation. Furthermore, the following result is proved in [1, Theorem  3].

Theorem 1.1. If the control 𝑢=𝐾𝑥 is the optimal feedback control of (1.9), then the solution {𝐻_𝑖}^∞_𝑖=1 of (1.4) and 𝐾 must satisfy (1.5).

By Theorem 1.1, it is necessary to solve (1.4) and (1.5) in order to find a solution of a cumulant control problem. Because of the nonlinearity of (1.4) and (1.5) in {𝐻_𝑖}^∞_𝑖=1, a global solution may not exist on the whole horizon [𝑡₀,𝑡_𝑓] of the cumulant problem. This can be illustrated by a scalar case of (1.6). Suppose 𝐴=0,𝐵=𝐺=𝜃=𝑅=𝑄=𝑄_𝑓=1, then (1.6) becomes 𝑃^′+𝑃²+1=0 and 𝑃(𝑡_𝑓)=1. The solution is 𝑃(𝑡)=tan (𝜋/4+𝑡_𝑓−𝑡), which is defined on (𝑠,𝑡_𝑓] with 𝑠=𝑡_𝑓−𝜋/4. So (1.6) has no solution unless 𝑡₀>𝑡_𝑓−𝜋/4.

By the local existence theory of differential equations, the solutions 𝐾 and {𝐻_𝑖}^∞_𝑖=1of (1.4) and (1.5) exist on a maximal subinterval (𝑠,𝑡_𝑓]⊂[𝑡₀,𝑡_𝑓]. Our interest is to give an estimate for this interval. In particular, we will obtain conditions that guarantee 𝑠=𝑡₀. The idea of our approach is to show that the trace tr(𝐻) of 𝐻=^∞_𝑖=1𝜇_𝑖𝐻_𝑖 (1.10) satisfies a scalar differential inequality:𝑧^′+𝑎𝑧²+𝑏𝑧+𝑐≥0;𝑧𝑡_𝑓=tr𝑄_𝑓 (1.11) with some functions 𝑎,𝑏,𝑐 on [𝑡₀,𝑡_𝑓]. A key of the proof is Proposition 2.1 below. It follows that tr(𝐻) is bounded by the solution of the Riccati equation: 𝑧′+𝑎𝑧²+𝑏𝑧+𝑐=0;𝑧𝑡_𝑓=𝑧_𝑓, (1.12) where 𝑧_𝑓=tr(𝑄_𝑓). Consequently, the existence interval of (1.12) gives an estimate for that of system (1.4) and (1.5); see Theorems 2.4 and 3.5 below. By a similar argument, we prove that the cumulant problem is well posed under appropriate conditions; see Theorems 2.3 and 3.4 below.

In [9] the norm of a solution of a coupled matrix Riccati equation was shown to satisfy a differential inequality similar to (1.11). Consequently, specific sufficient conditions were derived for the existence of solutions of the Riccati equation in [9]. Estimates for maximal existence interval of a classical Riccati equation had been obtained in [10] in terms of upper and lower solutions. For the coupled Riccati equation associated with the minimal cost variance control, some implicit sufficient conditions had been given in [11] for the existence of a solution. In this paper, we use the trace tr(𝐻) to bound the solution of system (1.4) and (1.5), which generally leads to a better estimate for the existence interval.

2. Comparison Results for Traces

We start with an assumption and some preparations. In this paper we assume that 𝑄_𝑓,𝑄(𝑡)≥0positivesemidefinitefor𝑡∈𝑡₀,𝑡_𝑓;𝑅(𝑡)>0positivedefinitefor𝑡∈𝑡₀,𝑡_𝑓. (2.1)

For the sequence 𝜇={𝜇_𝑖}^∞_𝑖=1 in (1.3), we will assume that 𝜇₁=1;𝜇_𝑖≥0for𝑖≥1;𝜌𝜇<∞, (2.2) where 𝜌𝜇𝜇=sup^𝑖𝜇_𝑗𝜇_𝑖−𝑗,𝑖≥2,𝑗=1,…,𝑖−1. (2.3)

Note that the assumption 𝜇₁=1 is not essential. The assumption that 𝜇_𝑖≥0 for 𝑖≥1 and Proposition 2.1 below imply that the matrix 𝐻 in (1.4) is a positive semidefinite series. The requirement that 𝜌(𝜇)<∞ imposes some growth condition for the sequence 𝜇; see the proofs of Theorems 2.3 and 2.4.

Also note that if 𝜃=𝜌(𝜇)<∞, then 𝜇_𝑖≤𝜃𝜇_𝑖−1 and 𝜇_𝑖≤𝜃^𝑖−1 for all 𝑖≥1. Theorem 3.4 below shows that the cumulant control problem is well posed for any sequence 𝜇 with a small 𝜌(𝜇). Some examples of 𝜌(𝜇) are as follows:(i)𝜌({1,𝜃,𝜃²,𝜃³,0,…})=𝜃; (ii)𝜌({1,𝜇₂,0,0,…})=𝜇₂;(iii)𝜌({1,𝜇₂,𝜇₃,0,…})=max{𝜇₂,𝜇₃/𝜇₂}.

We need the following properties of {𝐻_𝑖}^∞_𝑖=1.

Proposition 2.1. Suppose {𝐻_𝑖}^∞_𝑖=1 is a solution of (1.4) on some interval [𝑠,𝑡_𝑓] with a given 𝐾, then each 𝐻_𝑖(𝑡)≥0 for 𝑡∈[𝑠,𝑡_𝑓].

Proof. The formula of the 𝑖th cumulant 𝜅_𝑖 in [8] implies that 𝜅_𝑖 is nonnegative for all 𝑥₀. It follows from the representation (1.9) that 𝐻_𝑖(𝑡₀) must be positive semidefinite. This argument continues to hold with 𝑡₀ replaced by any 𝑠∈[𝑡₀,𝑡_𝑓].

Next we verify some properties related to matrix trace that are needed for our analysis. For 𝐴∈ℝ^𝑛×𝑛, denote by tr(𝐴)=∑^𝑛_𝑖=1𝑎_𝑖𝑖 the trace of 𝐴, and 𝜆₁(𝐴) and 𝜆_𝑛(𝐴) the smallest and largest eigenvalue of 𝐴, respectively.

Proposition 2.2. (a) For all 𝐴,𝐵∈ℝ^𝑛×𝑛, tr(𝐴)=tr(𝐴^𝑇), tr(𝐴𝐵)=tr(𝐵𝐴).
(b) For all 𝐴∈ℝ^𝑛×𝑛, 𝐵∈𝕊^𝑛,𝐵≥0, 𝜆₁(𝐴)tr(𝐵)≤tr(𝐴𝐵)≤𝜆_𝑛(𝐴)tr(𝐵). (2.4)
(c) If 𝐴,𝐵,𝐶∈𝕊^𝑛, then tr(𝐴𝐵𝐶)=tr(𝐴𝐶𝐵). (2.5)
(d) If 𝐴,𝐵,𝐶∈𝕊^𝑛 are all ≥0, then tr(𝐴𝐵)≤tr(𝐴)tr(𝐵), (2.6)tr(𝐴𝐵𝐶)≤tr(𝐴)tr(𝐵)tr(𝐶), (2.7)1𝑛𝜆₁(𝐴)tr(𝐵)²≤tr𝐴𝐵². (2.8)

Proof. The properties in (a) are obvious by the definitions of trace and matrix multiplication. Some of the inequalities in (b)–(d) might be known, but the authors were not able to find proofs in existing literature. So we include our proofs of (b)–(d) below for readers’ convenience.
To prove (b), let 𝑇 be a unitary matrix such that 𝐵=𝑇^∗𝐷𝑇 where 𝐷 is diagonal with eigenvalues 𝜆_𝑖(𝐵) of 𝐵,𝑖=1,…,𝑛. Then tr(𝐴𝐵)=tr𝐴𝑇^∗𝐷𝑇=^𝑛_𝑖=1𝑇𝐴𝑇^∗_𝑖𝑖𝜆_𝑖(𝐵), (2.9) where (𝑇𝐴𝑇^∗)_𝑖𝑖 is the entry of 𝑇𝐴𝑇^∗ at (𝑖,𝑖). Let 𝑣_𝑖 be the 𝑖th row of 𝑇, which is a unit vector. Then (𝑇𝐴𝑇^∗)_𝑖𝑖=𝑣_𝑖𝐴𝑣^∗_𝑖∈[𝜆₁(𝐴),𝜆_𝑛(𝐴)]. Since 𝜆_𝑖(𝐵)≥0,we have 𝜆₁(𝐴)𝜆_𝑖(𝐵)≤𝑇𝐴𝑇^∗_𝑖𝑖𝜆_𝑖(𝐵)≤𝜆_𝑛(𝐴)𝜆_𝑖(𝐵). (2.10) This implies (2.4).
To show (c), use the symmetry of 𝐴,𝐵, and 𝐶; we get that tr(𝐴𝐵𝐶)=∑_𝑖𝑗𝑘𝑎_𝑖𝑗𝑏_𝑗𝑘𝑐_𝑘𝑖=∑_𝑖𝑗𝑘𝑎_𝑗𝑖𝑐_𝑖𝑘𝑏_𝑘𝑗=tr(𝐴𝐶𝐵).
Inequality (2.6) follows from part (b) and the fact that 𝜆_𝑛(𝐴)≤tr(𝐴) since 𝐴≥0.
To show (2.7), first note that tr(𝐴𝐵𝐶)≤𝜆_𝑛(𝐴𝐵)tr(𝐶) by (2.4); then it remains to show that 𝜆_𝑛(𝐴𝐵)≤𝜆_𝑛(𝐴)𝜆_𝑛(𝐵). Choose a 𝑣∈ℝ^𝑛 with𝑣^𝑇𝑣=1 such that 𝜆_𝑛(𝐴𝐵)=𝑣^𝑇𝐴𝐵𝑣. By Schwartz inequality, 𝑣^𝑇𝐴𝐵𝑣≤𝑣^𝑇𝐴²𝑣𝑣^𝑇𝐵²𝑣≤𝜆_𝑛𝐴²𝜆_𝑛𝐵². (2.11) Since 𝐴,𝐵≥0, we have 𝜆_𝑛(𝐴²)=𝜆_𝑛(𝐴)² and 𝜆_𝑛(𝐵²)=𝜆_𝑛(𝐵)². Therefore, 𝜆_𝑛(𝐴𝐵)≤𝜆_𝑛(𝐴)𝜆_𝑛(𝐵).
For (2.8), using notations in the proof of (b), we first get tr𝐴𝐵²=tr𝐴𝑇^∗𝐷²𝑇=^𝑛_𝑖=1𝑇𝐴𝑇^∗_𝑖𝑖𝜆²_𝑖(𝐵)≥𝜆₁(𝐴)^𝑛_𝑖=1𝜆²_𝑖(𝐵). (2.12) Then (2.8) follows from the inequalities ^𝑛_𝑖=1𝜆²_𝑖(𝐵)≥1𝑛^𝑛_𝑖=1𝜆_𝑖(𝐵)²≥1𝑛tr(𝐵)². (2.13)

Now we estimate the existence intervals of solutions of (1.4) and (1.5). First, let 𝐾 be given and {𝐻_𝑖}^∞_𝑖=1 be the solution of system (1.4). We have the following result, which will be used in the proof of Theorem 3.5

Theorem 2.3. Suppose 𝜌(𝜇)<∞ and K in (1.4) is given. Let 𝑎₁, 𝑏₁, and 𝑐₁ be functions on [𝑡₀,𝑡_𝑓] satisfying 𝑎₁≥2𝜌𝜇tr(𝑊),𝑏₁≥2𝜆_𝑛(𝐴+𝐵𝐾),𝑐₁≥tr𝑄+𝐾^𝑇𝑅𝐾. (2.14)
(a) If {𝐻_𝑖}^∞_𝑖=1 is a solution of system (1.4), then 𝑧=tr(𝐻) satisfies the differential inequality −𝑧^′≤𝑎₁𝑧²+𝑏₁𝑧+𝑐₁,𝑧𝑡_𝑓=tr𝑄_𝑓. (2.15)
(b) If the equation −𝑧′=𝑎₁𝑧²+𝑏₁𝑧+𝑐₁,𝑧𝑡_𝑓=tr𝑄_𝑓 (2.16) has a solution on [𝑡₀,𝑡_𝑓], then system (1.4) has a solution {𝐻_𝑖}^∞_𝑖=1 on [𝑡₀,𝑡_𝑓] such that 𝐻=∑^∞_𝑖=1𝜇_𝑖𝐻_𝑖 is convergent.

Proof. (a) Denote 𝐹=𝐴+𝐵𝐾. Multiplying the equation in (1.4) for 𝐻_𝑖 by 𝜇_𝑖 and sum over 𝑖, we obtain 𝐻′+𝐹^𝑇𝐻+𝐻𝐹+𝑄+𝐾^𝑇𝑅𝐾+2^∞_𝑖=2𝜇_𝑖^𝑖−1_𝑗=1𝐻_𝑗𝑊𝐻_𝑖−𝑗=0. (2.17)
Taking traces of both sides of (2.17) gives −𝑧′=tr𝐹^𝑇𝐻+𝐻𝐹+tr𝑄+𝐾^𝑇𝑅𝐾+2^∞_𝑖=2𝜇_𝑖^𝑖−1_𝑗=1tr𝐻_𝑗𝑊𝐻_𝑖−𝑗. (2.18) Note that 𝑊=𝐺𝐺^𝑇≥0 and by Proposition 2.1, 𝐻_𝑖, 𝐻≥0 for 𝑡∈[𝑡_𝑓,𝑡_𝑓]. Proposition 2.2 (a), (b), and (c) imply that tr𝐹^𝑇𝐻+𝐻𝐹=2tr(𝐹𝐻)≤2𝜆_𝑛(𝐹)tr(𝐻)=𝑏₁𝐻𝑧,tr_𝑗𝑊𝐻_𝑖−𝑗≤tr(𝑊)tr𝐻_𝑗tr𝐻_𝑖−𝑗. (2.19)
By (1.4) and definition (2.3) of 𝜌(𝜇), we have 2^∞_𝑖=2𝜇_𝑖^𝑖−1_𝑗=1tr𝐻_𝑗𝑊𝐻_𝑖−𝑗≤2^∞_𝑖=2𝜇_𝑖^𝑖−1_𝑗=1tr𝐻_𝑗tr(𝑊)tr𝐻_𝑖−𝑗≤2𝜌𝜇tr(𝑊)^∞_𝑖=2^𝑖−1_𝑗=1𝜇_𝑗𝜇_𝑖−𝑗tr𝐻_𝑗tr𝐻_𝑖−𝑗=2𝜌𝜇tr(𝑊)^∞_𝑖=1𝜇_𝑖tr𝐻_𝑖²=2𝜌𝜇tr(𝑊)tr(𝐻)²≤𝑎₁𝑧². (2.20)
Substituting (2.19) and (2.20) into (2.18) and using the definition of 𝑐₁ in (2.14) we get −𝑧′≤𝑏₁𝑧+𝑐₁+𝑎₁𝑧². (2.21)(b) Suppose that (2.16) has a solution 𝑧^∗(𝑡) on [𝑡₀,𝑡_𝑓]. By local existence theory, system (1.4) has a solution {𝐻_𝑖}^∞_𝑖=1 on a maximal interval (𝑠,𝑡_𝑓]⊂[𝑡₀,𝑡_𝑓].By (a), tr(𝐻) satisfies inequality (2.15); that is tr(𝐻) is a lower solution of (2.16). By a comparison theorem of lower-upper solutions, tr(𝐻)≤𝑧^∗(𝑡) on (𝑠,𝑡_𝑓]. Since series (1.10) is positive semidefinite, it follows that 𝐻 and 𝐻_𝑖 are all bounded and (1.10) is convergent on (𝑠,𝑡_𝑓]. Since {𝐻_𝑖}^∞_𝑖=1 satisfies system (1.4), each 𝐻_𝑖 is in fact continuously differentiable on [𝑠,𝑡_𝑓]. If 𝑠>𝑡₀, then the local existence theory implies that {𝐻_𝑖}^∞_𝑖=1 can be extended further to the left of 𝑠, a contradiction to the maximality of (𝑠,𝑡_𝑓]. Therefore 𝑠=𝑡₀ and (1.4) has a solution on (𝑡₀,𝑡_𝑓], which can be extended to [𝑡₀,𝑡_𝑓].

Now consider system (1.4) and (1.5). We have

Theorem 2.4. Denote 𝑅=𝐵𝑅⁻¹𝐵^𝑇. Let 𝑎₂,𝑏₂, and 𝑐₂ be functions on [𝑡₀,𝑡_𝑓] satisfying 𝑎₂1≥−𝑛𝜆₁𝑅+2𝜌tr(𝑊),𝑏₂≥2𝜆_𝑛(𝐴),𝑐₂≥tr(𝑄). (2.22)(a) Suppose 𝐾 and {𝐻_𝑖}^∞_𝑖=1 are solutions of (1.4) and (1.5) on some (𝑠,𝑡_𝑓]. Then on (𝑠,𝑡_𝑓],𝑧=tr(𝐻) satisfies −𝑧′≤𝑎₂𝑧²+𝑏₂𝑧+𝑐₂,𝑧𝑡_𝑓=tr𝑄_𝑓. (2.23)(b) Suppose the equation −𝑧′=𝑎₂𝑧²+𝑏₂𝑧+𝑐₂,𝑧𝑡_𝑓=tr𝑄_𝑓 (2.24) has a solution on [𝑡₀,𝑡_𝑓], then system (1.4) and (1.5) have solutions 𝐾 and {𝐻_𝑖}^∞_𝑖=1 on [𝑡₀,𝑡_𝑓].

Proof. (a) Substituting 𝐾=−𝑅⁻¹𝐵^𝑇𝐻 into system (2.17) we get −𝐻′=𝐴^𝑇𝐻+𝐻𝐴−𝐻𝑅𝐻+𝑄+2^∞_𝑖=2𝜇_𝑖^𝑖−1_𝑗=1𝐻_𝑗𝑊𝐻_𝑖−𝑗, (2.25) where 𝑅=𝐵𝑅⁻¹𝐵^𝑇. Taking traces of both sides of (2.25) gives −tr𝐻′=tr𝐴^𝑇𝐻+𝐻𝐴−tr𝐻𝑅𝐻+tr(𝑄)+2^∞_𝑖=2𝜇_𝑖^𝑖−1_𝑗=1tr𝐻_𝑗𝑊𝐻_𝑖−𝑗. (2.26)
As in the proof of previous theorem, we have tr𝐴^𝑇𝐻+𝐻𝐴=2tr(𝐴𝐻)≤2𝜆_𝑛(𝐴)tr(𝐻)≤𝑏₂2𝑧,_∞𝑖=2𝜇_𝑖^𝑖−1_𝑗=1tr𝐻_𝑗𝑊𝐻_𝑖−𝑗≤2𝜌𝜇tr(𝑊)𝑧². (2.27) Using the fact that 𝐻,𝑅≥0and (2.8), we have tr𝐻𝑅𝐻=tr𝑅𝐻²≥𝜆₁𝑅tr(𝐻)²𝑛. (2.28)
By combining (2.28), (2.27), and (2.26) we obtain (2.23).
(b) Suppose that (2.24) has solution 𝑧^∗(𝑡) on [𝑡₀,𝑡_𝑓]. By local existence theory, system (1.5) and (1.4) have solutions 𝐾 and {𝐻_𝑖}^∞_𝑖=1 on a maximal interval (𝑠,𝑡_𝑓]⊂[𝑡₀,𝑡_𝑓]. By part (a), tr(𝐻) satisfies inequality (2.23) on (𝑠,𝑡_𝑓]. It follows that tr(𝐻)≤𝑧^∗ on (𝑠,𝑡_𝑓], which implies that 𝐾 and {𝐻_𝑖}^∞_𝑖=1 are continuously differentiable on [𝑠,𝑡_𝑓].If 𝑠>𝑡₀, then local existence theory implies that 𝐾 and {𝐻_𝑖}^∞_𝑖=1 can be extended further to the left of 𝑠, a contradiction to the maximality of (𝑠,𝑡_𝑓]. Therefore 𝑠=𝑡₀, and system (1.4) and (1.5) have solutions on [𝑡₀,𝑡_𝑓].

3. Well-Posedness and Sufficient Existence Conditions

In this section we will derive specific conditions that ensure that the scalar equations (2.16) and (2.24) have solutions on [𝑡₀,𝑡_𝑓]. Consequently we will obtain sufficient conditions for the well-posedness of the cumulant control and the existence of solutions of (1.4) and (1.5).

First we consider an autonomous scalar equation 𝑧^′+ℎ(𝑧)=0,𝑡≤𝑡_𝑓;𝑧𝑡_𝑓=𝑧_𝑓, (3.1) where 𝑧 is a polynomial with degree ≥2. Assume for some 𝑘≥0 that ℎ(𝑧) has 𝑘 distinct zeros 𝑧₁<⋯<𝑧_𝑘. Let 𝑧₀=−∞ and 𝑧_𝑘+1=∞. Since ℎ(𝑧) is locally Lipschitz, the solution 𝑧(𝑡) of (3.1) exists and is unique for every 𝑧_𝑓 for 𝑡 in a maximal interval, say (𝜎,𝑡_𝑓]. If 𝑧_𝑓=𝑧_𝑖 for some 𝑖=1,…,𝑘, then 𝑧(𝑡)=𝑧_𝑖 for all 𝑡. If 𝑧_𝑓∈(𝑧_𝑖,𝑧_𝑖+1) for some 𝑖=0,2,…,𝑘, then 𝑧(𝑡)∈(𝑧_𝑖,𝑧_𝑖+1) for 𝑡∈(𝜎,𝑡_𝑓]. This implies that for 𝑡∈(𝜎,𝑡_𝑓], −𝑧′(𝑡)=ℎ(𝑧(𝑡)) has the same sign as ℎ(𝑧_𝑓).In particular, as 𝑡∈(𝜎,𝑡_𝑓] decreases, 𝑧(𝑡) is strictly increasing if ℎ(𝑧_𝑓)>0 and decreasing if ℎ(𝑧_𝑓)<0. Denote 𝑧_𝑓=lim_𝑡→𝜎+𝑧(𝑡). Then 𝑧_𝑓=𝑧_𝑖+1ifℎ𝑧_𝑓𝑧>0,_𝑖ifℎ𝑧_𝑓<0. (3.2)

The following is a well-known fact in stability theory of differential equations. Ifℎ^′𝑧_𝑖<0,then𝑧_𝑓=𝑧_𝑖forevery𝑧_𝑓∈𝑧_𝑖−1,𝑧_𝑖+1, (3.3) where ^′=𝑑/𝑑𝑧. Indeed, ℎ^′(𝑧_𝑖)<0 implies that ℎ(𝑧_𝑓)<0 for 𝑧_𝑓∈(𝑧_𝑖,𝑧_𝑖+1) and ℎ(𝑧_𝑓)>0 for 𝑧_𝑓∈(𝑧_𝑖−1,𝑧_𝑖). In either cases, 𝑧_𝑓=𝑧_𝑖 by (3.2).

Consider (1.12) as an example. We have the following.

Proposition 3.1. Denote Δ=𝑏²−4𝑎𝑐 and 𝑧_1,2=(−𝑏±√Δ)/2𝑎 if Δ≥0. Then
(a) 𝑧_𝑓=−𝑐/𝑏 for all 𝑧_𝑓 if 𝑎=0 and 𝑏<0;
(b) 𝑧_𝑓=𝑧₂if Δ≥0 and 𝑎(𝑧_𝑓−𝑧₁)<0;
(c) 𝑧_𝑓=sgn(𝑎)∞ if Δ<0 or Δ≥0 and 𝑎(𝑧_𝑓−𝑧₁)>0.

Proof. If 𝑎=0, then ℎ(𝑧)=𝑏𝑧+𝑐, which has root 𝑧₁=−𝑐/𝑏. Since ℎ′(𝑧₁)=𝑏<0, 𝑧_𝑓=𝑧₁ by (3.3). This shows (a).
Next we prove (b) and (c). First assume 𝑎=1. If Δ<0, then ℎ(𝑧)>0 for all 𝑧. So 𝑧_𝑓=∞ by (3.2). Next consider the case Δ≥0, in which −∞<𝑧₂≤𝑧₁<∞. If 𝑧_𝑓>𝑧₁, then ℎ(𝑧_𝑓)>0, which implies that 𝑧_𝑓=∞. If 𝑧_𝑓<𝑧₁, then we have either ℎ(𝑧_𝑓)>0 when 𝑧_𝑓<𝑧₂ or ℎ(𝑧_𝑓)<0 when 𝑧₂<𝑧_𝑓<𝑧₁. In either cases, we have 𝑧_𝑓=∞ by (3.2). This finishes the proof of (b) and (c) when 𝑎=1. If 𝑎≠1, then consider 𝑦=𝑎𝑧, which satisfies 𝑦^′+𝑦²+𝑏𝑦+𝑎𝑐=0 and 𝑦(𝑡_𝑓)=𝑎𝑧_𝑓, and the conclusions follow from the special case just proved.

Write (3.1) as 𝑑𝑧/ℎ(𝑧)+𝑑𝑡=0 and integrate it against 𝑡from 𝑡_𝑓 to 𝜎, then we get 𝑡_𝑓𝑧−𝜎=^𝑓_𝑧𝑓𝑑𝑧ℎ(𝑧). (3.4)

Note that if 𝑧_𝑓 is finite, then 𝑧_𝑓 must be a zero of ℎ(𝑧). It follows that ∫_𝑓𝑧_𝑧𝑓[1/ℎ(𝑧)]𝑑𝑧=∞ and (𝜎,𝑡_𝑓]=(−∞,𝑡_𝑓]. If 𝑧_𝑓 is infinite, then ∫_𝑓𝑧_𝑧𝑓(𝑑𝑧/(ℎ(𝑧))) must converge because ℎ(𝑧) has a degree ≥2. In summary, we have the following.

Proposition 3.2. Suppose ℎ(𝑧) is a polynomial of degree ≥2. Then
(a)𝑧_𝑓 is finite if and only if the solution 𝑧(𝑡) of (3.1) exists on (−∞,𝑡_𝑓].
(b) 𝑧_𝑓=±∞ if and only the solution 𝑧(𝑡) of (3.1) exists on a finite maximal interval (𝜎,𝑡_𝑓] with length 𝑡_𝑓−𝜎=∫_𝑧^±∞_𝑓[1/ℎ(𝑧)]𝑑𝑧.

Applying Proposition 3.2 to (1.12) we obtain the following.

Proposition 3.3. (a) If either 𝑎=0 and 𝑏<0, or 𝑎(𝑧_𝑓−𝑧₁)<0 and Δ≥0, then the solution of (1.12) exists on (−∞,𝑡_𝑓].
(b) If either Δ<0 or Δ≥0 and 𝑎(𝑧_𝑓−𝑧₁)<0, then the solution of (1.12) exists on a finite interval (𝜎,𝑡_𝑓] with 𝑡_𝑓2−𝜎=|Δ|𝜋2−arctan2𝑎𝑧_𝑓+𝑏|Δ|2ifΔ<0,2𝑎𝑧_𝑓𝑧+𝑏ifΔ=0,𝑎_𝑓−𝑧₁1>0,√Δ𝑧ln^𝑓−𝑧₂𝑧_𝑓−𝑧₁ifΔ>0,𝑎𝑧_𝑓−𝑧₁>0. (3.5)

Proof. Part (a) directly follows from Proposition 3.1(a) and Proposition 3.2 (a) (b).
For part (b), first assume that Δ<0. Then 𝑧_𝑓=sgn(𝑎)∞ and 𝑎𝑧²+𝑏𝑧+𝑐=𝑎[(𝑧−𝑧₁)²+𝑑²], where 𝑑=|Δ|/2|𝑎|,𝑧₁=−b/2𝑎. So 𝑡_𝑓𝑧−𝜎=^𝑓_𝑧𝑓𝑑𝑧𝑎𝑧²=1+𝑏𝑧+𝑐𝑎𝑑arctan𝑧−𝑧₁𝑑_𝑧^±∞_𝑓=1±𝜋𝑎𝑑2𝑧−arctan^𝑓−𝑧₁𝑑=2|Δ|𝜋2−arctan2𝑎𝑧_𝑓+𝑏|Δ|. (3.6)
Next assume Δ=0 and 𝑎(𝑧_𝑓−𝑧₁)>0. Then 𝑧_𝑓=sgn(𝑎)∞ and 𝑎𝑧²+𝑏𝑧+𝑐=𝑎(𝑧−𝑧₁)², where 𝑧₁=−𝑏/(2𝑎). We have 𝑡_𝑓−𝜎=_𝑧^±∞_𝑓1𝑎𝑧−𝑧₁²1𝑑𝑧=−𝑎𝑧−𝑧_1𝑧^±∞_𝑓=1𝑎𝑧_𝑓−𝑧₁=22𝑎𝑧_𝑓+𝑏. (3.7)
Finally when Δ>0 and 𝑧_𝑓=sgn(𝑎)∞, we have 𝑎𝑧²+𝑏𝑧+𝑐=𝑎(𝑧−𝑧₁)(𝑧−𝑧₂) and 𝑡_𝑓−𝜎=_𝑧^±∞_𝑓1𝑎𝑧−𝑧₁𝑧−𝑧₂1𝑑𝑧=𝑎𝑧₁−𝑧₂ln𝑧−𝑧₁𝑧−𝑧_2𝑧^±∞_𝑓=1√Δ𝑧ln^𝑓−𝑧₂𝑧_𝑓−𝑧₁. (3.8)

Now we show that the cumulant control problem is well posed by Theorem 2.3 and Proposition 3.3.

Theorem 3.4. For any number 𝐿>0 there is 𝜌₀>0 such that the series ∑^∞_𝑖=1𝜇_𝑖𝜅_𝑖/𝑖! in (1.3) converges for each matrix 𝐾 and sequence 𝜇 with ||𝐾||_∞<𝐿 and 𝜌(𝜇)<𝜌₀.

Proof. Suppose that 𝐾 is a matrix function with ||𝐾||_∞<𝐿. Choose 𝑎₁, 𝑏₁, and 𝑐₁ as follows: 𝑎₁=2𝜌𝜇‖tr(𝑊)‖_∞,𝑏₁=2𝜆_𝑛(𝐴+𝐵𝐾)_∞+1,𝑐₁=tr𝑄+𝐾^𝑇𝑅𝐾_∞. (3.9) Note that 𝑐₁≤‖tr(𝑄)‖_∞+𝐿²‖𝑅‖_∞, which depends only on 𝐿. In addition, 𝑎₁→0 as 𝜌(𝜇)→0. It follows that when 𝜌(𝜇) is sufficiently small, ℎ(𝑧)=𝑎₁𝑧²+𝑏₁𝑧+𝑐₁ has two real roots 𝑧_1,2=−𝑏₁±𝑏²₁−4𝑎₁𝑐₁2𝑎₁<0 (3.10) with 𝑧₂→−𝑐₁/𝑏₁ and 𝑧₁→−∞ as 𝜌(𝜇)→0. In particular, since 𝑧_𝑓≥0, we have ℎ(𝑧_𝑓)>0 and so 𝑧_𝑓=∞. Proposition 3.3 implies that 𝑡_𝑓1−𝜎=𝑏²₁−4𝑎₁𝑐₁𝑧ln^𝑓−𝑧₂𝑧_𝑓−𝑧₁. (3.11)
So 𝑡_𝑓−𝜎→∞ as 𝜌(𝜇)→0. In particular, (2.16) has a solution 𝑧(𝑡) on [𝑡₀,𝑡_𝑓] when 𝜌(𝜇) is sufficiently small. By Theorem 2.3 (b), system (1.4) has a solution {𝐻_𝑖}^∞_𝑖=1 such that 𝐻 converges.
Finally we apply Proposition 3.3 to (2.24) to give a sufficient existence condition for (1.4) and (1.5) and the cumulant control problem. Choose 𝑎₂=max_{𝑡0≤𝑡≤𝑡𝑓}−1𝑛𝜆₁𝑅+2𝜌tr(𝑊),𝑏₂=2max_{𝑡0≤𝑡≤𝑡𝑓}𝜆_𝑛(𝐴),𝑐₂=‖tr(𝑄)‖_∞. (3.12)

Theorem 3.5. System (1.4) and (1.5) have solutions 𝐾 and {𝐻_𝑖}^∞_𝑖=1 on [𝑡₀,𝑡_𝑓] if 𝑡_𝑓−𝑡₀≤_𝑓𝑧_𝑧𝑓1𝑎₂𝑧²+𝑏₂𝑧+𝑐₂𝑑𝑧, (3.13) where 𝑧_𝑓 is defined as in (3.2) with ℎ(𝑧)=𝑎₂𝑧²+𝑏₂𝑧+𝑐₂. In particular, system (1.4) and (1.5) have solutions 𝐾 and {𝐻_𝑖}^∞_𝑖=1 on [𝑡₀,𝑡_𝑓] if one of the following holds.
(a) 𝑎₂=−𝜆₁(𝑅)/𝑛+2𝜌tr(𝑊)<0.
(b) 𝑎₂>0, 𝑏₂<0, Δ=𝑏²₂−4𝑎₂𝑐₂≥0 and 𝑧_𝑓<𝑧₁, where 𝑧₁=(−𝑏₂+𝑏²₂−4𝑎₂𝑐₂)/2𝑎₂.

Proof. The general conclusion follows directly from Proposition 3.3 and Theorem 2.4. In the case (a), ℎ(𝑧)=0 has two roots 𝑧₁<0<𝑧₂. Since 𝑧_𝑓=tr(𝑄_𝑓)≥0, 𝑎₂(𝑧_𝑓−𝑧₁)<0. In the case (b), ℎ(𝑧)=0 has two solutions 𝑧₂≤𝑧₁. So 𝑎₂(𝑧_𝑓−𝑧₁)<0 also holds. The conclusion follows from Propositions 3.1 and 3.2 and Theorem 2.4.

Note that in Theorem 3.5 condition (a) holds if 𝐵 has full rank (i.e., 𝜆(𝑅)>0) and 𝜌(𝜇) is sufficiently small, while condition (b) holds if the system in (1.1) is stable (i.e., 𝑏₂<0) and the product 𝜌(𝜇)‖tr(𝑄)‖_∞ is relatively small. The cumulant control problem has an optimal control under each of these conditions.

As an existence theorem, Theorem 3.5 gives one of the very few existence results for a Riccati differential system of infinitely many equations. In terms of the cumulant controls that lead to the system (3) and (4), Theorem 3.5 generalizes the corresponding results in [1, 5] for risk-sensitive controls (where 𝜇={1,𝜃,𝜃²,…}) and in [2, 4] for finite cumulant controls (where 𝜇 has only finite nonzero components). Numerical examples for risk-sensitive and finite cumulant controls satisfying the conditions in Theorem 3.5 may be found in [3–6].

4. Conclusions

In general it is very difficult to determine the existence interval of a differential Riccati equation (or system). By the approach in this paper, we can at least give an estimate for the existence interval of the Riccati system. Such an estimate leads to sufficient conditions for the existence of solutions to the Riccati system and the cumulant control problem.

Acknowledgments

The authors wish to express their sincere thanks to the reviewers for their valuable comments that helped improve this paper. The second author wishes to acknowledge the support of a Caterpillar Fellowship from Bradley University.