Optimal Control of Diffusions with Hard Terminal State Restrictions
A maximum principle is proved for certain problems of optimal control of diffusions where hard end constraints occur. The results apply to several dimensional problems, where some of the state equations involve Brownian motions, but not the equations corresponding to states being hard restricted at the terminal time.
Various types of maximum principles have been proved for problems of control of diffusions in case of no or soft terminal state restrictions; see for example, Kushner , Haussmann , Peng , and Yong and Zhou . Maximum principles for problem with hard terminal restrictions are proved for certain types of continuous time piecewise deterministic problems in Seierstad [5, 6]. Singular controls are sometimes introduced in various problems with certain types of hard restrictions, but below we merely consider problems where the controls appearing may be said to be absolutely continuous with respect to Lebesgue measure. The restriction to such controls makes it harder to operate with hard terminal state restrictions; in fact we can only work with such restrictions on states governed by differential equations not containing a Brownian motion. Brownian motion will only appear in differential equations of states unconstrained at the terminal time. So the problem we consider is a problem of control of diffusions where hard terminal restrictions are placed on states not governed by differential equations containing a Brownian motion; these states, however, can be influenced by other states directly influenced by Brownian motions. Below, a maximum principle is stated and proved for such problems. To the authors knowledge, maximum principles have not been stated for such problems before. Because the states are stochastic, the state space is infinite dimensional; so to obtain a maximum principle, one must impose a condition amounting to demand sufficient variability of the first order variations in the problem.
2. The Control Problem and the Statement of the Necessary Condition
Let and let be a given point in the Euclidean space , let be a projection from into , , such that () and let be a Borel subset of a Euclidean space. Furnish the interval with the Lebesgue measure. Let be a filtered probability space, (i.e., for , the ’s are sub--algebras of the given -algebra of subsets of , if , and is a probability measure on ), and assume that is complete, that contains all the -null sets in and that is right continuous. Let be the set of Lebesgue -measurable functions for which . Related to , let be a vector the components of which (denoted ) are independent one-dimensional Brownian motions all adapted to , such that is independent of for all , , , and is normally distributed with mean and covariances , with being the identity matrix. In applications where the ’s are the entities that can be observed, it is natural to take as the natural filtration generated by the ’s. There are given functions and , , from into ( independent of ). The following conditions are called the basic assumptions.
() The functions and have continuous derivatives and with respect to .
() The functions and have one-sided limits with respect to , and and are, separately, continuous in and in .
Write for the -matrix whose columns are ; let be the matrix with entries , and write . Also, write for the indicator function of the set . Let be a set of functions taking values in , such that , for each , when restricted to , is Lebesgue -measurable (i.e., progressively measurable), and such that when , , arbitrary, and , , is a measurable partition of , then belongs to (so-called switching closedness). We will also assume that is -closed, which means that if and is progressively measurable and takes values in and meas is not equal to a.s.} converges to zero when , then belongs to . Let be the -norm on .
The following assumptions are called the global assumptions.
(B1) is uniformly continuous in , uniformly in , .
(B2) For some constant , for all ,
(B3) A constant exists such that for all , (The symbol is used for the Euclidean norm in any Euclidean space , including , and, applied to matrices, it is the linear operator norm.)
(B4) One has
Let . The strong (unique) solution, continuous in , of the equation is denoted and is called a system solution.
Let ( fixed, ) such that ; let denote scalar product, and consider the problem subject to
Let be an optimal control in the problem and write . Let be the resolvent of the equation (so , with being the identity map).
In the subsequent necessary conditions, the following local linear controllability condition (2.10) is needed. Let be the set of progressively measurable functions in , and for , let and let co denote convex hull. There exists a number and a progressively measurable function , with , and a number such that
Theorem 2.1. Assume that is optimal in problems (2.5) and (2.6), that assumptions A and B hold (the basic and global assumptions), and that (2.10) is satisfied. Then there exists a number and a linear functional on , bounded on , such that, for all ,
Remark 2.2. If (2.10) holds for , then and is a continuous linear functional on .
Remark 2.3. Let and let be the transposed of . Note that for , is continuous in -norm and hence can be represented by an -function ( progressively measurable and continuous in ). Provided has the property that if and is -measurable then , we have that, for any , for a.e. in , a.s.
(a consequence of (2.11)).
When is continuous on , then (the limit being an -limit), in fact, when , in , where is the -function representing .
Assume that is the natural filtration generated by . Then the progressively measurable function satisfies the following condition: on , there exist -valued, progressively measurable functions , , , continuous in , such that , for all , such that and such that, for all , -a.s. In this case, if is continuous on , then the following additional properties hold: , , the -limit exists and equals , and .
3. Proof of Theorem 2.1
The proof consists of three lemmas and the five proof steps A–E and relies on an “abstract” maximum principle, Corollary I in the appendix.
Let and let , .
Lemma 3.1. Let be progressively measurable. For any there exists a function , with , , such that .
Proof. Using Dunford and Schwartz [7, III.11.16 Lemma] yields that a.e. For each there exists a function
- piecewise constant in -, , such that
Thus, there exists an open set , such that meas, and (note that meas, otherwise the inequality involving is contradicted). Let , and let if . For , so for , we have
If and , then for some , , so for this , , and (3.4) yields
Let be arbitrarily given. Assume now that is so small that ( = Lebesgue measure of ) and
Then, using successively, (3.6), (3.2), (3.3), (3.2), (3.7), and (3.8) yields
Lemma 3.2. Let be progressively measurable and let . Then for each there exists a set such that for all and such that ( measurable).
Proof. Apply Lemma 3.1 to obtain
where , , . Evidently, we can assume of the ’s that they satisfy the additional property
Hence, for any given ,
Moreover, by (3.15) and (3.11),
Finally, for any given , if is the largest such that , then, by (3.12), a.s.,
The conclusion of Lemma 3.2 then follows from (3.16) and (3.17) and the fact that .
Lemma 3.3. Let , and let be continuous in , and assume that . Let . When , then
Proof. Let an error function be a nonnegative function on such that when . By uniform continuity of in , uniformly in , there exists an increasing error function such that for all , . Suppose, by contradiction, that some exists, such that, for each , there exist such that
and . Then
Now, a.s. by the -assumption on in the Lemma. So converges a.s. to zero. Moreover, by (2.2), , the last function being an -function. By dominated convergence, when , and a contradiction of (3.19) is obtained.
(A) Growth Properties
Without loss of generality, from now on, let , . Let , the th component of . For , let (where of course ). For any , , let be the solution of (By general existence results and (2.1) and (2.2), , as well as , see (2.1), do exist, with both being unique (strong) solutions.)
By (2.1) and . By (A.3) in the appendix, (a consequence of Gronwall’s inequality), and (2.2), with , , , , , , and , we have that, for some constant independent of ,
Define . Note that by (2.2), . Then, when belong to , by (2.1) and (3.23), we get the following inequality: for all , ( independent of and ). For two -measurable functions and , let mean that . Define and define . Then a.s. .
Let . Using (2.4), (2.2), and (A.3) in the appendix, (with , , , , , , , and ), for some constant independent of and , we get (the last inequality by (3.24).)
Let , , . As explained below, we have ( independent of .) The inequalities (3.27) and (3.26) follow from and (A.3), respectively, in the appendix, (together with (2.2) and (3.22)), for , , , , , , and (and, for (3.26) for , ).
Similarly, for , , , ,
We need to prove that
This follows from (3.24), (3.25), (2.2), and (3.32), because, in a shorthand notation,
(B) Properties of the “Linear” Perturbations
Let be arbitrarily given, let be any number in , and let , . Let us first prove the following consequence of Lemma 3.2. (We drop writing for .) For all , where on , , is defined by , with the sets being as follows. They are obtained by replacing by , (hence by ), and by in (3.10), that is, in Lemma 3.2, and denoting the corresponding subset by , with in Lemma 3.2 being equal to . Here, (, so .
Let . Because (3.35) holds for some when is replaced by , we get that for some , for all ,
From (3.35) and (3.27) it follows that, for any , and similarly, from (3.36) and (3.29) it follows that
From (3.38) it follows, in a shorthand notation, that
To see this, note that
(by (2.2) and (3.38)), so
Note also that and
Then (3.39) follows from (3.36) and (3.41).
If , , arbitrary, and , , then for any , it is easily seen that we can obtain, for some , that (For ,by (3.39), we can first obtain a control such that , and then by (3.39) we can obtain a control such that , hence . Continuing this argument, we get (3.43) for general .)
Evidently, we can obtain for any , for some , that both
Finally, let the number satisfy , (for and , see (2.10)), and let . We want to prove the inequality (shorthand notation) whenever equals on .
Now, , see (3.26). Next, for , when , see (3.25), so .
Using the two inequalities involving , we get (3.45). And from this property and (2.10) it easily follows that (shorthand notation), for all , (cl = closure in -norm). To see this, apply Lemma 11.1 in Seierstad . (Intuitively this lemma says that if a ball is contained in the closed convex hull of a set, and the elements of the set are slightly perturbed then a slightly smaller ball is contained in the closed convex hull of the set of perturbed elements.)
(C) Relations between Exact and Linear Perturbations
Let , be given elements in . Let be arbitrarily given. Define where the constants and are specified in the proof below. Let us first prove that for any , small enough, for any , there exists a such that
Write . Define, in a shorthand notation,
There exists a such that by Lemmas B and C in the appendix.
In (3.35) let , and let (so ). We will prove (3.47) for this .
Let and let . On , , while, by (2.2),
So, by (3.49), uniformly in , where (recall , and ; see (3.26)).
By in the appendix, Lemma A, for some constants , ,
To see this, in Lemma A let , let , let , let , , and let