Abstract

A maximum principle is proved for certain problems of optimal control of diffusions where hard end constraints occur. The results apply to several dimensional problems, where some of the state equations involve Brownian motions, but not the equations corresponding to states being hard restricted at the terminal time.

1. Introduction

Various types of maximum principles have been proved for problems of control of diffusions in case of no or soft terminal state restrictions; see for example, Kushner [1], Haussmann [2], Peng [3], and Yong and Zhou [4]. Maximum principles for problem with hard terminal restrictions are proved for certain types of continuous time piecewise deterministic problems in Seierstad [5, 6]. Singular controls are sometimes introduced in various problems with certain types of hard restrictions, but below we merely consider problems where the controls appearing may be said to be absolutely continuous with respect to Lebesgue measure. The restriction to such controls makes it harder to operate with hard terminal state restrictions; in fact we can only work with such restrictions on states governed by differential equations not containing a Brownian motion. Brownian motion will only appear in differential equations of states unconstrained at the terminal time. So the problem we consider is a problem of control of diffusions where hard terminal restrictions are placed on states not governed by differential equations containing a Brownian motion; these states, however, can be influenced by other states directly influenced by Brownian motions. Below, a maximum principle is stated and proved for such problems. To the authors knowledge, maximum principles have not been stated for such problems before. Because the states are stochastic, the state space is infinite dimensional; so to obtain a maximum principle, one must impose a condition amounting to demand sufficient variability of the first order variations in the problem.

2. The Control Problem and the Statement of the Necessary Condition

Let and let be a given point in the Euclidean space , let be a projection from into , , such that () and let be a Borel subset of a Euclidean space. Furnish the interval with the Lebesgue measure. Let be a filtered probability space, (i.e., for , the ’s are sub--algebras of the given -algebra of subsets of , if , and is a probability measure on ), and assume that is complete, that contains all the -null sets in and that is right continuous. Let be the set of Lebesgue -measurable functions for which . Related to , let be a vector the components of which (denoted ) are independent one-dimensional Brownian motions all adapted to , such that is independent of for all , , , and is normally distributed with mean and covariances , with being the identity matrix. In applications where the ’s are the entities that can be observed, it is natural to take as the natural filtration generated by the ’s. There are given functions and , , from into ( independent of ). The following conditions are called the basic assumptions.

() The functions and have continuous derivatives and with respect to .

() The functions and have one-sided limits with respect to , and and are, separately, continuous in and in .

Write for the -matrix whose columns are ; let be the matrix with entries , and write . Also, write for the indicator function of the set . Let be a set of functions taking values in , such that , for each , when restricted to , is Lebesgue -measurable (i.e., progressively measurable), and such that when , , arbitrary, and , , is a measurable partition of , then belongs to (so-called switching closedness). We will also assume that is -closed, which means that if and is progressively measurable and takes values in and meas is not equal to a.s.} converges to zero when , then belongs to . Let be the -norm on .

The following assumptions are called the global assumptions.

(B1) is uniformly continuous in , uniformly in , .

(B2) For some constant , for all ,

(B3) A constant exists such that for all , (The symbol is used for the Euclidean norm in any Euclidean space , including , and, applied to matrices, it is the linear operator norm.)

(B4) One has

Let . The strong (unique) solution, continuous in , of the equation is denoted and is called a system solution.

Let ( fixed, ) such that ; let denote scalar product, and consider the problem subject to

Let be an optimal control in the problem and write . Let be the resolvent of the equation (so , with being the identity map).

In the subsequent necessary conditions, the following local linear controllability condition (2.10) is needed. Let be the set of progressively measurable functions in , and for , let and let co denote convex hull. There exists a number and a progressively measurable function , with , and a number such that

Theorem 2.1. Assume that is optimal in problems (2.5) and (2.6), that assumptions A and B hold (the basic and global assumptions), and that (2.10) is satisfied. Then there exists a number and a linear functional on , bounded on , such that, for all ,
Finally, .

Remark 2.2. If (2.10) holds for , then and is a continuous linear functional on .

Remark 2.3. Let and let be the transposed of . Note that for , is continuous in -norm and hence can be represented by an -function ( progressively measurable and continuous in ). Provided has the property that if and is -measurable then , we have that, for any , for a.e. in , a.s. (a consequence of (2.11)).
When is continuous on , then (the limit being an -limit), in fact, when , in , where is the -function representing .
Assume that is the natural filtration generated by . Then the progressively measurable function satisfies the following condition: on , there exist -valued, progressively measurable functions , , , continuous in , such that , for all , such that and such that, for all , -a.s. In this case, if is continuous on , then the following additional properties hold: , , the -limit exists and equals , and .

3. Proof of Theorem 2.1

The proof consists of three lemmas and the five proof steps A–E and relies on an “abstract” maximum principle, Corollary I in the appendix.

Let and let , .

Lemma 3.1. Let be progressively measurable. For any there exists a function , with , , such that .

Proof. Using Dunford and Schwartz [7, III.11.16 Lemma] yields that a.e. For each there exists a function - piecewise constant in -, , such that
Thus, there exists an open set , such that meas, and (note that meas, otherwise the inequality involving is contradicted). Let , and let if . For , so for , we have
Define
If and , then for some , , so for this , , and (3.4) yields
Let be arbitrarily given. Assume now that is so small that ( = Lebesgue measure of ) and
Then, using successively, (3.6), (3.2), (3.3), (3.2), (3.7), and (3.8) yields

Lemma 3.2. Let be progressively measurable and let . Then for each there exists a set such that for all and such that ( measurable).

Proof. Apply Lemma 3.1 to obtain where , , . Evidently, we can assume of the ’s that they satisfy the additional property
Define
Now,
Hence, for any given ,
Moreover, by (3.15) and (3.11),
Finally, for any given , if is the largest such that , then, by (3.12), a.s.,
The conclusion of Lemma 3.2 then follows from (3.16) and (3.17) and the fact that .

Lemma 3.3. Let , and let be continuous in , and assume that . Let . When , then

Proof. Let an error function be a nonnegative function on such that when . By uniform continuity of in , uniformly in , there exists an increasing error function such that for all , . Suppose, by contradiction, that some exists, such that, for each , there exist such that and . Then
Now, a.s. by the -assumption on in the Lemma. So converges a.s. to zero. Moreover, by (2.2), , the last function being an -function. By dominated convergence, when , and a contradiction of (3.19) is obtained.

(A) Growth Properties
Without loss of generality, from now on, let , . Let , the th component of . For , let (where of course ). For any , , let be the solution of (By general existence results and (2.1) and (2.2), , as well as , see (2.1), do exist, with both being unique (strong) solutions.)
By (2.1) and . By (A.3) in the appendix, (a consequence of Gronwall’s inequality), and (2.2), with , , , , , , and , we have that, for some constant independent of ,
Define . Note that by (2.2), . Then, when belong to , by (2.1) and (3.23), we get the following inequality: for all , ( independent of and ). For two -measurable functions and , let mean that . Define and define . Then a.s. .
Let . Using (2.4), (2.2), and (A.3) in the appendix, (with , , , , , , , and ), for some constant independent of and , we get (the last inequality by (3.24).)
Let , , . As explained below, we have ( independent of .) The inequalities (3.27) and (3.26) follow from and (A.3), respectively, in the appendix, (together with (2.2) and (3.22)), for , , , , , , and (and, for (3.26) for , ).
Similarly, for , , , ,
Define
Define also
We need to prove that
This follows from (3.24), (3.25), (2.2), and (3.32), because, in a shorthand notation,

(B) Properties of the “Linear” Perturbations
Let be arbitrarily given, let be any number in , and let , . Let us first prove the following consequence of Lemma 3.2. (We drop writing for .) For all , where on , , is defined by , with the sets being as follows. They are obtained by replacing by , (hence by ), and by in (3.10), that is, in Lemma 3.2, and denoting the corresponding subset by , with in Lemma 3.2 being equal to . Here, (, so .
Let . Because (3.35) holds for some when is replaced by , we get that for some , for all ,
From (3.35) and (3.27) it follows that, for any , and similarly, from (3.36) and (3.29) it follows that
From (3.38) it follows, in a shorthand notation, that
To see this, note that
(by (2.2) and (3.38)), so
Note also that and
Then (3.39) follows from (3.36) and (3.41).
If , , arbitrary, and , , then for any , it is easily seen that we can obtain, for some , that (For ,by (3.39), we can first obtain a control such that , and then by (3.39) we can obtain a control such that , hence . Continuing this argument, we get (3.43) for general .)
Evidently, we can obtain for any , for some , that both
Finally, let the number satisfy , (for and , see (2.10)), and let . We want to prove the inequality (shorthand notation) whenever equals on .
Now, , see (3.26). Next, for , when , see (3.25), so .
Using the two inequalities involving , we get (3.45). And from this property and (2.10) it easily follows that (shorthand notation), for all , (cl = closure in -norm). To see this, apply Lemma 11.1 in Seierstad [8]. (Intuitively this lemma says that if a ball is contained in the closed convex hull of a set, and the elements of the set are slightly perturbed then a slightly smaller ball is contained in the closed convex hull of the set of perturbed elements.)

(C) Relations between Exact and Linear Perturbations
Let , be given elements in . Let be arbitrarily given. Define where the constants and are specified in the proof below. Let us first prove that for any , small enough, for any , there exists a such that
Write . Define, in a shorthand notation,
There exists a such that by Lemmas B and C in the appendix.
In (3.35) let , and let (so ). We will prove (3.47) for this .
Let and let . On , , while, by (2.2),
So, by (3.49), uniformly in , where (recall , and ; see (3.26)).
Consider now
By in the appendix, Lemma A, for some constants , ,
To see this, in Lemma A let , let , let , let , , and let
Note that
Hence, we have
Observe that (see definition subsequent to (3.47)), and that , so , because satisfies (3.35) for . Using also (3.49), (3.52) yields (3.54) for ; hence, (3.47) for .
Next, let us prove that for any small enough, for any , there exists a such that
When is small enough, then , uniformly in when , by (3.26) and Lemma 3.3 above ( defined subsequent to (3.47), we use that ). Moreover, by (3.47), , so, by (2.2),
Hence, using the definition of referred to and , we get , where
Let
Then on . On , by (2.2), (3.25), and (3.26), for all , as . Hence, using the inequalities for and above and Jensen’s inequality,
Finally, recalling that , for we have that by (3.35). Hence, using and (3.63), so for , (3.58) has been proved.

(D) Continuity of at
Define . Let us first prove that
Now,
Using Ito’s isometry, Jensen’s inequality, and the algebraic inequality , then for some number (only dependent on the number of addends)
Hence, where
Now, when in -metric, see (3.25). Hence, by the Basic assumption , when in -metric, then for each , it is easily seen, using Lebesgue’s dominated convergence theorem (and, if necessary, Remark M in the appendix), that the terms in curly brackets converge to zero in , by the bound on and ; see (3.25) and (2.2). Since , then, for each , the product of the two terms in curly brackets with converge to zero in when . Hence, the expectation of the two products converge to zero when in -metric. Since is bounded, by (3.24), (3.25), and (3.26), then when in -metric (by dominated convergence again). By Gronwall’s inequality, for some constant ,
So (3.67) holds.
Next, we want to prove that satisfies where
Now, in a shorthand notation,
From this we get that
For some increasing nonnegative error function , the third integrand is smaller than ; see and in the global assumptions. We then get that
Now, when , by (3.67), and (see (3.25)). Then the term in -measure and then also in -norm (by the bound ), so in -norm, and then also in -norm, as the term is bounded by the -function , (see (3.26)). Thus , uniformly in ; that is, (3.73) holds.

(E) Final Proof Steps
For , define and , . Define for see (2.8). Furthermore, let be the subset of consisting of all element such that , and such that , (limit in -norm). It is easily seen that elements of the type , progressively measurable, , precisely make up the set . To see this, using Jensen’s inequality three times (and for any , that , note that for any interval , so, in particular, . This yields also, for , that so . Moreover, similarly, so is an -limit of . Hence, . Finally, if , then , for where , is progressively measurable.
Let be the linear map from into defined by . In (3.80) we have just proved that has norm for the norms and (or for , as we will express it). We also have that the norm on is for the norms , as we will see. Let and define . Then, by Jensen’s inequality, while for , so .
Now, for , ( a ball in -norm), we have
Hence, by (3.46), for , (cl = closure in ; note that is continuous in , as shown above). Observe, finally, that when satisfies , then, using (for ) and (3.24) yields
Hence, using (3.86) and the last string of inequalities and a simple argument (or Lemma 11.1 in Seierstad [8] again), we get (cl = closure in ) for , (the last fraction defined in connection with (3.45)).
To obtain the conclusion in Theorem 2.1, we will now apply Corollary I in the appendix, and for this end, we make the following identifications: , , , , , the norm on equal to , , , , , , , , and . By for and (3.58), it follows that the property is satisfied, and by and (3.44) it follows that is satisfied, both and in the manner required in Corollary I. Moreover, for , , by (3.88) for . By (3.33) and (3.25), and are continuous, and by (3.73), (3.67), is continuous at . The required boundedness of is satisfied because of (3.26). The space is complete by well-known arguments; see Lemma 5.1 in Seierstad [8] and Lemma 1, page 202 in Clarke [9]. Moreover, if is a Cauchy sequence in , then it is a Cauchy sequence in . Let be its -limit. Then, for all , . Now, for any , for , some , for all , , and so also . Thus the space is complete. Hence all conditions in Corollary I are satisfied. Thus, for some , some nonzero continuous linear functional on , , for all , . Because , the conclusion in Theorem 2.1 follows.

Proof of Remark 2.2. Note that (2.10) implies . See (3.88). If , the maximum condition (2.11) implies and hence , a contradiction. Observe that on (for , ). If we show for a -dense subset of that for for some independent of , then for , implying -continuity on . The maximum condition implies ; hence, for any , there exists a such that and , so , by (3.26).

Proofs of Remark 2.3 and the following remark can be found in the appendix.

Remark 3.4 (exact attainability). In Theorem 2.1, drop the assumption that is optimal (and the optimization problem). Then, for each , , , (cl and int = interior both corresponding to ), for all , for some number and some control , .

Example
Let , let , let , let , let be a nonzero constant, let , let , let , and let us maximize subject to , a.s. This trivial problem was solved in Seierstad [10] using the HJB equation. Let be the natural filtration generated by and let . Let us merely show that the solution presented in Seierstad [10], namely , (), satisfies the necessary conditions for . (So is deterministic and equals .) Evidently, the conditions in Remark 2.2 are satisfied, so can be put equal to , and is -continuous.
The -matrix satisfies where the -matrix consists of the partial derivatives of the drift terms in the three-state equation above, the only nonzero element in being the element . Hence, , , , , except for which equals . Now, , so the maximum condition (2.11) reduces to , . Writing , , the time-pointwise version (2.12) of this condition becomes , which evidently requires that . Now, for , , . This does satisfy . Moreover, . (Concerning (2.14), it is evidently satisfied by , , , , , , , ).

Appendix

The appendix includes a number of well-known results, included for the convenience of the reader. The first one concerns a result on comparison of solutions.

Lemma A. Assume that (an -vector and , (an matrix, with columns , ) are Lipschitz continuous in with rank and progressively measurable in . Assume that six progressively measurable functions , , , , , and exist (,   -matrices), satisfying where . (Assume that the eight integrands belong to -spaces). Then, for some constant , (applied to matrices, the index indicates columns), and for some constant , with and being only dependent on .

Proof of (A.3). We will use a shorthand notation. Using the algebraic inequality , then for some positive constant ,
The Burkholder-Davis-Gundy inequality yields, for a “universal” constant , that . Similar inequalities hold for the other terms involving . Hence (using also Jensen’s inequality) we get
Note that, by Gronwall’s inequality, for any functions , , if , and is increasing, then . Hence, for ,
Using the fact that the square root of a sum of positive numbers is less than or equal the sum of square roots of the numbers, we get
Note that , and that . Using this for the term containing , and a similar argument for the term containing , then (A.3) follows.

Proof of (A.2). Using Ito’s isometry,
Then, again using and Jensen’s inequality, for some positive constant ,
Thus, for , so (A.2) follows.

Simple results on Gâteaux derivatives appear in the next two lemmas.

Lemma B. For each , has, in the norm , a bounded linear Gâteaux derivative, which in “direction” , , equals . The derivative is uniform in ; see the first inequality below.

Proof. By Ito’s isometry,
The term in curly brackets converges to zero for each and is smaller than the -function . Hence, Lebesgue’s dominated convergence theorem gives that when .

Lemma C. For each , each has, in the norm , a bounded linear Gâteaux derivative, which in “direction” , , equals . In fact, for each , for then when .

Proof. Jensen’s inequality yields the inequality . The remaining arguments are as in the preceding proof.

Below, on product spaces, maximum norms (= maximum of norms) and maximum metrics are used. In the sequel, the following entities are used:

Theorem D (attainability). Let the entities in (A.13) be given. Let positive numbers , , , , and an element in be given. Assume that the following properties hold for all : for all with , for all , a exists, such that
Then, for all , there exists a pair , such that , where .

Corollary E. Assume that . Then, in (A.14), evidently can be replaced by the stronger inequality .

(On the other hand, when , then for ).

Central ideas in the proof of Theorem D stem from the proof of the multifunction inverse function theorem Theorem 4, page 431, in Aubin and Ekeland [11].

Proof of Theorem D. The property (A.14) also holds for in the set . To see this, let and let . Then for some , some such that . Now, for all , there exists a pair , , such that the inequalities in (A.14) hold. From these inequalities, for , using , it follows that and . Hence, (A.14) holds for .

Below, write . The following lemma is needed in the proof.

Lemma F. Let . Assume that the pair minimizes in . Then .

Proof of Lemma F. By contradiction, assume , . The vector satisfies , so belongs to . Hence, by the extended property (A.14), there exist an and a , such that
Moreover, , (use the first inequality for ), which implies . Define   . Then, using (A.16), , and the definition of , we get
Using and yields a contradiction of the optimality of .
Continued Proof of the Theorem. Let , let be as in the conclusion of the theorem, and let . Note that . Let the distance between and be in the complete space . By Aubin and Ekeland ([11, Theorem 1, page 255]) (Ekeland’s variational principle), there exists a such that which gives , . By Lemma F, , so , for .

Below, is a sort of Gâteaux derivative at of .

Corollary G. Let , let be a normed space, let be a complete pseudometric space with metric , let be a given element in , and let be continuous. Assume also the existence of a function from into and positive constants and such that, for each , for all , all , all , there exists a pair , , such that

Assume also that for all ,

Assume that is continuous at for any . Assume finally that is an interior point in , and that, for some , some , for all . Then, for some and some , .

Proof. Write , and let for some . Then, for some , . Define . Evidently, is an interior point in if . Then is an interior point in even if is only an approximate equality; in fact there exist positive numbers and such that for all . Because , by (A.21) there exists a , such that . By the continuity assumption on in the corollary, for small enough, for . We assume , . Evidently, . Hence, for all . Thus, for , , because and is convex. Hence, , . It follows that if , , then, for any , by (A.21), for some , ,
By (A.20), for , for some , and some arbitrarily small ,
Now, by (A.22), . Then, by (A.23), (, ). In Theorem D, replace by , by , by , and by and let , let , let , let , let , and let . Then the conditions in Theorem D are satisfied when, in (A.14), is replaced by , as just constructed, (, for , , and as ). Thus, we get that, for all small enough, for some , or .

Remark H. If , then can be taken to be arbitrary small ( can be replaced by for any ).

Corollary I. Let be a normed space with norm , let , and let be a complete pseudometric space with metric . Assume that is a given element in , let be continuous. Let , , , with for all . Assume that (A.20) and (A.21) are satisfied for replaced by and also that is continuous at for any . Assume, for some given , that . Assume also, for some , some , that for all . Assume, finally, that . Then, for some continuous non zero linear functional on , a number , we have for all .

Proof. Define , and . It is easily seen that the following inclusions hold for all : (if necessary, use the proof of Lemma 11.2 in Seierstad [8]). Assume by contradiction that belongs to int for some . Define , and for , , let and , and let , . Then (A.21) and (A.20) are evidently satisfied (the latter for when ). Obviously, for each , , where . Hence, by the preceding corollary, for some , for some . Hence, , , or , contradicting optimality. Thus the set is disjoint from int , so the convex set can be separated from the convex set int  by a nonzero continuous linear functional such that , , which implies .

Remark J. [a nonzero continuous linear functional on vanishing on all , ]. Let , be the natural filtration corresponding to some given . Choose a such that . Then , so belongs to the -boundary of the -ball . Then for some nonzero continuous linear functional on , . Let be any given integer. If , then for and for , so, . Then the inequality involving yields , that is, vanishes on , . To show in detail that such a exists, let , and let . Then , so for , , and for , . Letting , we get , so .

Proof of Remark 3.4. Let , and let . Corollary G will be applied. Let , let (for , see (3.88)), let , let , let the norm on be equal to , let , let , let , let , let , let , let , and let . Recall that (3.88) says that for , . By (3.58), it follows that the property (A.20) is satisfied, and by (3.43), it follows that (A.21) is satisfied. By (3.33) is continuous, and by (3.73), is continuous at . Let (the ball and cl corresponding to ). For any , it was shown earlier that there exists a such that , , so (cl corresponding to ). Hence all conditions in Corollary G are satisfied and the conclusion in Remark 3.4 follows.

Proof of Remark 2.3. Let , and let . Note that if , then , for , and hence, . On the other hand . Hence, on , the norms and are equivalent. Thus, the spaces , , are subspaces of . For , define
Then an application of Lemma A in the appendix, similar to the one yielding (3.27) gives that for some constant independent of . Let for some . Then . Because for , see (3.84),
Now, , and for some constant , for , so and . Hence, for any given , by the -continuity of on , , see (A.26), and hence , there exists an -function on such that for any -function , we have . In fact, the last equality yields that .
Let be any given element in and let . Moreover, let be an arbitrarily given Lebesgue point of . Then when . (Here -continuity of on and of , uniformly in , is used.) In fact, needs only be a Lebesgue point from the right. Replacing by in (2.6), we get (Here -convergence when of to and the -representation of is used). Now, if is a right Lebesgue point of , then is a right Lebesgue point of , for any . So , for any . Hence, for a.e. , a.s., we get . From this the property (2.12) follows.
Now, let be the natural filtration generated by , and let . Then, consider the pair of equations
By Theorem 2.2, page 349 in Yong and Zhou [4], there is a unique progressively measurable collection , , continuous in , satisfying these equations, , , and (by (2.20) in the proof of this theorem) for all , equals -a.s. The uniqueness in particular says that if two pairs , , and satisfy the pair of equations, then . We can let when , (, , increasing) and obtain functions , defined on . For , by the fact that a.s. and uniqueness, we have that on in the sense just stated. Then, evidently, there exists a unique pair on satisfying (A.29), with a.s. equal to for any .
Let and let , , be an increasing sequence with . Assume that is -continuous. For , it is easily seen, using the appendix, Lemma A, that in , uniformly in , . Hence, , uniformly in , , and thus in , where is the -function representing .

Remark K. Let us imagine that some of the states , are required to be softly constrained; that is, for given, . Then Theorem 2.1 would hold for in (2.11) replaced by , with being some vector in for which for , with . To obtain this result, only a slight modification of the proof is needed (all approximation tools needed are worked out, what is needed is a change in the separation argument).

Remark L. Assume that . Then, a least when first and second order derivatives of and with respect to and exist and are continuous and bounded and is a closed convex subset of for some , the following necessary condition, based on weak variations, holds: for some linear functional on , bounded on and some number , for all , where is the solution of

Moreover, .

We then need the linear controllability condition: for some , for some , some ,

Amoderate modification of the above proof works in this case. That, however, is another story.

Remark M. If (, Euclidean spaces) is continuous in and -measurable in , , , and in -measure, ( and -measurable), then in . This result, which is a special case of Krasnoselskii’s theorem (see page 20 in Aubin and Ekeland [11]), can be proved as follows. By contradiction, assume for some and for some subsequence that for all . A subsequence converges a.s. to . Then, by continuity, a.s. and even in , by Lebesgue is dominated convergence theorem. A contradiction has been obtained.