Research Article | Open Access

Xueping Zhu, Jianjun Zhou, "Infinite Horizon Optimal Control of Stochastic Delay Evolution Equations in Hilbert Spaces", *Abstract and Applied Analysis*, vol. 2013, Article ID 791786, 14 pages, 2013. https://doi.org/10.1155/2013/791786

# Infinite Horizon Optimal Control of Stochastic Delay Evolution Equations in Hilbert Spaces

**Academic Editor:**Juan J. Trujillo

#### Abstract

The aim of the present paper is to study an infinite horizon optimal control problem in which the controlled state dynamics is governed by a stochastic delay evolution equation in Hilbert spaces. The existence and uniqueness of the optimal control are obtained by means of associated infinite horizon backward stochastic differential equations without assuming the GÃ¢teaux differentiability of the drift coefficient and the diffusion coefficient. An optimal control problem of stochastic delay partial differential equations is also given as an example to illustrate our results.

#### 1. Introduction

In this paper, we consider a controlled stochastic evolution equation of the following form: where is the control process in a measurable spaceâ€‰â€‰ ,and is a cylindrical Wiener process in a Hilbert space is the generator of a strongly continuous semigroup of bounded linear operator in another Hilbert space , and the coefficients and , defined on , are assumed to satisfy Lipschitz conditions with respect to appropriate norms. We introduce the cost function Here, is a given real function, is large enough, and the control problem is understood in the weak sense. We wish to minimize the cost function over all admissible controls.

The particular form of the control system is essential for our results, but it covers numerous interesting cases. For example, in the particular cases and , the term in the state equation can be considered as a control affected by noise.

The stochastic optimal control problem was considered in 1977 by Bismut [1]. The optimal control problem for stochastic partial differential equations in the framework of a compact control state space has been studied in [2â€“5]. Buckdahn and RaÅŸcanu [6] considered an optimal control problem for a semilinear parabolic stochastic differential equation with a nonlinear diffusion coefficient, and the existence of a quasioptimal (nonrelaxed) control is showed without assuming convexity of the coefficients. In [7â€“11], the authors provided a direct (classical or mild) solution of the Hamilton-Jacobi-Bellman equation for the value function, which is then used to prove that the optimal control is related to the corresponding optimal trajectory by a feedback law. In Gozzi [10, 11], the existence and uniqueness of a mild solution of the associated Hamilton-Jacobi-Bellman equation are proved, when the diffusion term only satisfies weak nondegeneracy conditions. The proofs are based on the corresponding regularity properties of the transition semigroup of the associated Ornstein-Uhlenbeck process.

The main tools for the control problem are techniques from the theory of backward stochastic differential equations (BSDEs) in the sense of Pardoux and Peng, first considered in the nonlinear case in [12]; see [13, 14] as general references. BSDEs have been successfully applied to control problems; see, for example, [15, 16] and we also refer the reader to [17â€“20]. Fuhrman and Tessiture [19] considered the optimal control problem for stochastic differential equation in the strong form, assuming Lipschitz conditions and allowing degeneracy of the diffusion coefficient, under some structural constraint on the state equation. Existence of an optimal control for stochastic systems in infinite dimensional spaces also has been obtained in [21â€“27]. In [21], Fuhrman and Tessitore showed the regularity with respect to parameters and the regularity in the Malliavin spaces for the solution of the backward-forward system and defined the feedback law by Malliavin calculus. Finally, the optimal control is obtained by the feedback. Appealing to the Malliavin calculus, compared with Fuhrman et al. [23], the existence of optimal control for stochastic differential equations with delay is proved by the feedback law. Fuhrman and Tessiture [24] dealt with an infinite horizon optimal control problem for the stochastic evolution equation in Hilbert space, and the optimal control is showed by means of infinite horizon backward stochastic differential equation in infinite dimensional spaces and Malliavin calculus. In Masiero [25], the infinite horizon optimal control problem for stochastic evolution equation is also studied by means of the Hamilton-Jacobi-Bellman equation. In Fuhrman [26], a class of optimal control problems governed by stochastic evolution equations in Hilbert spaces which includes state constraints is considered, and the optimal control is obtained by the Fleming logarithmic transformation. Masiero [27] studied stochastic evolution equations evolving in a Banach space where is a constant and characterized the optimal control via a feedback law by avoiding use of Malliavin calculus. Since there is a lack of regularity of and , Malliavin calculus is not available in this case; the method in [27] also cannot be used as is not a constant, but we can prove a theorem similar to [26, Proposition 3.2], which will be used to define the feedback law.

In the present paper, we study the infinite horizon optimal control problem for stochastic delay evolution equations in Hilbert spaces, and by using Theorem 10, the optimal control is obtained. Since we do not relate the optimal feedback law with the gradient of the value function and do not consider the associated Hamilton-Jacobi-Bellman equation, we can drop the GÃ¢teaux differentiability of the drift term and the diffusion term.

The plan of the paper is as follows. In the next section, some notations are fixed, and the stochastic delay evolution equations are considered with an infinite horizon; in particular, continuous dependence on initial value is proved. In Section 3, we give the proof of Theorem 10, which is the key of many subsequent results. The addressed optimal control problem is considered, and the fundamental relation between the optimal control problem and BSDEs is established in Section 4. Section 5 is devoted to proving the existence and uniqueness of optimal control in the weak sense. Finally, an application is given in Section 6.

#### 2. Preliminaries

We list some notations that are used in this paper. We use the symbol to denote the norm in a Banach space , with a subscript if necessary. Let , , and denote real separable Hilbert spaces, with scalar products , , and , respectively. For fixed , denotes the space of continuous functions from to , endowed with the usual norm . Let denote the dual space of , with scalar product , and let denote the space of all bounded linear operators from into ; the subspace of Hilbert-Schmidt operators, with the Hilbert-Schmidt norm, is denoted by .

Let be a complete space with a filtration which satisfies the usual condition. By a cylindrical Wiener process with values in a Hilbert space , defined on , we mean a family of linear mappings such that for every , is a real Wiener process and . In the following, is a cylindrical Wiener process adapted to the filtration .

In this section and the next section, will denote the natural filtration of , augmented with the family of -null of . The filtration satisfies the usual conditions. For , we also use the following notations: By we denote the predictable -algebra, and by we denote the Borel -algebra of any topological space .

Similar to [24], we define several classes of stochastic processes with values in a Banach space as follows. (i) denotes the space of equivalence classes of processes , admitting a predictable version. is endowed with the norm
(ii), defined for and , denotes the space of equivalence classes of processes , with values in , such that the norm
â€‰is finite and admits a predictable version. (iii) denotes the space . The norm of an element is . Here, is a Hilbert space.(iv), defined for and , denotes the space of predictable processes with continuous paths in , such that the norm
is finite. Elements of are identified up to indistinguishability. (v), defined for and , denotes the space of predictable processes with continuous paths in , such that the norm
is finite. Elements of are identified up to indistinguishability. (vi)Finally, for and , we defined as the space , endowed with the norm
For simplicity, we denote *, *, , and by , , , and , respectively.

Now, for every fixed , we consider the following stochastic delay evolution equation: We make the following assumptions.

*Hypothesis 1. *(i) The operator is the generator of a strongly continuous semigroup of bounded linear operators in the Hilbert space . We denote by and two constants such that , for .

(ii) The mapping : is measurable and satisfies, for some constant and ,

(iii) is a mapping such that for every , the map : is measurable, for every , and , and
for some constants and .

We say that is a mild solution of (10) if it is a continuous, -predictable process with values in , and it satisfies -.,
To stress dependence on initial data, we denote the solution by *.* Note that is measurable, hence, independent of .

We first recall a well-known result on solvability of (10) on bounded interval.

Theorem 1. *Assume that Hypothesis 1 holds. Then, for all and , there exists a unique process as mild solution of (10). Moreover,
*

for some constant depending only on , and .

By Theorem 1 and the arbitrariness ofâ€‰â€‰ in its statement, the solution is defined for every . We have the following result.

Theorem 2. *Assume that Hypothesis 1 holds and the process is mild solution of (10) with initial value . Then, for every , there exists a constant such that the process . Moreover, for a suitable constant , one has
**
with the constant depending only on , and . *

* Proof. *We define a mapping from to by the formula
We are going to show that, provided is suitably chosen, is well defined and that it is a contraction in ; that is, there exists such that
For simplicity, we set , and we treat only the case , the general case being handled in a similar way. We will use the so called factorization method; see [28, Theorem 5.2.5]. Let us take and such that .

By the stochastic Fubini theorem,
where
Since , the process , , belongs to provided . Next, we estimate , where
setting , so that
Applying the Young inequality for convolutions, we have
and we conclude that
If we start again from (20) and apply the HÃ¶lder inequality, we obtain
So, we conclude that
On the other hand, by the Burkholder-Davis-Gundy inequalities, for some constant depending only on , we have
which implies that
so that
for suitable constants . Applying the Young inequality for convolutions, we obtain
This shows that is finite provided we assume that and ; so, the map is well defined.

If are processes belonging to and are defined accordingly, the entirely analogous passages show that
Recalling the inequalities (23) and (25) and noting that the map is linear, we obtain an explicit expression for the constant in (17), and it is immediate to verify that provided is chosen sufficiently large. We fix such a value of . The first result is a consequence of the contraction principle. The estimate (15) also follows from the contraction property of .

For investigating the dependence of the solution on the initial data and , we reformulate (13) as an equation on . We set and we consider the equation Under the assumptions of Hypothesis 1, by Theorem 2, it is easy to prove that equation (32) has a unique solution and for every . It clearly satisfies for , and its restriction to the time internal is the unique mild solution of (10). From now on, we denote by , the solution of (32).

We need the following parameter-depending contraction principle, which is stated in the following lemma and proved in [29, Theorems 10.1 and 10.2].

Lemma 3 (Parameter Depending Contraction Principle). *Let denote Banach spaces. Let be a continuous mapping satisfying
**
for some and every . Let denote the unique fixed point of the mapping . Then, is continuous. *

Theorem 4. *Assume that Hypothesis 1 holds true. Then, for every , the map is continuous from to .*

*Proof. *Clearly, it is enough to prove the claim for large. Let us consider the map defined in the proof of Theorem 2. In our present notation, can be seen as a mapping from to as follows:
By the arguments of the proof of Theorem 2, is a contraction in uniformly with respect to . The process is the unique fixed point of *.* So, by the parameter-depending contraction principle (Lemma 3), it suffices to show that is continuous from to . From the contraction property of mentioned earlier, we have that is continuous, uniformly in , . Moreover, for fixed , it is easy to verify that is continuous from to . The proof is finished.

*Remark 5. *By similar passages, we can show that, for fixed , Theorem 4 still holds true for large enough if the spaces and are replaced by the spaces and respectively, where denotes that the space of -measurable function with value in , such that the norm
is finite.

#### 3. The Backward-Forward System

In this section, we consider the system of stochastic differential equations, -., for varying on the time interval . As in Section 2, we extend the domain of the solution setting for .

We make the following assumptions.

*Hypothesis 2. *The mapping is Borel measurable such that, for all , is continuous, and for some , , and ,
for every , , .

We note that the third inequality in (37) follows from the first one taking but that the third inequality may also hold for different values of .

Firstly, we consider the backward stochastic differential equation
is a Hilbert space, the mapping is a given measurable function, is a predictable process with values in another Banach space , and is a real number.

Theorem 6. *Assume that Hypothesis 2 holds. Let and be given, and choose
**
Then, the following hold. *(i)*For and , (38) has a unique solution in that will be denoted by . *(ii)*The estimate
*â€‰*holds for a suitable constant . In particular, .*(iii)*The map is continuous from to , and is continuous from to . *(iv)*The statements of points (i), (ii), and (iii) still hold true if the space is replaced by the space .*

*Proof. *The theorem is very similar to Proposition 3.11 in [24]. The only minor difference is that the mapping is a given measurable function, while in [24], the measurable function is from to ; however, the same arguments apply.

Theorem 7. *Assume that Hypothesis 1 holds and that Hypothesis 2 holds true in the particular case . Then, for every satisfying (39) with , and for every , there exists a unique solution in of (36) that will be denoted by . Moreover, . The map is continuous from to , and the map is continuous from to .*

* Proof. *We first notice that the system is decoupled; the first does not contain the solution of the second one. Therefore, under the assumption of Hypothesis 1 by Theorem 2, there exists a unique solution and of the first equation. Moreover, from Theorem 4, it follows that the map is continuous from to .

Let ; from Theorem 6, we have that there exists a unique solution of the second equation, and the map is continuous from to while is continuous from to . We have proved that is the unique solution of (36), and the other assertions follow from composition.

*Remark 8. *From Remark 5, by similar passages, we can show that for fixed and for large enough, under the assumptions of Theorem 7, the map is continuous from to .

We also remark that the process is measurable, since is separable Banach space, we have that is measurable; So that is measurable with respect to both and , it follows that is deterministic.

For later use, we notice three useful identities; for , the equality, -a.s.,
is a consequence of the uniqueness of the solution of (13). Since the solution of the backward equation is uniquely determined on an interval by the values of the process on the same interval, for , we have, -a.s.,

Lemma 9 (see [30]). *Let be a metric space with metric , and let be strongly measurable. Then, there exists a sequence , of simple -valued functions (i.e., is / measurable and takes only a finite number of values) such that for arbitrary , the sequence , is monotonically decreasing to zero. *

Let now . By Lemma 9 we get the existence of a sequence of simple function , such that Hence, in by Lebesgue's dominated convergence theorem.

We are now in a position of showing the main result in this section.

Theorem 10. *Assume that Hypothesis 1 holds true and that Hypothesis 2 holds in the particular case . Then, there exist two Borel measurable deterministic functions and , such that for and , the solution of (36) satisfies
*

*Proof. *We apply the techniques introduced in [26, Proposition 3.2]. Let be a basis of , and let us define . Then, for every , and , we have that
From Theorem 7, we have that the map is continuous from to . By Remark 8, we also have that, for fixed , the map is continuous from to for large enough. Let us define
It is clear that is a Borel function.

We fix and . For , we denote , the random variable obtained by composing with the map .

By Lemma 9, there exists a sequence of -valued -measurable simple functions
where are pairwise distinct and *,* such that
For any , we have
and we get that
Fix and . Recalling that , by the Lebesgue theorem on differentiation, it follows that -a.s.
By the boundedness of , applying the dominated convergence theorem, we get that
Now, we have proved that for every ,
for every . Let denote the set of pairs such that exists and the series converges in . We define
Since satisfies
for every . From (53), it follows that for every , we have , -a.s., for almost all , and -a.s., for a.a. .

We define ; since is deterministic, so the map can be written as a composition with
From Theorem 7, it follows that is continuous. By
we have that is continuous. It is clear that is continuous. Then, the map is continuous from to ; therefore, is a Borel measurable function. From uniqueness of the solution of (36), it follows that *, *-a.s., for a.a. .

#### 4. The Fundamental Relation

Let be a given complete probability space with a filtration satisfying the usual conditions. is a cylindrical Wiener process in with respect to . We will say that an -predictable process with values in a given measurable space is an *admissible control*. The function is measurable and bounded. We consider the following controlled state equation:

Here, we assume that there exists a mild solution of (58) which will be denoted by or simply by . We consider a cost function of the form: Here, is function on with real values. Our purpose is to minimize the function over all admissible controls.

We define in a classical way the Hamiltonian function relative to the previous problem; for all ,