Necessary Conditions for Optimality for Stochastic Evolution Equations

Abstract

This paper is concerned with providing the maximum principle for a control problem governed by a stochastic evolution system on a separable Hilbert space. In particular, necessary conditions for optimality for this stochastic optimal control problem are derived by using the adjoint backward stochastic evolution equation. Moreover, all coefficients appearing in this system are allowed to depend on the control variable. We achieve our results through the semigroup approach.

1. Introduction

Consider a stochastic controlled problem governed by the following stochastic evolution equation (SEE): (𝑡)=((𝑡)(𝑋(𝑡)(𝑡(𝑋(𝑡)(𝑡(𝑡)(],𝑋(0)𝑑𝑋𝐴𝑋+𝑏,𝜈)))𝑑𝑡+𝜎,𝜈))𝑑𝑊,𝑡∈0,𝑇=𝑥₀. (1) We will be interested in trying to minimize the cost functional, which is given by (5), over a set of admissible controls.

This system is driven mainly by a possibly unbounded linear operator 𝐴 on a separable Hilbert space 𝐻 and a cylindrical Wiener process 𝑊 on 𝐻. Here, 𝜈(⋅) denotes a control process.

We will derive the maximum principle for this control problem. More precisely, we will concentrate on providing necessary conditions for optimality for this optimal control problem, which gives this minimization. For this purpose we will apply the theory of backward stochastic evolution equations (BSEEs) shortly as in (10) in Section 3. These equations together with backward stochastic differential equations (BSDEs) have become of great importance in a number of fields. For example, in [1–7], one can find applications of BSDEs to stochastic optimal control problems. Some of these references have also studied the maximum principle to find either necessary or sufficient conditions for optimality for stochastic differential equations (SDEs) or stochastic partial differential equations (SPDEs). Necessary conditions for optimality of the control process 𝜈(⋅) and its corresponding solution 𝑋^𝜈(⋅) but for the case when the noise term 𝜎 does not depend on 𝜈(𝑡) can be found in [3].

In our work here, we allow 𝜎 to depend on the control variable and study a stochastic control problem associated with the former SEE. This control problem is explained in details in Section 2, and the main theorem is stated in Section 3 and is proved together with all necessary estimates in Section 4. Sufficient conditions for optimality for this optimal control problem can be found in [2]. We refer the reader also to [1].

On the other hand, we recall that control problems governed by SPDEs that are driven by martingales are studied in [8]. In fact in [8], we derived the maximum principle (necessary conditions) for optimality of stochastic systems governed by SPDEs. The technique used there relies heavily on the variational approach. The reason beyond that is that the only known way until now to find solutions to the resulting adjoint BSPDEs is achieved through the same variational approach and is established in details in [9]. Thus, the semigroup approach to get mild solutions (as done here in Theorem 2 and in Section 3) cannot be used to study such adjoint BSPDEs considered in [8]. Moreover, it is not obvious how one can allow the control variable 𝜈(𝑡) to enter in the noise term and in particular in the mapping 𝐺 in equation (1.1) of [8] and obtain a result like Theorem 3. This problem is still open and is also pointed out in [8, Remark 6.4].

In the present work, we will show how to handle this open problem in great success, and as we stated earlier, we can and will allow all coefficients in (1) and especially in the diffusion term to depend on the control variable 𝜈(𝑡). We emphasize that our work here does not need to go through neither the technique of Hamilton-Jacobi-Bellman equations nor the technique of viscosity solutions. We refer the reader to [10] for this business and to [11] and some of the related references therein for the semigroup technique. Thus, our results here are new. In this respect, we thank the anonymous referee for pointing out the recent and relevant work of Fuhrman et al. in [12].

2. Statement of the Problem

Let (Ω,ℱ,ℙ) be a complete probability space, and denote by 𝒩 the collection of ℙ-null sets of ℱ. Let {𝑊(𝑡),0≤𝑡≤𝑇} be a cylindrical Wiener process on 𝐻 with its completed natural filtration ℱ_𝑡=𝜎{ℓ∘𝑊(𝑠),0≤𝑠≤𝑡,ℓ∈𝐻^∗}∨𝒩, 𝑡≥0; see [13] for more details.

For a separable Hilbert space 𝐸, denote by 𝐿²_ℱ(0,𝑇;𝐸) to the space of all {ℱ_𝑡,0≤𝑡≤𝑇} progressively measurable processes 𝑓 with values in 𝐸 such that 𝔼^𝑇₀𝑓(𝑡)²_𝐸𝑑𝑡<∞. (2)

This space is Hilbert with respect to the norm 𝑓=𝔼^𝑇₀𝑓(𝑡)²_𝐸𝑑𝑡^1/2. (3)

Moreover, if 𝑓∈𝐿²_ℱ(0,𝑇;𝐿₂(𝐻)), where 𝐿₂(𝐻) is the space of all Hilbert-Schmidt operators on 𝐻, the stochastic integral ∫𝑓(𝑡)𝑑𝑊(𝑡) can be defined and is a continuous stochastic martingale in 𝐻. The norm and inner product on 𝐿₂(𝐻) will be denoted, respectively, by ‖⋅‖₂ and ⟨⋅,⋅⟩₂.

Let us assume that 𝒪 is a separable Hilbert space equipped with an inner product ⟨⋅,⋅⟩_𝒪 and 𝑈 is a convex subset of 𝒪. We say that 𝜈(⋅)∶[0,𝑇]×Ω→𝒪 is admissible if 𝜈(⋅)∈𝐿²_ℱ(0,𝑇;𝒪) and 𝜈(𝑡)∈𝑈a.e., a.s. The set of admissible controls will be denoted by 𝒰_ad.

Suppose that 𝑏∶𝐻×𝒪→𝐻 and 𝜎∶𝐻×𝒪→𝐿₂(𝐻) are two continuous mappings, and consider the following controlled SEE: (𝑡)=((𝑡)(𝑋(𝑡)(𝑡(𝑋(𝑡)(𝑡(𝑡),𝑋(0)𝑑𝑋𝐴𝑋+𝑏,𝜈)))𝑑𝑡+𝜎,𝜈))𝑑𝑊=𝑥₀, (4) where 𝜈(⋅)∈𝒰_ad. A solution (in the sense of the following theorem) of (4) will be denoted by 𝑋^𝜈(⋅) to indicate the presence of the control process 𝜈(⋅).

Let ℓ∶𝐻×𝒪→ℝ and 𝜙∶𝐻→ℝ be two measurable mappings such that the following cost functional is defined: 𝐽(𝜈(⋅))∶=𝔼^𝑇₀ℓ𝑋^𝜈(⋅)(𝑡),𝜈(𝑡)𝑑𝑡+𝜙𝑋^𝜈(⋅)(𝑇),𝜈(⋅)∈𝒰_ad. (5)

For example, one can take ℓ and 𝜙 to satisfy the assumptions of Theorem 3 in Section 3.

The optimal control problem of system (4) is to find the value function𝐽^∗∶=inf𝐽(𝜈(⋅))∶𝜈(⋅)∈𝒰_ad (6) and an optimal control 𝜈^∗(⋅)∈𝒰_ad such that 𝐽^∗=𝐽𝜈^∗(⋅). (7)

If this happens, the corresponding solution 𝑋^𝜈∗(⋅) is called an optimal solution of the stochastic control problem (4)–(7) and (𝑋^𝜈∗(⋅),𝜈^∗(⋅)) is called an optimal pair.

We close this section by the following theorem.

Theorem 1. Assume that 𝐴 is an unbounded linear operator on 𝐻 that generates a 𝐶₀-semigroup {𝑆(𝑡),𝑡≥0} on 𝐻, 𝑏, 𝜎 are continuously Fréchet differentiable with respect to 𝑥, and their derivatives 𝑏_𝑥, 𝜎_𝑥 are uniformly bounded. Then, for every 𝜈(⋅)∈𝒰_𝑎𝑑 there exists a unique mild solution 𝑋^𝜈(⋅) on [0,𝑇] to (4). That is, 𝑋^𝜈(⋅) is a progressively measurable stochastic process such that 𝑋(0)=𝑥₀, and for all 𝑡∈[0,𝑇], 𝑋^𝜈(⋅)(𝑡)=𝑆(𝑡)𝑥₀+^𝑡₀𝑆(𝑡−𝑠)𝑏𝑋^𝜈(⋅)(𝑠),𝜈(𝑠)𝑑𝑠+^𝑡₀𝑆(𝑡−𝑠)𝜎𝑋^𝜈(⋅)(𝑠),𝜈(𝑠)𝑑𝑊(𝑠). (8)

The proof of this theorem can be derived in a similar way to those in [14, Chapter 7] or [15].

From here on, we will assume that 𝐴 is the infinitesimal generator of a 𝐶₀-semigroup {𝑆(𝑡),𝑡≥0} on 𝐻. Its adjoint operator 𝐴^∗∶𝒟(𝐴^∗)⊂𝐻→𝐻 is then the infinitesimal generator of the adjoint semigroup {𝑆^∗(𝑡),𝑡≥0} of {𝑆(𝑡),𝑡≥0}.

3. Stochastic Maximum Principle

It is known from the literature that BSDEs play a fundamental role in deriving the maximum principle for SDEs. In this section, we will search for such a role for SEEs like (4). To prepare for this business, let us first define the Hamiltonian by the following formula: ℋ∶𝐻×𝒪×𝐻×𝐿₂(𝐻)ℋ()+𝑏()⟶ℝ,𝑥,𝜈,𝑦,𝑧∶=ℓ𝑥,𝜈𝑥,𝜈,𝑦_𝐻+⟨𝜎(𝑥,𝜈),𝑧⟩₂. (9)

Then, we consider the following BSEE on 𝐻: −𝑑𝑌^𝜈(⋅)(𝑡)=𝐴^∗𝑌^𝜈(⋅)(𝑡)+∇_𝑥ℋ𝑋^𝜈(⋅)(𝑡),𝜈(𝑡),𝑌^𝜈(⋅)(𝑡),𝑍^𝜈(⋅)(𝑡)𝑑𝑡−𝑍^𝜈(⋅)(𝑡)(𝑡)𝑌𝑑𝑊,0≤𝑡<𝑇,_𝜈(⋅)(𝑇)=∇𝜙𝑋^𝜈(⋅)(𝑇), (10) where ∇𝜙 denotes the gradient of 𝜙, which is defined by using the directional derivative 𝐷𝜙(𝑥)(ℎ) of 𝜙 at a point 𝑥∈𝐻 in the direction of ℎ∈𝐻, as ⟨∇𝜙(𝑥),ℎ⟩_𝐻=𝐷𝜙(𝑥)(ℎ)(=𝜙_𝑥(ℎ)). This equation is the adjoint equation of (4).

As in the previous section, a mild solution (or a solution) of (10) is a pair (𝑌,𝑍)∈𝐿²_ℱ(0,𝑇;𝐻)×𝐿²_ℱ(0,𝑇;𝐿₂(𝐻)) such that we have ℙ—a.s. for all 𝑡∈[0,𝑇]𝑌^𝜈(⋅)(𝑡)=𝑆^∗(𝑇−𝑡)∇𝜙𝑋^𝜈(⋅)(𝑇)+^𝑇_𝑡𝑆^∗(𝑠−𝑡)∇_𝑥ℋ𝑋^𝜈(⋅)(𝑠)(𝑠),𝑌,𝜈_𝜈(⋅)(𝑠),𝑍^𝜈(⋅)(𝑠−)𝑑𝑠_𝑇𝑡𝑆^∗(𝑠−𝑡)𝑍^𝜈(⋅)(𝑠)𝑑𝑊(𝑠). (11)

Theorem 2. Assume that 𝑏, 𝜎, ℓ, and 𝜙 are continuously Fréchet differentiable with respect to 𝑥, the derivatives 𝑏_𝑥, 𝜎_𝑥, 𝜎_𝜈, and ℓ_𝑥 are uniformly bounded, and 𝜙_{𝑥𝐿(𝐻,𝐻)}≤𝑘1+|𝑥|_𝐻 (12) for some constant 𝑘>0.
Then, there exists a unique (mild) solution (𝑌^𝜈(⋅),𝑍^𝜈(⋅)) of BSEE (10).

The proof of this theorem can be found in [16] or [17]. An alternative proof by using finite dimensional framework through the Yosida approximation of 𝐴 can be found in [18].

Our main result is the following.

Theorem 3. Suppose that the following two conditions hold. (i)𝑏, 𝜎, and ℓ are continuously Fréchet differentiable with respect to 𝑥, 𝜈, 𝜙 is continuously Fréchet differentiable with respect to 𝑥, the derivatives 𝑏_𝑥, 𝑏_𝜈, 𝜎_𝑥, 𝜎_𝜈, ℓ_𝑥, and ℓ_𝜈 are uniformly bounded, and 𝜙_{𝑥𝐿(𝐻,𝐻)}≤𝑘1+|𝑥|_𝐻 (13) for some constant 𝑘>0.(ii)ℓ_𝑥 is Lipschitz with respect to 𝑢 uniformly in 𝑥.
If (𝑋^𝜈∗(⋅),𝜈^∗(⋅)) is an optimal pair for the control problem (4)–(7), then there exists a unique solution (𝑌^𝜈∗(⋅),𝑍^𝜈∗(⋅)) to the corresponding BSEE (10) such that the following inequality holds: ∇_𝜈ℋ𝑋^𝜈∗(⋅)(𝑡),𝜈^∗(𝑡),𝑌^𝜈∗(⋅)(𝑡),𝑍^𝜈∗(⋅)(𝑡),𝜈^∗(𝑡)−𝜈_𝒪[]≤0𝑎.𝑒.𝑡∈0,𝑇,𝑎.𝑠.∀𝜈∈𝑈. (14)

The proof of this theorem will be given in Section 4. Now, to illustrate this theorem, let us present an example.

Example 4. Let 𝐻 and 𝒪 be two separable Hilbert spaces as considered earlier, and let 𝑈=𝒪. We will study in this example a special case of the control problem (4)–(7). In particular, given 𝜙 as in Theorem 3, we would like to minimize the cost functional 𝐽(𝜈(⋅))=𝔼^𝑇₀|𝜈(𝑡)|²_𝒪𝑑𝑡+𝔼𝜙𝑋^𝜈(⋅)(𝑇) (15) subject to 𝑑𝑋^𝜈(⋅)(𝑡)=𝐴𝑋^𝜈(⋅)(𝑡)(𝑡)(𝑡)(𝑡)(],𝑋+𝐵𝜈𝑑𝑡+𝐷𝜈𝑑𝑊,𝑡∈0,𝑇_𝜈(⋅)(0)=𝑥₀∈𝐻, (16) where 𝐵 is a bounded linear operator from 𝒪 into 𝐻 and 𝐷 is another bounded linear operator from 𝒪 into 𝐿₂(𝐻).
The Hamiltonian is then given by the formula: ℋ=|𝜈|𝑥,𝜈,𝑦,𝑧_2𝒪+𝐵𝜈,𝑦_𝐻+⟨𝐷𝜈,𝑧⟩_𝐿2(𝐻), (17) where (𝑥,𝜈,𝑦,𝑧)∈𝐻×𝒪×𝐻×𝐿₂(𝐻) and the adjoint BSEE is −𝑑𝑌^𝜈(⋅)(𝑡)=𝐴^∗𝑌^𝜈(⋅)(𝑡)𝑑𝑡−𝑍^𝜈(⋅)(𝑡)(𝑡)[),𝑌𝑑𝑊,𝑡∈0,𝑇_𝜈(⋅)(𝑇)=∇𝜙𝑋^𝜈(⋅)(𝑇). (18)
From the construction of the solution of (18), as for example, in [16, Lemma 3.1], this BSEE attains an explicit solution: 𝑌^𝜈(⋅)(𝑡)=𝔼𝑆^∗(𝑇−𝑡)∇𝜙𝑋^𝜈(⋅)(𝑇)∣ℱ_𝑡,𝑍^𝜈(⋅)(𝑡)=𝑆^∗(𝑇−𝑡)𝑅^𝜈(⋅)(𝑡), (19) where 𝑅^𝜈(⋅) is the unique element of 𝐿²_ℱ(0,𝑇;𝐿₂(𝐻)) satisfying ∇𝜙𝑋^𝜈(⋅)(𝑇)=𝔼∇𝜙𝑋^𝜈(⋅)(𝑇)+^𝑇₀𝑅^𝜈(⋅)(𝑡)𝑑𝑊(𝑡). (20)
On the other hand, for fixed (𝑥,𝑦,𝑧), we note that the function 𝜈↦ℋ(𝑥,𝜈,𝑦,𝑧) attains its minimum at 𝜈=(1/2)(𝐵^∗𝑦+𝐷^∗𝑧)(∈𝑈), where 𝐵^∗∶𝐻→𝒪 and 𝐷^∗∶𝐿₂(𝐻)→𝒪 are the adjoint operators of 𝐵 and 𝐷, respectively. So, we elect 𝜈^∗()=1𝑡,𝜔2𝐵^∗𝑌^𝜈∗(⋅)(𝑡,𝜔)+𝐷^∗𝑍^𝜈∗(⋅)(𝑡,𝜔) (21) as a candidate optimal control.
It is easy to see that with these choices all the requirements of Theorem 3 are verified. Hence, this candidate 𝜈^∗(⋅) given in (21) is an optimal control for the problem (15)-(16), and its corresponding optimal solution 𝑋^𝜈∗(⋅) is the solution of the following SEE: 𝑑𝑋^𝜈∗(⋅)(𝑡)=𝐴𝑋^𝜈∗(⋅)(𝑡)+12𝐵𝐵^∗𝑌^𝜈∗(⋅)(𝑡)+𝐷^∗𝑍^𝜈∗(⋅)(𝑡)+1𝑑𝑡2𝐷𝐵^∗𝑌^𝜈∗(⋅)(𝑡)+𝐷^∗𝑍^𝜈∗(⋅)(𝑡)(𝑡)(],𝑋𝑑𝑊,𝑡∈0,𝑇_𝜈∗(⋅)(0)=𝑥₀. (22)
Finally, the value function attains the formula 𝐽^∗=14𝔼^𝑇₀𝐵^∗𝑌^𝜈∗(⋅)(𝑡)+𝐷^∗𝑍^𝜈∗(⋅)(𝑡)²_𝒪𝜙𝑋𝑑𝑡+𝔼^𝜈∗(⋅)(𝑇). (23)

Remark 5. A concrete example in the setting of Example 4 can be constructed by taking 𝐻=𝒪=𝐿²(ℝ^𝑑), 𝑑≥1, 𝐴=(1/2)Δ (half-Laplacian), 𝐵=𝑖𝑑_𝐻, 𝐷𝜈∶=⟨𝑣,ℎ⟩_𝐻𝒬^1/2, and 𝜙(𝑥)=⟨𝜌,𝑥⟩_𝐻, for some fixed elements ℎ, 𝜌 of 𝐻 and a positive definite nuclear operator 𝒬 on 𝐻.
The computations in this case of ℋ, 𝑌^∗, 𝑍^∗, 𝜈^∗(⋅), and 𝑋^∗ become direct from the corresponding equations in Example 4.

4. Proofs

Let 𝜈^∗(⋅) be an optimal control, and let 𝑋^∗≡𝑋^𝜈∗(⋅) be the corresponding solution of (4). Let 𝜈(⋅) be an element of 𝐿²_ℱ(0,𝑇;𝒪) such that 𝜈^∗(⋅)+𝜈(⋅)∈𝒰_ad. For a given 0≤𝜀≤1, consider the variational control: 𝜈_𝜀(𝑡)=𝜈^∗(𝑡)+𝜀𝜈(𝑡),𝑡∈[0,𝑇]. (24)

We note that the convexity of 𝑈 implies that 𝜈_𝜀(⋅)∈𝒰_ad. Considering this control 𝜈_𝜀(⋅), we will let 𝑋^{𝜈𝜀(⋅)} be the solution of the SEE (4) corresponding to 𝜈_𝜀(⋅) and denote it briefly by 𝑋_𝜀.

Let 𝑝 be the solution of the following linear equation: 𝑑𝑝(𝑡)=𝐴𝑝(𝑡)+𝑏_𝑥𝑋^∗(𝑡),𝜈^∗(𝑡)𝑝(𝑡)+𝑏_𝜈𝑋^∗(𝑡),𝜈^∗(𝑡)𝜈(𝑡)+𝜎𝑑𝑡_𝑥𝑋^∗(𝑡),𝜈^∗(𝑡)𝑝(𝑡)+𝜎_𝜈𝑋^∗(𝑡),𝜈^∗(𝑡)𝜈(𝑡)(𝑡),𝑝(0)𝑑𝑊=0. (25)

The following three lemmas contain estimates that will play a vital role in deriving the desired variational equation and the maximum principle for our control problem.

Lemma 6. Assume condition (i) of Theorem 3. Then, sup_{[]𝑡∈0,𝑇}𝔼𝑝(𝑡)²<∞. (26)

Proof. The solution of (25) is given by the formula 𝑝(𝑡)=^𝑡₀𝑆(𝑡−𝑠)𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)+𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)𝑑𝑠+^𝑡₀𝑆(𝑡−𝑠)𝜎_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)+𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)𝑑𝑊(𝑠). (27)
By using Minkowski's inequality (triangle inequality), Holder's inequality, Burkholder's inequality for stochastic convolution together with assumption (i), and Gronwall's inequality, we obtain easily sup_{[]𝑡∈0,𝑇}𝔼𝑝(𝑡)²≤𝐶 (28) for some constant 𝐶>0.

Lemma 7. Assuming condition (i) of Theorem 3, one has sup_{[]𝑡∈0,𝑇}𝔼𝑋_𝜀(𝑡)−𝑋^∗(𝑡)²=𝑂𝜀². (29)

Proof. Observe first from (8) that 𝑋_𝜀(𝑡)−𝑋^∗(𝑡)=^𝑡₀𝑆(𝑡−𝑠)𝑏𝑋_𝜀,𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈^∗(𝑠)𝑑𝑠+^𝑡₀𝑆(𝑡−𝑠)𝜎𝑋_𝜀,𝜈_𝜀(𝑠)−𝜎𝑋^∗(𝑠),𝜈^∗(𝑠)𝑑𝑊(𝑠). (30)
Hence, 𝔼𝑋_𝜀(𝑡)−𝑋^∗(𝑡)²≤2𝑀²𝑇𝔼^𝑡₀𝑏𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈^∗(𝑠)²𝑑𝑠+2𝑀²𝔼^𝑡₀𝜎𝑋_𝜀,𝜈_𝜀(𝑠)−𝜎𝑋^∗(𝑠),𝜈^∗(𝑠)²₂𝑑𝑠, (31) where 𝑀∶=sup_{𝑡∈[0,𝑇]}‖𝑆(𝑡)‖_{𝐿(𝐻,𝐻)}.
Secondly, from condition (i), we get 𝔼^𝑡₀𝑏𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈^∗(𝑠)²𝑑𝑠≤2𝔼^𝑡₀𝑏𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈_𝜀(𝑠)²𝑑𝑠+2𝔼^𝑡₀𝑏𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈^∗(𝑠)²𝑑𝑠=2𝔼^𝑡₀𝑏̃_𝑥(𝑠,𝜀)𝑋_𝜀(𝑠)−𝑋^∗(𝑠)²𝑑𝑠+2𝔼^𝑡₀𝛿_𝜀𝑏(𝑠)²𝑑𝑠≤2𝐶₁𝔼^𝑡₀𝑋_𝜀(𝑠)−𝑋^∗(𝑠)²𝑑𝑠+2𝐶₂𝜀², (32) where for 𝑦∈𝐻𝑏̃_𝑥()𝑦=𝑠,𝜀₁₀𝑏_𝑥𝑋^∗(𝑠)+𝜃𝑋_𝜀(𝑠)−𝑋^∗(𝑠),𝜈_𝜀(𝑠)𝑦𝛿𝑑𝜃,_𝜀𝑏(𝑠)=𝑏𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈^∗(𝑠), (33)𝐶₁ is a positive constant, and 𝐶₂ is another positive constant coming thanks to (i) from the following inequality: 𝔼^𝑇₀𝛿_𝜀𝑏(𝑠)²𝑑𝑠=𝔼^𝑇₀𝑏𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈^∗(𝑠)²𝑑𝑠=𝔼^𝑇₀¹₀𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)+𝜃𝜈_𝜀(𝑠)−𝜈^∗(𝑠)×𝜈_𝜀(𝑠)−𝜈^∗(𝑠)𝑑𝜃²𝑑𝑠≤𝐶₂𝜀². (34)
Similarly, 𝔼^𝑡₀𝜎𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−𝜎𝑋^∗(𝑠),𝜈^∗(𝑠)²𝑑𝑠≤2𝐶₃𝔼^𝑡₀𝑋_𝜀(𝑠)−𝑋^∗(𝑠)²𝑑𝑠+2𝐶₄𝜀², (35) for some positive constants 𝐶₃, 𝐶₄.
Finally, by applying (32), (35) in (30) and then using Gronwall's inequality, we find that 𝔼𝑋_𝜀(𝑡)−𝑋^∗(𝑡)²≤𝐶₅𝜀² (36) for some constant 𝐶₅>0 that depends in particular on 𝐶_𝑖, 𝑖=1,…,4, and 𝑀. Hence, the proof is complete.

Keeping the notations 𝑏̃_𝑥 and 𝛿_𝜀𝑏 used in the preceding proof, let us state the following lemma.

Lemma 8. Let 𝜂_𝜀(𝑡)=((𝑋_𝜀(𝑡)−𝑋^∗(𝑡))/𝜀)−𝑝(𝑡). Then, under condition (i) of Theorem 3, lim_𝜀→0+sup_{[]𝑡∈0,𝑇}𝔼𝜂_𝜀(𝑡)²=0. (37)

Proof. From the corresponding equations (4) and (25), we deduce that 𝜂_𝜀(𝑡)=^𝑡₀𝑆(1𝑡−𝑠)𝜀𝑏𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−𝑏𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠+)𝑑𝑠_𝑡0𝑆(1𝑡−𝑠)𝜀𝛿_𝜀𝑏(𝑠)−𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠+)𝑑𝑠_𝑡0𝑆(1𝑡−𝑠)𝜀𝜎𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−𝜎𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝜎_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠(𝑠)+)𝑑𝑊_𝑡0𝑆(1𝑡−𝑠)𝜀𝛿_𝜀𝜎(𝑠)−𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)𝑑𝑠=^𝑡₀𝑆(𝑡−𝑠)𝑏̃_𝑥(𝑠,𝜀)𝜂_𝜀(𝑠)+𝑏̃_𝑥(𝑠,𝜀)−𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)𝑑𝑠+^𝑡₀𝑆(1𝑡−𝑠)𝜀𝛿_𝜀𝑏(𝑠)−𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠+)𝑑𝑠_𝑡0𝑆(𝑡−𝑠)𝜎_𝑥(𝑠,𝜀)𝜂_𝜀(𝑠)+𝜎_𝑥(𝑠,𝜀)−𝜎_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠(𝑠)+)𝑑𝑊_𝑡0𝑆(1𝑡−𝑠)𝜀𝛿_𝜀𝜎(𝑠)−𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)𝑑𝑊(𝑠), (38) where 𝛿_𝜀𝜎(𝑠)=𝜎𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝜎𝑋^∗(𝑠),𝜈^∗(𝑠),𝜎_𝑥(𝑠,𝜀)𝑦=¹₀𝜎_𝑥𝑋^∗(𝑠)+𝜃𝑋_𝜀(𝑠)−𝑋^∗(𝑠),𝜈_𝜀(𝑠)×𝑦𝑑𝜃,𝑦∈𝐻. (39) Consequently, from (i) and as in the proof of Lemma 7, it follows that 𝔼𝜂_𝜀(𝑡)²≤𝐶₆^𝑡₀𝔼𝜂_𝜀(𝑠)²𝑑𝑠+𝜌(𝜀), (40) for all 𝑡∈[0,𝑇], where 𝜌(𝜀)=8𝑀𝑇𝔼^𝑇₀𝑏̃_𝑥(𝑠,𝜀)−𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)²𝑑𝑠+8𝑀𝔼^𝑇₀𝜎_𝑥(𝑠,𝜀)−𝜎_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)²₂𝑑𝑠+4𝑀𝑇𝔼^𝑇₀1𝜀𝛿_𝜀𝑏(𝑠)−𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)²𝑑𝑠+4𝑀𝔼^𝑇₀1𝜀𝛿_𝜀𝜎(𝑠)−𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)²₂𝑑𝑠. (41)
But (i), (28), and the dominated convergence theorem give 𝔼^𝑇₀𝑏̃_𝑥(𝑠,𝜀)−𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)²𝑑𝑠=𝔼^𝑇₀¹₀𝑏_𝑥𝑋^∗(𝑠)+𝜃𝑋_𝜀(𝑠)−𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)𝑑𝜃²≤𝑑𝑠_𝑇010𝔼𝑏_𝑥𝑋^∗(𝑠)+𝜃𝑋_𝜀(𝑠)−𝑋^∗(𝑠),𝜈_𝜀(𝑠)−𝑏_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)×𝑝(𝑠)²𝑑𝜃𝑑𝑠⟶0, (42) as 𝜀→0⁺.
Similarly, we have 𝔼^𝑇₀𝜎_𝑥(𝑠,𝜀)−𝜎_𝑥𝑋^∗(𝑠)𝑝(𝑠)²₂𝑑𝑠⟶0, (43) as 𝜀→0⁺.
On the other hand, as done for (34), 𝔼^𝑇₀1𝜀𝛿_𝜀𝑏(𝑠)−𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)²≤𝑑𝑠_𝑇010𝔼𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)+𝜃𝜈_𝜀(𝑠)−𝜈^∗(𝑠)−𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)²𝑑𝜃𝑑𝑠⟶0 (44) if 𝜀→0⁺, by using (i) and the dominated convergence theorem. Similarly, 𝔼^𝑇₀1𝜀𝛿_𝜀𝜎(𝑠)−𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)²₂𝑑𝑠⟶0 (45) if 𝜀→0⁺.
Finally applying (43)–(45) in (41) shows that 𝜌(𝜀)⟶0,as𝜀⟶0⁺. (46)
Hence, from (40) and Gronwall's inequality, we obtain sup_{[]𝑡∈0,𝑇}𝔼𝜂_𝜀(𝑡)²⟶0, (47) as 𝜀→0⁺.

The following theorem contains our main variational equation, which is one of the main tools needed for deriving the maximum principle stated in Theorem 3.

Theorem 9. Suppose that (i) and (ii) in Theorem 3 hold. For each 𝜀>0, we have 𝐽𝜈_𝜀(⋅)−𝐽𝜈^∗(⋅)=𝜀𝔼𝜙_𝑥𝑋^∗(𝑇)𝑝(𝑇)+𝜀𝔼^𝑇₀ℓ_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)×𝑝(𝑠)𝑑𝑠+𝔼^𝑇₀ℓ𝑋^∗(𝑠),𝜈_𝜀(𝑠)−ℓ𝑋^∗(𝑠),𝜈^∗(𝑠)(𝜀).𝑑𝑠+𝑜 (48)

Proof. We can write 𝐽(𝜈_𝜀(⋅))−𝐽(𝜈^∗(⋅)) as 𝐽𝜈_𝜀(⋅)−𝐽𝜈^∗(⋅)=𝐼₁(𝜀)+𝐼₂(𝜀), (49) with 𝐼₁(𝜀)=𝔼𝜙𝑋_𝜀(𝑇)−𝜙𝑋^∗(𝑇),𝐼₂(𝜀)=𝔼^𝑇₀ℓ𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−ℓ𝑋^∗(𝑠),𝜈^∗(𝑠)𝑑𝑠. (50)
Note that with the help of our assumptions and by making use of Lemmas 8, 7, and 6 and the dominated convergence theorem, we deduce that 1𝜀𝐼₁(𝜀)=1𝜀𝔼¹₀𝜙_𝑥𝑋^∗(𝑇)+𝜃𝑋_𝜀(𝑇)−𝑋^∗(𝑇)𝑋_𝜀(𝑇)−𝑋^∗(𝑇)𝑑𝜃=𝔼¹₀𝜙_𝑥𝑋^∗(𝑇)+𝜃𝑋_𝜀(𝑇)−𝑋^∗(𝑇)×𝑝(𝑇)+𝜂_𝜀(𝑇)𝜙𝑑𝜃⟶𝔼_𝑥𝑋^∗(𝑇)𝑝(𝑇),as𝜀⟶0⁺. (51) Hence, 𝐼₁(𝜀)=𝜀𝔼𝜙_𝑥𝑋^∗(𝑇)𝑝(𝑇)+𝑜(𝜀). (52) Similarly, 1𝜀𝐼₂(𝜀)=1𝜀𝔼^𝑇₀ℓ𝑋_𝜀(𝑠),𝜈_𝜀(𝑠)−ℓ𝑋^∗(𝑠),𝜈_𝜀(𝑠)+1𝑑𝑠𝜀𝔼^𝑇₀ℓ𝑋^∗(𝑠),𝜈_𝜀(𝑠)−ℓ𝑋^∗(𝑠),𝜈^∗(𝑠)𝑑𝑠=𝔼^𝑇₀¹₀ℓ_𝑥𝑋^∗(𝑠)+𝜃𝑋_𝜀(𝑠)−𝑋^∗(𝑠),𝜈_𝜀(𝑠)×𝑝(𝑠)+𝜂_𝜀(𝑠)+1𝑑𝜃𝑑𝑠𝜀𝔼^𝑇₀ℓ𝑋^∗(𝑠),𝜈_𝜀(𝑠)−ℓ𝑋^∗(𝑠),𝜈^∗(𝑠)𝑑𝑠. (53)
On the other hand, applying Lemmas 8, 7, and 6 and using the continuity and boundedness of ℓ_𝑥 in (i), (ii) and the dominated convergence theorem imply that 𝔼^𝑇₀¹₀ℓ_𝑥𝑋^∗(𝑠)+𝜃𝑋_𝜀(𝑠)−𝑋^∗(𝑠),𝜈^∗(𝑠)+𝜀𝜈(𝑠)𝑝(𝑠)+𝜂_𝜀(𝑠)𝑑𝜃𝑑𝑠⟶𝔼^𝑇₀ℓ_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)𝑑𝑠. (54) In particular, we obtain 𝐼₂(𝜀)=𝜀𝔼^𝑇₀ℓ_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)𝑑𝑠+𝔼^𝑇₀ℓ𝑋^∗(𝑠),𝜈_𝜀(𝑠)−ℓ𝑋^∗(𝑠),𝜈^∗(𝑠)𝑑𝑠+𝑜(𝜀). (55)
As a result, the theorem follows from (49), (52) and (55).

Let us next introduce an important variational inequality.

Lemma 10. Let hypotheses (i), (ii) in Theorem 3 hold. Let (𝑌^∗,𝑍^∗)≡(𝑌^𝜈∗(⋅),𝑍^𝜈∗(⋅)) be the solution of BSEE (10) corresponding to the optimal pair (𝑋^∗,𝜈^∗(⋅)). Then, 𝜀𝔼𝑌^∗(𝑇),𝑝(𝑇)+𝜀𝔼^𝑇₀ℓ_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)𝑑𝑠+𝔼^𝑇₀𝛿_𝜀ℋ(𝑠)−𝛿_𝜀𝑏(𝑠),𝑌^∗(𝑠−𝛿_𝜀)𝜎(𝑠),𝑍^∗(𝑠)₂𝑑𝑠≥𝑜(𝜀), (56) where 𝛿_𝜀ℋ(𝑠)=ℋ𝑋^∗(𝑠),𝜈_𝜀(𝑠),𝑌^∗(𝑠),𝑍^∗(𝑠)−ℋ𝑋^∗(𝑠),𝜈^∗(𝑠),𝑌^∗(𝑠),𝑍^∗(𝑠). (57)

Proof. Since 𝜈^∗(⋅) is an optimal control, then 𝐽(𝜈_𝜀(⋅))−𝐽(𝜈^∗(⋅))≥0. Hence, the result follows from (48) and (9).

The following duality relation between (25) and (10) is also needed in order to establish the proof of Theorem 3.

Lemma 11. Under hypothesis (i) in Theorem 3, we have 𝔼𝑌^∗(𝑇),𝑝(𝑇)=−𝔼^𝑇₀ℓ_𝑥𝑋^∗(𝑠),𝜈^∗(𝑠)𝑝(𝑠)𝑑𝑠+𝔼^𝑇₀𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠),𝑌^∗(𝑠)𝑑𝑠+𝔼^𝑇₀𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠),𝑍^∗(𝑠)₂𝑑𝑠. (58)

Proof. The proof is done by using Yosida approximation of the operator 𝐴 and Itô's formula for the resulting SDEs and can be gleaned directly from the proof of Theorem 2.1 in [18].

We are now ready to establish (or complete in particular) the proof of Theorem 3.

Proof of Theorem 3. Recall the BSEE (10): −𝑑𝑌^𝜈(⋅)(𝑡)=𝐴^∗𝑌^𝜈(⋅)(𝑡)+∇_𝑥ℋ𝑋^𝜈(⋅)(𝑡),𝜈(𝑡),𝑌^𝜈(⋅)(𝑡),𝑍^𝜈(⋅)(𝑡)𝑑𝑡−𝑍^𝜈(⋅)(𝑡)(𝑡)𝑌𝑑𝑊,0≤𝑡<𝑇,_𝜈(⋅)(𝑇)=∇𝜙𝑋^𝜈(⋅)(𝑇). (59) From Theorem 2, there exists a unique solution (𝑌^∗,𝑍^∗) to it. Thereby, it remains to prove (15).
Applying (56) and (58) gives 𝔼^𝑇₀𝛿_𝜀ℋ(𝑠)+𝜀𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)−𝛿_𝜀𝑏(𝑠),𝑌^∗(𝑠+)𝜀𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)−𝛿_𝜀𝜎(𝑠),𝑍^∗(𝑠)₂𝑑𝑠≥𝑜(𝜀). (60) But, as done for (44), by using the continuity and boundedness of 𝑏_𝜈 in assumption (i) and the dominated convergence theorem, one can find that 1𝜀𝔼^𝑇₀𝜀𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)−𝛿_𝜀𝑏(𝑠),𝑌^∗(𝑠)𝑑𝑠=−𝔼^𝑇₀𝑌^∗(𝑠),¹₀𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)+𝜃𝜈_𝜀(𝑠)−𝜈^∗(𝑠)−𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)×𝜈(𝑠)𝑑𝜃𝑑𝑠⟶0, (61) as 𝜀→0⁺. This means that 𝔼^𝑇₀𝜀𝑏_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)−𝛿_𝜀𝑏(𝑠),𝑌^∗(𝑠)𝑑𝑠=𝑜(𝜀). (62) Similarly, 𝔼^𝑇₀𝜀𝜎_𝜈𝑋^∗(𝑠),𝜈^∗(𝑠)𝜈(𝑠)−𝛿_𝜀𝜎(𝑠),𝑍^∗(𝑠)₂𝑑𝑠=𝑜(𝜀). (63)
Now, by applying (62) and (63) in (60), we deduce that 𝔼^𝑇₀𝛿_𝜀ℋ(𝑠)𝑑𝑠≥𝑜(𝜀). (64) Therefore, by dividing (64) by 𝜀 and letting 𝜀→0⁺, the following inequality holds: 𝔼^𝑇₀∇_𝜈ℋ𝑡,𝑋^∗(𝑡),𝜈^∗(𝑡),𝑌^∗(𝑡),𝑍^∗(𝑡),𝜈(𝑡)_𝒪𝑑𝑡≥0. (65)
Finally, (14) follows by arguing, if necessary, as in [19, page 280], for instance.

Acknowledgments

The author would like to thank the associate editor and anonymous referee(s) for their remarks and also for pointing out the recent work of Fuhrman et al., [12]. This work is supported by the Science College Research Center at Qassim University, Project no. SR-D-012-1610.