Maximum Principle for Stochastic Recursive Optimal Control Problems Involving Impulse Controls

Abstract

We consider a stochastic recursive optimal control problem in which the control variable has two components: the regular control and the impulse control. The control variable does not enter the diffusion coefficient, and the domain of the regular controls is not necessarily convex. We establish necessary optimality conditions, of the Pontryagin maximum principle type, for this stochastic optimal control problem. Sufficient optimality conditions are also given. The optimal control is obtained for an example of linear quadratic optimization problem to illustrate the applications of the theoretical results.

1. Introduction

The nonlinear backward stochastic differential equations (BSDEs for short) were first introduced by Pardoux and Peng [1]. Independently, Duffie and Epstein [2] introduced BSDEs under economic background. In [2], they presented a stochastic recursive utility which is an extension of the standard additive utility with the instantaneous utility depending not only on the instantaneous consumption rate but also on the future utility. Actually, it corresponds to the solution of a particular BSDE whose generator does not depend on the variable 𝑧. And then, El Karoui et al. [3] gave the formulation of recursive utilities from the BSDE point of view. The problem that the cost function of the control system is described by the solution of BSDE is called the stochastic recursive optimal control problem. In this case, the control systems become forward-backward stochastic differential equations (FBSDEs).

One fundamental research direction for optimal control problem is to establish the necessary optimality conditions—Pontryagin maximum principle. Stochastic maximum principle for forward, backward, and forward-backward systems has been studied by many authors, including Peng [4, 5], Tang and Li [6], Wang and Yu [7], Wu [8], and Xu [9] for full information and Huang et al. [10], Wang and Wu [11], Wang and Yu [12], and Wu [13] for partial information case. However, in these papers, there are only regular controls in the control systems and impulse controls are not included.

Stochastic impulse control problems have received considerable research attention in recent years due to wide applicability in a number of different areas, especially in mathematical finance; see, for example, [14–17]. In most cases, the optimal impulse control problem was studied through dynamic programming principle. It was shown in particular that the value function is a solution of some quasi-variational inequalities.

The first result in stochastic maximum principle for singular control problem was obtained by Cadenillas and Haussmann [18], in which linear dynamics, convex cost criterion, and convex state constraint are assumed. Bahlali and Chala [19] generalized [18] to the nonlinear dynamics case with a convex state constraint. Bahlali and Mezerdi [20] considered a stochastic singular control problem in which the control system is governed by a stochastic differential equation where the regular control enters the diffusion coefficient and the control domain is not necessarily convex. The stochastic maximum principle was obtained with the approach developed by Peng [4]. Dufour and Miller [21] studied a stochastic singular control problem in which the admissible control is of bounded variation. It is worth pointing out that the control systems in these works are stochastic differential equations with singular control, and few examples are given to illustrate the theoretical results. Wu and Zhang [22] were the first to study stochastic optimal control problems of forward-backward systems involving impulse controls, and they obtained both the maximum principle and sufficient optimality conditions for the optimal control problem.

In this paper, we continue to study stochastic optimal control problem involving impulse controls, in which the control system is described by a forward-backward stochastic differential equation and the control variable consists of regular control and impulse control. Different from [22], it is assumed in this paper that the domain of the regular controls is not necessarily convex and the control variable does not enter the diffusion coefficient. Thus the result of this paper and that of [22] do not contain each other. We obtain the stochastic maximum principle by using a spike variation on the regular part of the control and a convex perturbation on the impulsive one. Sufficient optimality conditions are also obtained which can help to find the optimal control in applications.

The rest of this paper is organized as follows. In Section 2 we give some preliminary results and the formulation of our stochastic optimal control problem. In Section 3 we obtain the maximum principle for our stochastic optimal control problem. Sufficient optimality conditions for the optimal control problem is established in Section 4, and an example of linear quadratic optimization problem is also given to illustrate the applications of our theoretical results.

2. Formulation of the Stochastic Optimal Control Problem

Firstly we introduce some notations. Let (Ω,ℱ,ℙ) be a probability space and 𝔼 the expectation with respect to ℙ. Let 𝑇 be a finite time horizon and ℱ_𝑡 the natural filtration of a 𝑑-dimensional standard Brownian motion {𝐵_𝑡,0≤𝑡≤𝑇} augmented by the ℙ-null sets of ℱ. For 𝑛∈ℕ and 𝑝>1, denote by 𝑆^𝑝(ℝ^𝑛) the set of 𝑛-dimensional adapted processes {𝜑_𝑡,0≤𝑡≤𝑇} such that 𝔼[sup_{0≤𝑡≤𝑇}|𝜑_𝑡|^𝑝]<∞, and denote by 𝐻^𝑝(ℝ^𝑛) the set of 𝑛-dimensional adapted processes {𝜓_𝑡,0≤𝑡≤𝑇} such that 𝔼[(∫^𝑇0|𝜓_𝑡|²𝑑𝑡)^𝑝/2]<∞.

Let 𝑈 be a nonempty subset of ℝ^𝑘 and 𝐾 a nonempty convex subset of ℝ^𝑛. Let {𝜏_𝑖} be a given sequence of increasing ℱ_𝑡-stopping times such that 𝜏_𝑖↑+∞ as 𝑖→∞. We denote by ℐ the class of right continuous processes 𝜂(⋅)=∑_𝑖≥1𝜂_𝑖𝟙_{[𝜏𝑖,𝑇]}(⋅) such that each 𝜂_𝑖 is an ℱ_𝜏𝑖-measurable random variable. It is worth noting that the assumption 𝜏_𝑖↑+∞ implies that at most finitely many impulses may occur on [0,𝑇]. Denote by 𝒰 the class of adapted processes 𝑣∶[0,𝑇]×Ω→𝑈 such that 𝔼[sup_{0≤𝑡≤𝑇}|𝑣_𝑡|³]<∞, and denote by 𝒦 the class of 𝐾-valued impulse processes 𝜂(⋅)∈ℐ such that 𝔼[(∑_𝑖≥1|𝜂_𝑖|)³]<∞. We call 𝒜∶=𝒰×𝒦 the admissible control set. In what follows, for a continuous function 𝑙(⋅), the integration ∫^𝑇0𝑙(𝑡)𝑑𝜂_𝑡 is understood as follows: ^𝑇0𝑙(𝑡)𝑑𝜂_𝑡=_{0≤𝜏𝑖≤𝑇}𝑙𝜏_𝑖𝜂_𝑖.(2.1)

Given 𝜂(⋅)∈ℐ and 𝑥∈ℝ^𝑛, we consider the following SDE with impulses: 𝑑𝑋_𝑡=𝑏𝑡,𝑋_𝑡𝑑𝑡+𝜎𝑡,𝑋_𝑡𝑑𝐵_𝑡+𝐶_𝑡𝑑𝜂_𝑡,𝑋₀=𝑥,(2.2) where 𝑏∶[0,𝑇]×Ω×ℝ^𝑛→ℝ^𝑛, 𝜎∶[0,𝑇]×Ω×ℝ^𝑛→ℝ^𝑛×𝑑, and 𝐶∶[0,𝑇]→ℝ^𝑛×𝑛 are measurable mappings. Similar to [22, Proposition 2.1], we have the following.

Proposition 2.1. Let 𝐶 be continuous and 𝑏, 𝜎 uniformly Lipschitz in 𝑥. Assume that 𝑏(⋅,0)∈𝐻^𝑝(ℝ^𝑛), 𝜎(⋅,0)∈𝐻^𝑝(ℝ^𝑛×𝑑), and 𝔼[(∑_𝑖≥1|𝜂_𝑖|)^𝑝]<∞ for some 𝑝≥2. Then SDE (2.2) admits a unique solution 𝑋(⋅)∈𝑆^𝑝(ℝ^𝑛).

For 𝜂(⋅)∈ℐ, let us consider the following BSDE with impulses: 𝑑𝑌_𝑡=−𝑓𝑡,𝑌_𝑡,𝑍_𝑡𝑑𝑡+𝑍_𝑡𝑑𝐵_𝑡−𝐷_𝑡𝑑𝜂_𝑡,𝑌_𝑇=𝜁,(2.3) where 𝜁∈ℱ_𝑇, 𝑓∶[0,𝑇]×Ω×ℝ^𝑚×ℝ^𝑚×𝑑→ℝ^𝑚 and 𝐷∶[0,𝑇]→ℝ^𝑚×𝑛 are measurable mappings. Similar to [22, Proposition 2.2], we have the following.

Proposition 2.2. Let 𝐷 be continuous and 𝑓 Lipschitz in (𝑦,𝑧). Assume that 𝔼|𝜁|^𝑝<∞, 𝔼[(∑_𝑖≥1|𝜂_𝑖|)^𝑝]<∞, and 𝑓(⋅,0,0)∈𝐻^𝑝(ℝ^𝑚) for some 𝑝≥2. Then BSDE (2.3) admits a unique solution (𝑌(⋅),𝑍(⋅))∈𝑆^𝑝(ℝ^𝑚)×𝐻^𝑝(ℝ^𝑚×𝑑).

The control system of our stochastic optimal control problem is subject to the following FBSDE: 𝑑𝑑𝑥𝑦^{𝑡𝑣,𝜂}_{𝑡𝑣,𝜂}=𝑏𝑡,𝑥^{𝑡𝑣,𝜂},𝑣_𝑡𝑑𝑡+𝜎𝑡,𝑥^{𝑡𝑣,𝜂}𝑑𝐵_𝑡+𝐶_𝑡𝑑𝜂_𝑡,𝑥_0𝑣,𝜂=−𝑓𝑡,𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑧^{𝑡𝑣,𝜂},𝑣_𝑡𝑑𝑡+𝑧^{𝑡𝑣,𝜂}𝑑𝐵_𝑡−𝐷_𝑡𝑑𝜂_𝑡,=𝑎∈ℝ^𝑛,𝑦^{𝑇𝑣,𝜂}=𝑔𝑥^{𝑇𝑣,𝜂},(2.4) where 𝑏∶[0,𝑇]×ℝ^𝑛×𝑈→ℝ^𝑛, 𝜎∶[0,𝑇]×ℝ^𝑛→ℝ^𝑛×𝑑, 𝑓∶[0,𝑇]×ℝ^𝑛×ℝ^𝑚×ℝ^𝑚×𝑑×𝑈→ℝ^𝑚, 𝑔∶ℝ^𝑛→ℝ^𝑚 are measurable mappings, and 𝐶∶[0,𝑇]→ℝ^𝑛×𝑛, 𝐷∶[0,𝑇]→ℝ^𝑚×𝑛 are continuous functions. The objective is to minimize the following cost functional over the class 𝒜: 𝐽(𝑣(⋅),𝜂(⋅))=𝔼𝜙𝑥^{𝑇𝑣,𝜂}+𝛾𝑦^0𝑣,𝜂+^𝑇0ℎ𝑡,𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑣_𝑡𝑑𝑡+_𝑖≥1𝑙𝜏_𝑖,𝜂_𝑖,(2.5) where 𝜙∶ℝ^𝑛→ℝ, 𝛾∶ℝ^𝑚→ℝ, ℎ∶[0,𝑇]×ℝ^𝑛×ℝ^𝑚×𝑈→ℝ, and 𝑙∶[0,𝑇]×ℝ^𝑛→ℝ are measurable mappings.

In what follows we assume the following.(𝐻1)𝑏, 𝜎, 𝑓, 𝑔 are continuous, and they are continuously differentiable in (𝑥,𝑦,𝑧), with derivatives continuous and uniformly bounded. Moreover, assume that 𝑏 and 𝑓 have linear growth in (𝑥,𝑦,𝑧,𝑣).(𝐻2)𝜙, 𝛾, ℎ, 𝑙 are continuous, and they are continuously differentiable in (𝑥,𝑦,𝜂), with derivatives continuous and bounded by 𝑐(1+|𝑥|), 𝑐(1+|𝑦|), 𝑐(1+|𝑥|+|𝑦|+|𝑣|), and 𝑐(1+|𝜂|), respectively. Moreover, we assume |ℎ(𝑡,0,0,𝑣)|≤𝑐(1+|𝑣|³) for any (𝑡,𝑣).

From Propositions 2.1 and 2.2, it follows that FBSDE (2.4) admits a unique solution (𝑥^𝑣,𝜂(⋅),𝑦^𝑣,𝜂(⋅),𝑧^𝑣,𝜂(⋅))∈𝑆³(ℝ^𝑛)×𝑆³(ℝ^𝑚)×𝐻³(ℝ^𝑚×𝑑) for any (𝑣(⋅),𝜂(⋅))∈𝒜, and the functional 𝐽 is well defined.

3. Stochastic Maximum Principle for the Optimal Control Problem

Let (𝑢(⋅),𝜉(⋅)=∑_𝑖≥1𝜉_𝑖𝟙_{[𝜏𝑖,𝑇]}(⋅))∈𝒜 be an optimal control and (𝑥^𝑢,𝜉(⋅),𝑦^𝑢,𝜉(⋅),𝑧^𝑢,𝜉(⋅)) the corresponding trajectory. We introduce the spike variation with respect to 𝑢(⋅) as follows: 𝑢^𝜀𝑡=𝑢𝑣,if𝜏≤𝑡≤𝜏+𝜀,_𝑡,otherwise,(3.1) where 𝜏∈[0,𝑇) is an arbitrarily fixed time, 𝜀>0 is a sufficiently small constant, and 𝑣 is an arbitrary 𝑈-valued ℱ_𝜏-measurable random variable such that 𝔼|𝑣|³<∞. Let 𝜂(⋅)∈ℐ be such that 𝜉(⋅)+𝜂(⋅)∈𝒦. Then it is easy to check that 𝜉^𝜀(⋅)∶=𝜉(⋅)+𝜀𝜂(⋅), 0≤𝜀≤1 is also an element of 𝒦. Let us denote by (𝑥^𝜀(⋅),𝑦^𝜀(⋅),𝑧^𝜀(⋅)) the trajectory associated with (𝑢^𝜀(⋅),𝜉^𝜀(⋅)). For convenience, denote 𝜑(𝑡)=𝜑(𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑢_𝑡), 𝜑(𝑢^𝜀𝑡)=𝜑(𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑢^𝜀𝑡) for 𝜑=𝑏,𝜎,𝑓,ℎ,𝑏_𝑥,𝜎_𝑥,𝑓_𝑥,𝑓_𝑦,𝑓_𝑧,ℎ_𝑥,ℎ_𝑦. In what follows, we use 𝑐 to denote a positive constant which can be different from line to line.

Let us introduce the following FBSDE (called the variational equation): 𝑑𝑑𝑥𝑦^1𝑡_1𝑡=𝑏_𝑥(𝑡)𝑥^1𝑡+𝑏𝑢^𝜀𝑡−𝑏(𝑡)𝑑𝑡+𝜎_𝑥(𝑡)𝑥^1𝑡𝑑𝐵_𝑡+𝜀𝐶_𝑡𝑑𝜂_𝑡,𝑥₁₀=−𝑓_𝑥(𝑡)𝑥^1𝑡+𝑓_𝑦(𝑡)𝑦^1𝑡+𝑓_𝑧(𝑡)𝑧^1𝑡+𝑓𝑢^𝜀𝑡−𝑓(𝑡)𝑑𝑡+𝑧^1𝑡𝑑𝐵_𝑡−𝜀𝐷_𝑡𝑑𝜂_𝑡,=0,𝑦^1𝑇=𝑔_𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇.(3.2) By Propositions 2.1 and 2.2, FBSDE (3.2) admits a unique solution (𝑥¹(⋅),𝑦¹(⋅),𝑧¹(⋅))∈𝑆³(ℝ^𝑛)×𝑆³(ℝ^𝑚)×𝐻³(ℝ^𝑚×𝑑).

Similar to [9, Lemma 1], we can easily obtain the following.

Lemma 3.1. We have sup_{0≤𝑡≤𝑇}𝔼||𝑥^1𝑡||³+sup_{0≤𝑡≤𝑇}𝔼||𝑦^1𝑡||³+𝔼^𝑇0|𝑧^1𝑡|²𝑑𝑡^3/2≤𝑐𝜀³.(3.3)

We proceed to give the following lemma.

Lemma 3.2. The following estimations hold: ssuupp_{0≤𝑡≤𝑇0≤𝑡≤𝑇}𝔼||𝑥^𝜀𝑡−𝑥^{𝑡𝑢,𝜉}−𝑥^1𝑡||²≤𝐶^𝜀𝜀²,(3.4)𝔼𝔼||𝑦^𝜀𝑡−𝑦^{𝑡𝑢,𝜉}−𝑦^1𝑡||²≤𝐶^𝜀𝜀²,(3.5)_𝑇0||𝑧^𝜀𝑡−𝑧^{𝑡𝑢,𝜉}−𝑧^1𝑡||²𝑑𝑡≤𝐶_𝜀𝜀²,(3.6) where 𝐶_𝜀→0 as 𝜀→0.

Proof. It is easy to check that 𝑥^𝜀𝑡−𝑥^{𝑡𝑢,𝜉}−𝑥^1𝑡=+^𝑡0𝐶^𝜀𝑠𝑥^𝜀𝑠−𝑥^{𝑠𝑢,𝜉}−𝑥^1𝑠+𝐴^𝜀𝑠𝑑𝑠_𝑡0𝐷^𝜀𝑠𝑥^𝜀𝑠−𝑥^{𝑠𝑢,𝜉}−𝑥^1𝑠+𝐵^𝜀𝑠𝑑𝐵^𝑠,(3.7) where 𝐵𝐴^𝜀𝑠_𝜀𝑠=¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑢^𝜀𝑠−𝑏^𝑥(𝑠)𝑑𝜆𝑥^1𝑠,𝐶_𝜀𝑠=¹⁰𝜎^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠−𝜎^𝑥(𝑠)𝑑𝜆𝑥^1𝑠,𝐷=¹⁰𝑏_𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝑥^1𝑠+𝜆𝑥^𝜀𝑠−𝑥^{𝑠𝑢,𝜉}−𝑥^1𝑠,𝑢^𝜀𝑠𝑑𝜆,_𝜀𝑠=¹⁰𝜎_𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝑥^1𝑠+𝜆𝑥^𝜀𝑠−𝑥^{𝑠𝑢,𝜉}−𝑥^1𝑠𝑑𝜆.(3.8) Since 𝑏_𝑥, 𝜎_𝑥 are uniformly bounded, we have sup_{0≤𝑠≤𝑇}(|𝐶^𝜀𝑠|+|𝐷^𝜀𝑠|)≤𝑐. Hence, if we can obtain sup_{0≤𝑡≤𝑇}𝔼^𝑡0𝐴^𝜀𝑠𝑑𝑠+^𝑡0𝐵^𝜀𝑠𝑑𝐵_𝑠²≤𝐶_𝜀𝜀²,(3.9) then the estimation (3.4) can be obtained from Gronwall's lemma and (3.7). Let us take the 𝐴^𝜀 term for example. By the definition of 𝑢^𝜀 and Hölder's inequality, we have sup_{0≤𝑡≤𝑇}𝔼^𝑡0𝐴^𝜀𝑠𝑑𝑠²≤2𝔼^𝑇0||||¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑢_𝑠−𝑏^𝑥(𝑠)𝑑𝜆𝑥^1𝑠||||𝑑𝑠²+2𝔼^{𝜏𝜏+𝜀}||||¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑣−𝑏_𝑥(𝑠)𝑑𝜆𝑥^1𝑠||||𝑑𝑠²=∶2𝐼+2𝐼𝐼.(3.10) From Hölder's inequality, Lemma 3.1, and the dominated convergence theorem, it follows that 𝐼≤𝑇𝔼^𝑇0||||¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑢_𝑠−𝑏^𝑥(𝑠)𝑑𝜆𝑥^1𝑠||||²≤𝑇_𝑇0𝑑𝑠𝔼||𝑥^1𝑠||³≤𝑇_5/3^2/3𝔼||||¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑢_𝑠−𝑏^𝑥(𝑠)𝑑𝜆||||⁶^1/3𝑑𝑠sup_{0≤𝑠≤𝑇}𝔼||𝑥^1𝑠||³^2/3^𝑇0𝔼||||¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑢_𝑠−𝑏^𝑥(𝑠)𝑑𝜆||||⁶𝑑𝑠^1/3≤𝐶_𝜀𝜀².(3.11) Since 𝑏_𝑥 is uniformly bounded, by Lemma 3.1 we get 𝐼𝐼≤𝜀^{𝜏𝜏+𝜀}𝔼||||¹⁰𝑏^𝑥𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑣−𝑏_𝑥(𝑠)𝑑𝜆𝑥^1𝑠||||²≤𝑐𝜀₂𝑑𝑠sup_{0≤𝑠≤𝑇}𝔼||𝑥^1𝑠||²≤𝑐𝜀⁴.(3.12) Thus we obtain sup_{0≤𝑡≤𝑇}𝔼[∫^𝑡0𝐴^𝜀𝑠𝑑𝑠]²≤𝐶_𝜀𝜀². In the same way we can get sup_{0≤𝑡≤𝑇}𝔼^𝑡0𝐵^𝜀𝑠𝑑𝐵_𝑠²≤𝐶_𝜀𝜀².(3.13) Hence, the estimation (3.4) is proved.
Now we prove (3.5) and (3.6). Set 𝑋^𝜀𝑠Π=_𝜀𝑠𝑥^𝜀𝑠−𝑥^{𝑠𝑢,𝜉}−𝑥^1𝑠,𝑌^𝜀𝑠=𝑦^𝜀𝑠−𝑦^{𝑠𝑢,𝜉}−𝑦^1𝑠,𝑍^𝜀𝑠=𝑧^𝜀𝑠−𝑧^{𝑠𝑢,𝜉}−𝑧^1𝑠,=𝑠,Λ𝑥^{𝑠𝑢,𝜉}_𝜀𝑠+𝑥^1𝑠+𝜆𝑋^𝜀𝑠,𝑦^{𝑠𝑢,𝜉}+𝑦^1𝑠+𝜆𝑌^𝜀𝑠,𝑧^{𝑠𝑢,𝜉}+𝑧^1𝑠+𝜆𝑍^𝜀𝑠,𝑢^𝜀𝑠,=𝑠,𝑥^{𝑠𝑢,𝜉}+𝜆𝑥^1𝑠,𝑦^{𝑠𝑢,𝜉}+𝜆𝑦^1𝑠,𝑧^{𝑠𝑢,𝜉}+𝜆𝑧^1𝑠,𝑢^𝜀𝑠.(3.14) It is easy to obtain 𝑌^𝜀𝑡=𝑔𝑥^𝜀𝑇−𝑔𝑥^{𝑇𝑢,𝜉}−𝑔^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇−^𝑇𝑡𝑍^𝜀𝑠𝑑𝐵_𝑠++^𝑇𝑡𝐸^𝑠1,𝜀𝑥^1𝑠+𝐸^𝑠2,𝜀𝑦^1𝑠+𝐸^𝑠3,𝜀𝑧^1𝑠𝑑𝑠_𝑇𝑡𝐹^𝑠1,𝜀𝑋^𝜀𝑠+𝐹^𝑠2,𝜀𝑌^𝜀𝑠+𝐹^𝑠3,𝜀𝑍^𝜀𝑠𝑑𝑠,(3.15) where 𝐸𝐸^𝑠1,𝜀=¹⁰𝑓_𝑥Λ^𝜀𝑠−𝑓_𝑥(𝑠)𝑑𝜆,𝐸^𝑠2,𝜀=¹⁰𝑓_𝑦Λ^𝜀𝑠−𝑓_𝑦(𝑠)𝑑𝜆,_𝑠3,𝜀𝐹=¹⁰𝑓_𝑧Λ^𝜀𝑠−𝑓_𝑧(𝑠)𝑑𝜆,𝐹^𝑠1,𝜀=¹⁰𝑓_𝑥Π^𝜀𝑠𝑑𝜆,_𝑠2,𝜀=¹⁰𝑓_𝑦Π^𝜀𝑠𝑑𝜆,𝐹^𝑠3,𝜀=¹⁰𝑓_𝑧Π^𝜀𝑠𝑑𝜆.(3.16) We have 𝑔𝑥^𝜀𝑇−𝑔𝑥^{𝑇𝑢,𝜉}−𝑔^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇==𝑔₁₀𝑥^𝜀𝑇−𝑔𝑥^{𝑇𝑢,𝜉}+𝑥^1𝑇+𝑔𝑥^{𝑇𝑢,𝜉}+𝑥^1𝑇−𝑔𝑥^{𝑇𝑢,𝜉}−𝑔^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇𝑔_𝑥𝑥^{𝑇𝑢,𝜉}+𝑥^1𝑇+𝜆𝑋^𝜀𝑇𝑑𝜆𝑋^𝜀𝑇+¹⁰𝑔^𝑥𝑥^{𝑇𝑢,𝜉}+𝜆𝑥^1𝑇−𝑔^𝑥𝑥^{𝑇𝑢,𝜉}𝑑𝜆𝑥^1𝑇=∶𝐼+𝐼𝐼.(3.17) Since 𝑔_𝑥 is uniformly bounded, it follows from (3.4) that 𝔼|𝐼|²≤𝑐𝔼|𝑋^𝜀𝑇|²≤𝐶_𝜀𝜀². Since 𝑔_𝑥 is continuous and uniformly bounded, from Lemma 3.1 and the dominated convergence theorem it follows that 𝔼||𝐼𝐼||²≤sup_{0≤𝑡≤𝑇}𝔼||𝑥^1𝑡||³^2/3𝔼||||¹⁰𝑔^𝑥𝑥^{𝑇𝑢,𝜉}+𝜆𝑥^1𝑇−𝑔^𝑥𝑥^{𝑇𝑢,𝜉}𝑑𝜆||||⁶^1/3≤𝐶_𝜀𝜀².(3.18) Consequently, 𝔼|||𝑔𝑥^𝜀𝑇−𝑔𝑥^{𝑇𝑢,𝜉}−𝑔^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇|||²≤2𝔼||𝐼||²+2𝔼||𝐼𝐼||²≤𝐶_𝜀𝜀².(3.19) From Lemma 3.1 and the dominated convergence theorem, it follows that sup_{0≤𝑡≤𝑇}𝔼^𝑇𝑡𝐸^𝑠1,𝜀𝑥^1𝑠+𝐸^𝑠2,𝜀𝑦^1𝑠+𝐸^𝑠3,𝜀𝑧^1𝑠𝑑𝑠²≤𝐶_𝜀𝜀².(3.20) Since 𝑓_𝑥, 𝑓_𝑦, and 𝑓_𝑧 are uniformly bounded, we have sup_{0≤𝑡≤𝑇}||𝐹^𝑠1,𝜀||+||𝐹^𝑠2,𝜀||+||𝐹^𝑠3,𝜀||≤𝑐.(3.21) Similar to the proof of Lemma 1 in [9] for the BSDE part, we can obtain (3.5) and (3.6) with the iterative method.

We are now ready to state the variational inequality.

Lemma 3.3. The following variational inequality holds: 𝔼𝜙_𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇+𝛾_𝑦𝑦^0𝑢,𝜉𝑦¹⁰+𝜀_𝑖≥1𝑙_𝜉𝜏_𝑖,𝜉_𝑖𝜂_𝑖+𝔼_𝑇0ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦(𝑡)𝑦^1𝑡+ℎ𝑢^𝜀𝑡−ℎ(𝑡)𝑑𝑡≥𝑜(𝜀).(3.22)

Proof. From the optimality of (𝑢(⋅),𝜉(⋅)), we have 𝐽(𝑢^𝜀(⋅),𝜉^𝜀(⋅))−𝐽(𝑢(⋅),𝜉(⋅))≥0.(3.23) From Lemmas 3.1 and 3.2, it follows that 𝔼𝜙𝑥𝔼𝜙𝑥^𝜀𝑇−𝜙𝑥^{𝑇𝑢,𝜉}+𝑥^1𝑇=𝑜(𝜀),_{𝑇𝑢,𝜉}+𝑥^1𝑇−𝜙𝑥^{𝑇𝑢,𝜉}=𝔼𝜙^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇+𝑜(𝜀).(3.24) Hence, 𝔼𝜙𝑥^𝜀𝑇−𝜙𝑥^{𝑇𝑢,𝜉}=𝔼𝜙^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^1𝑇+𝑜(𝜀).(3.25) Similarly we get 𝔼𝔼𝛾𝑦^𝜀0−𝛾𝑦^0𝑢,𝜉=𝔼𝛾^𝑦𝑦^0𝑢,𝜉𝑦¹⁰+𝑜(𝜀),_𝑖≥1𝑙𝜏_𝑖,𝜉_𝑖+𝜀𝜂_𝑖−_𝑖≥1𝑙𝜏_𝑖,𝜉_𝑖=𝜀𝔼_𝑖≥1𝑙_𝜉𝜏_𝑖,𝜉_𝑖𝜂_𝑖+𝑜(𝜀),(3.26) while 𝔼=^𝑇0𝔼ℎ𝑡,_𝑇0𝑥^𝜀𝑡,𝑦^𝜀𝑡,𝑢^𝜀𝑡−ℎ(𝑡)𝑑𝑡+𝔼_𝑇0ℎ𝑡,𝑥^𝜀𝑡,𝑦^𝜀𝑡,𝑢^𝜀𝑡−ℎ𝑡,𝑥^{𝑡𝑢,𝜉}+𝑥^1𝑡,𝑦^{𝑡𝑢,𝜉}+𝑦^1𝑡,𝑢^𝜀𝑡𝑑𝑡+𝔼_𝑇0ℎ𝑡,𝑥^{𝑡𝑢,𝜉}+𝑥^1𝑡,𝑦^{𝑡𝑢,𝜉}+𝑦^1𝑡,𝑢^𝜀𝑡−ℎ𝑢^𝜀𝑡𝑑𝑡ℎ𝑢^𝜀𝑡−ℎ(𝑡)𝑑𝑡∶=𝐼+𝐼𝐼+𝔼^𝑇0ℎ𝑢^𝜀𝑡−ℎ(𝑡)𝑑𝑡.(3.27) Since ℎ_𝑥,ℎ_𝑦,ℎ_𝑧 have linear growth, it follows from Lemma 3.2 and Hölder's inequality that 𝐼=𝔼^𝑇0¹⁰ℎ_𝑥Π^𝜀𝑡𝑋^𝜀𝑡+ℎ_𝑦Π^𝜀𝑡𝑌^𝜀𝑡𝑑𝜆𝑑𝑡=𝑜(𝜀).(3.28) By Lemma 3.1 and the dominated convergence theorem, we have 𝐼𝐼=𝔼^𝑇0¹⁰ℎ_𝑥Λ^𝜀𝑡𝑥^1𝑡=𝔼_𝑇0+ℎ_𝑦Λ^𝜀𝑡𝑦^1𝑡𝑑𝜆𝑑𝑡=𝔼_𝑇0ℎ_𝑥𝑢^𝜀𝑡𝑥^1𝑡+ℎ_𝑦𝑢^𝜀𝑡𝑦^1𝑡𝑑𝑡+𝑜(𝜀)+𝔼_𝑇0ℎ_𝑥𝑢^𝜀𝑡−ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦𝑢^𝜀𝑡−ℎ_𝑦(𝑡)𝑦^1𝑡𝑑𝑡=𝔼_{𝜏𝜏+𝜀}ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦(𝑡)𝑦^1𝑡𝑑𝑡+𝑜(𝜀)+𝔼_𝑇0ℎ_𝑥(𝑡,𝑣)−ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦(𝑡,𝑣)−ℎ_𝑦(𝑡)𝑦^1𝑡𝑑𝑡ℎ_𝑥(𝑡)𝑥¹+ℎ_𝑦(𝑡)𝑦¹𝑑𝑡+𝑜(𝜀),(3.29) where 𝜑(𝑡,𝑣)=𝜑(𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑣), 𝜑=ℎ_𝑥,ℎ_𝑦. It follows from Hölder's inequality that 𝐼𝐼≤𝔼^{𝜏𝜏+𝜀}||ℎ_𝑥(𝑡,𝑣)−ℎ_𝑥(𝑡)||²𝑑𝑡^1/2𝔼^𝑇0||𝑥^1𝑡||²𝑑𝑡^1/2+𝔼^{𝜏𝜏+𝜀}||ℎ_𝑦(𝑡,𝑣)−ℎ_𝑦(𝑡)||²𝑑𝑡^1/2𝔼^𝑇0||𝑦^1𝑡||²𝑑𝑡^1/2+𝔼^𝑇0ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦(𝑡)𝑦^1𝑡𝑑𝑡+𝑜(𝜀).(3.30) Using Lemma 3.1 again, we get 𝐼𝐼≤𝔼^𝑇0ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦(𝑡)𝑦^1𝑡𝑑𝑡+𝑜(𝜀).(3.31) Consequently, 𝔼^𝑇0ℎ𝑡,𝑥^𝜀𝑡,𝑦^𝜀𝑡,𝑢^𝜀𝑡−ℎ(𝑡)𝑑𝑡=𝔼^𝑇0ℎ_𝑥(𝑡)𝑥^1𝑡+ℎ_𝑦(𝑡)𝑦^1𝑡+ℎ𝑢^𝜀𝑡−ℎ(𝑡)𝑑𝑡+𝑜(𝜀).(3.32) The variational inequality follows from (3.25)–(3.32).

Now we introduce the following FBSDE (called the adjoint equation): 𝑑𝑞_𝑡𝑑𝑝_𝑡=𝑓^∗𝑦(𝑡)𝑝_𝑡−ℎ^∗𝑦(𝑡)𝑑𝑡+𝑓^∗𝑧(𝑡)𝑝_𝑡𝑑𝐵_𝑡,𝑝₀=𝑓^∗𝑥(𝑡)𝑝_𝑡−𝑏^∗𝑥(𝑡)𝑞_𝑡−𝜎^∗𝑥(𝑡)𝑘_𝑡−ℎ^∗𝑥(𝑡)𝑑𝑡+𝑘_𝑡𝑑𝐵_𝑡,=−𝛾^∗𝑦𝑦^0𝑢,𝜉,𝑞^𝑇=−𝑔^∗𝑥𝑥^{𝑇𝑢,𝜉}𝑝^𝑇+𝜙^∗𝑥𝑥^{𝑇𝑢,𝜉}.(3.33)

It is easy to check that the adjoint equation admits a unique solution (𝑝(⋅),𝑞(⋅),𝑘(⋅))∈𝑆³(ℝ^𝑚)×𝑆³(ℝ^𝑛)×𝐻³(ℝ^𝑚×𝑑).

We are now in a position to state the stochastic maximum principle.

Theorem 3.4. Let (𝑢(⋅),𝜉(⋅)) be an optimal control, (𝑥^𝑢,𝜉(⋅),𝑦^𝑢,𝜉(⋅),𝑧^𝑢,𝜉(⋅)) the corresponding trajectory, and (𝑝(⋅),𝑞(⋅),𝑘(⋅)) the solution of the adjoint equation. Then for any 𝑣∈𝑈 and 𝜂(⋅)∈𝒦 it holds that 𝐻𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉}𝔼,𝑧^{𝑡𝑢,𝜉},𝑣,𝑝_𝑡,𝑞_𝑡,𝑘_𝑡−𝐻𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑢_𝑡,𝑝_𝑡,𝑞_𝑡,𝑘_𝑡≥0,a.e.,a.s.,(3.34)_𝑖≥1𝑙_𝜉𝜏_𝑖,𝜉_𝑖+𝑞^∗𝜏_𝑖𝐶_𝜏𝑖−𝑝^∗𝜏_𝑖𝐷_𝜏𝑖𝜂_𝑖−𝜉_𝑖≥0,(3.35) where 𝐻∶[0,𝑇]×ℝ^𝑛×ℝ^𝑚×ℝ^𝑚×𝑑×𝑈×ℝ^𝑚×ℝ^𝑛×ℝ^𝑛×𝑑→ℝ is defined by 𝐻(𝑡,𝑥,𝑦,𝑧,𝑣,𝑝,𝑞,𝑘)=+−⟨⟨𝑘𝑝,,𝜎𝑓((𝑡𝑡,,𝑥𝑥),⟩𝑦,+𝑧,ℎ𝑣()𝑡⟩,𝑥+,𝑦⟨,𝑞𝑣,)𝑏.(𝑡,𝑥,𝑣)⟩(3.36)

Proof. Applying Itô's formula to ⟨𝑝_𝑡,𝑦^1𝑡⟩+⟨𝑞_𝑡,𝑥^1𝑡⟩, by Lemma 3.3 we derive 𝔼+^𝑇0𝜀𝔼𝐻𝑡,𝑥^{𝑡𝑢,𝜉}_𝑖≥1,𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑢^𝜀𝑡,𝑝_𝑡,𝑞_𝑡,𝑘_𝑡−𝐻𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑢_𝑡,𝑝_𝑡,𝑞_𝑡,𝑘_𝑡𝑑𝑡𝑙_𝜉𝜏_𝑖,𝜉_𝑖+𝑞^∗𝜏_𝑖𝐶_𝜏𝑖−𝑝^∗𝜏_𝑖𝐷_𝜏𝑖𝜂_𝑖≥𝑜(𝜀),(3.37) where 𝜂(⋅)∈ℐ satisfies 𝜉(⋅)+𝜂(⋅)∈𝒦. Dividing (3.37) by 𝜀 and letting 𝜀 go to 0, we obtain 𝔼𝐻+𝔼𝜏,𝑥^{𝜏𝑢,𝜉},_𝑖≥1𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑣,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏−𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑢_𝜏,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏𝑙_𝜉𝜏_𝑖,𝜉_𝑖+𝑞^∗𝜏_𝑖𝐶_𝜏𝑖−𝑝^∗𝜏_𝑖𝐷_𝜏𝑖𝜂_𝑖≥0,a.e.𝜏∈[0,𝑇].(3.38) By choosing 𝑣=𝑢_𝜏 in (3.38) we obtain the conclusion (3.35). If we choose 𝜂(⋅)≡0, then for 𝑣∈ℱ_𝜏 satisfying 𝔼|𝑣|³<∞ we have 𝔼𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑣,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏−𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑢_𝜏,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏≥0.(3.39) Now let us set 𝑣_𝜏=𝑣𝟙_𝐴+𝑢_𝜏𝟙_A for any 𝑣∈𝑈 and 𝐴∈ℱ_𝜏. Then it is obvious that 𝑣_𝜏∈ℱ_𝜏 and 𝔼|𝑣_𝜏|³<∞. So from (3.39) it follows that, for any 𝐴∈ℱ_𝜏, 𝔼𝟙_𝐴𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑣,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏−𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑢_𝜏,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏≥0.(3.40) Hence, 𝔼𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑣,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏−𝐻𝜏,𝑥^{𝜏𝑢,𝜉},𝑦^{𝜏𝑢,𝜉},𝑧^{𝜏𝑢,𝜉},𝑢_𝜏,𝑝_𝜏,𝑞_𝜏,𝑘_𝜏∣ℱ^𝜏≥0,∀𝑣∈𝑈.(3.41) Since the quantity inside the conditional expectation is ℱ_𝜏-measurable, the conclusion (3.34) can be obtained easily.

Similar to [22, Corollary 3.1], by Theorem 3.4 we can easily obtain the following

Corollary 3.5. Assume 𝐾=ℝ^𝑛. Then for the optimal control (𝑢(⋅),𝜉(⋅)) it holds that 𝐻𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑣,𝑝𝑙^𝑡,𝑞_𝑡,𝑘_𝑡−𝐻𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑧^{𝑡𝑢,𝜉},𝑢_𝑡,𝑝_𝑡,𝑞_𝑡,𝑘_𝑡≥0,∀𝑣∈𝑈,a.e.,a.s.,_𝜉𝜏_𝑖,𝜉_𝑖+𝑞^∗𝜏_𝑖𝐶_𝜏𝑖−𝑝^∗𝜏_𝑖𝐷_𝜏𝑖=0,𝑖≥1,a.𝑠..(3.42)

Remark 3.6. We can still obtain the stochastic maximum principle if the assumptions are relaxed in the following way.(i)The regular control process 𝑣(⋅) and the impulse control process 𝜂(⋅) are assumed to satisfy 𝔼[sup_{0≤𝑡≤𝑇}|𝑣_𝑡|^𝑝]<∞ and 𝔼[∑_𝑖≥1|𝜂_𝑖|^𝑝]<∞ for some 𝑝∈(2,3).(ii)The assumption |ℎ(𝑡,0,0,𝑣)|≤𝑐(1+|𝑣|³) in Hypothesis (H2) can be weakened as |ℎ(𝑡,0,0,𝑣)|≤𝑐(1+|𝑣|^𝑝).(iii)In the spike variation setting, the random variable 𝑣 is assumed to satisfy 𝔼|𝑣|^𝑝<∞.
In fact, under these new assumptions both the solutions of the control system (2.4) and the variational equation (3.2) belong to 𝑆^𝑝(ℝ^𝑛)×𝑆^𝑝(ℝ^𝑚)×𝐻^𝑝(ℝ^𝑚×𝑑). The conclusion of Lemma 3.1 becomes sup_{0≤𝑡≤𝑇}𝔼||𝑥^1𝑡||^𝑝+sup_{0≤𝑡≤𝑇}𝔼||𝑦^1𝑡||^𝑝+𝔼^𝑇0||𝑧^1𝑡||²𝑑𝑡^𝑝/2≤𝑐𝜀^𝑝.(3.43) And Lemmas 3.2 and 3.3 still hold true.

4. Sufficient Optimality Conditions for Optimal Controls

We still denote by (𝑥^𝑣,𝜂(⋅),𝑦^𝑣,𝜂(⋅),𝑧^𝑣,𝜂(⋅)) the trajectory corresponding to (𝑣(⋅),𝜂(⋅))∈𝒜. Let us first introduce an additional assumption.

(H3) The control domain 𝑈 is a convex body in ℝ^𝑘. The maps 𝑏, 𝑓, and ℎ are locally Lipschitz in the regular control variable 𝑣.

Theorem 4.1. Let (H1)–(H3) hold. Assume that the functions 𝜙, 𝛾, 𝜂→𝑙(𝑡,𝜂) and (𝑥,𝑦,𝑧,𝑣)→𝐻(𝑡,𝑥,𝑦,𝑧,𝑣,𝑝,𝑞,𝑘) are convex. Moreover, 𝑦^{𝑇𝑣,𝜂} has the following particular form: 𝑦^{𝑇𝑣,𝜂}=𝐾𝑥^{𝑇𝑣,𝜂}+𝜁 for 𝐾∈ℝ^𝑚×𝑛 and 𝜁∈𝐿³(Ω,ℱ_𝑇,ℙ;ℝ^𝑚). Let (𝑝^𝑢,𝜉,𝑞^𝑢,𝜉,𝑘^𝑢,𝜉) be the solution of the adjoint equation associated with (𝑢,𝜉)∈𝒜. Then (𝑢,𝜉) is an optimal control of the stochastic optimal control problem if it satisfies (3.34) and (3.35).

Proof. Set 𝐽=𝐽(𝑣(⋅),𝜂(⋅))−𝐽(𝑢(⋅),𝜉(⋅)). Since 𝜙, 𝛾, 𝜂→𝑙(𝑡,𝜂) are convex, we have 𝜙𝛾𝑥𝑦^{𝑇𝑣,𝜂}_0𝑣,𝜂−𝜙𝑥^{𝑇𝑢,𝜉}≥𝜙^𝑥𝑥^{𝑇𝑢,𝜉}𝑥^{𝑇𝑣,𝜂}−𝑥^{𝑇𝑢,𝜉},_𝑖≥1−𝛾𝑦^0𝑢,𝜉≥𝛾^𝑦𝑦^0𝑢,𝜉𝑦^0𝑣,𝜂−𝑦^0𝑢,𝜉,𝑙𝜏_𝑖,𝜂_𝑖−_𝑖≥1𝑙𝜏_𝑖,𝜉_𝑖≥_𝑖≥1𝑙_𝜉𝜏_𝑖,𝜉_𝑖𝜂_𝑖−𝜉_𝑖.(4.1) Thus, 𝐽≥𝔼𝜙_𝑥+𝔼_𝑇0𝑥^{𝑇𝑢,𝜉}𝑥^{𝑇𝑣,𝜂}−𝑥^{𝑇𝑢,𝜉}+𝛾^𝑦𝑦^0𝑢,𝜉𝑦^0𝑣,𝜂−𝑦^0𝑢,𝜉+𝔼_𝑖≥1ℎ𝑡,𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑣_𝑡−ℎ𝑡,𝑥^{𝑡𝑢,𝜉},𝑦^{𝑡𝑢,𝜉},𝑢_𝑡𝑑𝑡𝑙_𝜉𝜏_𝑖,𝜉_𝑖𝜂_𝑖−𝜉_𝑖.(4.2) Set ℋ^𝑣,𝜂(𝑡)∶=𝐻(𝑡,𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑧^{𝑡𝑣,𝜂},𝑣_𝑡,𝑝^{𝑡𝑢,𝜉},𝑞^{𝑡𝑢,𝜉},𝑘^{𝑡𝑢,𝜉}). Then by Itô's formula applied to ⟨𝑞^{𝑡𝑢,𝜉},(𝑥^{𝑡𝑣,𝜂}−𝑥^{𝑡𝑢,𝜉})⟩+⟨𝑝^{𝑡𝑢,𝜉},(𝑦^{𝑡𝑣,𝜂}−𝑦^{𝑡𝑢,𝜉})⟩, we get 𝐽≥Ξ+Θ, where ΘΞ==𝔼𝔼_{𝑖≥1𝑇0}𝑙_𝜉𝜏_𝑖,𝜉_𝑖+𝑞^∗𝜏_𝑖𝐶_𝜏𝑖−𝑝^∗𝜏_𝑖𝐷_𝜏𝑖𝜂_𝑖−𝜉_𝑖,−ℋℋ^𝑣,𝜂_{𝑦𝑢,𝜉}(𝑡)−ℋ^𝑢,𝜉(𝑡)−ℋ^{𝑥𝑢,𝜉}(𝑡)𝑥^{𝑡𝑣,𝜂}−𝑥^{𝑡𝑢,𝜉}(𝑡)𝑦^{𝑡𝑣,𝜂}−𝑦^{𝑡𝑢,𝜉}−ℋ^{𝑧𝑢,𝜉}(𝑡)𝑧^{𝑡𝑣,𝜂}−𝑧^{𝑡𝑢,𝜉}𝑑𝑡.(4.3) From (3.35) we have Ξ≥0. By (3.34) and [23, Lemma 2.3-(iii); Chapter 3], we have 0∈𝜕_𝑢ℋ^𝑢,𝜉(𝑡). By [23, Lemma 2.4; Chapter 3], we further conclude that ℋ^{𝑥𝑢,𝜉}(𝑡),ℋ^{𝑦𝑢,𝜉}(𝑡),ℋ^{𝑧𝑢,𝜉}(𝑡),0∈𝜕_{𝑥,𝑦,𝑧,𝑢}ℋ^𝑢,𝜉(𝑡).(4.4) Then, by [23, Lemma 2.3-(v); Chapter 3] and the convexity of 𝐻(𝑡,.,.,.,.,𝑝,𝑞,𝑘), we obtain ℋ^𝑣,𝜂(𝑡)−ℋ^𝑢,𝜉(𝑡)≥ℋ^{𝑥𝑢,𝜉}(𝑡)𝑥^{𝑡𝑣,𝜂}−𝑥^{𝑡𝑢,𝜉}+ℋ^{𝑦𝑢,𝜉}(𝑡)𝑦^{𝑡𝑣,𝜂}−𝑦^{𝑡𝑢,𝜉}+ℋ^{𝑧𝑢,𝜉}(𝑡)𝑧^{𝑡𝑣,𝜂}−𝑧^{𝑡𝑢,𝜉},(4.5) from which it follows immediately that Θ≥0. Thus we obtain 𝐽≥0 and the proof is complete.

We now give an example of linear quadratic optimal control problem involving impulse controls to illustrate the application of our theoretical results.

Example 4.2. For simplicity, assume that the variables and coefficients are scalar-valued. Let us take 𝑈={−1,1} and 𝐾=ℝ. There are only two values −1 and 1 in 𝑈 which is a usual case in practice and represents only two control states: “on” and “off”. For (𝑣(⋅),𝜂(⋅))∈𝒜, the controlled system is subject to the following linear FBSDE:𝑑𝑦_𝑡𝑑𝑥_𝑡=𝐴𝑥_𝑡+𝐵𝑣_𝑡𝑑𝑡+𝐶𝑥_𝑡𝑑𝐵_𝑡+𝐻𝑑𝜂_𝑡,=−𝐷𝑥_𝑡+𝑥𝐸₀𝑦_𝑡+𝐹𝑧_𝑡+𝐺𝑣_𝑡𝑑𝑡+𝑧_𝑡𝑑𝐵_𝑡−𝑅𝑑𝜂_𝑡,=𝑎,𝑦_𝑇=𝑔𝑥_𝑇,(4.6) and the cost functional is given by𝐽(𝑣(⋅),𝜂(⋅))=12𝔼𝑊𝑥^2𝑇+𝛾𝑦²⁰+^𝑇0𝑀𝑥^2𝑡+𝑁𝑦^2𝑡+𝑄𝑣^2𝑡𝑑𝑡+𝐿_𝑖≥1𝜂^2𝑖.(4.7) The coefficients are deterministic constants such that 𝑊,𝛾,𝑀,𝑁≥0 and 𝑄,𝐿>0. By Propositions 2.1 and 2.2 we know that the control system admits a unique solution (𝑥(⋅),𝑦(⋅),𝑧(⋅))∈𝑆³(ℝ)×𝑆³(ℝ)×𝐻³(ℝ) for any (𝑣,𝜂)∈𝒜. And the functional 𝐽 is well defined from 𝒜 into ℝ.
Let (𝑢(⋅),𝜉(⋅)=∑_𝑖≥1𝜉_𝑖𝟙_{[𝜏𝑖,𝑇]}(⋅))∈𝒜 be an optimal control and (𝑥(⋅),𝑦(⋅),𝑧(⋅)) the corresponding trajectory. Then the following adjoint equation 𝑑𝑞_𝑡𝑑𝑝_𝑡=𝐸𝑝_𝑡−𝑁𝑦_𝑡𝑑𝑡+𝐹𝑝_𝑡𝑑𝐵_𝑡,=𝑝₀𝐷𝑝_𝑡−𝐴𝑞_𝑡−𝐶𝑘_𝑡−𝑀𝑥_𝑡𝑑𝑡+𝑘_𝑡𝑑𝐵_𝑡,=−𝛾𝑦₀,𝑞_𝑇=−𝑔𝑝_𝑇+𝑊𝑥_𝑇(4.8) admits a unique solution (𝑝(⋅),𝑞(⋅),𝑘(⋅))∈𝑆³(ℝ)×𝑆³(ℝ)×𝐻³(ℝ). The Hamiltonian 𝐻 is given by 𝐻(𝑡,𝑥,𝑦,𝑧,𝑣,𝑝,𝑞,𝑘)=+−𝑝𝑘𝐶(𝐷𝑥𝑥++1𝐸𝑦+𝐹𝑧+𝐺𝑣)+𝑞(𝐴𝑥+𝐵𝑣)2𝑀𝑥²+𝑁𝑦²+𝑄𝑣².(4.9) Then by Corollary 3.5 we obtain −𝐺𝑝_𝑡+𝐵𝑞_𝑡𝑣+𝐿12𝑄𝜉_𝑖𝑣²≥−𝐺𝑝_𝑡+𝐵𝑞_𝑡𝑢_𝑡+12𝑄𝑢^2𝑡,∀𝑣∈𝑈,a.e.,a.s.,(4.10)+𝐻𝑞_𝜏𝑖−𝑅𝑝_𝜏𝑖=0,𝑖≥1,a.s..(4.11) From (4.10) we get 𝑢_𝑡=1,if𝐺𝑝^𝑡−1,otherwi−se𝐵.𝑞_𝑡≥0,(4.12) From (4.11) we obtain that 𝜉_𝑖=𝐿⁻¹𝑅𝑝_𝜏𝑖−𝐻𝑞_𝜏𝑖,𝑖≥1,a.s..(4.13) Hence, if (𝑢,𝜉)∈𝒜 is an optimal control of this linear quadratic control problem, then it satisfies (4.12) and (4.13).
We can prove that (𝑢(⋅),𝜉(⋅)) obtained in (4.12) and (4.13) is indeed an optimal control of this linear quadratic optimization problem. Note that Theorem 4.1 does not hold now since 𝑈 is not convex in this example. In what follows, we use the same notations as those in the proof of Theorem 4.1. In fact, as in the proof of Theorem 4.1, we can still derive 𝐽(𝑣(⋅),𝜂(⋅))−𝐽(𝑢(⋅),𝜉(⋅))≥Ξ+Θ. On the one hand, it follows from (4.13) that Ξ=0. On the other hand, we have Θ=𝔼^𝑇0ℋ^𝑣,𝜂(𝑡)−𝐻𝑡,𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑧^{𝑡𝑣,𝜂},𝑢_𝑡,𝑝^{𝑡𝑢,𝜉},𝑞^{𝑡𝑢,𝜉},𝑘^{𝑡𝑢,𝜉}+Φ^𝑡𝑑𝑡,(4.14) where Φ_𝑡=−𝐻ℋ𝑡,_{𝑥𝑢,𝜉}𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑧^{𝑡𝑣,𝜂},𝑢_𝑡,𝑝^{𝑡𝑢,𝜉},𝑞^{𝑡𝑢,𝜉},𝑘^{𝑡𝑢,𝜉}−ℋ^𝑢,𝜉(𝑡)(𝑡)𝑥^{𝑡𝑣,𝜂}−𝑥^{𝑡𝑢,𝜉}−ℋ^{𝑦𝑢,𝜉}(𝑡)𝑦^{𝑡𝑣,𝜂}−𝑦^{𝑡𝑢,𝜉}−ℋ^{𝑧𝑢,𝜉}(𝑡)𝑧^{𝑡𝑣,𝜂}−𝑧^{𝑡𝑢,𝜉}.(4.15) From (4.12) and the definition of 𝐻, it is easy to get ℋ=^𝑣,𝜂(𝑡)−−𝐺𝐻𝑝_𝑡𝑡,𝑥^{𝑡𝑣,𝜂},𝑦^{𝑡𝑣,𝜂},𝑧^{𝑡𝑣,𝜂},𝑢_𝑡,𝑝^{𝑡𝑢,𝜉},𝑞^{𝑡𝑢,𝜉},𝑘^{𝑡𝑢,𝜉}+𝐵𝑞_𝑡𝑣_𝑡+12𝑄𝑣^2𝑡−−𝐺𝑝^𝑡+𝐵𝑞_𝑡𝑢_𝑡+12𝑄𝑢^2𝑡≥0.(4.16) Since 𝑀, 𝑁≥0, 𝐻 is convex in (𝑥,𝑦,𝑧), and thus Φ_𝑡≥0, so we obtain Θ≥0. Consequently, it follows that 𝐽(𝑣(⋅),𝜂(⋅))−𝐽(𝑢(⋅),𝜉(⋅))≥0 and the optimality of (𝑢(⋅),𝜉(⋅)) is proved.

Remark 4.3. For the classical linear quadratic optimal control problem, one can usually obtain an optimal control in a linear state feedback form by virtue of the so-called Riccati equation, and along this line the solvability of the Riccati equation leads to that of the linear quadratic problem. However, it is difficult to obtain a state feedback optimal control in terms of the Riccati equation in Example 4.2 mainly due to the particular form of the regular control domain and the appearance of the impulse control in the control system.

Acknowledgments

The authors would like to thank the referees for valuable suggestions which helped to improve the first version of this paper. Z. Wu acknowledges the financial support from the National Natural Science Foundation of China (10921101 and 61174092) and the Natural Science Fund for Distinguished Young Scholars of China (11125102). F. Zhang acknowledges the financial support from the Natural Science Foundation of Shandong Province, China (ZR2011AQ018), and the Foundation of Doctoral Research Program, Shandong University of Finance and Economics.