Backward Stochastic Differential Equations Coupled with Value Function and Related Optimal Control Problems

Abstract

We get a new type of controlled backward stochastic differential equations (BSDEs), namely, the BSDEs, coupled with value function. We prove the existence and the uniqueness theorem as well as a comparison theorem for such BSDEs coupled with value function by using the approximation method. We get the related dynamic programming principle (DPP) with the help of the stochastic backward semigroup which was introduced by Peng in 1997. By making use of a new, more direct approach, we prove that our nonlocal Hamilton-Jacobi-Bellman (HJB) equation has a unique viscosity solution in the space of continuous functions of at most polynomial growth. These results generalize the corresponding conclusions given by Buckdahn et al. (2009) in the case without control.

1. Introduction

In the recent years, many authors (see [1–6]) have studied models of large stochastic particle systems with mean-field interaction. Lasry and Lions studied mean-field limits of problems of stochastic differential games ([4] and the references inside). Inspired by them, Buckdahn et al. [1] got in a purely stochastic approach, a new type of backward stochastic differential equations (BSDEs), namely, mean-field BSDEs. In [7], Buckdahn et al. deepened the investigation of such mean-field BSDEs. They obtained some central results for the mean-field BSDEs such as the existence and the uniqueness theorem, as well as a comparison theorem.

On the other hand, the modern optimal control theory has been developing very quickly since the works on the maximum principle by Pontryagin et al. [8] and the dynamic programming approach proposed by Bellman [9]. Since then, there have been a lot of works published on the stochastic maximum principle; refer to, for example, Kushner [10, 11], Bensoussan [12], Haussmann [13], Peng [14], Tang and Li [15], and Zhou [16]. There are also many works on the stochastic maximum principle for optimal control problems in the mean-field case; see, for example, Bensoussan et al. [17], Buckdahn et al. [18], Li [19], Meyer-Brandis et al. [20], and Yong [21]. There have also been a lot of works published on the dynamic programming approach, which gives with the help of dynamic programming principle (DPP) a stochastic interpretation to the associated partial differential equations (PDEs); we refer, for instance, to Buckdahn and Li [22], Peng [23, 24], and Yong and Zhou [25]. But to the best of our knowledge, there are no works relating optimal control problems in the mean-field case to nonlocal PDEs of Hamilton-Jacobi-Bellman (HJB) type.

In [7], the authors also considered the following decoupled forward-backward stochastic differential equation (FBSDE) with the initial given data and frozen : and they defined the value function which turns out to deterministic when all coefficients are deterministic. In (1), we have denoted by an independent copy of , and by the expectation taken only with respect to ; for more details, refer to Section 2. The authors of [7] proved that is the unique viscosity solution of the following nonlocal PDE in the space (the space of continuous functions over with polynomial growth): Here we have defined

In this paper we will investigate an optimal stochastic control problem involving mean-field BSDEs. Two major obstacles can be observed; let us explain them. We consider the following controlled decoupled FBSDE with the initial given data and frozen and the control : where is an admissible control over the time interval . In order to determine , we consider in a first step (4) for . The thus obtained equation is a mean-field one and was studied in [7]; for more details, the reader is referred to (22) and (26).

A first idea for the above introduced control problem could be to consider as a value function. Consider But, in fact, such a value function does not satisfy the DPP because of the expectation terms in (4). For this reason, similar to [7] we have to freeze, here, not only but also , and we consider the following value function: However, the fact that the control is frozen in (4) has as a consequence that the value function is a viscosity solution of the following classical PDE (there are many references, such as [22–24]): with

But is not a viscosity solution of the following nonlocal HJB equation:

However, it is this latter nonlocal HJB equation which we want to give a stochastic interpretation. PDE (9) is nonlocal in its solution . Indeed; that is, this coefficient depends nonlinearly not only of the value of at but also on the whole function .

On the other hand, we observe that, in the case without control, that is, in [7], we have , and then we can rewrite the backward SDE of (1) as follows: which can be regarded as an equation with the solution in some sense.

Inspired by this idea, we change to study the following BSDE coupled with value function: Here and satisfy (H3.2). Notice that if one of the coefficients is not deterministic, then usually the value function is also not deterministic. On the other hand, the assumption that is bounded in plays an important role in our work. However, in the case without control, that is, when the control state space is a singleton, the boundedness assumption on can be deleted (see Remark 5(i) or [7]). The solution of BSDE coupled with value function (12) is a triplet (see Theorem 9). We use a new iteration method to prove that (12) has a unique solution .

One of the main objectives of our paper is to study the stochastic interpretation of our nonlocal HJB equation (9). Firstly, unlike [7, 22, 24] in which the authors use the BSDE method to prove the existence and the uniqueness of the viscosity solution for the related PDEs, our approach here is quite different and more direct. Secondly, in [7], the authors have to consider the uniqueness of the viscosity solution in a smaller space in which the continuous functions are of at most polynomial growth. But, in our work, since is bounded in , we have the uniqueness of viscosity solution in     such that  , uniformly in . On the other hand, for the existence and the uniqueness of the viscosity solution of our nonlocal HJB equation (9), we do not need the monotonicity assumption on in , because once knowing , the driving coefficient satisfies the usual assumptions for classical BSDEs. From this point of view, we generalize Theorem 6.1 and Theorem 7.1 in [7] for the case without control (see Remark 23).

Our paper is organized as follows. Section 2 introduces the theory of mean-field SDEs and mean-field BSDEs which are used in what follows. In Section 3, a new type of BSDEs, namely, BSDEs, coupled with value function is studied. The existence and the uniqueness theorem, as well as a comparison theorem, for this type of BSDEs are proved (Theorems 9 and 11). We also show that is Lipschitz and has linear growth in (Theorem 9). Section 4 is devoted to prove the DPP and to show that is -Hölder continuous in . The existence and the uniqueness of the viscosity solution of our nonlocal HJB equation in the space is studied in Section 5. Finally, we give two examples.

2. Mean-Field SDEs and Mean-Field BSDEs

Our probability space is the classical Wiener space; that is, is the set of all continuous functions from to beginning from 0; is the Wiener measure such that the coordinate process , , , becomes a -dimensional Brownian motion; is the Borel -field over , completed by the set of all -null sets, and is the natural filtration generated by and completed by ; that is, .

We introduce the following spaces which will be used frequently: for ,

real-valued -adapted càdlàg process: ;

-valued -progressively measurable process: .

For the reader's convenience, let us first introduce the framework of mean-field SDEs—also called McKean-Vlasov SDEs (MV SDEs for short) and mean-field BSDEs which we will use in our work. For more details about them we refer to [1, 7].

Let be the (noncompleted) product of with itself. In this space, we use the filtration . A random variable (the space of all real-valued random variables over ) defined on can be extended to by putting , . For any (the space of integrable random variables of ), is in ,  -a.s., and we define . Then we can calculate the expectation of with the help of the Fubini Theorem:

We suppose that the following are given measurable functions: and , which satisfy the following:

(H2.1) (i) are -progressively measurable processes;

(ii) and have linear growth and are globally Lipschitz in ; that is, there exist some constant , such that, for all -a.s.,(1);(2).

For any , the mean-field SDE has a unique strong solution . For the proof, we refer to Theorem 4.1 in [7]. Notice that

Let us now introduce the mean-field BSDEs (see [7] for more details). We suppose that is -progressively measurable, for all and satisfies the following assumptions:

(H2.2) (i) there exists a constant such that, -a.s., for all ,,

(ii) .

Lemma 1. Under the assumption (H2.2), for any random variable the mean-field BSDE has a unique adapted solution .

Lemma 2 (comparison theorem). Let ,  , be two generators satisfying the assumption (H2.2). Furthermore, we assume the following:(i)one of the two coefficients is independent of ;(ii)one of the two coefficients is nondecreasing with respect to .
Let and and be the solutions of the mean-field BSDE (17) with data and , respectively. Then, if ,  -a.s., and ,   -a.s., we have ,   ,  -a.s.

Now, we want to introduce the decoupled forward-backward SDEs in the mean-field case. We suppose is -progressively measurable, for all and satisfies the following:

(H2.3) (i) is Lipschitz with respect to ; that is, there exists a constant such that, -a.s., for all ,

(ii) .

We suppose that and satisfy (H2.1) and (H2.3), respectively. Let be arbitrarily given. For any data , we consider the following decoupled forward-backward SDE in the mean-field case:

Notice that with the choice of the initial data , (19) becomes a decoupled mean-field FBSDE, the forward equation is a mean-field SDE which has a unique strong solution . Then, from Lemma 1, it follows that the backward equation is the mean-field BSDE which has a unique solution . Once having got , (19) becomes a classical decoupled forward-backward SDE with the coefficients , , and , respectively, which means that (19) has a unique solution under the assumptions (H2.1) and (H2.3).

Lemma 3. For and , we let and , be the solutions of FBSDE (19) with the initial data and , respectively. Then there exists a constant such that Here the constant depends only on the Lipschitz and the linear growth constants of , and .

For Lemmas 1–3, the reader is referred to [7].

3. BSDEs Coupled with Value Function

In this section, we will investigate a new type of BSDEs, namely, the BSDEs, coupled with value function. We will first prove the existence and the uniqueness theorem of the solution for this type of BSDEs. For this we first consider the associate forward equation and we study later the BSDEs coupled with value function by an iteration approach.

Let be a compact metric space. An admissible control process on is an -progressively measurable process taking its values in . The set of all admissible controls on is denoted by .

We assume that the coefficients and satisfy the following conditions:

(H3.1) (i) for every fixed , , and are continuous in ;

(ii) there exists a such that, for all , , ,

From the above assumption (H3.1), we get immediately that, for all  ,   ,   ,   .

For what follows is chosen arbitrarily but fixed. Under the assumption (H3.1), for any , , and , the SDE has a unique strong solution. We emphasize that, for , SDE (22) is a mean-field equation with as the unique solution. Once having , SDE (22) becomes a classical SDE with the coefficients and , which satisfy the linear growth and the Lipschitz assumptions.

Remark 4. For any , there exist the constants and such that, for all , and . Consider , for all , where only depend on the Lipschitz constant and the linear growth constant of and (for it also depends on ). The reader is referred to Remark  4.1 in Buckdahn et al. [7].

We assume that the both mappings and satisfy the following conditions:

(H3.2) (i) for each fixed , is continuous in , and there exists a constant , such that, for all , , , , and ,

(ii) is -measurable, for all ; and there exists a constant , such that, for all ,

(iii) there exists a constant , such that , for all .

From (H3.2)-(i), we get directly that there exists some constant such that, for all , , and , , .

For and satisfying the assumption (H3.2), we consider the following new type of BSDE, namely, the BSDE, coupled with value function. Consider

Remark 5. (i) When the coefficients , and do not depend on controls, we consider (19) as in [7] with the initial data . We define . From [7], we have ,-a.s.; that is, the backward equation of (19) becomes now coupled with the associated value function . It means that (27) (i.e., (26)) has a unique solution . However, this approach is not possible anymore in the case we study here. Moreover, we emphasize that, for the case without control, we do not need to be bounded in to make sure that (26) has a unique solution . On the other hand, when all the coefficients are deterministic, is the unique viscosity solution of the associated nonlocal PDE under the standard assumptions; refer to [7].
(ii) When the coefficients , and do not depend on , and is deterministic and does not depend on , then the SDE (22) becomes the classical SDE: The BSDE (26) becomes the following classical BSDE: decoupled with the associated value function , where is the solution of SDE (28). is the unique viscosity solution of the associated HJB equation under the standard assumptions; we refer to [24] or [22].

Remark 6. Recall that due to (H3.2) (ii) the terminal condition is a random variable. This has as a consequence that, here, in general, cannot be expected to be a deterministic function, but is an -measurable random variable.

In order to make things clearer and to show the existence of a solution, we choose an iterative approach. Putting ; we consider, for , ,

Lemma 7. For all , (30) admits a unique solution . Moreover, is a measurable random field such that(i) is -measurable, ;(ii)there exists a constant independent of , such that, a.s., for all ,

Proof. For and ; that is, the driving coefficient of (30) takes the form it satisfies the Lipschitz and the growth conditions for classical BSDEs (see [26, 27] or refer to Lemma 1); therefore, (30) admits a unique solution . Moreover, from (20), for some constant , we have -a.s., for all , Consequently, -a.s., for all , and, obviously, is -measurable.
We suppose now that, for , is a measurable random field with, for some constant , -a.s., for all , Then, putting it follows from (H3.2) that, for some constant independent of , -a.s., for all . Consequently, due to (20), for some constant independent of , -a.s., for all , we have Therefore, with respect to the same constant , we have, -a.s., for all , and, obviously, is -measurable. The proof of the lemma is complete.

Theorem 8. There are processes and a measurable random field which is -measurable, for all , such that converges to in , for all , and converges to in ; solves BSDE (26). Moreover, there exists some constant , such that, -a.s., for all , we have

Proof. Putting by applying Itô's formula to and taking the conditional expectation, we get for all ,  -a.s., where is arbitrarily small and is a constant depending on the Lipschitz constant and on . As is independent of , we can choose such that . Then, for some constant depending only on and , .
Thus, in particular, for , And, from the definition of  , and , it follows that Therefore, which means . Iterating this inequality and denoting the bound of by , we obtain that Therefore, It follows that there exists some process such that On the other hand, from (49) and (45), we get