Stochastic Maximum Principle of Near-Optimal Control of Fully Coupled Forward-Backward Stochastic Differential Equation

Tang, Maoning

doi:https://doi.org/10.1155/2014/361259

Abstract and Applied Analysis

On this page

Abstract Introduction Conclusion Acknowledgments References Copyright Related Articles

Special Issue

Advanced Stochastic Control Systems with Engineering Applications

View this Special Issue

Research Article | Open Access

Volume 2014 | Article ID 361259 | https://doi.org/10.1155/2014/361259

Stochastic Maximum Principle of Near-Optimal Control of Fully Coupled Forward-Backward Stochastic Differential Equation

Maoning Tang¹

Academic Editor: Shen Yin

Received30 Dec 2013

Revised29 Jan 2014

Accepted06 Feb 2014

Published16 Mar 2014

Abstract

This paper first makes an attempt to investigate the near-optimal control of systems governed by fully nonlinear coupled forward-backward stochastic differential equations (FBSDEs) under the assumption of a convex control domain. By Ekeland’s variational principle and some basic estimates for state processes and adjoint processes, we establish the necessary conditions for any -near optimal control in a local form with an error order of exact . Moreover, under additional convexity conditions on Hamiltonian function, we prove that an -maximum condition in terms of the Hamiltonian in the integral form is sufficient for near-optimality of order .

1. Introduction

Bismut [1] first investigated linear backward stochastic differential equations (BSDEs in short) as the adjoint equation of the forward stochastic system. The existence and uniqueness of BSDEs with nonlinear generators under Lipschitz condition were first proved by Pardoux and Peng [2] in 1990. Since then, the theory of BSDEs has extensive applications in both mathematical finance and stochastic control. Forward-backward stochastic differential equations (FBSDEs in short) consist of forward stochastic differential equations (SDEs in short) of Itô type and BSDEs of Pardoux-Peng. Forward-backward stochastic equations (FBSDEs) not only are widely used in stochastic control and differential games but also have profound applications in mathematical economics and mathematical finance. Therefore, it is natural to investigate control problems for systems governed by this kind of stochastic equations. In mathematical finance, FBSDEs can be formulated as the price equations of financial assets under model uncertainty. In the stochastic optimal control problem, FBSDEs arise as the Hamilton system which is composed of the optimality conditions, the adjoint equation, and the state equation and which completely characterizes the optimal control.

A classical approach for optimal control problems is to derive necessary conditions satisfied by an optimum, such as Pontryagin’s maximum principle. Now the maximum principles for optimal controls of FBSDEs have rich literatures which can be referred to [3–12] and references therein.

The references stated in the above are all concerned with (exact) optimal control. But, in fact, the (exact) optimal control may not exist in many situations. So it becomes very important to study near-optimal controls which are more available and much easier to be obtained than optimal ones, both analytically and numerically. The near-optimal deterministic control problems have been investigated in [13–15]. Near-optimal control problems for SDEs with controlled diffusion coefficients were first investigated in 1998 by Zhou [16], where necessary and sufficient conditions are established by introducing second adjoint equation, for all near-optimal controls. Inspired by Zhou [16], we refer to [16–20] on the near-optimal control of other forward stochastic systems.

For forward-backward stochastic systems, Huang et al. [21] in 2010 and Bahlali et al. [22] in 2009, respectively, established the corresponding stochastic maximum principle for the near-optimal control of linear systems and nonlinear systems, where diffusion coefficients and control variables are each independently based on Ekeland’s principle and spike variation. In 2011, Hui et al. [23] studied the near-optimal control of nonlinear FBSDEs, where diffusion coefficients can be dependent on the control variable, with the assumption that the control domain is convex. In 2012, for linear FBSDEs, Zhang et al. [24] extended the results of [21–23] to the general case of control domains based on the Ekeland’s principle, spike variation, reduction technique developed recently by Yong [25], and the methodology recently introduced by Wu [26].

The control systems of FBSDEs studied in references [21–24] are nonfully coupled which are only coupled in BSDE and not in SDE. For the control systems of fully coupled FBSDEs, the existing literatures mainly focused on exact optimal control problems and few on near-optimal control problems. The purpose of the present paper is to make the first attempt to discuss the near-optimal control for fully coupled FBSDEs. Its main contribution is the developments of maximum principle and verification theorem of the near-optimal control in a uniform manner by Ekeland’s variational principle. Compared with references [21–24], this paper mainly has three advantages as follows. Firstly, our systems studied are fully coupled FBSDEs, which are coupled not only in BSDEs but also in SDEs. Secondly, we get necessary optimality conditions for near-optimal control with an error order of exact , which is better than all in the existing literature on the cases of FBSDEs, where the error orders are almost . In fact, by Ekeland’s variational principle, we know that the error order of exact for the near-optimal control is the best error order. Thirdly, different from [21–24], by continuous dependence theorem of FBSDEs (see Lemma 4), we obtain directly the basic estimates for state processes and adjoint processes (see Lemmas 10, 11, 12, and 14) which play a very important role in proving our main results. Therefore, our approach is simpler and more quickly.

The paper is organized as follows. In Section 2, we present the notations and give main theory on FBSDEs. In Section 3, the problem studied is formulated and basic assumptions are given. In Section 4, we prove some prior estimates for state trajectories and adjoint equation. In Section 5, we obtain a variational formula for the performance functional. Sections 6 and 7 are devoted to deriving verification theorem and stochastic maximum principle by Ekeland’s variational principle. In Section 8, we conclude our paper.

2. Preliminary Notations and Basic Theory for FBSDEs

Now we first introduce some preliminary notations which will be used throughout this paper. Let be a probability space. Let be a -dimensional Brownian motion. Let be -completed natural filtration generated by . Let be a Euclidean space, where the inner product and norm are denoted by and , respectively. For a given function, , we denote its gradient and Hessian by and , respectively. If (with ), then is the corresponding Jacobian matrix. By we denote the predictable field on and by the Borel -algebra of any topological space . Denote by the space of all -measurable -valued stochastic processes satisfying , by , the space of all -adapted -valued stochastic càdlàg processes such that , by , and the set of all -valued random variables on such that . Finally, we define the space Then is a Banach space with respect to the norm given by for .

Now we are in position to present the preliminary results of fully coupled FBSDEs. Consider a general FBSDE as follows: Here , , , and are given mappings and . For a given full-rank matrix , we use the notations and , where is the transpose matrix of .

Definition 1. A stochastic process for the coefficients is said to be an adapted solution of (3) if, for any , it follows almost surely Furthermore, FBSDE (3) is said to be solvable if it has an adapted solution. An FBSDE is said to be nonsolvable if it is not solvable.

In order to get the solvability of FBSDE (3), we make the basic assumptions as follows.

Assumption 2. (i) The random mappings , , and are measurable with , , and . And is measurable with . Moreover, , , and are uniformly Lipschitz continuous in and is uniformly Lipschitz continuous in .
(ii) Monotonicity conditions or for all and , , , , where , and are nonnegative constants with , . Moreover, we have , (resp., ), if (resp., ).

The following two lemmas present the solvability results and continuous dependence theorem of FBSDE (3), respectively, which will be used to demonstrate the basic estimates for the state equation and adjoint equation (see Lemmas 10, 11, 12, and 14).

Lemma 3. Let Assumption 2 be satisfied. Then (3) admits a unique solution .

The proof can be found in Peng and Wu [27].

Lemma 4. Let and be the solutions of the FBSDE (3) corresponding to two given coefficients and which both satisfy Assumption 2, respectively. Then there exists constant such that Particularly, if , we have

The proof can be found in Lin [28].

3. Statement of the Problem and Basic Assumptions

Suppose that is a given compact convex subset of . The stochastic process is said to be admissible, if it is an -adopted process taking values in . We denote all admissible controls by the set .

For any admissible control , we consider the following controlled FBSDE: with the performance functional In the above, , , , , , , and are given random mappings. , , , , , , and are given measurable mappings.

The basic assumptions on coefficients are given as follows.

Assumption 5. (i) For any , satisfy Assumption 2. Moreover, , , and are differentiable in , is differentiable in , and the corresponding derivatives are continuous and uniformly bounded for all .
(ii) is continuous differentiable in , is continuous differentiable in , and is continuous differentiable in . For all , there is a constant such that, for all ,

Under Assumption 5, from Lemma 3, we know that, for every , (9) has a unique solution. The corresponding strong solution is denoted by or . Then is said to be the state processes associated with the admissible control and is called the admissible control pair. Moreover, under Assumption 5, using a priori estimates (8), we can deduce the fact that

The so-called stochastic optimal control problem is to minimize the cost function , over all . The corresponding value function is defined as We denote the above problem (9)–(13) by . Any is said to be an optimal control of Problem , if achieves the infimum of over . The state process is said to be the optimal state. And is called an optimal pair of Problem .

Since this paper is devoted to discussing the near-optimal problem of FBSDEs, we recall the definition of the near-optimal control, following [16].

Definition 6. An admissible control pair is said to be an -optimal control for some , if

Definition 7. The set of parameterized admissible control pairs is said to be near-optimal for sufficient small , if Here is a function with respect to satisfying as . We call the estimate an error bound. If for some independent of the constant , then we call the near-optimal control with order .

Before we conclude this section, let us recall the definition of the Clarke generalized gradient as well as Ekeland’s variational principle which will be used to prove our main results.

Definition 8 (see Zhou [16]). Let be a convex set in and let be a locally Lipschitz function. At any given , we define the generalized gradient of as a set given by

Lemma 9 (Ekeland’s variational principle [29]). Suppose that is a complete metric space and is bounded from below and lower-semi-continuous. For , let satisfy the following inequality: Then, for any , there exists such that(1),(2),(3), for all .

4. Some Prior Estimates for State Trajectories and Adjoint Equations

In order to apply Ekeland’s variational principle to obtain our main result, we must define a distance on the space of admissible controls such that is a complete metric space. For any given , we define To simplify our notation, for any admissible control pair , we set The following is devoted to proving the boundedness and continuity of the state and adjoint processes with the control processes under the metric (18). Note that, in the following, is a generic constants, which may change from line to line.

Lemma 10. Let Assumption 5 be satisfied. Then there exists a constant s.t. that, for every admissible pair ,

Proof. Under Assumption 5, by the estimate (8), we have where the last inequality is obtained by the boundedness of the control domain . The proof is complete.

Lemma 11. Let Assumption 5 be satisfied. Then there is a positive constant s.t. for any given two admissible pairs and ,

Proof. Under Assumption 5, from the estimate (7), we have where the second inequality is obtained by the mean value theorem and the boundedness of , , and . The proof is complete.

We know that the adjoint process plays a key role in establishing stochastic maximum principle. In the following, we will study certain boundedness and continuity of adjoint processes with the control variable under the metric .

For a given admissible pair , corresponding adjoint process is defined as the solution to the following FBSDE:

The adjoint equation (24) is a linear FBSDE whose solution consists of . Under Assumption 5, by Lemma 3, the adjoint equation has a unique solution , , .

Next, the Hamiltonian is defined as follows: Then (24) can be rewritten in Hamiltonian system as follows:

Lemma 12. Let Assumption 5 be satisfied. Then there is a constant s.t. for all control pairs ; it holds

Proof. Under Assumption 5, by the estimate (8), we have where the last inequality is obtained by Lemma 10. The proof is complete.

Assumption 13. There is a constant s.t. for every , and a.s. , where , , , .

Lemma 14. Let Assumptions 5 and 13 be satisfied. Let and be adjoint processes corresponding to two admissible pairs and , respectively. Then we have

Proof. Under Assumptions 5 and 13, from the estimate (7), we have where the last inequality is obtained by Lemma 11 directly.

5. A Variational Formula

The purpose of this section is to obtain a variational formula for the cost functional (10). For any two given control pairs and , from the convex property of the control domain , we can define an admissible control process as follows:

We denote the corresponding state process by .

In the following, using the Hamiltonian (see (25)) and adjoint process , , associated with the admissible control pair , we will state and prove a presentation for the difference .

Lemma 15. Let Assumption 5 be satisfied. Then we get

Proof. Applying the definitions of and Hamilton , we obtain Applying Itô formula to , we have Now putting (35) into (34), we deduce the fact that (33) holds. The proof is complete.

Remark 16. According to the above proof, it is easy to check that can be changed as any admissible control and need not have the form of the convex variation .

Now we state and prove the variational formula for the cost functional (10) as follows.

Theorem 17. Suppose that Assumption 5 holds. Let be any given admissible control. Then we have where is any given admissible control and .

Proof. Define ; by Lemma 15, we have Applying Lemma 11 and Assumption 5, we get Hence, by (38) and (37), we get The proof is complete.

6. Necessary Conditions for Near-Optimality

In this section, we will state and prove our main results, that is, the stochastic maximum principle of the near-optimal control of Problem . Moreover, we give the additional assumption as follows.

Assumption 18. There is a constant s.t. for all and a.s. ,

Theorem 19. Suppose that Assumptions 5 and 13 hold. Let be -optimal pair of problem . Then, for any given , there is a positive constant s.t.: where is the adjoint process corresponding to .

Proof. By Lemma 11 and Assumption 13, we can deduce the fact that is continuous on with respect to the metric (18). Using Ekeland’s variational principle (see [16]) with , there exists an admissible pair such that Now we define a convex perturbed control of as where is an arbitrary given admissible control and .
Then by the variational formula (36), (43), and the fact that we have where is the adjoint process corresponding to .
Now in order to obtain the optimal condition (41), we now have to estimate the following formula: First, by adding and subtracting , we have
Next, using Lemmas 11 and 14 and (42), we have Then, combining Schwarz’s inequality and Lemmas 10 and 12 and (42), we have
Therefore, combining (46), (47), (49), and (50), we have which implies that (41) holds. The proof is complete.

7. Sufficient Optimality Conditions

In this section, we will show that, under certain convex conditions, the near-maximum condition of the Hamiltonian function in the integral form is sufficient for near-optimality.

Theorem 20. Under Assumption 5, let be an admissible pair with , . Let be the adjoint process associated with . Assume that for almost all , is convex in , is convex in , and is convex in , respectively, and for some , the optimality conditions, hold. Then where is a constant independent of .

Proof. In the following, is a constant which may change from line to line and is independent of .
According to Lemma 15, we deduce the fact that where are any given admissible control pairs. By the convexity of , , and , we have Putting (55) into (54), we have
Therefore, the rest of the proof is only to estimate the term . To this end, for a given , let us introduce a new metric on as follows: where Now on we define a new functional by It is easy to check that Therefore is continuous on with respect to metric . Using (52) and Ekeland’s variational principle, we can find an admissible control such that where By standard methods, the maximum condition (52) implies that Applying Proposition 2.3.2 in [30], we have which implies that there exists such that Therefore, under Assumptions 13 and 5, Then, applying Holder’s inequality and Lemma 12 and (61), we deduce By (56) and (68), we get Since is arbitrary, is a near-optimal control with order .

8. Conclusion

This paper is the near-optimal control problem for a stochastic system driven by fully coupled FBSDEs. Stochastic maximum principle and verification theory of the near-optimal control are obtained. The control variable appears in both drift and diffusion coefficients of the FBSDEs. The control domain is assumed to be convex. The reviewers suggest that the data-driven control has extensive applications in industry and finance (see, e.g., [31–33] and the references therein) and the model discussed in this present paper may has the potential to achieve more practical oriented results under data-driven framework. Some investigations on this topic will be studied and carried out in our future publications.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

Financial supports by the National Natural Science Foundation of China (Grant nos. 11101140 and 11301177), China Postdoctoral Science Foundation (2011M500721 and 2012T50391), and the Natural Science Foundation of Zhejiang (Grant nos. Y6110775 and Y6110789) are gratefully acknowledged.

References

J.-M. Bismut, “Conjugate convex functions in optimal stochastic control,” Journal of Mathematical Analysis and Applications, vol. 44, no. 3, pp. 384–404, 1973.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
É. Pardoux and S. G. Peng, “Adapted solution of a backward stochastic differential equation,” Systems & Control Letters, vol. 14, no. 1, pp. 55–61, 1990.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. Peng, “Backward stochastic differential equations and applications to optimal control,” Applied Mathematics and Optimization, vol. 27, no. 2, pp. 125–144, 1993.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
W. S. Xu, “Stochastic maximum principle for optimal control problem of forward and backward system,” Journal of the Australian Mathematical Society B, vol. 37, no. 2, pp. 172–185, 1995.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
Z. Wu, “Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems,” Systems Science and Mathematical Sciences, vol. 11, no. 3, pp. 249–259, 1998.
View at: Google Scholar | Zentralblatt MATH | MathSciNet
J. T. Shi and Z. Wu, “Maximum principle for partially-observed optimal control of fully-coupled forward-backward stochastic systems,” Journal of Optimization Theory and Applications, vol. 145, no. 3, pp. 543–578, 2010.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
J.-T. Shi and Z. Wu, “The maximum principle for fully coupled forward-backward stochastic control system,” Acta Automatica Sinica, vol. 32, no. 2, pp. 161–169, 2006.
View at: Google Scholar | MathSciNet
G. Wang and Z. Wu, “The maximum principles for stochastic recursive optimal control problems under partial information,” IEEE Transactions on Automatic Control, vol. 54, no. 6, pp. 1230–1242, 2009.
View at: Publisher Site | Google Scholar | MathSciNet
G. Wang, Z. Wu, and J. Xiong, “Maximum principles for forward-backward stochastic control systems with correlated state and observation noises,” SIAM Journal on Control and Optimization, vol. 51, no. 1, pp. 491–524, 2013.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. Ji and Q. Wei, “A maximum principle for fully coupled forward-backward stochastic control systems with terminal state constraints,” Journal of Mathematical Analysis and Applications, vol. 407, no. 2, pp. 200–210, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
Q. Meng, “A maximum principle for optimal control problem of fully coupled forward-backward stochastic systems with partial information,” Science in China A, vol. 52, no. 7, pp. 1579–1588, 2009.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
M. Tang and Q. Meng, “Stochastic differential games of fully coupled forward-backward stochastic systems under partial information,” in Proceedings of the 29th Chinese Control Conference (CCC '10), pp. 1150–1155, IEEE, July 2010.
View at: Google Scholar
X. Y. Zhou, “Deterministic near-optimal control. I. Necessary and sufficient conditions for near-optimality,” Journal of Optimization Theory and Applications, vol. 85, no. 2, pp. 473–488, 1995.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
X. Y. Zhou, “Deterministic near-optimal controls. II. Dynamic programming and viscosity solution approach,” Mathematics of Operations Research, vol. 21, no. 3, pp. 655–674, 1996.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
I. Ekeland, “On the variational principle,” Journal of Mathematical Analysis and Applications, vol. 47, pp. 324–353, 1974.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
X. Y. Zhou, “Stochastic near-optimal controls: necessary and sufficient conditions for near-optimality,” SIAM Journal on Control and Optimization, vol. 36, no. 3, pp. 929–947, 1998.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
F. Chighoub and B. Mezerdi, “Near optimality conditions in stochastic control of jump diffusion processes,” Systems & Control Letters, vol. 60, no. 11, pp. 907–916, 2011.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
M. Hafayed, P. Veverka, and S. Abbas, “On maximum principle of near-optimality for diffusions with jumps, with application to consumption-investment problem,” Differential Equations and Dynamical Systems, vol. 20, no. 2, pp. 111–125, 2012.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
M. Hafayed and S. Abbas, “On near-optimal mean-field stochastic singular controls: necessary and sufficient conditions for near-optimality,” Journal of Optimization Theory and Applications, 2013.
View at: Publisher Site | Google Scholar
M. Hafayed, S. Abbas, and P. Veverka, “On necessary and sufficient conditions for near-optimal singular stochastic controls,” Optimization Letters, vol. 7, no. 5, pp. 949–966, 2013.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
J. Huang, X. Li, and G. Wang, “Near-optimal control problems for linear forward-backward stochastic systems,” Automatica, vol. 46, no. 2, pp. 397–404, 2010.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
K. Bahlali, N. Khelfallah, and B. Mezerdi, “Necessary and sufficient conditions for near-optimality in stochastic control of FBSDEs,” Systems & Control Letters, vol. 58, no. 12, pp. 857–864, 2009.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
E. Hui, J. Huang, X. Li, and G. Wang, “Near-optimal control for stochastic recursive problems,” Systems & Control Letters, vol. 60, no. 3, pp. 161–168, 2011.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
L. Zhang, J. Huang, and X. Li, “Stochastic maximum principle for nearoptimal control of fbsdes,” 2012, http://arxiv.org/pdf/1203. 1774.pdf.
View at: Google Scholar
J. Yong, “Optimality variational principle for controlled forward-backward stochastic differential equations with mixed initial-terminal conditions,” SIAM Journal on Control and Optimization, vol. 48, no. 6, pp. 4119–4156, 2010.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
Z. Wu, “A general maximum principle for optimal control of forward-backward stochastic systems,” Automatica, vol. 49, no. 5, pp. 1473–1480, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
S. Peng and Z. Wu, “Fully coupled forward-backward stochastic differential equations and applications to optimal control,” SIAM Journal on Control and Optimization, vol. 37, no. 3, pp. 825–843, 1999.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
Q. Lin, “Optimal control of coupled forward-backward stochastic system with jumps and related hamilton-jacobi-bellman equations,” 2011, http://arxiv.org/pdf/1111.4642.pdf.
View at: Google Scholar
J. Yong and X. Y. Zhou, Stochastic Controls, vol. 43 of Applications of Mathematics, Springer, New York, NY, USA, 1999.
View at: MathSciNet
F. H. Clarke, Optimization and Nonsmooth Analysis, vol. 5 of Classics in Applied Mathematics, SIAM, Philadelphia, Pa, USA, Second edition, 1990.
View at: Publisher Site | MathSciNet
S. Yin, S. X. Ding, A. H. A. Sari, and H. Hao, “Data-driven monitoring for stochastic systems and its application on batch process,” International Journal of Systems Science, vol. 44, no. 7, pp. 1366–1376, 2013.
View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
S. Yin, S. X. Ding, A. Haghani, H. Hao, and P. Zhang, “A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark tennessee eastman process,” Journal of Process Control, vol. 22, no. 9, pp. 1567–1581, 2012.
View at: Publisher Site | Google Scholar
S. Yin, H. Luo, and S. Ding, “Real-time implementation of fault-tolerant control systems with performance optimization,” IEEE Transactions on Industrial Electronics, vol. 61, no. 5, pp. 2402–2411, 2013.
View at: Google Scholar

Copyright

Copyright © 2014 Maoning Tang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

908

Downloads

1442

Citations