About this Journal Submit a Manuscript Table of Contents 
Abstract and Applied Analysis
Volume 2012 (2012), Article ID 709682, 16 pages
doi:10.1155/2012/709682
Research Article

Maximum Principle for Stochastic Recursive Optimal Control Problems Involving Impulse Controls

Zhen Wu1 and Feng Zhang2

1School of Mathematics, Shandong University, Jinan 250100, China
2School of Mathematics and Quantitative Economics, Shandong University of Finance and Economics, Jinan 250014, China

Received 13 January 2012; Accepted 11 April 2012

Academic Editor: Zhenya Yan

Copyright © 2012 Zhen Wu and Feng Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

We consider a stochastic recursive optimal control problem in which the control variable has two components: the regular control and the impulse control. The control variable does not enter the diffusion coefficient, and the domain of the regular controls is not necessarily convex. We establish necessary optimality conditions, of the Pontryagin maximum principle type, for this stochastic optimal control problem. Sufficient optimality conditions are also given. The optimal control is obtained for an example of linear quadratic optimization problem to illustrate the applications of the theoretical results.

1. Introduction

The nonlinear backward stochastic differential equations (BSDEs for short) were first introduced by Pardoux and Peng [1]. Independently, Duffie and Epstein [2] introduced BSDEs under economic background. In [2], they presented a stochastic recursive utility which is an extension of the standard additive utility with the instantaneous utility depending not only on the instantaneous consumption rate but also on the future utility. Actually, it corresponds to the solution of a particular BSDE whose generator does not depend on the variable . And then, El Karoui et al. [3] gave the formulation of recursive utilities from the BSDE point of view. The problem that the cost function of the control system is described by the solution of BSDE is called the stochastic recursive optimal control problem. In this case, the control systems become forward-backward stochastic differential equations (FBSDEs).

One fundamental research direction for optimal control problem is to establish the necessary optimality conditions—Pontryagin maximum principle. Stochastic maximum principle for forward, backward, and forward-backward systems has been studied by many authors, including Peng [4, 5], Tang and Li [6], Wang and Yu [7], Wu [8], and Xu [9] for full information and Huang et al. [10], Wang and Wu [11], Wang and Yu [12], and Wu [13] for partial information case. However, in these papers, there are only regular controls in the control systems and impulse controls are not included.

Stochastic impulse control problems have received considerable research attention in recent years due to wide applicability in a number of different areas, especially in mathematical finance; see, for example, [14–17]. In most cases, the optimal impulse control problem was studied through dynamic programming principle. It was shown in particular that the value function is a solution of some quasi-variational inequalities.

The first result in stochastic maximum principle for singular control problem was obtained by Cadenillas and Haussmann [18], in which linear dynamics, convex cost criterion, and convex state constraint are assumed. Bahlali and Chala [19] generalized [18] to the nonlinear dynamics case with a convex state constraint. Bahlali and Mezerdi [20] considered a stochastic singular control problem in which the control system is governed by a stochastic differential equation where the regular control enters the diffusion coefficient and the control domain is not necessarily convex. The stochastic maximum principle was obtained with the approach developed by Peng [4]. Dufour and Miller [21] studied a stochastic singular control problem in which the admissible control is of bounded variation. It is worth pointing out that the control systems in these works are stochastic differential equations with singular control, and few examples are given to illustrate the theoretical results. Wu and Zhang [22] were the first to study stochastic optimal control problems of forward-backward systems involving impulse controls, and they obtained both the maximum principle and sufficient optimality conditions for the optimal control problem.

In this paper, we continue to study stochastic optimal control problem involving impulse controls, in which the control system is described by a forward-backward stochastic differential equation and the control variable consists of regular control and impulse control. Different from [22], it is assumed in this paper that the domain of the regular controls is not necessarily convex and the control variable does not enter the diffusion coefficient. Thus the result of this paper and that of [22] do not contain each other. We obtain the stochastic maximum principle by using a spike variation on the regular part of the control and a convex perturbation on the impulsive one. Sufficient optimality conditions are also obtained which can help to find the optimal control in applications.

The rest of this paper is organized as follows. In Section 2 we give some preliminary results and the formulation of our stochastic optimal control problem. In Section 3 we obtain the maximum principle for our stochastic optimal control problem. Sufficient optimality conditions for the optimal control problem is established in Section 4, and an example of linear quadratic optimization problem is also given to illustrate the applications of our theoretical results.

2. Formulation of the Stochastic Optimal Control Problem

Firstly we introduce some notations. Let ( ) be a probability space and the expectation with respect to . Let be a finite time horizon and the natural filtration of a -dimensional standard Brownian motion augmented by the -null sets of . For and , denote by the set of -dimensional adapted processes such that , and denote by the set of -dimensional adapted processes such that .

Let be a nonempty subset of and a nonempty convex subset of . Let be a given sequence of increasing -stopping times such that as . We denote by the class of right continuous processes such that each is an -measurable random variable. It is worth noting that the assumption implies that at most finitely many impulses may occur on . Denote by the class of adapted processes such that , and denote by the class of -valued impulse processes such that . We call the admissible control set. In what follows, for a continuous function , the integration is understood as follows:

Given and , we consider the following SDE with impulses: where , , and are measurable mappings. Similar to [22, Proposition 2.1], we have the following.

Proposition 2.1. Let be continuous and , uniformly Lipschitz in . Assume that , , and for some . Then SDE (2.2) admits a unique solution .

For , let us consider the following BSDE with impulses: where , and are measurable mappings. Similar to [22, Proposition 2.2], we have the following.

Proposition 2.2. Let be continuous and Lipschitz in . Assume that , , and for some . Then BSDE (2.3) admits a unique solution .

The control system of our stochastic optimal control problem is subject to the following FBSDE: where , , , are measurable mappings, and , are continuous functions. The objective is to minimize the following cost functional over the class : where , , , and are measurable mappings.

In what follows we assume the following. , , , are continuous, and they are continuously differentiable in , with derivatives continuous and uniformly bounded. Moreover, assume that and have linear growth in . , , , are continuous, and they are continuously differentiable in , with derivatives continuous and bounded by , , , and , respectively. Moreover, we assume for any .

From Propositions 2.1 and 2.2, it follows that FBSDE (2.4) admits a unique solution for any , and the functional is well defined.

3. Stochastic Maximum Principle for the Optimal Control Problem

Let be an optimal control and the corresponding trajectory. We introduce the spike variation with respect to as follows: where is an arbitrarily fixed time, is a sufficiently small constant, and is an arbitrary -valued -measurable random variable such that . Let be such that . Then it is easy to check that , is also an element of . Let us denote by the trajectory associated with . For convenience, denote , for . In what follows, we use to denote a positive constant which can be different from line to line.

Let us introduce the following FBSDE (called the variational equation): By Propositions 2.1 and 2.2, FBSDE (3.2) admits a unique solution .

Similar to [9, Lemma 1], we can easily obtain the following.

Lemma 3.1. We have

We proceed to give the following lemma.

Lemma 3.2. The following estimations hold: where as .

Proof. It is easy to check that where Since , are uniformly bounded, we have . Hence, if we can obtain then the estimation (3.4) can be obtained from Gronwall's lemma and (3.7). Let us take the term for example. By the definition of and Hölder's inequality, we have From Hölder's inequality, Lemma 3.1, and the dominated convergence theorem, it follows that Since is uniformly bounded, by Lemma 3.1 we get Thus we obtain . In the same way we can get Hence, the estimation (3.4) is proved.
Now we prove (3.5) and (3.6). Set It is easy to obtain where We have Since is uniformly bounded, it follows from (3.4) that . Since is continuous and uniformly bounded, from Lemma 3.1 and the dominated convergence theorem it follows that Consequently, From Lemma 3.1 and the dominated convergence theorem, it follows that Since , , and are uniformly bounded, we have Similar to the proof of Lemma 1 in [9] for the BSDE part, we can obtain (3.5) and (3.6) with the iterative method.

We are now ready to state the variational inequality.

Lemma 3.3. The following variational inequality holds:

Proof. From the optimality of , we have From Lemmas 3.1 and 3.2, it follows that Hence, Similarly we get while Since have linear growth, it follows from Lemma 3.2 and Hölder's inequality that By Lemma 3.1 and the dominated convergence theorem, we have where , . It follows from Hölder's inequality that Using Lemma 3.1 again, we get Consequently, The variational inequality follows from (3.25)–(3.32).

Now we introduce the following FBSDE (called the adjoint equation):

It is easy to check that the adjoint equation admits a unique solution .

We are now in a position to state the stochastic maximum principle.

Theorem 3.4. Let be an optimal control, the corresponding trajectory, and the solution of the adjoint equation. Then for any and it holds that where is defined by

Proof. Applying Itô's formula to , by Lemma 3.3 we derive where satisfies . Dividing (3.37) by and letting go to 0, we obtain By choosing in (3.38) we obtain the conclusion (3.35). If we choose , then for satisfying we have Now let us set for any and . Then it is obvious that and . So from (3.39) it follows that, for any , Hence, Since the quantity inside the conditional expectation is -measurable, the conclusion (3.34) can be obtained easily.

Similar to [22, Corollary 3.1], by Theorem 3.4 we can easily obtain the following

Corollary 3.5. Assume . Then for the optimal control it holds that

Remark 3.6. We can still obtain the stochastic maximum principle if the assumptions are relaxed in the following way.(i)The regular control process and the impulse control process are assumed to satisfy and for some .(ii)The assumption in Hypothesis (H2) can be weakened as .(iii)In the spike variation setting, the random variable is assumed to satisfy .
In fact, under these new assumptions both the solutions of the control system (2.4) and the variational equation (3.2) belong to . The conclusion of Lemma 3.1 becomes And Lemmas 3.2 and 3.3 still hold true.

4. Sufficient Optimality Conditions for Optimal Controls

We still denote by the trajectory corresponding to . Let us first introduce an additional assumption.

(H3) The control domain is a convex body in . The maps , , and are locally Lipschitz in the regular control variable .

Theorem 4.1. Let (H1)–(H3) hold. Assume that the functions , , and are convex. Moreover, has the following particular form: for and . Let be the solution of the adjoint equation associated with . Then is an optimal control of the stochastic optimal control problem if it satisfies (3.34) and (3.35).

Proof. Set . Since , , are convex, we have Thus, Set . Then by Itô's formula applied to , we get , where From (3.35) we have . By (3.34) and [23, Lemma 2.3-(iii); Chapter 3], we have . By [23, Lemma 2.4; Chapter 3], we further conclude that Then, by [23, Lemma 2.3-(v); Chapter 3] and the convexity of , we obtain from which it follows immediately that . Thus we obtain and the proof is complete.

We now give an example of linear quadratic optimal control problem involving impulse controls to illustrate the application of our theoretical results.

Example 4.2. For simplicity, assume that the variables and coefficients are scalar-valued. Let us take and . There are only two values −1 and 1 in which is a usual case in practice and represents only two control states: “on” and “off”. For , the controlled system is subject to the following linear FBSDE: and the cost functional is given by The coefficients are deterministic constants such that and . By Propositions 2.1 and 2.2 we know that the control system admits a unique solution for any . And the functional is well defined from into .
Let be an optimal control and the corresponding trajectory. Then the following adjoint equation admits a unique solution . The Hamiltonian is given by Then by Corollary 3.5 we obtain From (4.10) we get From (4.11) we obtain that Hence, if is an optimal control of this linear quadratic control problem, then it satisfies (4.12) and (4.13).
We can prove that obtained in (4.12) and (4.13) is indeed an optimal control of this linear quadratic optimization problem. Note that Theorem 4.1 does not hold now since is not convex in this example. In what follows, we use the same notations as those in the proof of Theorem 4.1. In fact, as in the proof of Theorem 4.1, we can still derive . On the one hand, it follows from (4.13) that . On the other hand, we have where From (4.12) and the definition of , it is easy to get Since , , is convex in , and thus , so we obtain . Consequently, it follows that and the optimality of is proved.

Remark 4.3. For the classical linear quadratic optimal control problem, one can usually obtain an optimal control in a linear state feedback form by virtue of the so-called Riccati equation, and along this line the solvability of the Riccati equation leads to that of the linear quadratic problem. However, it is difficult to obtain a state feedback optimal control in terms of the Riccati equation in Example 4.2 mainly due to the particular form of the regular control domain and the appearance of the impulse control in the control system.

Acknowledgments

The authors would like to thank the referees for valuable suggestions which helped to improve the first version of this paper. Z. Wu acknowledges the financial support from the National Natural Science Foundation of China (10921101 and 61174092) and the Natural Science Fund for Distinguished Young Scholars of China (11125102). F. Zhang acknowledges the financial support from the Natural Science Foundation of Shandong Province, China (ZR2011AQ018), and the Foundation of Doctoral Research Program, Shandong University of Finance and Economics.

References

  1. E. Pardoux and S. G. Peng, “Adapted solution of a backward stochastic differential equation,” Systems & Control Letters, vol. 14, no. 1, pp. 55–61, 1990. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  2. D. Duffie and L. G. Epstein, “Stochastic differential utility,” Econometrica, vol. 60, no. 2, pp. 353–394, 1992. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  3. N. El Karoui, S. G. Peng, and M. C. Quenez, “Backward stochastic differential equations in finance,” Mathematical Finance, vol. 7, no. 1, pp. 1–71, 1997. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  4. S. G. Peng, “A general stochastic maximum principle for optimal control problems,” SIAM Journal on Control and Optimization, vol. 28, no. 4, pp. 966–979, 1990. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  5. S. G. Peng, “Backward stochastic differential equations and applications to optimal control,” Applied Mathematics and Optimization, vol. 27, no. 2, pp. 125–144, 1993. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  6. S. J. Tang and X. J. Li, “Necessary conditions for optimal control of stochastic systems with random jumps,” SIAM Journal on Control and Optimization, vol. 32, no. 5, pp. 1447–1475, 1994. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  7. G. C. Wang and Z. Y. Yu, “A Pontryagin's maximum principle for non-zero sum differential games of BSDEs with applications,” IEEE Transactions on Automatic Control, vol. 55, no. 7, pp. 1742–1747, 2010. View at Publisher · View at Google Scholar
  8. Z. Wu, “Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems,” Systems Science and Mathematical Sciences, vol. 11, no. 3, pp. 249–259, 1998. View at Zentralblatt MATH
  9. W. S. Xu, “Stochastic maximum principle for optimal control problem of forward and backward system,” Journal of the Australian Mathematical Society B, vol. 37, no. 2, pp. 172–185, 1995. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  10. J. H. Huang, G. C. Wang, and J. Xiong, “A maximum principle for partial information backward stochastic control problems with applications,” SIAM Journal on Control and Optimization, vol. 48, no. 4, pp. 2106–2117, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  11. G. C. Wang and Z. Wu, “The maximum principles for stochastic recursive optimal control problems under partial information,” IEEE Transactions on Automatic Control, vol. 54, no. 6, pp. 1230–1242, 2009. View at Publisher · View at Google Scholar
  12. G. C. Wang and Z. Y. Yu, “A partial information non-zero sum differential game of backward stochastic differential equations with applications,” Automatica, vol. 48, no. 2, pp. 342–352, 2012. View at Publisher · View at Google Scholar
  13. Z. Wu, “A maximum principle for partially observed optimal control of forward-backward stochastic control systems,” Science China information sciences, vol. 53, no. 11, pp. 2205–2214, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  14. A. Bensoussan and J.-L. Lions, Impulse Control and Quasivariational Inequalities, Gauthier-Villars, Paris, France, 1984.
  15. M. H. A. Davis and A. R. Norman, “Portfolio selection with transaction costs,” Mathematics of Operations Research, vol. 15, no. 4, pp. 676–713, 1990. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  16. M. Jeanblanc-Piqué, “Impulse control method and exchange rate,” Mathematical Finance, vol. 3, pp. 161–177, 1993.
  17. R. Korn, “Some applications of impulse control in mathematical finance,” Mathematical Methods of Operations Research, vol. 50, no. 3, pp. 493–518, 1999. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  18. A. Cadenillas and U. G. Haussmann, “The stochastic maximum principle for a singular control problem,” Stochastics and Stochastics Reports, vol. 49, no. 3-4, pp. 211–237, 1994. View at Zentralblatt MATH
  19. S. Bahlali and A. Chala, “The stochastic maximum principle in optimal control of singular diffusions with non linear coefficients,” Random Operators and Stochastic Equations, vol. 13, no. 1, pp. 1–10, 2005. View at Publisher · View at Google Scholar
  20. S. Bahlali and B. Mezerdi, “A general stochastic maximum principle for singular control problems,” Electronic Journal of Probability, vol. 10, no. 30, pp. 988–1004, 2005. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  21. F. Dufour and B. Miller, “Maximum principle for singular stochastic control problems,” SIAM Journal on Control and Optimization, vol. 45, no. 2, pp. 668–698, 2006. View at Publisher · View at Google Scholar
  22. Z. Wu and F. Zhang, “Stochastic maximum principle for optimal control problems of forward-backward systems involving impulse controls,” IEEE Transactions on Automatic Control, vol. 56, no. 6, pp. 1401–1406, 2011. View at Publisher · View at Google Scholar
  23. J. M. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, vol. 43, Springer, New York, NY, USA, 1999.