Linear Quadratic Stochastic Optimal Control of Forward Backward Stochastic Control System Associated with Lévy Process

Huang, Hong; Wang, Xiangrong; Hou, Ting; Xu, Lu

doi:https://doi.org/10.1155/2017/2541687

Mathematical Problems in Engineering

On this page

Abstract Introduction Preliminaries Conclusion Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Stochastic Systems and Control: Theory and Applications

View this Special Issue

Research Article | Open Access

Volume 2017 | Article ID 2541687 | https://doi.org/10.1155/2017/2541687

Linear Quadratic Stochastic Optimal Control of Forward Backward Stochastic Control System Associated with Lévy Process

Hong Huang,^1,2Xiangrong Wang ,¹Ting Hou,³and Lu Xu⁴

Academic Editor: Zhongwei Lin

Received05 Apr 2017

Accepted17 Aug 2017

Published25 Sept 2017

Abstract

This paper analyzes one kind of linear quadratic (LQ) stochastic control problem of forward backward stochastic control system associated with Lévy process. We obtain the explicit form of the optimal control, then prove it to be unique, and get the linear feedback regulator by introducing one kind of generalized Riccati equation. Finally, we discuss the solvability of the generalized Riccati equation, and its existence and uniqueness of the solutions are proved in a special case.

1. Introduction

LQ stochastic optimal control is a kind of special optimal control problem, which not only can be used to model many linear optimal problems practically, but also can reasonably be used to approach and solve many nonlinear problems. In 1962, Kushner [1] firstly established a forward random stochastic LQ model with a dynamic programming method and Wonham [2] firstly studied a LQ stochastic optimal control problem by introducing a Riccati equation in 1968. Then a lot of works have been done for forward or backward stochastic LQ control problems, the corresponding Riccati equation, and its application in finance, such as Li and Zhang [3], Ma and Hou [4], Liu et al. [5], Wang et al. [6], and Shen and Wang [7]. In 2003, Wang et al. [8] discussed a special kind of forward backward stochastic LQ problem and got the existence and uniqueness of the optimal control for the control system. Subsequently, Wu [9] extended this conclusion to the fully coupled forward backward stochastic LQ problem.

The optimal control problem with random jumps was first considered by Boel and Varaiya [10]; in this case, the control system is often described by Brownian motion and Poisson processes. On the basis of proving the existence and uniqueness of solutions of a kind of forward backward stochastic differential equation with Poisson jumps (FBSDEP), Wu and Wang [11] got the explicit form of the optimal control for LQ stochastic control problem where the state variable was described by a stochastic differential equation with a Poisson process (SDEP). In 2009, Shi and Wu [12] extended Wu and Wang’s results in [11] to a fully coupled LQ stochastic control problem of forward backward stochastic control system with Poisson jumps. Moreover, Lin and Zhang [13] considered the control problem for linear stochastic systems driven by both Brownian motion and Poisson jumps. In 2016, Li et al. [14] studied a stochastic differential equations driven by G-Brownian motion and got the existence and uniqueness of the solution for these equations.

In 2000, Nualart and Schoutens [15] introduced a class of Lévy processes with exponential moments satisfying some conditions. Using these exponential moments and the standard orthogonalization process, they constructed a series of orthogonal normal martingales called Teugels martingale. And they also proved a martingale representation theorem associated with Teugels martingale. In the next year, Nualart and Schoutens [16] considered a backward stochastic differential equation (BSDE) driven by Teugels martingale and proved the existence and uniqueness theory of this BSDE. In 2003, Bahlali et al. [17] studied a BSDE driven by Teugels martingale and an independent Brownian motion; they got the existence, uniqueness, and comparison of solutions for these equations, having a Lipschitz or locally Lipschitz coefficient. El Otmani [18] considered a kind of generalized BSDE (GBSDE) associated with Teugels martingale and Brownian motion associated with a pure jump-independent Lévy process. They got the existence and uniqueness theory of this GBSDE when the coefficient verifies some conditions of Lipschitz. More results about BSDE associated with Teugels martingale can be found in the theses of El Otmani [19], Ren and Fan [20], Tang and Zhang [21], and Huang and Wang [22]. On the basis of these results, in 2008, Mitsui and Tabata [23] studied a LQ regulation stochastic control problem with Lévy process and obtained the optimal control for the nonhomogeneous case. In [24], Tang and Wu considered the following LQ stochastic control problem in a given finite horizon with Lévy process:and the cost function wasThey show that the solvability of one kind of generalized Riccati equation is sufficient to the well-posedness of this LQ problem and proved the existence of the optimal control.

In this paper, we consider one kind of LQ stochastic control problem where the controlled system is driven by a fully coupled linear forward backward stochastic differential equation associated with Lévy process (FBSDEL).where are adapted stochastic processes taking values in and is adapted stochastic process called admissible control process. Assume the control process set and define the admissible control set as follows:

The cost functional we considered isAnd the optimal control problem is to find , such that

Note that (3) is a fully coupled FBSDEL. In 2012, Pereira and Shamarova [25] firstly considered this kind of FBSDEL, obtained a solution to this FBSDEL via a partial integrodifferential equation, and proved the uniqueness. Under some monotonicity assumptions, Baghery et al. [26] proved the existence and uniqueness of solutions of fully coupled FBSDEL and then obtained the existence of an open-loop Nash equilibrium point for nonzero sum stochastic differential games by using this result. Based on [25], Wang and Huang [27] got the maximum principle for forward backward stochastic control system driven by Lévy process; then they discussed a kind of LQ stochastic control problem of forward backward stochastic control system and got a necessary condition for the optimal control.

We extend the result of Shi and Wu [12] to the fully coupled linear forward backward stochastic control system driven by Brownian motion and an independent Teugels martingale. Since Teugels martingale is more complex than the Poisson process, we also need more general formula about càdlàg semimartingale. The rest of this paper is organized as follows. In Section 2, we provide a list of notations and results of the existence and uniqueness of solutions of fully coupled FBSDEL. In Section 3, we prove the existence and uniqueness of the optimal control of LQ stochastic control problem (6) and give the linear feedback regulator for the optimal control by the solution of a kind of generalized matrix-valued Riccati equation when assuming the coefficient matrices are deterministic. In Section 4, the solvability of this kind of matrix-valued Riccati equation is discussed.

2. Preliminaries and Notations

Let be a complete probability space satisfying the usual conditions; is a right continuous increasing family of complete sub-algebra which is generated by the following two mutually independent processes: a one-dimensional standard Brownian motion and an valued Lévy process with a standard Lévy measure satisfy(i),(ii), for every and for some .

Naluart and Schoutens denoted Teugels martingale associated with the Lévy process by , and is given bywhere is the compensated power-jump process of order and is power-jump processes: Coefficients correspond to orthonormalization of the polynomials with respect the measure . Please refer to Naluart and Schoutens [15] for more details about Teugels martingale.

Introduce the following notations adopted in this paper: : the inner product in , : the norm in , : the space of valued measurable random variable satisfies : the space of valued measurable process satisfies : the space of valued satisfies : the space of valued measurable processes satisfies : the space of valued measurable càdlàg process satisfies

For notational brevity, we set

Next, consider the following fully coupled FBSDELwhere , , ,

For a given full rank matrix , set

Assumption 1. (i) , and are uniformly Lipschitz continuous with respect to .
(ii) For each , and , where , respectively.
(iii) is uniformly Lipschitz continuous with respect to and , .

Assumption 2. where , , , , , , and ,, are nonnegative constants with and . Moreover, we have (resp., ) when (resp., ).

Lemma 3 (existence and uniqueness theorem of FBSDEL [25]). Under Assumptions 1 and 2, FBSDEL (10) admits a unique solution in .

In the following sections we also need the more general Ito’s formula about a càdlàg semimartingales.

Lemma 4 (Ito’s formula [27]). Let be càdlàg semimartingales, denote as the quadratic variation process, is a real valued function, then is also a semimartingales, and the following Ito’s formula holds where is the continuous part of .
In particular, when and , where are two càdlàg semimartingales, we getHere is the quadratic covariation of .

3. Linear Quadratic Stochastic Optimal Control Problem

Let us consider the LQ stochastic optimal control problem (6). First of all, we give some necessary explanations for the coefficients in the system:

, , and are all bounded progressively measurable matrix-valued processes. are nonnegative symmetric bounded progressively measurable matrix-valued processes, and is a positive bounded progressively measurable matrix-valued process; the inverse is , which is also bounded. is a adapted nonnegative symmetric bounded matrix-valued random variable.

For a given admissible control , under assumptions of the coefficients above, we can verify that FBSDEL (3) satisfies Assumptions 1 and 2. Therefore, there exists a unique solution satisfying the control system (3) from Lemma 3.

Then we get the explicit form of the optimal control for the LQ stochastic optimal control problem (6).

Theorem 5. There exists a unique optimal control for LQ stochastic optimal control problem (6), and is given by the following equation.

Proof. As we know, for a given admissible control , the control system (15) has a unique solution .
Existence. For any admissible control , assume the corresponding trajectory is ; thenApplying Ito’s formula to we have Since , and are nonnegative and is positive, we can get Then the admissible control defined by (15) is the optimal control of LQ stochastic control problem (6).
Unique. Assume admissible control is an optimal control; the corresponding trajectories are and is another optimal control; the corresponding trajectories are . So the trajectories corresponding to areand the trajectories corresponding to areSince and are both optimal controls, is positive, and , are nonnegative, we haveHere is a constant and ; thenhence, in .

Assume , , and are all deterministic matrices, denoted as , and for convenience. Introducing the following generalized matrix-valued Riccati equation (23),

Then we can get the following conclusions.

Theorem 6. Suppose the generalized matrix-valued Riccati equation (23) has solution for all ; then the optimal linear feedback regulator for LQ stochastic optimal control problem (6) is and the optimal value function is

Proof. If is the solution of the matrix-valued Riccati equation (23), then we can check that the solution of (6) satisfiesAs we have proved that the optimal control has the form of (15), take (26) into (15); then the optimal control can be written byFor the optimal value function, using Ito’s formula to , then On the other hand, from the relationship of and , we can verify that and then By the definition of cost function (5), we prove that the optimal value function is

Now consider a special case of stochastic LQ control problem when , and the control system is reduced toThe cost functional now is

Remark 7. Comparing the LQ stochastic optimal control system (32) and control system (1) which was considered in [22] by Tang and Wu, we know that control system (1) is a special case of control system (32) when .

We can get the following Corollary 8 easily from Theorem 5.

Corollary 8. There exists a unique optimal control for LQ stochastic optimal control problem (32)-(33), andwhere the is the solution of the following BSDE driven by Lévy process.

Assume , , , and are all deterministic; then Riccati equation (23) changes toThen from Theorem 6 we can get Corollary 9.

Corollary 9. For LQ stochastic optimal control problem (32)-(33), if, for all , there exist matrices satisfying (36), then the optimal linear feedback regulator isand the optimal value function is

4. Solvability of the Generalized Riccati Equation

From the discussion of the previous section, we can see that the key to get the optimal linear feedback regulator for LQ stochastic optimal control problem is the solvability of the generalized Riccati equation (23). But (23) is so complicated that we cannot prove its existence and uniqueness at this moment. Using technique introduced by Shi and Wu [12], we only discuss a special case: ; in this case Riccati equation (23) becomesEquivalently, consider the following equation:

Compare (39) and (40); we can find that if we can prove the solution of (40), then is the solution of the Riccati equation (39).

In the following, we will focus on the existence and uniqueness of solutions of (40). Firstly, let denote the space of all nonnegative symmetric matrices, and is a Banach space of valued continuous functions on . We have the following uniqueness result.

Theorem 10. The Riccati equation (40) admits at most one solution .

Proof. Suppose satisfying is another solution of (40). Let ; thenwhere and are uniformly bounded as they are continuously in ; apply Gronwall’s inequality; we can get, for all , . Then we prove the uniqueness of solution.

For the existence part, first of all, if we let then from the conventional Riccati equation theory, for all , the following conventional Riccati equationhas a unique solution , when

Let be the subspace of which is formed by the symmetric matrices satisfying (46). Obviously, as the definition of is reasonable. Define a mapping ; we can get Lemma 11 about and .

Lemma 11. The operators are monotonously increasing when , and the operator is continuous and monotonously increasing.

Proof. When , from the definition of , we haveSo if , then ; that is, is monotonously increasing when .
As , set ; then the conventional Riccati equation (45) can be rewrittenFrom the conclusion of above in this lemma and Lemma in [28], if then ; the operator is monotonously increasing. On the other hand, by Gronwall’s inequality, we know that if , then , so the operator is also continuous.

For (45), it is easy to know that if there exists satisfyingthen Riccati equation (40) admits a unique solution. So the following task is to find the suitable satisfying (49). We need the following lemma.

Lemma 12. If there exist , which satisfythen Riccati equation (40) admits a solution .

Proof. For given which satisfied (49), define the sequences as follows: From (50) and Lemma 11, by induction, we obtain and ; we have So is a solution of (45) corresponding to ; then where is a solution of Riccati equation (40). By the same step, we can get and is also a solution of Riccati equation (40). From Theorem 10, .

From Lemma 12, in order to get the existence of solution for Riccati equation (40), we only need to find and satisfying (50). Obviously, we can let and for the existence of , we need the following Assumption 13.

Assumption 13. There exists , such thatwhere is the unique solution of the following equation:

It is easy to known that when matrix is invertible and , Assumption 13 is satisfied. Then we get the main result of this section.

Theorem 14. Let and Assumption 13 holds; the Riccati equation (40) has a unique solution .

At last, we give a simple example of the Riccati equation which has a unique solution.

Example 15. Assume the dimensions of the state and control are the same; that is, , when in Riccati equation (39); let ; then check Assumption 13.

In fact, , so ; here is the solution of, and then . From Theorem 14, the Riccati equation (39) has a unique solution when , .

5. Conclusion

In this paper, we discussed one kind of LQ stochastic control problem with Lévy process as noise source where the control system is described by a linear FBSDEL. Explicit form of optimal control is obtained, and it can be proved to be unique. When assuming that all the coefficient matrices in this control problem are deterministic, it has been shown that the linear feedback regulator for this LQ problem has a close relation to the solutions of a kind of generalized Riccati equation. Finally, we discuss the solvability of the generalized Riccati equation and prove the existence and uniqueness of the solution for it in a special case.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (no. 11271007, no. 61673013), the Natural Science Foundation of Shandong Province (no. ZR2016JZ022), the SDUST Research Fund (no. 2014JQJH103), and the Graduate Student Innovation Fund of SDUST (no. SDKDYC170345).

References

H. Kushner, “Optimal Stochastic Control,” IRE Transactions on Automatic Control, vol. 7, no. 5, pp. 120–122, 1962.
View at: Publisher Site | Google Scholar
W. M. Wonham, “On a matrix Riccati equation of stochastic control,” SIAM Journal on Control and Optimization, vol. 6, pp. 681–697, 1968.
View at: Publisher Site | Google Scholar | MathSciNet
G. Li and W. Zhang, “Study on indefinite stochastic linear quadratic optimal control with inequality constraint,” Journal of Applied Mathematics, vol. 2013, Article ID 805829, 9 pages, 2013.
View at: Publisher Site | Google Scholar
H.-j. Ma and T. Hou, “A separation theorem for stochastic singular linear quadratic control problem with partial information,” Acta Mathematicae Applicatae Sinica. English Series, vol. 29, no. 2, pp. 303–314, 2013.
View at: Publisher Site | Google Scholar | MathSciNet
X. Liu, Y. Li, and W. Zhang, “Stochastic linear quadratic optimal control with constraint for discrete-time systems,” Applied Mathematics and Computation, vol. 228, pp. 264–270, 2014.
View at: Publisher Site | Google Scholar | MathSciNet
X. Wang, X. Zhang, and P. Zhao, “Binary nonlinearization for AKNS-KN coupling system,” Abstract and Applied Analysis, Article ID 253102, 12 pages, 2014.
View at: Publisher Site | Google Scholar | MathSciNet
C. H. Shen and X. R. Wang, “Nonlinear analysis on the pattern structures of connection between final markets,” Journal of Management sciences in China, vol. 18, no. 2, pp. 66–75, 2015.
View at: Google Scholar
X.-R. Wang, Z.-Y. Gao, and Z. Wu, “Forward-backward stochastic differential equation and the linear quadratic stochastic optimal control,” Acta Automatica Sinica, vol. 29, no. 1, pp. 32–37, 2003.
View at: Google Scholar | MathSciNet
Z. Wu, “Forward-backward stochastic differential equations, linear quadratic stochastic optimal control and nonzero sum differential games,” Journal of Systems Science and Complexity, vol. 2, pp. 179–192, 18.
View at: Google Scholar
R. Boel and P. Varaiya, “Optimal control of jump processes,” SIAM Journal on Control and Optimization, vol. 15, no. 1, pp. 92–119, 1977.
View at: Publisher Site | Google Scholar | MathSciNet
Z. Wu and X. Wang, “FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps,” Acta Automatica Sinica, vol. 29, no. 6, pp. 821–826, 2003.
View at: Google Scholar | MathSciNet
J. T. Shi and Z. Wu, “One kind of fully coupled linear quadratic stochastic control problem with random jumps,” Acta Automatica Sinica. Zidonghua Xuebao, vol. 35, no. 1, pp. 92–97, 2009.
View at: Publisher Site | Google Scholar | MathSciNet
X. Lin and R. Zhang, “H∞ control for stochastic systems with Poisson jumps,” Journal of Systems Science & Complexity, vol. 24, no. 4, pp. 683–700, 2011.
View at: Publisher Site | Google Scholar | MathSciNet
X. Li, X. Lin, and Y. Lin, “Lyapunov-type conditions and stochastic differential equations driven by G-Brownian motion,” Journal of Mathematical Analysis and Applications, vol. 439, no. 1, pp. 235–255, 2016.
View at: Publisher Site | Google Scholar | MathSciNet
D. Nualart and W. Schoutens, “Chaotic and predictable representations for Lévy processes,” Stochastic Processes and their Applications, vol. 90, no. 1, pp. 109–122, 2000.
View at: Publisher Site | Google Scholar | MathSciNet
D. Nualart and W. Schoutens, “Backward stochastic differential equations and Feynman-Kac formula for Lévy processes, with applications in finance,” Bernoulli. Official Journal of the Bernoulli Society for Mathematical Statistics and Probability, vol. 7, no. 5, pp. 761–776, 2001.
View at: Publisher Site | Google Scholar | MathSciNet
K. Bahlali, M. Eddahbi, and E. Essaky, “BSDE associated with Lévy processes and application to PDIE,” Journal of Applied Mathematics and Stochastic Analysis, vol. 16, no. 1, pp. 1–17, 2003.
View at: Publisher Site | Google Scholar | MathSciNet
M. El Otmani, “Generalized BSDE driven by a Lévy process,” Journal of Applied Mathematics and Stochastic Analysis, Art. ID 85407, 25 pages, 2006.
View at: Publisher Site | Google Scholar | MathSciNet
M. El Otmani, “Backward stochastic differential equations associated with Lévy processes and partial integro-differential equations,” Communications on Stochastic Analysis, vol. 2, no. 2, pp. 277–288, 2008.
View at: Google Scholar | MathSciNet
Y. Ren and X. Fan, “Reflected backward stochastic differential equations driven by a lvy process,” ANZIAM Journal, vol. 50, no. 4, pp. 486–500, 2009.
View at: Publisher Site | Google Scholar
M. Tang and Q. Zhang, “Optimal variational principle for backward stochastic control systems associated with Lévy processes,” Science China. Mathematics, vol. 55, no. 4, pp. 745–761, 2012.
View at: Publisher Site | Google Scholar | MathSciNet
H. Huang and X. Wang, “LQ stochastic optimal control of forward-backward stochastic control system driven by Lévy process,” in Proceedings of the 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference, IMCEC 2016, pp. 1939–1943, chn, October 2016.
View at: Publisher Site | Google Scholar
K.-i. Mitsui and Y. Tabata, “A stochastic linear-quadratic problem with Lévy processes and its application to finance,” Stochastic Processes and their Applications, vol. 118, no. 1, pp. 120–152, 2008.
View at: Publisher Site | Google Scholar | MathSciNet
H. Tang and Z. Wu, “Stochastic differential equations and stochastic linear quadratic optimal control problem with Lévy processes,” Journal of Systems Science & Complexity, vol. 22, no. 1, pp. 122–136, 2009.
View at: Publisher Site | Google Scholar | MathSciNet
R. S Pereira and E. Shamarova, “Forward backward SDEs driven by Lévy processes and application to option pricing,” Random Operators and Stochastic Equations, vol. 2, no. 1, pp. 1–20, 2012.
View at: Google Scholar
F. Baghery, N. Khelfallah, B. Mezerdi, and I. Turpin, “Fully coupled forward backward stochastic differential equations driven by Lévy processes and application to differential games,” Random Operators and Stochastic Equations, vol. 22, no. 3, pp. 151–161, 2014.
View at: Publisher Site | Google Scholar | MathSciNet
X. Wang and H. Huang, “Maximum principle for forward-backward stochastic control system driven by Lévy process,” Mathematical Problems in Engineering, Article ID 702802, Art. ID 702802, 12 pages, 2015.
View at: Publisher Site | Google Scholar | MathSciNet
S. Peng, “Problem of eigenvalues of stochastic Hamiltonian systems with boundary conditions,” Stochastic Processes and their Applications, vol. 88, no. 2, pp. 259–290, 2000.
View at: Publisher Site | Google Scholar | MathSciNet

Copyright

Copyright © 2017 Hong Huang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1220

Downloads

799

Citations