Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015 (2015), Article ID 892304, 13 pages
Research Article

Maximum Principle for Optimal Control Problems of Forward-Backward Regime-Switching Systems Involving Impulse Controls

School of Mathematics, Shandong University, Jinan, Shandong 250100, China

Received 15 April 2014; Accepted 28 August 2014

Academic Editor: Guangchen Wang

Copyright © 2015 Shujun Wang and Zhen Wu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


This paper is concerned with optimal control problems of forward-backward Markovian regime-switching systems involving impulse controls. Here the Markov chains are continuous-time and finite-state. We derive the stochastic maximum principle for this kind of systems. Besides the Markov chains, the most distinguishing features of our problem are that the control variables consist of regular and impulsive controls, and that the domain of regular control is not necessarily convex. We obtain the necessary and sufficient conditions for optimal controls. Thereafter, we apply the theoretical results to a financial problem and get the optimal consumption strategies.

1. Introduction

Maximum principle was first formulated by Pontryagin et al.’s group [1] in the 1950s and 1960s, which focused on the deterministic control system to maximize the corresponding Hamiltonian instead of the optimization problem. Bismut [2] introduced the linear backward stochastic differential equations (BSDEs) as the adjoint equations, which played a role of milestone in the development of this theory. The general stochastic maximum principle was obtained by Peng in [3] by introducing the second order adjoint equations. Pardoux and Peng also proved the existence and uniqueness of solution for nonlinear BSDEs in [4], which has been extensively used in stochastic control and mathematical finance. Independently, Duffie and Epstein introduced BSDEs under economic background, and in [5] they presented a stochastic recursive utility which was a generalization of the standard additive utility with the instantaneous utility depending not only on the instantaneous consumption rate but also on the future utility. Then El Karoui et al. gave the formulation of recursive utilities from the BSDE point of view. As found by [6], the recursive utility process can be regarded as a solution of BSDE. Peng [7] first introduced the stochastic maximum principle for optimal control problems of forward-backward control system as the control domain is convex. Since BSDEs and forward-backward stochastic differential equations (FBSDEs) are involved in a broad range of applications in mathematical finance, economics, and so on, it is natural to study the control problems involving FBSDEs. To establish the necessary optimality conditions, Pontryagin maximum principle is one fundamental research direction for optimal control problems. Rich literature for stochastic maximum principle has been obtained; see [812] and the references therein. Recently, Wu [13] established the general maximum principle for optimal controls of forward-backward stochastic systems in which the control domains were nonconvex and forward diffusion coefficients explicitly depended on control variables.

The applications of regime-switching models in finance and stochastic control also have been researched in recent years. Compared to the traditional system based on the diffusion processes, it is more meaningful from the empirical point of view. Specifically, it modulates the system with a continuous-time finite-state Markov chain with each state representing a regime of the system or a level of economic indicator. Based on the switching diffusion model, much work has been done in the fields of option pricing, portfolio management, risk management, and so on. In [14], Crépey focused on the pricing equations in finance. Crépey and Matoussi [15] investigated the reflected BSDEs with Markov chains. For the controlled problem with regime-switching model, Donnelly studied the sufficient maximum principle in [16]. Using the results about BSDEs with Markov chains in [14, 15], Tao and Wu [17] derived the maximum principle for the forward-backward regime-switching model. Moreover, in [18] the weak convergence of BSDEs with regime switching was studied. For more results of Markov chains, readers can refer to the references therein.

In addition, stochastic impulse control problems have received considerable research attention due to their wide applications in portfolio optimization problems with transaction costs (see [19, 20]) and optimal strategy of exchange rates between different currencies [21, 22]. Korn [23] also investigated some applications of impulse control in mathematical finance. For a comprehensive survey of theory of impulse controls, one is referred to [24]. Wu and Zhang [25] first studied stochastic optimal control problems of forward-backward systems involving impulse controls, in which they assumed the domain of the regular controls was convex and obtained both the maximum principle and sufficient optimality conditions. Later on, in [26] they considered the forward-backward system in which the domain of regular controls was not necessarily convex and the control variable did not enter the diffusion coefficient.

In this paper, we consider a stochastic control system, in which the control system is described by a forward-backward stochastic differential equation, all the coefficients contain Markov chains, and the control variables consist of regular and impulsive parts. This case is more complicated than [17, 25, 26]. We obtain the stochastic maximum principle by using spike variation on the regular control and convex perturbation on the impulsive one. Applying the maximum principle to a financial investment-consumption model, we also get the optimal consumption processes and analyze the effects on consumption by various economic factors.

The rest of this paper is organized as follows. In Section 2, we give preliminaries and the formulation of our problems. A necessary condition in the form of maximum principle is established in Section 3. Section 4 aims to investigate sufficient optimality conditions. An example in finance is studied in Section 5 to illustrate the applications of our theoretical results and some figures are presented to give more explanations. In the end, Section 6 concludes the novelty of this paper.

2. Preliminaries and Problem Formulation

Let be a complete filtered probability space equipped with a natural filtration generated by , , where is a -dimensional standard Brownian motion defined on the space, is a finite-state Markov chain with the state space given by , and is a fixed time horizon. The transition intensities are for with nonnegative and bounded. . For , denote by the set of -dimensional adapted processes such that and denote by the set of -dimensional adapted processes such that .

Define as the integer-valued random measure on which counts the jumps from to state between time 0 and . The compensator of is , which means is a martingale (compensated measure). Then the canonical special semimartingale representation for is given by Define . Denote by the set of measurable functions from to endowed with the topology of convergence in measure and the norm of ; denote by the space of -measurable functions such that .

Let be a nonempty subset of and nonempty convex subset of . Let be a given sequence of increasing -stopping times such that as . Denote by the class of right continuous processes such that each is an -measurable random variable. It’s worth noting that, the assumption implies that at most finitely many impulses may occur on . Denote by the class of adapted processes such that and denote by the class of -valued impulse processes such that . is called the admissible control set. For notational simplicity, in what follows we focus on the case where all processes are 1-dimensional.

Now we consider the forward regime-switching systems modulated by continuous-time, finite-state Markov chains involving impulse controls. Let , , and be measurable mappings. Given and , the system is formulated byThe following result is easily obtained.

Proposition 1. Assume that are Lipschitz with respect to , , , , and is a continuous function. Then SDE (2) admits a unique solution .

Given and , consider the following backward regime-switching system modulated by Markov chains involving impulse controls:where and are measurable mappings and is a measurable function such that .

Proposition 2. Assume that is Lipschitz with respect to , , , and is a continuous function. Then BSDE (3) admits a unique solution .

Proof. Define and , . It is easy to check thatSince is uniformly bounded, we haveHere are positive constants. Then is Lipschitz with respect to . We also get that and . Hence, the following BSDEadmits a unique solution (see [15, 18] for details). Now define , , and . Then it is easy to check that solves BSDE (3).
Let and be two solutions of (3). Applying Itô’s formula to and combining Gronwall’s inequality, we get the uniqueness of solution.

Now, we consider the following stochastic control system:where , , , and are deterministic measurable functions and , are continuous functions. In what follows will be written as for short. The objective is to maximize, over class , the cost functionalwhere , , and are deterministic measurable functions. A control which solves this problem is called an optimal control.

In what follows, we make the following assumptions.(H1), , , , , , and are continuous and continuously differentiable with respect to . have linear growth with respect to . is continuous and continuously differentiable with respect to .(H2)The derivatives of , , , and are bounded.(H3)The derivatives of , and are bounded by , , , and , respectively. Moreover, for any .

From Propositions 1 and 2, it follows that, under (H1)–(H3), FBSDE (7) admits a unique solution for any .

3. Stochastic Maximum Principle

In this section, we will derive the stochastic maximum principle for optimal control problem (7) and (8). We give the necessary conditions for optimal controls.

Let and be an optimal control of this stochastic control problem and let be the corresponding trajectory. Now, we introduce the spike variation with respect to as follows:where is an arbitrarily fixed time, is a sufficiently small constant, and is an arbitrary -valued -measurable random variable such that . Let be such that . For the reason that domain is convex, we can check that , , is also an element of . Let be the trajectory corresponding to . For convenience, we denote , for , where , .

Introduce the following FBSDE which is called the variational equation:Obviously, this FBSDE admits a unique solution .

We have the following lemma. In what follows, we denote by a positive constant which can be different from line to line.

Lemma 3. Consider

Proof. By the boundedness of and using Hölder’s inequality, we have. Noting the definition of , we getHere we apply Hölder’s inequality for , , and the growth condition of in (H1). Since is bounded on , then (11) is obtained by applying Gronwall’s inequality.
By the result of Section  5 in [6] and noting that the predictable covariation of is we obtainOn the one hand, since is bounded, by (11), we have On the other hand, since is bounded, using the basic inequality and (11), we haveFrom the growth condition of in (H1) and the same technique as above, it follows thatBesides, is bounded on ; then (12) is obtained. The proof is complete.

Denote , , , and , and then we have the following.

Lemma 4. Considerwhere as .

Proof. It is easy to check that satisfieswhereThen we have. Since , by the boundedness of , we have . Further we get On the other hand, since , we have whereSince is bounded, by Lemma 3 we getFor , by Hölder’s inequality, Lemma 3, and the dominated convergence theorem, it follows thatThen we get and obtainIn the same way, we haveFrom (24), (31), and (32) it follows thatFinally, applying Gronwall’s inequality implies (20).

To get estimate (21), for simplicity, we introduceIt is easy to check that satisfieswhereSimilar to the proof above, we haveThen for BSDE (35), by the estimates of BSDEs, we obtainApplying Hölder’s inequality, Cauchy-Schwartz inequality, the dominated convergence theorem, Lemma 3, and (20) and noting the boundedness of , we obtain (21).

Now, we are ready to state the variational inequality.

Lemma 5. The following variational inequality holds:

Proof. From the optimality of , we have By Lemmas 3 and 4, we haveSimilarly, we obtainNext, we aim to get the first term of (39). For convenience, we introduce two notations as follows:Applying the same technique to the proof of Lemma 4, we obtain HenceThus, variational inequality (39) follows from (41)–(45).
Let us introduce the following adjoint equations:where for . It is easy to check that SDE (46) admits a unique solution . Besides, the generator of BSDE (47) does not contain . Therefore, the Lipschitz condition is satisfied obviously. Hence (47) admits a unique solution . Now we establish the stochastic maximum principle.

Theorem 6. Let assumptions (H1)–(H3) hold. Suppose is an optimal control, is the corresponding trajectory, and is the solution of adjoint equations (46) and (47). Then, , it holds thatwhere is the Hamiltonian defined bywhere .

Proof. Applying Itô’s formula to and combining with Lemma 5, we obtainwhere such that . Then it follows thatLetting , we obtainBy choosing we get (49). Setting , then for any we haveLet for and . Obviously and . Then it follows that for any which impliesThe proof is complete.

4. Sufficient Optimality Conditions

In this section, we add additional assumptions to obtain the sufficient conditions for optimal controls. Let us introduce the following.(H4)The control domain is a convex body in . The measurable functions , and are locally Lipschitz with respect to , and their partial derivatives are continuous with respect to .

Theorem 7. Let (H1)–(H4) hold. Suppose that the functions , , , and are concave and is the solution of adjoint equations (46) and (47) corresponding to control . Moreover, assume that is of the special form , , where is a deterministic measurable function and . Then is an optimal control if it satisfies (48) and (49).

Proof. Let be the trajectory corresponding to . By the concavity of and , we deriveDefine Applying Itô’s formula to and noting , we obtainBy (48) and Lemma 2.3 of Chapter  3 in [27], we have By Lemma 2.4 of Chapter  3 in [27], we further conclude that Finally, by the concavity of and (49), we obtain , . Thus, it follows that . We complete the proof.

5. Application in Finance

This section is devoted to studying an investment and consumption model under the stochastic recursive utility arising from financial markets, which naturally motivates the study of the problem (7) and (8).

5.1. An Example in Finance

In a financial market, suppose there are two kinds of securities which can be invested: a bond, whose price is given byand a stock, whose price isHere, is the standard Brownian motion and , , and are bounded deterministic functions. For the sake of rationality, we assume , . Here, stands for a positive constant, which ensures that is nondegenerate. In reality, in order to get stable profit and avoid risk of bankruptcy, many small companies and individual investors usually make a plan at the beginning of a year or a period, in which the weight invested in stock was fixed. Denote by the weight invested in stock which is called the portfolio strategy. It means no matter how much the wealth is, the portfolio strategy is fixed, which is a bounded deterministic function with respect to . Then the wealth dynamics are given aswhere , , and . Here, is a continuous consumption process, is a piecewise consumption process, and is a weight factor. Not only in the mode of continuous consumption, but also in reality society, one consumes piecewise. Hence our setting of consumption process is practical.

Besides, if the macroeconomic conditions are also taken into account in this model, above model has obvious imperfections because it lacks the flexibility to describe the changing stochastically of investment environment. One can modulate the uncertainty of the economic situation by a continuous-time finite-state Markov chain. Then the wealth is formulated by a switching process asLet be a nonempty subset of and a nonempty convex subset of . Suppose is the natural filtration generated by the Brownian motion and the Markov chains, is an -progressively measurable process satisfying is a fixed sequence of increasing -stopping times, and each is an -measurable random variable satisfyingWe consider the following stochastic recursive utility, which is described by a BSDE with the Markov chain :where , . The recursive utility is meaningful in economics and theory. Details can be found in Duffie and Epstein [5] and El Karoui et al. [6].

Define the associated utility functional aswhere , , and are positive constants, is a discount factor, and is also called Arrow-Pratt index of risk aversion (see, e.g., Karatzas and Shreve [28]). To get the explicit solution, we also assume . The first and second terms in (69) measure the total utility from and , while the third term characterizes the initial reserve . It is natural to desire to maximize the expected utility functional representing cumulative consumption and the recursive utility , which means to find satisfying (66) and (67), respectively, to maximize in (69).

We solve the problem by the maximum principle derived in Section 3. The Hamiltonian corresponding to this model iswhere is the solution of the following adjoint equations: