Chinese Journal of Mathematics

Volume 2017, Article ID 3596037, 10 pages

https://doi.org/10.1155/2017/3596037

## Value Function and Optimal Rule on the Optimal Stopping Problem for Continuous-Time Markov Processes

College of Economics and Management and Zhejiang Provincial Research, Center for Ecological Civilization, Zhejiang Sci-Tech University, Hangzhou 310018, China

Correspondence should be addressed to Lu Ye; moc.liamtoh@lyjazwjz

Received 30 May 2017; Accepted 5 September 2017; Published 9 October 2017

Academic Editor: Antonio Di Crescenzo

Copyright © 2017 Lu Ye. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This paper considers the optimal stopping problem for continuous-time Markov processes. We describe the methodology and solve the optimal stopping problem for a broad class of reward functions. Moreover, we illustrate the outcomes by some typical Markov processes including diffusion and Lévy processes with jumps. For each of the processes, the explicit formula for value function and optimal stopping time is derived. Furthermore, we relate the derived optimal rules to some other optimal problems.

#### 1. Introduction

Let be a complete probability space; the problem studied in this paper is to find the optimum where is the discount rate and is a stopping time. The process is a continuous-time Markov process with starting state . When denotes the stock price and is the payoff function, is the pricing expression for American option (e.g., see Wong [1]). When is an investor’s utility function about the stock price , the optimum solution indicates the best time to buy or sell the stock (e.g., see McDonald and Siegel [2]).

Because of the Markov property, the value function can be written as where denotes the conditional expectation . The function is called the reward function. The possibility that is allowed, and making the convention that it means that if an option is never exercised, then its reward payment is valueless for the investor.

In the paper, we are interested in determining both an optimal stopping time and value function for a large class of reward functions. We show the conditions for reward functions and deduce the explicit optimal rules for general continuous-time Markov processes including diffusion and Lévy processes with jumps.

The study of optimal stopping time for stochastic processes, especially geometric Brownian motion, has a long history in finance literature. Under the assumption that is geometric Brownian motion, the seminal paper by McDonald and Siegel [2] puts forward the problem with the reward function as a model to illustrate the financial decision making. Hu and Øksendal [3] solved the problem in multidimensional cases, when , but they restricted the stopping time in a bounded interval. Recently, Nishide and Rogers [4] extended the problem by relaxing the restriction on the stopping time. For the other forms of value functions, Pedersen and Peskir [5] solved the problem by taking some special diffusion processes.

The purpose of our work is threefold. Firstly, we intend to make clear the assumptions on reward function , in such a way that the explicit value can be generalized to a larger class of issues. For this purpose, throughout this article we assume that the function is nonincreasing, concave and twice continuous differentiable. These properties are very powerful in the following proof, as we will see. Secondly, we pay attention to general Markov processes including diffusion and Lévy processes with jumps. With the help of the infinitesimal generator, we obtain an explicit formula for the value function and the stopping time. Thirdly, we find that the optimal problem (2) is equivalent to other optimal problems like This work is inspired by Pedersen and Peskir [5], who verified the equivalence of problems (2) and (4) for , where is an Ornstein-Uhlenbeck process. Note that we do not consider the case , so our approach to deal with the optimal problem is different from that of Pedersen and Peskir. Moreover, our work naturally explore the explicit solutions of the new optimal problem (4) for a larger class of reward functions and underlying processes .

The paper is organized as follows. In Section 2, the explicit value function and optimal stopping time are derived for a general Markov process along with the condition for the reward functions. Section 3 discusses some applications to diffusion, which include Brownian motion with drift, geometric Brownian motion, and the Ornstein-Uhlenbeck process. Section 4 displays some concrete examples of Lévy processes with jumps. In Section 5, we will link the outcomes with other optimal problems such that explicit solutions for the new problems can also be feasible to a general Markov process with a large class of reward functions. Finally, concluding remarks are given in Section 6.

#### 2. Optimal Rule for Continuous-Time Markov Processes

For a Markov process , the infinitesimal generator of is defined as where is twice continuous differentiable. In the diffusion case, namely, where is a standard Brownian motion, , the infinitesimal generator is equivalent to In the case of Lévy process with jumps, driven by the equation where is a homogeneous Poisson process, , the infinitesimal generator is where is the Lévy measure (for jump diffusion and its generators, e.g., we can refer to Gihman and Skorohod [6]).

First, we make the assumption about the reward function.

*Assumption 1. *The reward function is nonincreasing, concave, and twice continuous differentiable; that is, , , and is .

Under Assumption 1, we present the explicit solutions for general continuous-time Markov processes.

Theorem 2. *For a Markov process with infinitesimal generator , let be the solution of satisfying Given the reward function in Assumption 1, if there exists a point such that then the optimal problem has an explicit expression for the value function The optimal stopping time is *

*Proof. *Define a functionand then we get Hence, attains the minimum at , So we have the inequality The value function is , exists, and is continuous except at . By Itô’s formula, for any time , where is the infinitesimal generator of the process .

Because the value function is bounded, the local martingale term on the right-hand side of (21) is also bounded (all the other terms in (21) are clearly bounded), implying that it is in fact a martingale with zero expectation. Hence, by the optimal sampling theorem, we have, for any stopping time , (a) When , , we claim that For diffusion, , , , and the inequality in (23) holds naturally. For Lévy processes with jumps, As and is decreasing on states, then .

(b) When , , the function is the solution of so

Therefore, from (a) and (b), we have , and the equality holds for . It turns to be From which we can see that is an upper bound for the value starting from . This bound is achieved when . When the starting state is smaller than , the optimal stopping time is . That is, as it reaches its upper bound. If the starting state is greater than the point , it must wait until , which is the first hitting time to the point . At the stopping time , , and , so , reaching its upper bound.

*Remark 3. *Actually, for all diffusion, Theorem 2 holds for the drift term satisfying . We calculate that so for , As satisfies , we can arrive at The rest of the proof is the same as that in Theorem 2.

However, the condition for Lévy process will be more complicated. In order to have a uniform style, we restrict to the case . Moreover, we can see from (19) that if the point exists, then it is unique. Next we present results on some classical Markov processes as applications of Theorem 2.

#### 3. Diffusion

##### 3.1. Brownian Motion with Drift

For Brownian motion with drift , and variance , namely, the process is driven by the SDE by using Theorem 2, we get the following proposition.

Proposition 4. *Let be the function in Assumption 1; if there exists a point such that then the value function has the form **The optimal stopping time is *

*Proof. *The infinitesimal generator for Brownian motion with drift is . It is well known that the ordinary differential equation (ODE) of has two linearly independent solutions So ( and are constants). Considering the boundary condition and , then must be equal to zero, , and . Equation (13) in Theorem 2 tells us that the point is determined by since . Thus, the expression for the value function is easily obtained by (15).

As the simplest example, we take the reward function and we take the parameters , , and . Then, the problem has the solution The optimal stopping time is We draw the value function and the point in Figure 1.