Abstract

Let be a controlled one-dimensional standard Brownian motion starting from . The problem of optimally controlling until for the first time is solved explicitly in a particular case. The maximal value that the instantaneous reward given for survival in can take is determined.

1. Introduction

Consider the one-dimensional controlled standard Brownian motion process defined by the stochastic differential equation where is the control variable, , and is a standard Brownian motion. Assume that and define the first passage time Our aim is to find the control that minimizes the expected value of the cost function where and are positive constants.

In the case when , Lefebvre and Whittle [1] were able to find the optimal control by making use of a theorem in Whittle [2, page 289] that enables us to express the value function in terms of a mathematical expectation for the uncontrolled Brownian motion obtained by setting in (1.1). Moreover, Lefebvre [3] has also obtained the value of when if the cost function in (1.3) is replaced by Although we cannot appeal to the theorem in Whittle [2] in that case, the author was able to express the function in terms of a mathematical expectation for an uncontrolled geometric Brownian motion.

In Section 2, we will find when . The problem cannot then be reduced to the computation of a mathematical expectation for an uncontrolled diffusion process. Therefore, we will instead find the optimal control by considering the appropriate dynamic programming equation. Moreover, if the instantaneous reward given for survival in the interval is too large, then the value function becomes infinite. We will determine the maximal value that can take in Section 3.

2. Optimal Control

The value function satisfies the following dynamic programming equation: It follows that the optimal control is given by Substituting this value into (2.1), we find that we must solve the nonlinear ordinary differential equation The boundary conditions are

Next, let Making use of a mathematical software program, we find that the solution of (2.3) can be expressed as where and are Bessel functions and is a constant that must be chosen so that . Unfortunately, it seems very difficult to evaluate the integral explicitly. Notice however that actually we do not need to find , but only to determine the optimal value of the control variable .

We will prove the following proposition.

Proposition 2.1. The control that minimizes the expected value of the cost function defined in (1.3), when in (1.1), is given by

Proof. We deduce from (2.6) that Moreover, from the formula (see Abramowitz and Stegun [4, page 358]) which is valid for , we find that the function may be rewritten as
Now, because the optimizer is trying to maximize the time spent by in the interval , taking the quadratic control costs into account, we can assert, by symmetry, that should be equal to zero when . One can check that it is indeed the case for any value of the constant . Furthermore, the function must have a minimum (that is, a maximum in absolute value) at , so that as well.
With the help of the formula (see Abramowitz and Stegun [5, page 360]) if and , we find that Hence, we deduce that the constant must be equal to 1, so that Formula (2.7) for the optimal control then follows at once from (2.2).

3. Maximal Value of

Because the optimizer wants to remain in the interval as long as possible and because is multiplied by (with ) in (1.1), we can state that the optimal control should always be negative when . However, if we plot against for particular values of the constants , , , and , we find that it is sometimes positive. This is due to the fact that the formula in Proposition 2.1 is actually only valid for less than a critical value . This depends on the other parameters. Conversely, if we fix the value of , then we can find the largest value that can take.

One way to determine is to find out for what value of the value function becomes infinite. However, because we were not able to obtain an explicit expression for (without an integral sign), we must proceed differently.

Another way that can be used to obtain the value of is to determine the smallest value of (positive) for which the denominator in (2.13) vanishes.

Let Using a mathematical software program, we find that at (approximately) . Hence, we deduce that we must have

We can now state the following proposition.

Proposition 3.1. For fixed values of , , and , the value of is given by
To conclude, one gives the value of when is fixed.

Corollary 3.2. For fixed values of , , and , the maximal value that can take is

4. Conclusion

We have solved explicitly a problem of the type that Whittle [2] termed “LQG homing.” Actually, the expression “LQG homing” corresponds to the case when the parameter in the cost function is negative, so that the optimizer wants the controlled process to leave the continuation region as soon as possible.

The author has studied LQG homing problems in a number of papers (see Lefebvre [4], in particular). They were also considered recently by Makasu [6]. However, in the present paper, we did not appeal to the theorem in Whittle [2] to obtain the optimal solution by reducing the optimal control problem to a purely probabilistic problem. Although this interpretation is very interesting, it only works when a certain relation holds between the noise and control terms. In practice, the relation in question is seldom verified in more than one dimension.

We could determine the optimal control when the parameter in (1.3) is negative, so that the optimizer wants to leave the interval as soon as possible. This time, the function would have a maximum at . We could also consider other values of the constant in (1.1). However, the value is probably the most important one in practice, apart from .