International Journal of Stochastic Analysis

Volume 2011, Article ID 916952, 17 pages

http://dx.doi.org/10.1155/2011/916952

## Control of Dams Using Policies When the Input Process Is a Nonnegative Lévy Process

Department of Statistics, College of Business and Economics, United Arab Emirates University, Al-Ain 17555, UAE

Received 28 April 2011; Accepted 17 July 2011

Academic Editor: Ho Lee

Copyright © 2011 Mohamed Abdel-Hameed. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

We consider policy of a dam in which the water input is an increasing Lévy process. The release rate of the water is changed from 0 to and from to 0 at the moments when the water level upcrosses level and downcrosses level , respectively. We determine the potential of the dam content and compute the total discounted as well as the long-run average cost. We also find the stationary distribution of the dam content. Our results extend the results in the literature when the water input is assumed to be a Poisson process.

#### 1. Introduction and Summary

Lam and Lou [1] consider the control of a finite dam where the water input is a Wiener process, using policies. In these policies, the water release rate is assumed to be zero until the water reaches level as soon as this happens the water is released at rate until the water content reaches level Abdel-Hameed and Nakhi [2] discuss the optimal control of a finite dam using policies, using the total discounted as well as the long-run average costs. They consider the cases where the water input is a Wiener process and a geometric Brownian motion process. Lee and Ahn [3] consider the long-run average cost case when the water input is a compound Poisson process. Abdel-Hameed [4] treats the case where the water input is a compound Poisson process with a positive drift. He obtains the total discounted cost as well as the long-run average cost. Bae et al*. *[5] consider the policy in assessing the workload of an M/G/1 queuing system. Bae et al*. *[6] consider the log-run average cost for policy in a finite dam, when the input process is a compound Poisson process. In this paper, we consider the policy for the more general case where the water input is assumed to be an increasing Lévy process. At any time, the release rate can be increased from 0 to with a starting cost or decreased from to zero with a closing cost . Moreover, for each unit of output, a reward is received. Furthermore, there is a penalty cost which accrues at a rate , where is a bounded measurable function on the state space of the content process.

We will use the term “increasing” to mean “nondecreasing” throughout this paper.

In Section 2, we discuss the potentials of the processes of interest as well as the other results that are needed to compute the total discounted and long-run average costs. In Section 3, we obtain formulas for the cost functionals using the total discounted as well as the long-run average cost cases. In Section 4, we discuss the special cases where the water input is an increasing compound Poisson process as well as inverse Gaussian process.

#### 2. Basic Results

The content process is best described by the bivariate process , where and describe the dam content and the release rate, respectively. We define the following sequence of stopping times: The process has as its state space the pair of line segments

Let be an increasing Lévy process with drift . For each , we let . From the definition of the policy, it follows that, for each , , Furthermore, . It follows that the content process is a delayed regenerative process with the regeneration points being the The penalty cost rate function is defined as follows: where and are bounded measurable functions.

For any process with state space ,any Borel set and any functional , denotes the expectation of conditional on ,denotes the corresponding probability measure, and is the indicator function of the set . Throughout, we let , , , and . For , we define and . Throughout, we define and . For any and , let and be the expected discounted penalty costs, during the intervals and , respectively. Furthermore, let and be the expected nondiscounted penalty costs during the same intervals. It follows that The functionals above, which we aim to evaluate, are basic ingredients in computing the total discounted and long-run average costs associated with the policy as discussed in Section 3.

Let and be the drift term and the Lévy measure of input process respectively, then, for all ,and the Laplace transform of is of the form, The function is known as the Lévy component and is given by where is a measure on satisfying Increasing Lévy processes include increasing compound Poisson processes, inverse Gaussian processes, gamma processes, and stable processes.

We assume that the expected value of is finite throughout this paper.

To evaluate the cost functionals and other parameters of the content process, we define the Lévy process killed at as follows: From Theorem 3.3.12 of Blumenthal and Getoor [7], it follows that the process is a strong Markov process.

*Definition 2.1. *Let be a Markov process with a state space . For each , the -potential of (denoted by ) is defined for any bounded measurable function on and every via ((1.8.9), p.41 of [8])

*Remark 2.2. *Throughout, we denote the -potential of the process by . Since the process has stationary independent increments, it follows that , for each and in the state space of the process satisfying . We denote by , throughout.

Since the process is increasing and has stationary independent increments, it follows that

The following lemma follows by taking for all in (2.11) and (2.12), respectively.

Lemma 2.3. *For one has
*

The following Lemma gives the Laplace transform of as well as the expected value of .

Lemma 2.4. *
(a) For and ,
**
**
(b) For ,
*

*Proof of (a). *For and , since the process has stationary independent increments, we have
where the second equation follows from (8) of Alili and Kyprianou [9], and the fourth equation follows from the definition of and .

*Proof of (b). *For ,
where the first equation follows since the process is a Lévy process, the third equation follows from (2.15), the fourth equation follows because , the fifth equation follows since , and the last equation follows from (2.14).

To derive and , we define Clearly, the state space of the process is . From Theorem 3.3.12 of Blumenthal and Getoor [7], it follows that the process is a strong Markov process.

Throughout, we assume that . Using Doob's optional sampling theorem, the following is easy to see.

Lemma 2.5. *For ,
**
where is the solution of the integral equation
*

The following Lemma gives, among other things, a formula for computing and condition under which this expectation is finite.

Lemma 2.6. *
(a) if and only if .**
(b) The function is a concave increasing function on .**
(c) For ,*

*Proof of (a). *From (2.20), it follows that is an increasing function on and . Let , using (2.21) it follows that . Furthermore, is the largest root of , and 0 is indeed a root of and, since is an increasing function, is an increasing function on the domain . It follows that the only root of the function above is zero if and only if . Observe that
where the interchange of the differentiation and integration in the second equation is permissible using the Lebesgue dominated convergence theorem, since for each and . The rest of the proof follows since .

*Proof of (b). *To prove part (b), first we observe that is an increasing function in its argument, and hence is a convex function in its argument. Since , it follows that is a concave function.

*Proof of (c). *If the proof of part (c) follows since, from (2.20), almost everywhere if and only if , in this case, .

*Remark 2.7. *The equation given in part (c) of Lemma 2.6 is consistent with the well-known fact about the expected busy period of the M/G/1 queue.

Let be the potential of the process . To find , we first need to introduce the following definition.

*Definition 2.8. *A Lévy process is said to be spectrally positive (negative) if it has no negative (positive) jumps.

Clearly, a Lévy process is spectrally positive if and only if the process is spectrally negative. Furthermore, the process is spectrally positive with bounded variation.

For , we have where We note that the function is the right-hand inverse of the function .

We now define the -scale function, which plays a major role in the applications of spectrally positive (negative) Lévy processes. This function is closely connected to the two-sided exit problem of such processes (cf. Bertion [10]).

*Definition 2.9. *For , the -scale* function* (of the process ) is the unique function whose restriction to is continuous and has Laplace transform
and is defined to be identically zero on the interval .

Letting , we get the 0-*scale* function, which is referred to as the “scale function” in the literature. We denote this function by (instead of ) throughout. We note that , where . Let . For every , let be the equilibrium distribution function corresponding to . Let and assume that . It follows that
where is the th convolution of . Furthermore, we note that for ,
where is the *k*th convolution of .

We are now in a position to state and prove a lemma that characterizes .

Lemma 2.10. * is absolutely continuous with respect to the Lebesgue measure on , and its density is given as follows:
*

*Proof. *Define the process to be equal to ; it follows that is a spectrally negative Lévy process. For , we let

Supurn [11] proved that (for ) the -potential of the process obtained by killing the process at is absolutely continuous with respect to the Lebesgue measure on , and its density is equal to
It follows that, for , , the -potential of the process obtained by killing the process at is absolutely continuous with respect to the Lebesgue measure on , and its density is equal to
From Lemma 4 of Pistorious [12], we have as . Letting in the last density above, then the the potential of the process obtained by killing the process at is absolutely continuous with respect to the Lebesgue measure on , and its density (denoted by is as follows:

Observe that for any and ,
Thus,

It is seen that, for ,

Theorem 2.11. *For any and ,*(a)*for ,
*(b)*for ,
*

*Proof of (a). *Let be the sigma algebra generated by , then we have
where the third equation follows from the second equation, since given almost everywhere, the fourth equation follows from (2.20) above, the seventh equation follows from (8) Alili and Kyprianou [9], and the last equation follows from (2.21) above.

*Proof of (b). *The proof of the part (b) of the theorem follows from (2.20), since for , and almost everywhere.

#### 3. The Total Discounted, Long-Run Average Costs and the Stationary Distribution of the Dam Content

We now discuss the computations of the cost functionals using the total discounted cost as well as the long-run average cost criteria. Let be the length of the first cycle, that is, ,and let be the expected cost during the interval when . Since the content process is a delayed regenerative process with regeneration points , using the delayed regeneration property, it follows that the total discounted cost associated with an policy is given by where is the total discounted cost during the interval . From the definitions of , it follows that, for , To compute for , we let be the sigma algebra generated by and proceed as follows: where the second equation follows from the definition of the process , the third equation follows from the definition of ,the fourth equation follows from the definition of the content process and since, given , almost everywhere, and the last equation follows from the definition of .

We note that

The following lemma shows how (given in (3.1)) can be computed and also gives a formula for computing the expected value of , which we will need later on to compute the long-run average cost.

Lemma 3.1. *Let be the length of the first cycle as defined above, then**
(a) **
(b) *

*Proof of (a). *We note that, given , almost everywhere. Thus, for each ,
where the second equation follows (2.37) upon substituting for , the third equation follows from the definition of the and (2.21).

*Proof of (b). *From (3.5), it is evident that, starting at , is finite almost everywhere if and only if . From part (a) of Lemma 2.6, it follows that is finite almost everywhere if and only if . From (2.14) and (3.5), we have
The proof of (b) is complete, since as shown in the proof of part (c) of Lemma 2.6

Now, we turn our attention to computing the long-run average cost per a unit of time. Let and assume that . From (3.1), (3.3), and (3.4), it follows, by a Tauberian theorem, that the long-run average cost per unit of time, denoted by is given by where, and the second equation follows from (3.6) and the first equation.

*Remark 3.2. *Assume that both penalty functions and are identically zero on their domains, and defined above is greater than zero. The following follows from (3.10) above:
Letting and and in (3.10), we get the following proposition which generalizes the results obtained by Lee and Ahn [3], where they assumed that the input process is a compound Poisson process and .

Proposition 3.3. *Assume that . Let , and, be the distribution function of the process , then, for ,
*

#### 4. Special Cases

In this section, we give the basic identities needed to compute the cost functionals when the input process is an inverse Gaussian process and a compound Poisson process, respectively.

*Case 1. *Assume that is an inverse Gaussian process with transition function defined for , and , by
It follows that the process is an increasing Lévy process with state space , Lévy measure
and Lévy component
Furthermore, .

Substituting this Lévy component above in (2.21), it is seen that the solution of this equation is as follows (we omit the proof):

To find the -potential of the process , for each and , we define , and it is easily seen that

Throughout we let be as the standard normal density function and let and be the well-known error and complimentary error functions, respectively. Inverting the above function with respect to , we have where

From (2.13), it follows that, for , where the last equation follows by integrating over the interval .

Inverting the right hand side of (4.8) with respect to , it follows that, given , the distribution function of (denoted by ) is given by Furthermore, for , where the third equation follows from the second equation upon tedious calculations which we omit.

We now turn our attention to computing the distribution function of (denoted by ). We first need the following identity which expresses the Lévy component given in (4.3) in a form suitable for computing . The proof of this identity follows from (4.3) after some simple algebraic manipulations which we omit: For each , we write where the first equation follows from the second equation given the proof of part (a) of Lemma 2.4 by letting , the second equation follows from (4.11), the third equation follows from (4.5) upon letting , and the fourth equation follows from the third equation through integration by parts.

From (4.12), it follows that, for each ,

*Case 2. *Assume that is an increasing compound Poison process with intensity and as the distribution function of the size of each jump. This model is treated in details in references [3, 4, 6]. Here, we give the basic entities involved when the drift terms . For the proof of these entities and more in depth analysis of this case, the reader is referred to the above- mentioned references.

It is obvious that
, where is the expected jump size of the compound Poisson process.

Define, for any and , .For , we let be the th convolution of , where for all . For each , we define to be the renewal function corresponding to .It follows that
Furthermore, for ,

Also,
where the first equation follows from the second equation in the proof of part (a) of Lemma 2.4, by letting . Furthermore, the second equation follows from (4.14) and (4.15).

Inverting (4.17) with respect to , the distribution function of , denoted by , is given through

#### Acknowledgment

This Research was supported, in part, by a 2010 Summer Research Grant from the College of Business and Economics, UAE University.

#### References

- Y. Lam and J. H. Lou, “Optimal control of a finite dam: Wiener process input,”
*Journal of Applied Probability*, vol. 24, no. 1, pp. 186–199, 1987. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - M. Abdel-Hameed and Y. Nakhi, “Optimal control of a finite dam using ${P}_{\lambda ,\tau}^{M}$ policies and penalty cost: total discounted and long run average cases,”
*Journal of Applied Probability*, vol. 28, no. 4, pp. 888–898, 1990. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - E. Y. Lee and S. K. Ahn, “${P}_{\lambda}^{M}$-policy for a dam with input formed by a compound Poisson process,”
*Journal of Applied Probability*, vol. 35, no. 2, pp. 482–488, 1998. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - M. Abdel-Hameed, “Optimal control of a dam using ${P}_{\lambda ,\tau}^{M}$ policies and penalty cost when the input process is a compound Poisson process with positive drift,”
*Journal of Applied Probability*, vol. 37, no. 2, pp. 408–416, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - J. Bae, S. Kim, and E. Y. Lee, “A ${P}_{\lambda}^{M}$ policy for M/G/1 queueing system,”
*Applied Mathematical Modelling*, vol. 26, pp. 929–939, 2002. View at Google Scholar - J. Bae, S. Kim, and E. Y. Lee, “Average cost under the ${P}_{\lambda ,\tau}^{M}$ policy in a finite dam with compound Poisson inputs,”
*Journal of Applied Probability*, vol. 40, no. 2, pp. 519–526, 2003. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - R. M. Blumenthal and R. K. Getoor,
*Markov Processes and Potential Theory*, Pure and Applied Mathematics, Vol. 29, Academic Press, New York, NY, USA, 1968. - J. Lamperti,
*Stochastic Processes: A survey of the Mathematical Theory*, Springer, New York, NY, USA, 1977. - L. Alili and A. E. Kyprianou, “Some remarks on first passage of Lévy processes, the American put and pasting principles,”
*The Annals of Applied Probability*, vol. 15, no. 3, pp. 2062–2080, 2005. View at Publisher · View at Google Scholar · View at Zentralblatt MATH - J. Bertion,
*Lévy Processes*, Cambridge University Press, Cambridge, UK, 1996. - V. N. Supurn, “The ruin problem and the resolvent of a killed independent increment process,”
*Ukrainian Mathematical Journal*, vol. 28, pp. 39–45, 1976. View at Google Scholar - M. R. Pistorius, “On exit and ergodicity of the spectrally one-sided Lévy process reflected at its infimum,”
*Journal of Theoretical Probability*, vol. 17, no. 1, pp. 183–220, 2004. View at Publisher · View at Google Scholar · View at Zentralblatt MATH