Control of Dams Using 𝑃^𝐌_𝝀,𝝉 Policies When the Input Process Is a Nonnegative Lévy Process

Abstract

We consider 𝑃^𝑀_𝜆,𝜏 policy of a dam in which the water input is an increasing Lévy process. The release rate of the water is changed from 0 to 𝑀 and from 𝑀 to 0 (𝑀>0) at the moments when the water level upcrosses level 𝜆 and downcrosses level 𝜏(𝜏<𝜆), respectively. We determine the potential of the dam content and compute the total discounted as well as the long-run average cost. We also find the stationary distribution of the dam content. Our results extend the results in the literature when the water input is assumed to be a Poisson process.

1. Introduction and Summary

Lam and Lou [1] consider the control of a finite dam where the water input is a Wiener process, using 𝑃^𝑀_𝜆,𝜏policies. In these policies, the water release rate is assumed to be zero until the water reaches level 𝜆>0,as soon as this happens the water is released at rate 𝑀>0until the water content reaches level 𝜏>0,𝜆>𝜏.Abdel-Hameed and Nakhi [2] discuss the optimal control of a finite dam using 𝑃^𝑀_𝜆,𝜏policies, using the total discounted as well as the long-run average costs. They consider the cases where the water input is a Wiener process and a geometric Brownian motion process. Lee and Ahn [3] consider the long-run average cost case when the water input is a compound Poisson process. Abdel-Hameed [4] treats the case where the water input is a compound Poisson process with a positive drift. He obtains the total discounted cost as well as the long-run average cost. Bae et al. [5] consider the 𝑃^𝑀_𝜆,0policy in assessing the workload of an M/G/1 queuing system. Bae et al. [6] consider the log-run average cost for 𝑃^𝑀_𝜆,𝜏policy in a finite dam, when the input process is a compound Poisson process. In this paper, we consider the 𝑃^𝑀_𝜆,𝜏policy for the more general case where the water input is assumed to be an increasing Lévy process. At any time, the release rate can be increased from 0 to 𝑀 with a starting cost𝐾₁𝑀 or decreased from𝑀 to zero with a closing cost 𝐾₂𝑀. Moreover, for each unit of output, a reward 𝑅is received. Furthermore, there is a penalty cost which accrues at a rate 𝑓, where 𝑓 is a bounded measurable function on the state space of the content process.

We will use the term “increasing” to mean “nondecreasing” throughout this paper.

In Section 2, we discuss the potentials of the processes of interest as well as the other results that are needed to compute the total discounted and long-run average costs. In Section 3, we obtain formulas for the cost functionals using the total discounted as well as the long-run average cost cases. In Section 4, we discuss the special cases where the water input is an increasing compound Poisson process as well as inverse Gaussian process.

2. Basic Results

The content process is best described by the bivariate process 𝐵=(𝑍,𝑅), where 𝑍={𝑍_𝑡,𝑡≥0} and 𝑅={𝑅_𝑡,𝑡≥0}describe the dam content and the release rate, respectively. We define the following sequence of stopping times: 𝑇₀=inf𝑡≥0∶𝑍_𝑡≥𝜆,𝑇^∗₀=inf𝑡≥𝑇₀∶𝑍_𝑡,𝑇≤𝜏_𝑛=inf𝑡≥𝑇_𝑛−1∶𝑍_𝑡≥𝜆,𝑇^∗_𝑛=inf𝑡≥𝑇_𝑛∶𝑍_𝑡≤𝜏,𝑛=1,2,…. (2.1) The process 𝐵 has as its state space the pair of line segments 𝑆=[[0,𝜆)×{0}]∪[(𝜏,∞)×{𝑀}]. (2.2)

Let 𝐼={𝐼_𝑡,𝑡≥0} be an increasing Lévy process with drift 𝑎≥0. For each 𝑡≥0, we let 𝐼^∗_𝑡=𝐼_𝑡−𝑀𝑡. From the definition of the 𝑃^𝑀_𝜆,𝜏policy, it follows that, for each 𝑡∈[0,𝑇₀), 𝑍_𝑡=𝐼_𝑡, 𝑍_𝑡=𝐼_𝑡,𝑡∈𝑇^∞_𝑛=0^∗_𝑛,𝑇_𝑛+1,𝐼^∗_𝑡,𝑡∈^∞_𝑛=0𝑇_𝑛,𝑇^∗_𝑛. (2.3) Furthermore, 𝐼^∗𝑇_𝑛𝑇=𝐼_𝑛,𝑛=0,1,…. It follows that the content process 𝑍 is a delayed regenerative process with the regeneration points being the 𝑇^∗_𝑛,𝑛=1,2,….The penalty cost rate function is defined as follows: 𝑓()=𝑔(𝑧)∈[)×{0},𝑔𝑧,𝑟)(𝑧,𝑟0,𝜆_∗(𝑧)(𝑧,𝑟)∈(𝜏,∞)×{𝑀}, (2.4) where 𝑔∶[0,𝜆)→𝑅₊ and 𝑔^∗∶(𝜏,∞)∶→𝑅₊ are bounded measurable functions.

For any process 𝑌={𝑌_𝑡,𝑡≥0}with state space 𝐸,any Borel set 𝐴⊂𝐸and any functional 𝑓, 𝐸_𝑦(𝑓)denotes the expectation of 𝑓 conditional on 𝑌₀=𝑦,𝑃_𝑦(𝐴)denotes the corresponding probability measure, and 𝐼_𝐴()is the indicator function of the set 𝐴. Throughout, we let 𝑅=(−∞,∞), 𝑅₊=[0,∞), 𝑁={1,2,…}, and 𝑁₊={0,1,…}. For 𝑥,𝑦∈𝑅, we define 𝑥∨𝑦=𝑥max𝑦and 𝑥∧𝑦=𝑥min𝑦. Throughout, we define 𝑊_𝜆=inf{𝑡≥0∶𝐼_𝑡≥𝜆} and 𝑊^∗_𝜏=inf{𝑡≥0∶𝐼^∗_𝑡≤𝜏}. For any 𝑥<𝜆 and 𝑦>𝜏, let 𝐶^𝛼_𝑔(0,𝑥,𝜆) and 𝐶^𝛼_𝑔∗(𝑀,𝑦,𝜏)be the expected discounted penalty costs, during the intervals (0,𝑊_𝜆)and (0,𝑊^∗_𝜏), respectively. Furthermore, let 𝐶_𝑔(0,𝑥,𝜆) and 𝐶_𝑔∗(𝑀,𝑦,𝜏)be the expected nondiscounted penalty costs during the same intervals. It follows that 𝐶^𝛼_𝑔(0,𝑥,𝜆)=𝐸_𝑥^𝑊𝜆₀𝑒^−𝛼𝑡𝑔𝐼_𝑡𝑑𝑡,𝐶^𝛼_𝑔∗𝑀,𝑦,𝜏=𝐸_𝑦^𝑊∗𝜏₀𝑒^−𝛼𝑡𝑔^∗𝐼^∗_𝑡𝐶𝑑𝑡,_𝑔(0,𝑥,𝜆)=𝐸_𝑥^𝑊𝜆₀𝑔𝐼_𝑡𝑑𝑡,𝐶_𝑔∗𝑀,𝑦,𝜏=𝐸_𝑦^𝑊∗𝜏₀𝑔^∗𝐼^∗_𝑡𝑑𝑡. (2.5) The functionals above, which we aim to evaluate, are basic ingredients in computing the total discounted and long-run average costs associated with the 𝑃^𝑀_𝜆,𝜏 policy as discussed in Section 3.

Let 𝑎≥0 and 𝜈 be the drift term and the Lévy measure of input process 𝐼, respectively, then, for all 𝑡≥0,𝑥≥0,and 𝛼≥0the Laplace transform of 𝐼_𝑡 is of the form, 𝐸_𝑥𝑒^{−𝛼𝐼𝑡}=𝑒^{−𝑡[𝑥+𝜙(𝛼)]}. (2.6) The function 𝜙(𝛼) is known as the Lévy component and is given by𝜙(𝛼)=𝛼𝑎+^∞₀1−𝑒^−𝛼𝑥𝜈(𝑑𝑥), (2.7) where 𝜈 is a measure on [0,∞)satisfying^∞₀(𝑥∧1)𝜈(𝑑𝑥)<∞,𝜈({0})=0. (2.8) Increasing Lévy processes include increasing compound Poisson processes, inverse Gaussian processes, gamma processes, and stable processes.

We assume that the expected value of 𝐼₁ is finite throughout this paper.

To evaluate the cost functionals and other parameters of the content process, we define the Lévy process killed at 𝑊_𝜆 as follows:𝑋=𝐼_𝑡,𝑡<𝑊_𝜆. (2.9) From Theorem  3.3.12 of Blumenthal and Getoor [7], it follows that the process 𝑋 is a strong Markov process.

Definition 2.1. Let 𝑌 be a Markov process with a state space 𝐸. For each 𝛼≥0, the 𝛼-potential of 𝑌 (denoted by 𝑈^𝛼_𝑌) is defined for any bounded measurable function on 𝐸and every 𝑥∈𝐸 via ((1.8.9), p.41 of [8]) 𝑈^𝛼_𝑌𝑓(𝑥)=^def_𝐸𝑓(𝑧)𝑈^𝛼_𝑌(𝑥,𝑑𝑧)=𝐸_𝑥^∞₀𝑒^−𝛼𝑡𝑓𝑌_𝑡𝑑𝑡. (2.10)

Remark 2.2. Throughout, we denote the 𝛼-potential of the process 𝐼 by 𝑈^𝛼. Since the process 𝐼 has stationary independent increments, it follows that 𝑈^𝛼(𝑥,𝑑𝑦)=𝑈^𝛼(0,𝑑𝑦−𝑥), for each 𝑥and 𝑦 in the state space of the process 𝐼 satisfying 𝑦≥𝑥. We denote 𝑈^𝛼(0,𝑑𝑦)by 𝑈^𝛼(𝑑𝑦), throughout.

Since the process 𝐼is increasing and has stationary independent increments, it follows that 𝐶^𝛼_𝑔(0,𝑥,𝜆)=𝑈^𝛼𝑔(𝑥)=^𝜆_𝑥𝑔𝑦𝑈^𝛼𝑥,𝑑𝑦=₀^𝜆−𝑥𝑔𝑥+𝑦𝑈^𝛼𝑑𝑦,𝐶_𝑔(0,𝑥,𝜆)=𝑈⁰𝑔(𝑥)=^𝜆_𝑥𝑔𝑦𝑈⁰𝑥,𝑑𝑦=₀^𝜆−𝑥𝑔𝑥+𝑦𝑈⁰𝑑𝑦. (2.11)(2.12)

The following lemma follows by taking 𝑔(𝑥)=1 for all 𝑥∈[0,𝜆)in (2.11) and (2.12), respectively.

Lemma 2.3. For 𝑥≤𝜆 one has 𝐸_𝑥exp−𝛼𝑊_𝜆=1−𝛼𝑈^𝛼𝐼_{[0,𝜆−𝑥)}(0)=𝛼𝑈^𝛼𝐼_{[𝜆−𝑥,∞)}(0),𝐸_𝑥𝑊_𝜆=𝑈⁰𝐼_{[0,𝜆−𝑥)}(0). (2.13)(2.14)

The following Lemma gives the Laplace transform of 𝐼_𝑊𝜆 as well as the expected value of 𝐼_𝑊𝜆.

Lemma 2.4. (a) For 𝑥<𝜆 and 𝛼≥0, 𝐸_𝑥exp−𝛼𝐼_𝑊𝜆=exp(−𝛼𝑥)1−𝜙(𝛼)_{[0,𝜆−𝑥)}exp(−𝛼𝑧)𝑈⁰(𝑑𝑧). (2.15)  
(b) For 𝑥<𝜆, 𝐸_𝑥𝐼_𝑊𝜆=𝑥+𝐸₀𝐼₁𝐸₀𝑊_𝜆−𝑥. (2.16)

Proof of (a). For 𝑥<𝜆and 𝛼≥0, since the process 𝐼 has stationary independent increments, we have 𝐸_𝑥exp−𝛼𝐼_𝑊𝜆=𝐸₀exp−𝛼𝑥+𝐼_{𝑊𝜆−𝑥}=exp(−𝛼𝑥)𝜙(𝛼)_{[𝜆−𝑥,∞)}exp(−𝛼𝑧)𝑈⁰((𝜙(𝛼𝑑𝑧)=exp−𝛼𝑥))_[0,∞)exp(−𝛼𝑧)𝑈⁰(𝑑𝑧)−_{[0,𝜆−𝑥)}exp(−𝛼𝑧)𝑈⁰((𝜙(𝛼1𝑑𝑧)=exp−𝛼𝑥))𝜙(𝛼)−_{[0,𝜆−𝑥)}exp(−𝛼𝑧)𝑈⁰(((𝛼)𝑑𝑧)=exp−𝛼𝑥)1−𝜙_{[0,𝜆−𝑥)}exp(−𝛼𝑧)𝑈⁰(𝑑𝑧), (2.17) where the second equation follows from (8) of Alili and Kyprianou [9], and the fourth equation follows from the definition of 𝜙(𝛼)and 𝑈⁰.

Proof of (b). For 𝑥<𝜆, 𝐸_𝑥𝐼_𝑊𝜆=𝑥+𝐸₀𝐼_{𝑊𝜆−𝑥}=𝑥+lim_𝛼→01−𝐸₀exp−𝛼𝐼_{𝑊𝜆−𝑥}𝛼=𝑥+lim_𝛼→0𝜙(𝛼)𝛼_{[0,𝜆−𝑥)}exp(−𝛼𝑧)𝑈⁰(𝑑𝑧)=𝑥+𝜙^′(0)𝑈⁰𝐼_{[0,𝜆−𝑥)}(0)=𝑥+𝐸₀𝐼₁𝑈⁰𝐼_{[0,𝜆−𝑥)}(0)=𝑥+𝐸₀𝐼₁𝐸₀𝑊_𝜆−𝑥, (2.18) where the first equation follows since the process 𝐼 is a Lévy process, the third equation follows from (2.15), the fourth equation follows because 𝜙(0)=0, the fifth equation follows since 𝜙^′(0)=𝐸₀(𝐼₁), and the last equation follows from (2.14).

To derive 𝐶^𝛼_𝑔∗(𝑀,𝑦,𝜏),𝐶_𝑔∗(𝑀,𝑦,𝜏),𝐸_𝑦(exp(−𝛼𝑊^∗_𝜏)),and 𝐸_𝑦(𝑊^∗_𝜏), we define𝑋^∗=𝐼^∗_𝑡,𝑡<𝑊^∗_𝜏. (2.19) Clearly, the state space of the process 𝑋^∗is (𝜏,∞). From Theorem  3.3.12 of Blumenthal and Getoor [7], it follows that the process 𝑋^∗ is a strong Markov process.

Throughout, we assume that 𝑀≥𝑎. Using Doob's optional sampling theorem, the following is easy to see.

Lemma 2.5. For 𝑥≥𝜏, 𝐸_𝑥exp−𝛼𝑊^∗_𝜏=exp−(𝑥−𝜏)𝜂(𝛼), (2.20) where 𝜂(𝛼) is the solution of the integral equation 𝑀𝜂(𝛼)=𝛼+𝜙𝜂(𝛼). (2.21)

The following Lemma gives, among other things, a formula for computing 𝐸_𝑥(𝑊^∗_𝜏)and condition under which this expectation is finite.

Lemma 2.6. (a)   𝜂(0+)=0 if and only if 𝑀−𝐸₀(𝐼₁)>0.
(b) The function 𝜂(𝛼) is a concave increasing function on 𝑅₊.
(c) For 𝑥≥𝜏,𝐸_𝑥𝑊^∗_𝜏=𝑥−𝜏𝑀−𝐸₀𝐼₁if𝑀−𝐸₀𝐼₁>0,=∞otherwise. (2.22)

Proof of (a). From (2.20), it follows that 𝜂(𝛼) is an increasing function on 𝑅₊ and lim_𝛼→∞𝜂(𝛼)=∞. Let 𝑓(𝑥)=𝜂⁻¹(𝑥), using (2.21) it follows that 𝑓(𝑥)=𝑀𝑥−𝜙(𝑥). Furthermore, 𝜂(0⁺)is the largest root of 𝑓, and 0 is indeed a root of 𝑓 and, since 𝜂(𝛼) is an increasing function, 𝑓is an increasing function on the domain [𝜂(0⁺),∞). It follows that the only root of the function 𝑓 above is zero if and only if 𝑓^′(0)>0. Observe that 𝑓^′(𝑥)=𝑀−𝜙^′(𝑥)=𝑀−𝑎−^∞₀𝑦𝑒^−𝑥𝑦𝜈𝑑𝑦, (2.23) where the interchange of the differentiation and integration in the second equation is permissible using the Lebesgue dominated convergence theorem, since for each 𝑥≥0,𝑦≥0,𝑦𝑒^−𝑥𝑦<𝑦 and ∫^∞₀𝑦𝜈(𝑑𝑦)=𝐸₀(𝐼₁)−𝑎<∞. The rest of the proof follows since 𝑓^′(0)=𝑀−𝐸₀(𝐼₁).

Proof of (b). To prove part (b), first we observe that 𝑓^′(𝑥)is an increasing function in its argument, and hence 𝑓(𝑥) is a convex function in its argument. Since 𝑓(𝑥)=𝜂⁻¹(𝑥), it follows that 𝜂(𝛼) is a concave function.

Proof of (c). If the proof of part (c) follows since, from (2.20), 𝑊^∗_𝜏<∞almost everywhere if and only if 𝜂(0⁺)=0, in this case, 𝐸_𝑥(𝑊^∗_𝜏)=(𝑥−𝜏)𝜂^′(0⁺)=(𝑥−𝜏)/𝑓^′(0)=(𝑥−𝜏)/(𝑀−𝐸₀(𝐼₁)).

Remark 2.7. The equation given in part (c) of Lemma 2.6 is consistent with the well-known fact about the expected busy period of the M/G/1 queue.

Let 𝑈^∗𝛼 be the potential of the process 𝑋^∗. To find 𝑈^∗𝛼, we first need to introduce the following definition.

Definition 2.8. A Lévy process is said to be spectrally positive (negative) if it has no negative (positive) jumps.

Clearly, a Lévy process 𝐿 is spectrally positive if and only if the process −𝐿is spectrally negative. Furthermore, the process 𝐼^∗ is spectrally positive with bounded variation.

For 𝜃,𝑡∈𝑅₊, we have𝐸𝑒^{−𝜃∗𝐼𝑡}=𝑒^{𝑡𝜓(𝜃)}, (2.24) where𝜓(𝜃)=𝑀𝜃−𝜙(𝜃). (2.25) We note that the function 𝜂 is the right-hand inverse of the function 𝜓.

We now define the 𝛼-scale function, which plays a major role in the applications of spectrally positive (negative) Lévy processes. This function is closely connected to the two-sided exit problem of such processes (cf. Bertion [10]).

Definition 2.9. For 𝛼≥0, the 𝛼-scale function (of the process 𝐼^∗_𝑡) 𝑊^(𝛼)∶𝑅→𝑅₊ is the unique function whose restriction to 𝑅₊ is continuous and has Laplace transform ^∞₀𝑒^−𝜃𝑥𝑊^(𝛼)(𝑥)1𝑑𝑥=𝜓(𝜃)(𝛼),−𝛼,𝜃>𝜂 (2.26) and is defined to be identically zero on the interval (−∞,0).

Letting 𝛼=0, we get the 0-scale function, which is referred to as the “scale function” in the literature. We denote this function by 𝑊 (instead of 𝑊⁽⁰⁾) throughout. We note that 𝜓(𝜃)=𝑀𝜃−𝜙(𝜃)=(𝑀𝜃−𝑎𝜃)−∫^∞₀(1−𝑒^−𝛼𝑥)𝜈(𝑑𝑥)=𝑁𝜃−𝜃∫^∞₀𝑒^−𝛼𝑥𝜈[𝑥,∞)𝑑𝑥, where 𝑁=𝑀−𝑎>0. Let 𝜇=∫^∞₀𝑥𝜈(𝑑𝑥)=∫^∞₀𝜈[𝑥,∞)𝑑𝑥. For every 𝑥∈𝑅₊, let 𝐹(𝑥)=∫^𝑥₀𝜈[𝑦,∞)𝑑𝑦/𝜇 be the equilibrium distribution function corresponding to 𝜈. Let 𝜌=𝜇/𝑁 and assume that 𝜌<1. It follows that𝑊(𝑥)=1𝑁^∞_𝑘=0𝜌^𝑘𝐹^(𝑘)(𝑥), (2.27) where 𝐹^(𝑘) is the 𝑘th convolution of 𝐹. Furthermore, we note that for 𝛼,𝑥∈𝑅₊,𝑊^(𝛼)(𝑥)=^∞_𝑘=0𝛼^𝑘𝑊^(𝑘+1)(𝑥), (2.28) where 𝑊^(𝑘) is the kth convolution of 𝑊.

We are now in a position to state and prove a lemma that characterizes 𝑈^∗𝛼.

Lemma 2.10. 𝑈^∗𝛼 is absolutely continuous with respect to the Lebesgue measure on [𝜏,∞), and its density is given as follows: 𝑈^∗𝛼𝑥,𝑦=𝑒^{−𝜂(𝛼)(𝑥−𝜏)}𝑊^(𝛼)𝑦−𝜏−𝑊^(𝛼)𝑥−𝑦,𝑥,𝑦∈[𝜏,∞). (2.29)

Proof. Define the process 𝐼^∧ to be equal to −𝐼^∗; it follows that 𝐼^∧ is a spectrally negative Lévy process. For 𝑎,𝑏∈𝑅, we let 𝑇⁺_𝑏=inf𝑡≥0∶𝐼^∧_𝑡,𝑇≥𝑏_−𝑎=inf𝑡≥0∶𝐼^∧_𝑡≤𝑎. (2.30)
Supurn [11] proved that (for 𝑏>0) the 𝛼-potential of the process obtained by killing the process 𝐼^∧at 𝑇⁺_𝑏∧𝑇⁻₀ is absolutely continuous with respect to the Lebesgue measure on [0,𝑏], and its density is equal to 𝑊^(𝛼)(𝑥)𝑊^(𝛼)𝑏−𝑦𝑊^(𝛼)(𝑏)−𝑊^(𝛼)𝑥−𝑦,𝑥,𝑦∈[0,𝑏]. (2.31) It follows that, for 𝑎,𝑏∈𝑅, 𝑎<𝑏, the 𝛼-potential of the process obtained by killing the process 𝐼^∧ at 𝑇⁺_𝑏∧𝑇⁻_𝑎 is absolutely continuous with respect to the Lebesgue measure on [𝑎,𝑏], and its density is equal to 𝑊^(𝛼)(𝑥−𝑎)𝑊^(𝛼)𝑏−𝑦𝑊^(𝛼)(𝑏−𝑎)−𝑊^(𝛼)𝑥−𝑦,𝑥,𝑦∈[𝑎,𝑏]. (2.32) From Lemma  4 of Pistorious [12], we have 𝑊^(𝛼)(𝑥)=𝑂(𝑒^{𝜂(𝛼)𝑥})as 𝑥→∞. Letting 𝑎→−∞ in the last density above, then the the 𝛼 potential of the process obtained by killing the process 𝐼^∧ at 𝑇⁺_𝑏is absolutely continuous with respect to the Lebesgue measure on (−∞,𝑏), and its density (denoted by 𝑢^𝛼_𝑏(𝑥,𝑦)) is as follows: 𝑢^𝛼_𝑏𝑥,𝑦=𝑒^{−𝜂(𝛼)(𝑏−𝑥)}𝑊^(𝛼)𝑏−𝑦−𝑊^(𝛼)𝑥−𝑦,𝑥,𝑦∈(−∞,𝑏]. (2.33)
Observe that for any 𝐴⊂(𝜏,∞) and 𝑥∈(𝜏,∞), 𝑃_𝑥𝑋^∗_𝑡∈𝐴=𝑃𝐼^∗_𝑡∈𝐴,𝑡<𝑊^∗_𝜏∣𝐼^∗₀𝐼=𝑥=𝑃_𝑡∈−𝐴,𝑡<𝑇⁺_−𝜏∣𝐼₀=−𝑥. (2.34) Thus, 𝑢^∗𝛼𝑥,𝑦=𝑢^𝛼_−𝜏−𝑥,−𝑦=𝑒^{−𝜂(𝛼)(𝑥−𝜏)}𝑊^(𝛼)𝑦−𝜏−𝑊^(𝛼)𝑦−𝑥,𝑥,𝑦∈[𝜏,∞). (2.35)

It is seen that, for 𝑥≥𝜏,𝐶^𝛼_𝑔∗(𝑀,𝑥,𝜏)=𝑈^∗𝑔^𝛼∗(𝑥)=^∞_𝜏𝑔^∗𝑦𝑈^∗𝛼,𝐶𝑥,𝑑𝑦_0𝑔∗(𝑀,𝑥,𝜏)=𝑈^∗𝑔^0∗(𝑥)=^∞_𝜏𝑔^∗𝑦𝑈^∗0𝑥,𝑑𝑦. (2.36)

Theorem 2.11. For any 𝛼≥0 and 𝑥≥0,(a)for 𝑥≤𝜆, 𝐸_𝑥exp−𝛼𝑇^∗₀=𝑀𝜂(𝛼)exp−𝜂(𝛼)(𝑥−𝜏)_{[𝜆−𝑥,∞)}exp−𝑧𝜂(𝛼)𝑈^𝛼(𝑑𝑧), (2.37)(b)for 𝑥>𝜆, 𝐸_𝑥exp−𝛼𝑇^∗₀=exp−𝜂(𝛼)(𝑥−𝜏). (2.38)

Proof of (a). Let ϝbe the sigma algebra generated by (𝑊_𝜆,𝐼_𝑊𝜆), then we have 𝐸_𝑥exp−𝛼𝑇^∗₀=𝐸_𝑥exp−𝛼𝑊_𝜆+𝑇^∗₀−𝑊_𝜆=𝐸_𝑥𝐸_𝑥exp−𝛼𝑊_𝜆+𝑇^∗₀−𝑊_𝜆∣ϝ=𝐸_𝑥exp−𝛼𝑊_𝜆𝐸_𝐼𝑊𝜆exp−𝛼𝑊^∗_𝜏=𝐸_𝑥exp−𝛼𝑊_𝜆exp−𝜂(𝛼)𝐼_𝑊𝜆−𝜏=𝐸₀exp−𝛼𝑊_𝜆−𝑥exp−𝜂(𝛼)𝐼_{𝑊𝜆−𝑥}(𝛼)𝐸+𝑥−𝜏=exp−𝜂)(𝑥−𝜏₀exp−𝛼𝑊_𝜆−𝑥exp−𝜂(𝛼)𝐼_{𝑊𝜆−𝑥}=𝛼+𝜙𝜂(𝛼)exp−𝜂(𝛼)(𝑥−𝜏)_{[𝜆−𝑥,∞)}exp−𝑧𝜂(𝛼)𝑈^𝛼()(𝛼)(𝛼)𝑑𝑧=𝑀𝜂exp−𝜂)(𝑥−𝜏_{[𝜆−𝑥,∞)}exp−𝑧𝜂(𝛼)𝑈^𝛼(𝑑𝑧), (2.39)where the third equation follows from the second equation, since given ϝ,𝑇^∗₀−𝑊_𝜆=𝑊^∗_𝜏almost everywhere, the fourth equation follows from (2.20) above, the seventh equation follows from (8) Alili and Kyprianou [9], and the last equation follows from (2.21) above.

Proof of (b). The proof of the part (b) of the theorem follows from (2.20), since for 𝑥>𝜆, 𝑊_𝜆=0 and 𝑇^∗₀=𝑊^∗_𝜏 almost everywhere.

3. The Total Discounted, Long-Run Average Costs and the Stationary Distribution of the Dam Content

We now discuss the computations of the cost functionals using the total discounted cost as well as the long-run average cost criteria. Let 𝑊 be the length of the first cycle, that is, 𝑊=𝑇^∗₁−𝑇^∗₀,and let 𝐶_𝛼(𝑥)be the expected cost during the interval [0,𝑇^∗₀),when 𝑍₀=𝑥. Since the content process 𝑍 is a delayed regenerative process with regeneration points 𝑇^∗₀,𝑇^∗₁,…, using the delayed regeneration property, it follows that the total discounted cost associated with an 𝑃^𝑀_𝜆,𝜏 policy is given by𝐶_𝛼(𝜆,𝜏)=𝐶_𝛼(𝑥)+𝐸_𝑥exp−𝛼𝑇^∗₀𝐸_𝜏𝐶_𝛼(1)1−𝐸_𝜏(),exp−𝛼𝑊 (3.1) where 𝐶_𝛼(1) is the total discounted cost during the interval (0,𝑊). From the definitions of 𝐶_𝛼(𝑥), it follows that, for 𝑥>𝜆, 𝐶_𝛼(𝑥)=𝑀𝐾₁−𝑅𝐸_𝑥^𝑊∗𝜏₀𝑒^−𝛼𝑡𝑑𝑡+𝐶^𝛼_𝑔∗(𝑀,𝑥,𝜏). (3.2) To compute 𝐶_𝛼(𝑥) for 𝑥≤𝜆, we let ϝ be the sigma algebra generated by (𝑊_𝜆,𝐼_𝑊𝜆) and proceed as follows:𝐶_𝛼(𝑥)=𝑀𝐾₂+𝐾₁𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝑅𝐸_𝑥^∗𝑇0_𝑊𝜆𝑒^−𝛼𝑡𝑑𝑡+𝐸_𝑥^𝑊𝜆₀𝑒^−𝛼𝑡𝑔𝑍_𝑡𝑑𝑡+𝐸_𝑥^∗𝑇0_𝑊𝜆𝑒^−𝛼𝑡𝑔^∗𝑍_𝑡𝐾𝑑𝑡=𝑀₂+𝐾₁𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝑅𝛼𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝐸_𝑥𝑒^{−𝛼∗𝑇0}+𝐸_𝑥^𝑊𝜆₀𝑒^−𝛼𝑡𝑔𝐼_𝑡𝑑𝑡+𝐸_𝑥^∗𝑇0_𝑊𝜆𝑒^−𝛼𝑡𝑔^∗𝑍_𝑡𝐾𝑑𝑡=𝑀₂+𝐾₁𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝑅𝛼𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝐸_𝑥𝑒^{−𝛼∗𝑇0}+𝐶^𝛼_𝑔(0,𝜏,𝜆)+𝐸_𝑥𝐸_𝑥^∗𝑇0_𝑊𝜆𝑒^−𝛼𝑡𝑔^∗𝑍_𝑡𝐾𝑑𝑡∣ϝ=𝑀₂+𝐾₁𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝑅𝛼𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝐸_𝑥𝑒^{−𝛼∗𝑇0}+𝐶^𝛼_𝑔(0,𝜏,𝜆)+𝐸_𝑥𝑒^{−𝛼𝑊𝜆}𝐸_𝐼𝑊𝜆^∗𝑊𝜏₀𝑒^−𝛼𝑡𝑔^∗𝐼^∗_𝑡𝐾𝑑𝑡=𝑀₂+𝐾₁𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝑅𝛼𝐸_𝑥𝑒^{−𝛼𝑊𝜆}−𝐸_𝑥𝑒^{−𝛼∗𝑇0}+𝐶^𝛼_𝑔(0,𝜏,𝜆)+𝐸_𝑥𝑒^{−𝛼𝑊𝜆}𝐶^𝛼_𝑔∗𝑀,𝐼_𝑊𝜆,𝜆, (3.3) where the second equation follows from the definition of the process 𝑍, the third equation follows from the definition of 𝐶^𝛼_𝑔(0,𝜏,𝜆),the fourth equation follows from the definition of the content process 𝑍 and since, given ϝ, 𝑇^∗₀−𝑊_𝜆=𝑊^∗_𝜏 almost everywhere, and the last equation follows from the definition of C^𝛼_𝑔∗(𝑀,𝑥,𝜆).

We note that𝐸_𝜏𝐶_𝛼(1)=𝐶_𝛼(𝜏). (3.4)

The following lemma shows how 𝐸_𝜏(exp(−𝛼𝑊))(given in (3.1)) can be computed and also gives a formula for computing the expected value of 𝑊, which we will need later on to compute the long-run average cost.

Lemma 3.1. Let 𝑊 be the length of the first cycle as defined above, then
(a) 𝐸_𝜏𝑒^−𝛼𝑊=1−𝑀𝜂(𝛼)₀^𝜆−𝜏exp−𝑧𝜂(𝛼)𝑈^𝛼(𝑑𝑧), (3.5)
(b) E_𝜏(𝑊)=𝑀𝐸₀𝑊_𝜆−𝜏𝑀−𝐸₀𝐼₁if𝐸₀𝐼₁<𝑀,=∞otherwise. (3.6)

Proof of (a). We note that, given 𝑍₀=𝜏, 𝑇^∗₀=𝑊 almost everywhere. Thus, for each 𝛼≥0, 𝐸_𝜏𝑒^−𝛼𝑊=𝐸_𝜏𝑒^{−𝛼𝑇∗0}=𝑀𝜂(𝛼)_{[𝜆−𝜏,∞)}exp−𝑧𝜂(𝛼)𝑈^𝛼()(𝛼𝑑𝑧=𝑀𝜂)^∞₀exp−𝑧𝜂(𝛼)𝑈^𝛼(𝑑𝑧)−₀^𝜆−𝜏exp−𝑧𝜂(𝛼)𝑈^𝛼((𝛼1𝑑𝑧)=𝑀𝜂)(𝛼)−𝑀𝜂₀^𝜆−𝜏exp−𝑧𝜂(𝛼)𝑈^𝛼((𝛼)𝑑𝑧)=1−𝑀𝜂₀^𝜆−𝜏exp−𝑧𝜂(𝛼)𝑈^𝛼(𝑑𝑧), (3.7)where the second equation follows (2.37) upon substituting 𝜏 for 𝑥, the third equation follows from the definition of the 𝑈^𝛼 and (2.21).

Proof of (b). From (3.5), it is evident that, starting at 𝜏, 𝑊 is finite almost everywhere if and only if 𝜂^′(0+)=0. From part (a) of Lemma 2.6, it follows that 𝑊 is finite almost everywhere if and only if 𝐸₀(𝐼₁)<𝑀. From (2.14) and (3.5), we have 𝐸_𝜏(𝑊)=𝑀𝜂^′(0)𝐸₀𝑊_𝜆−𝜏if𝐸₀𝐼₁<𝑀,=∞otherwise. (3.8) The proof of (b) is complete, since as shown in the proof of part (c) of Lemma 2.6𝜂^′(0)=1𝑀−𝐸₀𝐼₁if𝐸₀𝐼₁<𝑀. (3.9)

Now, we turn our attention to computing the long-run average cost per a unit of time. Let 𝑀−𝐸₀(𝐼₁)=𝑀^∗and assume that 𝑀^∗>0. From (3.1), (3.3), and (3.4), it follows, by a Tauberian theorem, that the long-run average cost per unit of time, denoted by 𝐶(𝜆,𝜏),is given by𝐶()=𝑀𝜆,𝜏𝐾+𝑅𝐸₀𝑊_𝜆−𝜏+𝐶_𝑔(0,𝜆,𝜏)+𝐸_𝜏𝐶_𝑔∗𝑀,𝐼_𝑊𝜆,𝜏𝐸_𝜏(𝑊)=−𝑅𝑀𝐾𝑀^∗+𝑀^∗/𝑀𝐶_𝑔(0,𝜆,𝜏)+𝐸_𝜏𝐶_𝑔∗𝑀,𝐼_𝑊𝜆,𝜏𝐸₀𝑊_𝜆−𝜏−𝑅𝐸₀𝐼₁, (3.10) where𝐾=𝐾₁+𝐾₂, and the second equation follows from (3.6) and the first equation.

Remark 3.2. Assume that both penalty functions 𝑔 and 𝑔^∗ are identically zero on their domains, and 𝑀^∗ defined above is greater than zero. The following follows from (3.10) above: 𝐶(𝜆,𝜏)=𝐾𝑀^∗𝐸₀𝑊_𝜆−𝜏−𝑅𝐸₀𝐼₁. (3.11) Letting 𝑅=0,𝐾=0and 𝑔(𝑥)=𝐼_[𝜏,𝑧](𝑥),𝑥∈[0,𝜆)and 𝑔^∗(𝑥)=𝐼_[𝜏,𝑧](𝑥),𝑥∈[𝜏,∞)in (3.10), we get the following proposition which generalizes the results obtained by Lee and Ahn [3], where they assumed that the input process is a compound Poisson process and 𝜏=0.

Proposition 3.3. Assume that 𝑀>𝐸₀(𝐼₁). Let 𝑍=lim_𝑡→∞𝑍_𝑡, and, 𝐻(𝑧) be the distribution function of the process 𝑍, then, for 𝑧∈[𝜏,∞), 𝐻(𝑧)=𝑀^∗𝑀𝐸₀𝑊_{(𝜆∧𝑧)−𝜏}𝐸₀𝑊_𝜆−𝜏+𝑀^∗𝑀𝐸𝑈_𝜏^∗0_{𝐼[𝜏,𝑧]}𝐼_𝑊𝜆𝐸₀𝑊_𝜆−𝜏. (3.12)

4. Special Cases

In this section, we give the basic identities needed to compute the cost functionals when the input process is an inverse Gaussian process and a compound Poisson process, respectively.

Case 1. Assume that 𝐼 is an inverse Gaussian process with transition function defined for 𝑥≥0,𝑦≥0,𝜇>0, and 𝜎²>0, by 𝑝=𝑡𝑡,𝑥,𝑦𝜎2𝜋𝑦−𝑥³−𝜇exp𝑦−𝑥−𝑡²2𝑦−𝑥𝜎²,𝑦≥𝑥.=0𝑦<𝑥. (4.1) It follows that the process 𝐼 is an increasing Lévy process with state space 𝑅₊, Lévy measure 𝜈=1𝑑𝑦𝜎𝑒2𝜋𝑦3^{−(𝑦𝜇2/2𝜎2)}, (4.2) and Lévy component 𝜙(𝛼)=2𝛼𝜎²+𝜇²−𝜇𝜎². (4.3) Furthermore, 𝐸₀(𝐼₁)=1/𝜇.

Substituting this Lévy component above in (2.21), it is seen that the solution of this equation is as follows (we omit the proof):𝜂(𝛼)=𝛼𝑀+1−𝑀𝜇+2𝛼𝑀𝜎²+1−𝑀𝜇²𝑀²𝜎². (4.4)

To find the 𝛼-potential of the process 𝐼, for each 𝑥≥0 and 𝛽≥0, we define 𝑓_𝛽(𝑥)=exp(−𝛽𝑥), and it is easily seen that𝑈^𝛼𝑓_𝛽(0)=𝜎²𝛼𝜎²+2𝛽𝜎²+𝜇².−𝜇 (4.5)

Throughout we let 𝜑_𝑍(⋅) be as the standard normal density function and let erf() and erf𝑐() be the well-known error and complimentary error functions, respectively. Inverting the above function with respect to 𝛽, we have 𝑈^𝛼=𝜎𝑑𝑦√𝑦𝜑_𝑍√𝑦𝜇𝜎𝑑𝑦+𝜇−𝛼𝜎²2𝑒^{𝛼𝑦((𝛼𝜎2/2)−𝜇)}erf𝑐√𝑦𝛼𝜎²−𝜇√2𝜎²𝑑𝑦=𝑢^𝛼𝑦𝑑𝑦, (4.6) where𝑢^𝛼𝑦=𝜎√𝑦𝜑_𝑍√𝑦𝜇𝜎+𝜇−𝛼𝜎²2𝑒^{𝛼𝑦((𝛼𝜎2/2)−𝜇)}erf𝑐√𝑦𝛼𝜎²−𝜇√2𝜎². (4.7)

From (2.13), it follows that, for 𝑥≤𝜆,𝐸_𝑥exp−𝛼𝑊_𝜆=𝛼𝑈^𝛼𝐼_{[𝜆−𝑥,∞)}(0)=𝛼𝜎²−𝜇𝛼𝜎²𝑒−2𝜇^{𝛼(𝜆−𝑥)((𝛼𝜎2/2)−𝜇)}erf𝑐√𝜆−𝑥𝛼𝜎²−𝜇√2𝜎²−𝜇𝛼𝜎²√−2𝜇erf𝑐𝜆−𝑥𝜇√2𝜎², (4.8) where the last equation follows by integrating 𝑈^𝛼(𝑑𝑦)over the interval [𝜆−𝑥,∞).

Inverting the right hand side of (4.8) with respect to 𝛼, it follows that, given 𝐼₀=𝑥≤𝜆, the distribution function of 𝑊_𝜆 (denoted by 𝐹_𝑊𝜆()) is given by𝐹_𝑊𝜆(𝑡)=12()erf𝑐𝜆−𝑥𝜇−𝑡√2𝜎²−12𝑒^{2𝜇𝑡/𝜎2}()erf𝑐𝜆−𝑥𝜇+𝑡√2𝜎²,𝑡≥0. (4.9) Furthermore, for 𝑥≤𝜆,𝐸_𝑥𝑊_𝜆=𝑈⁰𝐼_[0,𝜆)(𝑥)=𝜎₀^𝜆−𝑥1√𝑦𝜑_𝑍√𝑦𝜇𝜎𝜇𝑑𝑦+2₀^𝜆−𝑥erf𝑐−𝑦2𝜇𝜎=()𝜇𝑑𝑦𝜆−𝑥2+𝜎√𝜆−𝑥𝜑_𝑍√𝜆−𝑥𝜇𝜎+(𝜆−𝑥)𝜇²+𝜎²2𝜇erf𝜆−𝑥2𝜇𝜎, (4.10) where the third equation follows from the second equation upon tedious calculations which we omit.

We now turn our attention to computing the distribution function of 𝐼_𝑊𝜆 (denoted by 𝐹_𝐼𝑊𝜆(𝑥)). We first need the following identity which expresses the Lévy component 𝜙(𝛼) given in (4.3) in a form suitable for computing 𝐹_𝐼𝑊𝜆. The proof of this identity follows from (4.3) after some simple algebraic manipulations which we omit:𝜙(𝛼)=2𝛼𝜎²+𝜇²−𝜇𝜎²=2𝜎²𝛼𝜙(𝛼)−𝜇. (4.11) For each 𝛽∈𝑅₊, we write^∞_𝜆𝑒^−𝛽𝑥𝐹_𝐼𝑊𝜆(𝑥)𝜙𝛽𝑑𝑥=𝛽^∞_𝜆𝑒^−𝛼𝑥𝑢⁰(𝑥)=2𝑑𝑥𝜎²1𝜙𝛽^∞_𝜆𝑒^−𝛽𝑥𝑢⁰(𝑥)𝜇𝑑𝑥−𝛽^∞_𝜆𝑒^−𝛽𝑥𝑢⁰(𝑥)=2𝑑𝑥𝜎^2∞₀𝑒^−𝛽𝑥𝑢⁰(𝑥)𝑑𝑥^∞_𝜆𝑒^−𝛽𝑥𝑢⁰(𝑥)𝜇𝑑𝑥−𝛽^∞_𝜆𝑒^−𝛽𝑥𝑢⁰(𝑥)=2𝑑𝑥𝜎^2∞_𝜆𝑒^{−𝛽𝑥𝑥}_𝜆𝑢⁰𝑥−𝑦−𝜇𝑢⁰𝑦𝑑𝑦𝑑𝑥, (4.12) where the first equation follows from the second equation given the proof of part (a) of Lemma 2.4 by letting 𝑥=0, the second equation follows from (4.11), the third equation follows from (4.5) upon letting 𝛼=0, and the fourth equation follows from the third equation through integration by parts.

From (4.12), it follows that, for each 𝑥≥𝜆,𝐹_𝐼𝑊𝜆(𝑥)=2𝜎^2𝑥_𝜆𝑢⁰𝑥−𝑦−𝜇𝑢⁰𝑦𝑑𝑦. (4.13)

Case 2. Assume that 𝐼 is an increasing compound Poison process with intensity 𝑢 and 𝐹 as the distribution function of the size of each jump. This model is treated in details in references [3, 4, 6]. Here, we give the basic entities involved when the drift terms 𝑎=0. For the proof of these entities and more in depth analysis of this case, the reader is referred to the above- mentioned references.
It is obvious that 𝜙(𝛼)=𝑢^∞₀1−𝑒^−𝛼𝑥𝐹(𝑑𝑥), (4.14)𝐸₀(𝐼₁)=𝑢𝜇, where 𝜇is the expected jump size of the compound Poisson process.
Define, for any 𝛼≥0 and 𝑦≥0, 𝐹_𝛼(𝑦)=(𝑢/𝑢+_𝛼)𝐹(𝑦).For 𝑛∈𝑁₊, we let 𝐹_𝛼^(𝑛)(𝑦)be the 𝑛th convolution of 𝐹_𝛼(𝑦), where 𝐹_𝛼⁽⁰⁾(𝑦)=1 for all 𝑦≥0. For each 𝑦≥0, we define 𝑅_𝛼(𝑦)=∑^∞_𝑛=0𝐹_𝛼^(𝑛)(𝑦) to be the renewal function corresponding to 𝐹_𝛼(𝑦).It follows that 𝑈^𝛼=1𝑑𝑦𝑅𝑢+𝛼^𝛼𝑑𝑦. (4.15) Furthermore, for 𝑥≤𝜆, 𝐸_𝑥exp−𝛼𝑊_𝜆=1−𝛼𝑈^𝛼𝐼_{[0,𝜆−𝑥)}(0)𝛼=1−𝑅𝑢+𝛼^𝛼(),𝐸𝜆−𝑥_𝑥𝑊_𝜆=𝑈⁰𝐼_{[0,𝜆−𝑥)}(0)=1𝑢𝑅₀(𝜆−𝑥). (4.16)
Also, 𝐸₀𝑒^{−𝛼𝐼𝑊𝜆}=𝜙(𝛼)^∞_𝜆𝑒^−𝛼𝑥𝑈⁰()=𝑑𝑥_∞𝜆𝑒^−𝛼𝑥𝑅₀(𝑑𝑥)−^∞₀𝑒^−𝛼𝑥𝐹(𝑑𝑥)^∞_𝜆𝑒^−𝛼𝑥𝑅₀(d𝑥)=^∞_𝜆𝑒^−𝛼𝑥𝑅₀(𝑑𝑥)−^∞_𝜆𝑒^{−𝛼𝑥𝑥}_𝜆𝐹𝑑𝑥−𝑦𝑅₀𝑑𝑦, (4.17) where the first equation follows from the second equation in the proof of part (a) of Lemma 2.4, by letting 𝑥=0. Furthermore, the second equation follows from (4.14) and (4.15).
Inverting (4.17) with respect to 𝛼, the distribution function of 𝐼_𝑊𝜆, denoted by 𝐺, is given through 𝐺(𝑑𝑥)=𝑅₀(𝑑𝑥)−^𝑥_𝜆𝐹𝑑𝑥−𝑦𝑅₀𝑑𝑦𝐼_[𝜆,∞)(𝑥)=𝐹(𝑑𝑥)+^𝑥_𝜆𝐹𝑑𝑥−𝑦𝑅₀𝑑𝑦𝐼_[𝜆,∞)(𝑥). (4.18)

Acknowledgment

This Research was supported, in part, by a 2010 Summer Research Grant from the College of Business and Economics, UAE University.