First Hitting Place Probabilities for a Discrete Version of the Ornstein-Uhlenbeck Process

Abstract

A Markov chain with state space {0,…,𝑁} and transition probabilities depending on the current state is studied. The chain can be considered as a discrete Ornstein-Uhlenbeck process. The probability that the process hits 𝑁 before 0 is computed explicitly. Similarly, the probability that the process hits 𝑁 before −𝑀 is computed in the case when the state space is {−𝑀,…,0,…,𝑁} and the transition probabilities 𝑝_𝑖,𝑖+1 are not necessarily the same when 𝑖 is positive and 𝑖 is negative.

1. Introduction

The Ornstein-Uhlenbeck process {𝑋(𝑡),𝑡≥0} is defined by the stochastic differential equation

𝑑𝑋(𝑡)=−𝑐𝑋(𝑡)𝑑𝑡+𝑑𝑊(𝑡), (1.1) where {𝑊(𝑡),𝑡≥0} is a standard Brownian motion and 𝑐 is a positive constant. Discrete versions of this very important diffusion process have been considered by various authors. In particular, Larralde [1, 2] studied the discrete-time process {𝑋_𝑛,𝑛=0,1,…} for which

𝑋_𝑛+1=𝛾𝑋_𝑛+𝑌_𝑛+1,(𝑋₀=𝑥₀), (1.2) where the random variables 𝑌_𝑛+1 are i.i.d. with zero mean and a common probability distribution. Larralde computed the probability that {𝑋_𝑛,𝑛=0,1,…} will hit the negative semiaxis for the first time at the 𝑛th step, starting from 𝑋₀=0. The problem was solved exactly in the case when the distribution of the random variables 𝑌_𝑛 is continuous and such that

𝑓_𝑌𝑛1(𝑦)=2𝑒^−|𝑦| (1.3) for 𝑦∈ℝ and for all 𝑛∈{1,2,…}.

Versions of the discrete Ornstein-Uhlenbeck process have also been studied by, among others, Renshaw [3], Anishchenko et al. [4, page 53], Bourlioux et al. [5, page 236], Sprott [6, page 234], Kontoyiannis and Meyn [7], and Milstein et al. [8]. In many cases, the distribution of the 𝑌_𝑛's is taken to be 𝑁(0,𝜎²).

For discrete versions of diffusion processes, in general, see Kac [9] and the references therein. A random walk leading to the Ornstein-Uhlenbeck process is considered in Section 4 of Kac's paper.

Next, consider a Markov chain for which the displacements take place every Δ𝑡 units of time. When the process is in state 𝑥, it moves to 𝑥+Δ𝑥 (resp., 𝑥−Δ𝑥) with probability 𝜃(𝑥) (resp., 𝜙(𝑥)) and remains in 𝑥 with probability 1−𝜃(𝑥)−𝜙(𝑥).

Assume that (Δ𝑥)²=𝐴Δ𝑡, and let

1𝜃(𝑥)=12𝐴[𝛼(𝑥)+𝛽(𝑥)Δ𝑥],𝜙(𝑥)=2𝐴[𝛼(𝑥)−𝛽(𝑥)Δ𝑥], (1.4) where 𝐴 is a positive constant such that 𝛼(𝑥)<𝐴 for all 𝑥. Then, when Δ𝑥 and Δ𝑡 decrease to zero, the Markov chain converges to a diffusion process having infinitesimal mean 𝛽(𝑥) and infinitesimal variance 𝛼(𝑥) (see [10, page 213]). In the case of the Ornstein-Uhlenbeck process, 𝛽(𝑥)=−𝑐𝑥 (with 𝑐>0) and 𝛼(𝑥)≡1. Hence, with 𝐴=2, we have that

1𝜃(𝑥)=41(1−𝑐𝑥Δ𝑥),𝜙(𝑥)=4(1+𝑐𝑥Δ𝑥). (1.5)

In the present paper, we first consider the Markov chain with state space {0,…,𝑁} and

𝑝_𝑖,𝑖+1=14(1−𝑐𝑖),𝑝_{𝑖,𝑖−1}=14(1+𝑐𝑖),𝑝_𝑖,𝑖=12 (1.6) for 𝑖=1,…,𝑁−1. Notice that 𝑝_𝑖,𝑖+1 (resp., 𝑝_{𝑖,𝑖−1}) could be denoted by 𝜃_𝑖 (resp., 𝜙_𝑖). To respect the condition 𝑝_𝑖,𝑗∈[0,1]  for all 𝑖,𝑗, the positive constant 𝑐 must be such that

1𝑐<.𝑁−1 (1.7)

This Markov chain with state-dependent transition probabilities may also clearly be regarded as a discrete version of the Ornstein-Uhlenbeck process. It corresponds to the case when 𝛾=1 in (1.2) and

𝑌_𝑛+1=−1withprobability𝑝_{𝑋𝑛,𝑋𝑛−1},10withprobability2,1withprobability𝑝_{𝑋𝑛,𝑋𝑛+1}, (1.8) for 𝑋_𝑛∈{1,2,…,𝑁−1}.

In Section 2, the probability

𝑝_𝑖∶=𝑃[𝑋_𝜏=𝑁∣𝑋₀=𝑖], (1.9) where

𝜏∶=inf{𝑛>0∶𝑋_𝑛=0or𝑁} (1.10) and 𝑖∈{1,2,…,𝑁−1}, will be computed explicitly. In Section 3, the problem will be extended by assuming that the state space of the Markov chain {𝑋_𝑛,𝑛=0,1,…} is {−𝑀,…,0,…,𝑁}. Furthermore, the transition probabilities 𝑝_𝑖,𝑗 will be assumed to be (possibly) sign-dependent (see [11]). Finally, some concluding remarks will be made in Section 4.

2. First Hitting Place Probabilities

To obtain the first hitting place probability defined in (1.9), we may try to solve the following difference equation:

𝑝_𝑖=(1−𝑐𝑖)4𝑝_𝑖+1+12𝑝_𝑖+(1+𝑐𝑖)4𝑝_𝑖−1⇔𝑝_𝑖=(1−𝑐𝑖)2𝑝_𝑖+1+(1+𝑐𝑖)2𝑝_𝑖−1 (2.1) for 𝑖=1,…,𝑁−1, subject to the boundary conditions

𝑝₀=0,𝑝_𝑁=1. (2.2) For 𝑁 small, it is a relatively simple task to calculate explicitly 𝑝_𝑖 for all 𝑖 by solving a system of linear equations. However, we want to obtain an exact expression for any positive 𝑁.

Next, setting 𝑥=𝑖−1 and letting 𝑦(𝑥)=𝑝_𝑖−1, (2.1) can be rewritten as

[1−𝑐(𝑥+1)2]𝑦(𝑥+2)−𝑦(𝑥+1)+[1+𝑐(𝑥+1)2]𝑦(𝑥)=0 (2.3) for 𝑥=0,…,𝑁−2 (with 𝑦(0)=0 and 𝑦(𝑁)=1). This second-order homogeneous equation with linear coefficients is called the hypergeometric difference equation, due to the fact that its solutions can be expressed in terms of the hypergeometric function (see [12, page 68]).

Equation (2.3) can be transformed into its normal form, namely,

(𝑥+𝛽₁+𝛽₂+2)𝑦(𝑥+2)−[(𝜌₁+𝜌₂)(𝑥+1)+𝛽₁𝜌₂+𝛽₂𝜌₁]𝑦(𝑥+1)+𝜌₁𝜌₂𝑥𝑦(𝑥)=0. (2.4) In our case, we have (see [12, pages 68-69])

𝜌₁=1,𝜌₂=−1,𝛽₁=−1,𝛽₂2=−1−𝑐, (2.5) so that we must solve

(𝑥−𝑎)𝑦(𝑥+2)+𝑎𝑦(𝑥+1)−𝑥𝑦(𝑥)=0, (2.6) where

2𝑎∶=𝑐. (2.7) Furthermore, the variable 𝑥 now belongs to the set {1+1/𝑐,…,𝑁−1+1/𝑐} (because the new argument of the function 𝑦 is 𝑥^′=𝑥+𝛽₃, where 𝛽₃=1+1/𝑐 in our problem).

Using the results in Batchelder [12, Chapter III], we can state that a fundamental system of solutions of (2.6) is

𝑦₁,𝑦(𝑥)≡𝛾∈ℝ₂(𝑥)=(−1)^𝑥Γ(𝑥)1Γ(𝑥−𝑎)𝐹(−𝑎,1,𝑥−𝑎,2), (2.8) where 𝐹(⋅,⋅,⋅,⋅) is the hypergeometric function defined by (see [13, page 556])

𝐹(𝛼,𝛽,𝛾,𝑧)=^∞_𝑛=0(𝛼)_𝑛(𝛽)_𝑛(𝛾)_𝑛𝑧^𝑛𝑛! (2.9) with

(𝛼)_𝑛∶=𝛼(𝛼+1)⋯(𝛼+𝑛−1),(and(𝛼)₀=1). (2.10)

Remark 2 s. (i) The function 𝐹 is sometimes denoted by ₂𝐹₁. It can also be expressed as (see, again, [13, page 556]) 𝐹(𝛼,𝛽,𝛾,𝑧)=Γ(𝛾)Γ(𝛼)Γ(𝛽)^∞_𝑛=0Γ(𝛼+𝑛)Γ(𝛽+𝑛)𝑧Γ(𝛾+𝑛)^𝑛.𝑛! (2.11)
 (ii) The ratio 𝐹(𝛼,𝛽,𝛾,𝑧₀)/Γ(𝛾) is an entire function of 𝛼, 𝛽, and 𝛾 if 𝑧₀ is fixed and such that |𝑧₀|<1 (see [14, page 68]).
Now, because of the term (−1)^𝑥, the function 𝑦₂(𝑥) defined previously is generally complex-valued. Since the function 𝑦(𝑥) in our application is obviously real, we can take the real part of 𝑦₂(𝑥). That is, we simply have to replace (−1)^𝑥 by cos(𝜋𝑥). Alternatively, because [𝑥] = [𝑖+1+1/𝑐] = 𝑖+1+[1/𝑐], where [] denotes the integer part, we can write that
(−1)^𝑥=(−1)^{[𝑥]+1/𝑐−[1/𝑐]}. (2.12) With the difference equation (2.6) being homogeneous, we can state that 𝑦₂(𝑥)=(−1)^[𝑥]Γ(𝑥)1Γ(𝑥−𝑎)𝐹(−𝑎,1,𝑥−𝑎,2) (2.13) is a real-valued function that is also a solution of this equation. Hence, the general solution of (2.6) can be expressed as 𝑦(𝑥)=𝛾₁+𝛾₂(−1)^[𝑥]Γ(𝑥)1Γ(𝑥−𝑎)𝐹(−𝑎,1,𝑥−𝑎,2), (2.14) where 𝛾₁ and 𝛾₂ are arbitrary (real) constants.
(iii) We must be careful when the constant 𝑐 is of the form
1𝑐=,𝑁+𝑗 (2.15) where 𝑗∈{0,1,…}. Indeed, because 𝛽₁=−1, (2.6) is reducible. Moreover, it is completely reducible if 𝛽₂ is also a negative integer (see [12, pages 123-124]), that is, if 𝛽₂2=−1−𝑐=−𝑘 (2.16) with 𝑘∈{1,2,…}. Since 𝑐 must be smaller than 1/(𝑁−1) (see (1.7)), this condition translates into 1𝑐=2𝑁+𝑗or𝑐=,2(𝑁+𝑗)−1 (2.17) where 𝑗∈{0,1,…}. We find that the case when 𝑐=2/[2(𝑁+𝑗)−1] does not really cause any problem. However, when 𝑐=1/(𝑁+𝑗), we can show that although 𝑦₁(𝑥)≡𝛾 and 𝑦₂(𝑥) defined in (2.13) are obviously linearly independent when we consider all possible values of the argument 𝑥, it turns out that in our problem 𝑦₂(𝑥) always takes on the same value. More precisely, we can show that 𝑦₂(𝑥)=(−1)^[𝑥]𝑃_𝑎(𝑥), (2.18) where 𝑃_𝑎(𝑥) is a polynomial of degree 𝑎, with 2𝑎=𝑐=2(𝑁+𝑗), (2.19) given by 𝑃_𝑎(𝑥)=𝑎!𝑥1(−2)^𝑎𝑎_𝑛=0(−2)^𝑛𝑥^(𝑛+1),𝑛! (2.20) with 𝑥^(𝑖)∶=𝑥(𝑥−1)⋯(𝑥−𝑖+1) (2.21) for any natural number 𝑖.

RemarkThe formula for 𝑃_𝑎(𝑥) is valid if 𝑎=2(𝑁+𝑗)−1 as well, so that we can set 𝑎 equal to 2𝑁+𝑗, with 𝑗∈{−1,0,1,…}, above.

Now, we find that

𝑦₂(𝑥)=(−1)^𝑎+1𝑎!2^𝑎if𝑥=1,2,…,𝑎+1. (2.22) For example, suppose that 𝑁=3, so that the state space of the Markov chain is {0,1,2,3}, and that 𝑐=1/3. Because 𝑥=𝑖+4, the possible values of 𝑥 are 4,5,6,7. Furthermore, 𝑎=2/𝑐=6. The solution 𝑦₂(𝑥) can be written as

𝑦₂(𝑥)=(−1)^[𝑥]{𝑥⁶−24𝑥⁵+4552𝑥⁴−1080𝑥³+2674𝑥²−3216𝑥+57154}. (2.23) It is a simple matter to show that this function satisfies (2.6) with 𝑎=6. However, we calculate

𝑦₂(𝑥)=−454for𝑥∈{1,2,…,7}. (2.24) Thus, 𝑦₁(𝑥) and 𝑦₂(𝑥) are both constant for the values of interest of 𝑥 in our problem.

Actually we easily find that 𝑝₁=1/13 and 𝑝₂=3/13 in this example. Therefore, we cannot make use of 𝑦₁(𝑥) and 𝑦₂(𝑥) to obtain 𝑝_𝑖. Nevertheless, because 𝑦₂(𝑥) is a continuous function of the parameter 𝑐, we simply have to take the limit as 𝑐 tends to 1/(𝑁+𝑗) to get the solution we are looking for.

Next, we have obtained the general solution of (2.6) in (2.14). We must find the constants 𝛾₁ and 𝛾₂ for which the boundary conditions

𝑎𝑦(1+2𝑎)=0,𝑦(𝑁+1+2)=1 (2.25) are satisfied. We can state the following proposition.

Proposition 2.1. When 𝑐≠2/[2(𝑁+𝑗)−1] for 𝑗∈{0,1,…}, the probability 𝑝_𝑖 defined in (1.9) is given by 𝑝_𝑖1=𝑦(𝑖+1+𝑐), (2.26) where the function 𝑦(⋅) is defined in (2.14), with 𝛾₂=(−1)^[𝑎/2]{𝑎2√𝜋Γ(𝑎/2)Γ((1−𝑎)/2)−(−1)^𝑁Γ(𝑁+1+𝑎/2)𝑎Γ(𝑁+1−𝑎/2)𝐹(−𝑎,1,𝑁+1−2,12)}⁻¹,𝛾₁=𝛾₂(−1)^[𝑎/2]𝑎2√𝜋Γ(𝑎/2).Γ((1−𝑎)/2) (2.27) In the case when 𝑐=2/[2(𝑁+𝑗)−1], the constants 𝛾₁ and 𝛾₂ become 𝛾₁=0,𝛾₂=(−1)^𝑁Γ(𝑁+1−𝑎/2).Γ(𝑁+1+𝑎/2)𝐹(−𝑎,1,𝑁+1−𝑎/2,1/2) (2.28)

Proof. We find (see [13, page 557]) that 𝑦₂(𝑥) evaluated at 𝑥=1+𝑎/2 (i.e., 𝑖=0) can be expressed as 𝑦₂𝑎(1+2)=(−1)^1+[𝑎/2]𝑎2√𝜋Γ(𝑎/2).Γ((1−𝑎)/2) (2.29) This is actually obtained as a limit when 𝑐=1/(𝑁+𝑗) with 𝑗∈{0,1,…}. Moreover, it follows that as 𝑐 tends to 2/[2(𝑁+𝑗)−1], we have 𝑦₂𝑎(1+2)→0. (2.30) Hence, for any 𝑐≠2/[2(𝑁+𝑗)−1], the constants 𝛾₁ and 𝛾₂ are uniquely determined from the boundary conditions (2.25), while 𝑐=2/[2(𝑁+𝑗)−1] immediately yields that 𝛾₁=0 and that 𝛾₂ is as in (2.28).

Remark 2 s. (i) We see that the case when the difference equation is completely reducible is rather special. When 𝑐=2/[2(𝑁+𝑗)−1], the constant 𝛾₁ vanishes, while when 𝑐=1/(𝑁+𝑗), the probability 𝑝_𝑖 is obtained by taking the limit of the previous solution when 𝑐 tends to this particular value.
(ii) We can obtain an approximate formula for the probability 𝑝_𝑖, valid for 𝑁 large, by proceeding as follows. First, because (by assumption) 𝑐<1/(𝑁−1), we can write that
1𝑐=⇔1𝑁−1+𝜅𝑐=𝑁−1+𝜅, (2.31) where 𝜅>0. Hence, 1𝑐1≈𝑁−1+[𝜅]+21=𝑁+[𝜅]−2. (2.32) Notice that the relative error 𝜖_𝑟 committed by replacing 1/𝑐 by its approximate value is such that 𝜖_𝑟≤1/2,𝑁−1+𝜅 (2.33) so that it is negligible when 𝑁 is large. Moreover, for this approximate value of the constant 𝑐, we can express the solution in terms of the polynomial in (2.20), with 𝑎=2(𝑁+[𝜅])−1. Making use of the boundary conditions, we deduce that 𝑦(𝑥)≈(−1)^[𝑥]𝑃_𝑎(𝑥)(−1)^𝑁𝑃_𝑎.(𝑁+1+𝑎/2) (2.34) We can simply write that 𝑃𝑦(𝑥)≈|^𝑎(𝑥)𝑃_𝑎(𝑁+1+𝑎/2)|. (2.35) Since 𝑃_𝑎(𝑥) is a polynomial of degree 𝑎, we find that we have approximated the function 𝑦(𝑥) by a polynomial of degree 2[1/𝑐]+1.
In the next section, the state space of the Markov chain will be extended to {−𝑀,…,0,…,𝑁} and the (possibly) asymmetric case will be treated.

3. The Asymmetric Case

We extend the problem considered in the previous section by assuming that the state space of the Markov chain {𝑋_𝑛,𝑛=0,1,…} is the set

𝑆∶={−𝑀,…−1,0,1…,𝑁}, (3.1) where 𝑀∈{1,2,…}. Furthermore, we set

𝑝_0,1=𝑝₀,𝑝_0,−1=𝑞₀,𝑝_0,0=12, (3.2) where 𝑝₀,𝑞₀∈(0,1) and 𝑝₀+𝑞₀=1/2.

When 𝑖 is a negative state, we define

𝑝_𝑖,𝑖+1=1−𝑑𝑖4,𝑝_{𝑖,𝑖−1}=1+𝑑𝑖4,𝑝_𝑖,𝑖=12 (3.3) for 𝑖∈{−𝑀+1,…,−1}. In order to respect the condition 𝑝_𝑖,𝑗∈[0,1]  for all 𝑖,𝑗, we find that the positive constant 𝑑 must be such that

1𝑑<.𝑀−1 (3.4)

Let

𝑇=inf{𝑛>0∶𝑋_𝑛=𝑁or−𝑀}. (3.5) We want to compute the first hitting place probability

𝜋_𝑖∶=𝑃[𝑋_𝑇=𝑁∣𝑋₀=𝑖] (3.6) for 𝑖∈{−𝑀+1,…,0,…,𝑁−1}. We have

𝜋_𝑁=1,𝜋_−𝑀=0. (3.7)

Let us denote the probability 𝑝_𝑖 defined in (1.9) by 𝑝_𝑁(𝑖) and define

𝑝_𝑀(𝑖)=𝑃[𝑋_𝜎=−𝑀∣𝑋₀=𝑖], (3.8) where 𝑖∈{−𝑀+1,…,−1}, and

𝜎∶=inf{𝑛>0∶𝑋_𝑛=−𝑀or0}. (3.9) Proceeding as in Section 2, we can show that

𝑝_𝑀(𝑖)=𝑙₁+𝑙₂(−1)^{𝑖+1+[1/𝑑]}Γ(𝑖+1+1/𝑑)2Γ(𝑖+1−1/𝑑)𝐹(−𝑑1,1,𝑖+1−𝑑,12), (3.10) where the constants 𝑙₁ and 𝑙₂ are uniquely determined from the boundary conditions

𝑝_𝑀(−𝑀)=1,𝑝_𝑀(0)=0. (3.11) Again, we must be careful in the case when the difference equation is completely reducible.

Next, we define the events

𝐸_𝑖𝐹=theprocesshits𝑁before−𝑀from𝑖∈𝑆;_𝑖𝐺=theprocesshits𝑁before0from𝑖>0;_𝑖=theprocesshits−𝑀before0from𝑖<0. (3.12) Assume first that 𝑖 is positive. Then, we can write that

𝜋_𝑖≡𝑃[𝐸_𝑖]=𝑃[𝐸_𝑖∩𝐹_𝑖]+𝑃[𝐸_𝑖∩𝐹^𝑐_𝑖]=𝑝_𝑁(𝑖)+2[1−𝑝_𝑁(𝑖)]{𝜋₁𝑝₀+𝜋₋₁𝑞₀}. (3.13) When 𝑖 is negative, we have

𝜋_𝑖=𝑃[𝐸_𝑖∩𝐺_𝑖]+𝑃[𝐸_𝑖∩𝐺^𝑐_𝑖]=𝑃[𝐸_𝑖∩𝐺^𝑐_𝑖]=2𝑝_𝑀(𝑖){𝜋₁𝑝₀+𝜋₋₁𝑞₀}. (3.14)

Setting 𝑖=1 (resp., −1) in (3.13) (resp., (3.14)), we obtain a system of two linear equations for 𝜋₁ and 𝜋₋₁:

𝜋₁=𝑝_𝑁(1)+2[1−𝑝_𝑁(1)]{𝜋₁𝑝₀+𝜋₋₁𝑞₀𝜋},₋₁=2𝑝_𝑀(−1){𝜋₁𝑝₀+𝜋₋₁𝑞₀}. (3.15)

Proposition 3.1. The probability 𝜋_𝑖 defined in (3.6) is given for 𝑖>0 (resp., 𝑖<0) by (3.13) (resp., (3.14)), in which 𝜋₁=𝑝_𝑁(1)[1−2𝑞₀𝑝_𝑀(−1)]1−2𝑞₀𝑝_𝑀(−1)−2𝑝₀[1−𝑝_𝑁,𝜋(1)]₋₁=2𝑝₀𝑝_𝑀(−1)𝑝_𝑁(1)1−2𝑞₀𝑝_𝑀(−1)−2𝑝₀[1−𝑝_𝑁.(1)] (3.16)

Remark 3 s. (i) If 𝑝₀=𝑞₀=1/4, the formulas for 𝜋₁ and 𝜋₋₁ reduce to 𝜋₁=𝑝_𝑁(1)[2−𝑝_𝑀(−1)]1−𝑝_𝑀(−1)+𝑝_𝑁(1),𝜋₋₁=𝑝_𝑀(−1)𝑝_𝑁(1)1−𝑝_𝑀(−1)+𝑝_𝑁.(1) (3.17) Moreover, if 𝑀=𝑁 and 𝑑=𝑐, then (by symmetry) 𝑝_𝑀(−1)=𝑝_𝑁(1) and 𝜋₁=𝑝_𝑁(1)[2−𝑝_𝑁(1)],𝜋₋₁=𝑝²_𝑁(1). (3.18)
 (ii) The probability
𝜈_𝑖∶=𝑃[𝑋_𝑇=−𝑀∣𝑋₀=𝑖] (3.19) is of course given by 1−𝜋_𝑖, for 𝑖=−𝑀+1,…,0,…,𝑁−1.

4. Concluding Remarks

In Section 2, we computed the probability 𝑝_𝑖 that a Markov chain with transition probabilities given by (1.6) and state space {0,1,…,𝑁} will hit 𝑁 before 0, starting from 𝑖∈{1,…,𝑁−1}. If we let 𝑐 decrease to 0 in (1.6), we obtain that

𝑝_𝑖,𝑖+1=𝑝_{𝑖,𝑖−1}=14,𝑝_𝑖,𝑖=12for𝑖∈{1,2,…,𝑁−1}. (4.1) That is, the Markov chain {𝑋_𝑛,𝑛=0,1,…} is a (generalized) symmetric random walk having a probability 𝑝_𝑖,𝑖=1/2 of remaining in its current state on each transition. The fact that 𝑝_𝑖,𝑖>0 should not influence the probability 𝑝_𝑖. Taking the limit as 𝑐 decreases to 0 (i.e., 𝑎→∞) in Proposition 2.1, we indeed retrieve the well-known formula

𝑝_𝑖=𝑖𝑁for𝑖=0,1,…,𝑁. (4.2)

In Section 3, we were able to compute explicitly the probability 𝜋_𝑖 defined in (3.6) for a (possibly) asymmetric Markov chain with state space {−𝑀,…,0,…,𝑁}. This type of Markov chain could have applications in mathematical finance, in particular. Indeed, if one is looking for the probability that the value of a certain stock reaches a given level before a lower one, it can be more realistic to assume that the stock price does not vary in the same way when the price is high or low. Hence, the assumption that the transition probabilities may be different when 𝑋_𝑛>0 and 𝑋_𝑛<0 seems plausible in some applications. In the application we have just mentioned, 0 could be the centered current value of the stock.

Next, another problem of interest is the determination of the average time 𝐷_𝑖 the process, starting from 𝑖, takes to hit either 0 or 𝑁 (in Section 2), or −𝑀 or 𝑁 (in Section 3). To obtain an explicit expression for 𝐷_𝑖, we must solve a nonhomogeneous linear difference equation. Finding a particular solution to this equation (in order to obtain the general solution by using the solution to the homogeneous equation obtained in the present work) is a surprisingly difficult problem.

Finally, we could try to take the limit of the Markov chain {𝑋_𝑛,𝑛=0,1,…} in such a way as to obtain the Ornstein-Uhlenbeck process as a limiting process. We should retrieve the known formula for the probability 𝑝_𝑖 in the case of this process considered in the interval [0,𝑁] and generalize this formula to the asymmetric case, based on Section 3.

Acknowledgments

This work was supported by the Natural Sciences and Engineering Research Council of Canada. The authors are also grateful to the referee for constructive comments.