Remarks on a monotone Markov chain

Abstract

In applications, considerations on stochastic models often involve a Markov chain {ζn}0∞ with state space in R+, and a transition probability Q. For each x  R+ the support of Q(x,.) is [0,x]. This implies that ζ0≥ζ1≥…. Under certain regularity assumptions on Q we show that Qn(x,Bu)→1 as n→∞ for all u>0 and that 1−Qn(x,Bu)≤[1−Q(x,Bu)]n where Bu=[0,u). Set τ0=max{k;ζk=ζ0}, τn=max{k;ζk=ζτn−1+1} and write Xn=ζτn−1+1, Tn=τn−τn−1. We investigate some properties of the imbedded Markov chain {Xn}0∞ and of {Tn}0∞. We determine all the marginal distributions of {Tn}0∞ and show that it is asymptotically stationary and that it possesses a monotonicity property. We also prove that under some mild regularity assumptions on β(x)=1−Q(x,Bx), ∑1n(Ti−a)/bn→dZ∼N(0,1).