Subgeometric Ergodicity under Random-Time State-Dependent Drift Conditions

Abstract

Motivated by possible applications of Lyapunov techniques in the stability of stochastic networks, subgeometric ergodicity of Markov chains is investigated. In a nutshell, in this study we take a look at 𝑓-ergodic general Markov chains, subgeometrically ergodic at rate 𝑟, when the random-time Foster-Lyapunov drift conditions on a set of stopping times are satisfied.

1. Introduction

Ergodicity of Markov processes has, in recent years, received quite substantial attention. Of the three standard types of ergodicity, being ordinary, geometric (often referred to as exponential), and strong ergodicity, ordinary ergodicity stands out as the most well studied of the three, especially on discrete time Markov chains. However for subgeometric ergodicity, which can be referred to as a kind of convergence that is faster than ordinary ergodicity but slower than geometric ergodicity, much study is needed especially for continuous Markov processes [1].

Ergodicity in the context of state-dependent drift conditions on a deterministic time index has been studied by many authors, [2–4]. Connor and Fort [2] and Yüksel and Meyn [3] studied ergodicity with a drift condition taking the form 𝑃^𝑛(𝑥)𝑉(𝑥)≤𝛽𝑉(𝑥)+𝑏𝟏_𝐶(𝑥) for some deterministic function 𝑛∶𝒳→[1,∞) and a constant 𝛽∈(0,1). According to Theorem 2.1(ii) of [5] a deterministic sequence of functions 𝑉_𝑛 exists 𝑉∶𝒳→[1,∞) and satisfies a Foster-Lyapunov drift condition: 𝑃𝑉_𝑛+1≤𝑉_𝑛−𝑟(𝑛)𝑓+𝑏𝑟(𝑛)𝟏_𝐶,𝑛∈ℤ₊ (1) for a petite set 𝐶∈ℬ(𝒳) and a constant 𝑏<∞ such that 𝑉₀ is bounded on 𝐶.

It has been proved that this Foster-Lyapunov condition holds not only for every 𝑛∈ℤ₊ but also for a sequence of stopping times {𝒯_𝑛}, 𝑛∈ℤ₊ for some discrete time Markov chain (Φ_𝑛)_𝑛∈ℤ+, [4]. The results of Zurkowski [4] relied heavily on the work of Connor and Fort [2], Meyn and Tweedie [6], and Tuominen and Tweedie [5]. The aim of this study is to investigate and refine these random-time state-dependent drift conditions results with our emphasis on the subgeometric ergodicity for a general Markov chain.

As observed by Yüksel and Meyn [3], there is a function 𝑉 on the state space 𝒳 taking positive values and an increasing sequence of stopping times {𝒯_𝑖∶𝑖∈ℤ₊}, with 𝒯₀=0, such that, for each 𝑖, 𝐸𝑉Φ_𝒯𝑖+1∣ℱ_𝒯𝑖≤𝑉Φ_𝒯𝑖−𝛿Φ_𝒯𝑖, (2) where the function 𝛿∶𝒳→ℝ is positive (bounded away from zero) outside a small set and ℱ_𝒯𝑖 denotes the filtration of events up to time 𝒯_𝑖. The reader needs to take note that although inequality (2) is important to this study, it will not be of much use because we will only use its refinement which is availed as (23) in Theorem 7.

Throughout the paper the sequence of stopping times {𝒯_𝑖∶𝑖∈ℤ₊} is assumed to be nondecreasing, with 𝒯₀=0. On state-dependent criteria for stability, traditionally it is assumed that the stopping times take the form 𝒯_𝑖+1=𝒯_𝑖+𝑛Φ𝒯_𝑖,𝑖≥0, (3) where 𝑛∶𝒳→ℕ is a deterministic function of the state.

Although our work will mostly be confined to working with petite sets, we know that for 𝜓-irreducible and aperiodic processes petite sets are in fact small sets [5]. Thus we need not worry that the techniques that rely on small sets in some of the previous studies may become unavailable in the random-time drift setting thinking that a small set for Φ_𝑛 may not necessarily be small for Φ_𝒯𝑛. Small sets are analogous to compact sets in the stability theory of 𝜓-irreducible Markov chains. Also in most applications of 𝜓-irreducible Markov chains we find that any compact set is small.

The paper is organized as follows. In Section 2, we have the preliminaries which introduces basic notations, definitions, and theorems. The main results are availed in Section 3 which is mostly concerned with (𝑓,𝑟)-ergodicity. In Section 4 we make the conclusions of this study.

2. Preliminaries

Let ℤ₊={0,1,2,…}, ℕ₊={1,2,…}, and ℝ₊=[0,∞). Let (Φ_𝑛)_𝑛∈ℤ+(resp., (Φ_𝑡)_𝑡∈ℝ+) denote a discrete time Markov chain (DTMC), (resp., a continuous time Markov chain (CTMC) on a countable space, as given in Liu et al. [7]). The state space for all chains is denoted (𝒳,ℬ(𝒳)). Unless otherwise specified 𝜓-irreducibility, aperiodicity, and positive recurrence of all the chains we will be dealing with is assumed. The corresponding transition function of the DTMC process is denoted by 𝑃^𝑛(𝑥,𝑦)=𝑃_𝑥(Φ_𝑛=𝑦)=𝐸_𝑥[𝟏_Φ𝑛=𝑦], where 𝟏_𝐴, here and hereafter, is the indicator function of set 𝐴 and 𝑃_𝑥 and 𝐸_𝑥, respectively, denote the probability and expectation of the chain under the condition that Φ₀=𝑥. Analogously we get 𝑃^𝑡(𝑥,𝑦)=𝑃_𝑥(Φ_𝑡=𝑦)=𝐸_𝑥[𝟏_Φ𝑡=𝑦] for the CTMC.

2.1. Subgeometric Rate Function

Let function 𝑟∈Λ₀, where Λ₀ is the family of measurable increasing functions 𝑟∶ℝ₊→[1,∞) satisfying log𝑟(𝑛)/𝑛↓0 as 𝑛↑∞. Let Λ denote the class of positive functions 𝑟∶ℝ₊→(0,∞) such that for some 𝑟∈Λ₀ we have 0<lim_𝑛inf𝑟(𝑛)𝑟(𝑛)≤lim_𝑛sup𝑟(𝑛)𝑟(𝑛)<∞. (4) Then Λ is referred to as the class of subgeometric rate functions [8]. Indeed (4) implies the equivalence of the class of functions Λ₀ with the class of functions Λ. Example of functions in the class 𝑟∈Λ is the rate 𝑟(𝑛)=exp(𝑠𝑛^1/(1+𝛼)), 𝛼>0, 𝑠>0, which has been discussed only recently in literature. Without loss to generality we suppose that 𝑟(0)=1 whenever 𝑟∈Λ.

The properties of 𝑟∈Λ₀ which follow from (4) and are to be used frequently in this study are 𝑟𝑥+𝑦≤𝑟(𝑥)𝑟𝑦∀𝑥,𝑦∈ℝ₊𝑟(𝑥+𝑎)𝑟(𝑥)⟶1as𝑥→∞,foreach𝑎∈ℝ₊. (5)(6)

2.2. Subgeometric Rate Ergodicity

Let 𝑟∈Λ; then, the ergodic chain Φ_𝑛 is said to be subgeometric ergodic of order 𝑟 in the 𝑓-norm or simply (𝑓,𝑟)-ergodic if for all 𝑥∈𝒳, then lim_𝑡→+∞𝑟(𝑛)𝑃^𝑛(𝑥,⋅)−𝜋(⋅)_𝑓=0, (7) where 𝜋 is the unique invariant distribution of the process for a (signed) measure 𝜎 and ‖𝜎‖_𝑓=sup_{|𝑔|≤𝑓}|𝜎(𝑔)| and 𝑓∶𝒳→[1,∞) is a measurable function.

In the analysis of ergodicity, for any nonempty set 𝐶∈ℬ(𝒳) the two first hitting times are given as 𝛿_𝐶=inf{𝑛≥𝐽₁∶Φ_𝑛∈𝐶}, where 𝐽₁ denotes the first jump-time of the chain Φ_𝑛 and 𝜏_𝐶=inf{𝑛≥0∶Φ_𝑛∈𝐶}. If 𝐶 is a singleton consisting of state 𝑥, we write 𝛿_𝑥 for 𝛿_𝐶. We also note that 𝜏_𝐶=0≠𝛿_𝐶 if Φ₀∈𝐶; however, 𝜏_𝐶=𝛿_𝐶, if Φ₀∉𝐶. For hitting times of the continuous chain Φ_𝑡 we need only replace 𝑛∈ℤ₊ with 𝑡∈ℝ₊ in the above notations and we are done. It is known that (7) holds if and only if the Foster-Lyapunov (or just Lyapunov) drift condition holds. It is for this reason that we cannot do without the Lyapunov drift conditions in our study. Let 𝑟(̂𝑡)=∫^𝑡₀𝑟(𝑠)𝑑𝑠, for the case of Φ_𝑡.

2.3. -Ergodicity

For ∈ℝ such that ≥1, the chain Φ_𝑛 is called -ergodic if max_𝑥∈𝐶𝐸_𝑥[𝛿_𝐶]<∞ for some finite nonempty 𝐶⊆𝒳.

2.4. Small Set

A set 𝐶∈ℬ(𝒳) is said to be small (or (𝑛,𝜖,𝜈)-small) if there exists some 𝑛∈ℤ₊, 𝜖>0, and a probability measure 𝜈(⋅) on 𝒳 such that 𝑃^𝑛(𝑥,⋅)≥𝜖𝜈(⋅),∀𝑥∈𝐶. (8)

2.5. Petite Set

A set 𝐶∈ℬ(𝒳) is said to be petite if there exists some nontrivial measure 𝜈_𝑎 on ℬ(𝒳) and some probability distribution 𝑎={𝑎_𝑛∶𝑛∈ℤ₊} such that ^∞_𝑛=1𝑎_𝑛𝑃^𝑛(𝑥,⋅)≥𝜈_𝑎(⋅),∀𝑥∈𝐶. (9) Petite sets generalize small sets. If Ψ-irreducibility and aperiodicity of the process are assumed (as is the case throughout this study) then set 𝐶∈ℬ⁺(𝒳) is a petite set if and only if (8) is satisfied. Thus petite sets in this case are in fact small sets.

Next we state condition 𝐷 and Theorems 1 and 2 which will be useful in this study especially in the proof of Theorem 8.

2.6. Condition 𝐷(𝜙,𝑉,𝐶)

There exist a function 𝑉∶𝒳→[1,∞), a concave monotone function 𝜙∶[1,∞]→(0,∞], a measurable set 𝐶, and a constant 𝑏<∞ such that 𝑃𝑉(𝑥)−𝑉(𝑥)≤𝜙∘𝑉+𝑏𝟏_𝐶. (10)

Theorem 1 (Theorem 14.0.1 in [6]). If condition 𝐷 holds for some petite set 𝐶 and there exists 𝑥₀∈𝒳 such that 𝑉(𝑥₀)<∞, then there exists a unique invariant distribution 𝜋, 𝜋(𝜙∘𝑉)<∞ and lim_𝑛→∞𝑃^𝑛(𝑥,⋅)−𝜋(⋅)_𝜙∘𝑉=0,𝜋--𝑎.𝑒., (11) where 𝜙∘𝑉(𝑥)≥1, for all 𝑥∈𝒳.

Theorem 2 (Proposition 2.5 of Douc et al. [9]). Let 𝑃 be a 𝜓-irreducible and aperiodic kernel. Assume that 𝐷(𝜙,𝑉,𝐶) holds for function 𝜙 with lim_𝑡→+∞𝜙^′(𝑡)=0, a petite set 𝐶, and a function 𝑉 with {𝑉<+∞}≠∅. Then there exists an invariant probability measure 𝜋, and for all 𝑥 in the full and absorbing set {𝑉<∞}, we got lim_𝑛→∞𝑟_𝜙(𝑛)𝑃^𝑛(𝑥,⋅)−𝜋(⋅)=0,𝜋--𝑎.𝑒., (12) where 𝑟_𝜙(𝑛)=𝜙∘𝐻_𝜙⁻¹(𝑛), 𝐻_𝜙(𝑣)∶=∫^𝑣₁(1/𝜙(𝑥))𝑑𝑥.

3. Main Results

The core results in this Section are presented as Theorems 4, 7, and 8. Theorem 4 is a modified version of Theorem 5.1.3 in [4], while Theorem 7 is a modified version of both Theorem 2.1 in Yüksel and Meyn [3] and Proposition 5.1.2 in Zurkowski [4]. Then we have the refinement of Theorem 7 in the light of the work of [9] given as Theorem 8.

The skeleton chain method will be employed in the proof of Proposition 3, so next we define it. Let the chain {Φ_𝑛}_𝑛∈ℤ+ be aperiodic and irreducible. Let ℎ>0; for the skeleton chain Φ_𝑛ℎ we define, 𝛿_𝑥(ℎ)=inf𝑛≥1∶Φ_𝑛ℎ=𝑥; (13) then 𝛿_𝑥(ℎ)∶=ℎ𝛿_𝑥(ℎ) denotes the first hitting time on 𝑥 of the skeleton chain Φ_𝑛ℎ. Proposition 3 will be employed in the proof of Theorem 4.

Proposition 3. For some (and equivalently for any) finite nonempty set 𝐴⊂𝒳, some constant ℎ>0 and any subgeometric rate function 𝑟, we have 𝐸_𝑦^{𝛿𝑦(ℎ)}_𝑘=0𝑟(𝑘ℎ)<∞. (14)

Proof. For any ℎ>0, when Φ₀=𝑥, there is the possibility that the skeleton chain Φ_𝑛ℎ can miss visits of the continuous time process Φ_𝑡 to 𝑦 resulting in 𝛿_𝑦≤𝛿_𝑦(ℎ). Supposing that Φ₀=𝑥, we let 𝐷_𝑘 be the 𝑘th sojourn time at 𝑦 and 𝑊_𝑘 be the interval length between the 𝑘th exit from 𝑦 and the next visit to 𝑦 for the continuous time Markov process Φ_𝑡. We note the 𝐷_𝑘 are independent and identically distributed, having an exponential distribution with parameter 𝜆. Also we note that the 𝑊_𝑘 are independent of each other and the 𝐷_𝑘.
We define 𝑁=min𝑛≥1∶theℎ-skeletonchainisinstate𝑦duringtheinterval𝐷_𝑛. (15) Then 𝛿_𝑦(ℎ)≤𝜏_𝑦+∑^𝑁−1_𝑘=1(𝐷_𝑘+𝑊_𝑘)+ℎ, so by the submultiplicative property (5) we have 𝐸_𝑥𝑟𝛿_𝑦(ℎ(ℎ)𝐸)≤𝑟_𝑥𝑟𝜏_𝑦×^∞_𝑛=1𝐸_𝑦𝑟^𝑛−1_𝑘=1𝐷_𝑘+𝑊_𝑘𝟏_{𝑁=𝑛}≤𝑟(ℎ)𝐸_𝑥𝑟𝜏_𝑦×^∞_𝑛=1𝐸_𝑦𝑟^𝑛−1_𝑘=1𝐷_𝑘+𝑊_𝑘𝟏_⋂^𝑛−1_{𝑘=1{𝐷𝑘≤ℎ}}≤𝑟(ℎ)𝐸_𝑥𝑟𝜏_𝑦×^∞_𝑛=1𝐸_𝑦𝑟((𝑛−1)ℎ)𝑟^𝑛−1_𝑘=1𝑊_𝑘1−𝑒^{𝜆ℎ𝑛−1}≤𝑟(ℎ)𝐸_𝑥𝑟𝜏_𝑦×^∞_𝑛=1𝑟((𝑛−1)ℎ)𝐸_𝑦𝑟𝛿_𝑦^𝑛−11−𝑒^{𝜆ℎ𝑛−1}. (16) We define 𝐷(ℎ)=∑^∞_𝑛=1𝑟((𝑛−1)ℎ)(𝐸_𝑦[𝑟(𝛿_𝑦)])^𝑛−1(1−𝑒^𝜆ℎ)^𝑛−1. Also we have lim_ℎ→0(1−𝑒^𝜆ℎ)=0; by hypothesis we get 𝐸_𝑦[𝑟(𝛿_𝑦)]<∞, and by property (6) we get lim_𝑛→∞(𝑟(𝑛ℎ−ℎ)/𝑟(𝑛ℎ))=1; therefore we can choose ℎ₀>0 small enough such that 𝐷(ℎ₀)<∞. By (16) we get the following recursive formula: 𝐸_𝑦^{𝛿𝑦(ℎ0)}_𝑘=0𝑟𝑘ℎ₀=𝐸_𝑦𝑟𝛿_𝑦ℎ₀+𝐸_𝑦^{𝛿𝑦(ℎ0)−1}_𝑘=0𝑟𝑘ℎ₀=𝛿_𝑦ℎ₀𝐸_𝑦𝑟𝛿_𝑦ℎ₀+𝑟(0)<∞. (17)

Theorem 4. Suppose that there exist a function 𝑉∶𝒳→(0,∞) and constants 𝑏∈ℝ and 𝛽∈(0,1) such that for an increasing sequence of stopping times {𝒯_𝑖}𝐸𝑉Φ_𝒯𝑖+1∣ℱ_𝒯𝑖≤𝛽𝑉Φ_𝒯𝑖+𝑏𝟏_{{Φ𝒯𝑖∈𝐶}}. (18) Then for any 𝑟∈Λ the chain Φ_𝑛 is 𝑟-ergodic.

Proof. It is worth noting that Φ_𝒯𝑖 is also an aperiodic and irreducible Markov chain that satisfies the univariate condition of (Lemma 14 of [10]); hence, 𝑉 is bounded on 𝐶. For any 𝑟∈Λ and 𝜉^𝑧_𝐶=min{𝑛≥𝑧∶Φ_𝑛∈𝐶}, with 𝜏_𝐶≤𝜉^𝑧_𝐶 for all 𝐶∈ℬ⁺(𝒳), we have sup_𝑥∈𝐶𝐸_𝑥^𝜏𝐶−1_𝑘=0𝑟(𝑘)≤sup_𝑥∈𝐶𝐸_𝑥^𝛿𝐶−1_𝑘=0𝑟(𝑘)≤sup_𝑥∈𝐶𝐸_𝑥^{𝜉𝑧𝐶−1}_𝑘=0𝑟(𝑘). (19) Owing to the equivalence of Λ₀ and Λ we suppose that 𝑟∈Λ₀ therefore rate 𝑟 satisfies the submultiplicative property (5). We have established by Proposition 3 that 𝐸_𝑦[∑^{𝛿𝑥(ℎ)}_𝑘=0𝑟(𝑘ℎ)]<∞ which satisfies (19) for some ℎ=1 and so by Theorem 2.1 in [1] implies that 𝐸_𝑦[𝑟(̂𝛿_𝑦)]<∞. Then by Theorem 3.3 of Liu et al. [7], the set 𝐶 is (1,𝑟)-regular hence we conclude that the chain is 𝑟-ergodic.

As a consequence of studying subgeometric ergodicity in the total variation norm (that is in the 𝟏-norm, when we let 𝑓≡𝟏), it is worth noting that the results of Theorem 4 above are applicable to continuous time Markov chains, (Φ_𝑡)_𝑡∈ℝ+ on a countable state space studied in [7], by letting 𝑡=𝑛ℎ. Mao [11] investigated -ergodicity when is restricted to ℤ₊. Liu et al. [7] extended the results to the case when ∈ℝ₊. Under random-time state-dependent drift function, we state the following corollary which is an extension of Corollary 2.1 of Liu et al. [7].

Corollary 5. Let ∈ℝ₊ with ≥1. Then the following are equivalent.(i)The chain Φ_𝑡 is -ergodic, that is, max_𝑥∈𝐶𝐸_𝑥[𝛿_𝐶]<∞ for some finite nonempty set 𝐶.(ii)For some nonempty finite set 𝐶 and an increasing sequence of stopping times {𝒯_𝑖}, there exist ⌊⌋+1 finite nonnegative functions 𝑉_𝑛(Φ_𝒯𝑖), 0≤𝑛≤⌊⌋ and a constant 𝑏 such that 𝑉₀(Φ_𝒯𝑖)≥𝐸_Φ𝒯𝑖[𝜏^̂₀] and _{Φ𝒯𝑖+1∈𝐶}𝑞_{Φ𝒯𝑖,Φ𝒯𝑖+1}𝑉_𝑚+1Φ_𝒯𝑖+1≤−̂+𝑚+1𝑉_𝑚Φ_𝒯𝑖+𝑏𝟏_𝐶Φ_𝒯𝑖 (20)for 0≤𝑚≤⌊⌋−1, where ̂=−⌊⌋ and ⌊⌋ denote the integer-part function, that is, the largest integer which is smaller than or equal to .

Proof. The proof is analogous to the proof of Corollary 2.1 of Liu et al [7] and follows naturally therefrom.

For stochastically monotone CTMCs, some computable bounds on exponential convergence based on first hitting times and the drift function were investigated in Corollary 2.2 in Liu et al. [7] which leads us to the next corollary. The following corollary is analogous to Corollary 2.2 of Liu et al. [7] for a chain of the form (Φ_𝑡)_𝑡∈ℝ+ and also follows naturally consequent to the results of Theorem 4.

Corollary 6. Let 𝒳=ℤ₊. Suppose that the regular and totally stable intensity matrix 𝑄 is monotone and 𝑟(𝑡) is strictly increasing. If 𝐸₀[𝑟(̂𝛿₀)]<∞ then Φ_𝑡 is subgeometrically ergodic of order 𝑟 and for any 𝑥∈𝒳 and any 𝑡∈ℝ₊ we have 𝑟(𝑡)𝑃^𝑡(𝑥,⋅)−𝜋(⋅)≤𝐷_𝑥, (21) where 𝐷_𝑥=2𝐸_𝑥[𝑟(𝜏₀)]+2𝐸_𝜋[𝑟(𝜏₀)]<∞. In the specific case of 𝑟(𝑡)=𝑡, >1 such that ∈ℤ₊ and the drift criterion (ii) of Corollary 5 holds for the singleton set 𝐶={0}, 𝑉_𝑛(Φ_𝒯𝑖), 0≤𝑛≤⌊⌋ and 𝑏; then, for any Φ_𝒯𝑖=𝑥∈𝒳 and any 𝑡∈ℝ₊𝑡⁻¹𝑃^𝑡(𝑥,⋅)−𝜋(⋅)≤𝐷_𝑥, (22) where 𝐷_𝑥=2𝑉_⌊⌋−1(𝑥)+(2𝑏𝜋₀/)≤2𝑉_⌊⌋−1(𝑥)+(2𝑏/)≤∞.

Proof. The proof is analogous to the proof of Corollary 2.2 of Liu et al. [7] and follows naturally therefrom.

Theorem 7 which follows deals with subgeometric rate convergence of 𝑓-ergodic chains, in the case when 𝑓 isn’t a total variation norm. In a nutshell we’ll be dealing with (𝑓,𝑟)-regularity of the petite subset 𝐶⊂𝒳, or simply (𝑓,𝑟)-ergodicity of the chain Φ_𝑛.

Theorem 7. Suppose that Φ_𝑛 is a 𝜓-irreducible and aperiodic Markov chain. Suppose further that there are functions 𝑉∶𝒳→(0,∞), 𝛿∶𝒳→[1,∞), 𝑓∶𝒳→[1,∞), a small set 𝐶, constant 𝑏∈ℝ and 𝑟∈Λ such that for an increasing sequence of stopping times {𝒯_𝑖}𝐸𝑉Φ_𝒯𝑖+1∣ℱ_𝒯𝑖≤𝑉Φ_𝒯𝑖−𝛿Φ_𝒯𝑖+𝑏𝟏_{{Φ𝒯𝑖∈𝐶}};𝐸^{𝒯𝑖+1−1}_{𝑘=𝒯𝑖}𝑓Φ_𝑘𝑟(𝑘)∣ℱ_𝒯𝑖≤𝛿Φ_𝒯𝑖,𝑖≥0 (23) then, Φ_𝑛 is (𝑓,𝑟)-ergodic.

Proof. We define a sequence {𝑀_𝑖∶𝑖≥0} by 𝑀_𝑖=𝑉Φ_𝒯𝑖−^𝑖−1_𝑘=0𝛿Φ_𝒯𝑘−𝑏𝟏_{{Φ𝒯𝑘∈𝐶}}, (24) where by convention ∑^𝑛_𝑘=𝑚(⋅)=0 whenever 𝑚>𝑛; hence, 𝑀₀=𝑉(Φ₀).
Then under the assumed drift conditions we have 𝐸𝑀_𝑗∣ℱ_𝒯𝑗−1≤𝑉Φ_𝒯𝑗−1−^𝑗−2_𝑘=0𝛿Φ_𝒯𝑘−𝑏𝟏_{{Φ𝒯𝑘∈𝐶}}=𝑀_𝑗−1,𝑗≥1 (25) which implies that {𝑀_𝑖∶𝑖≥0} is a supermartingale. For a measurable subset 𝐶⊂𝒳 we denote the first hitting time for the sampled chain delayed by a constant 𝑛∈ℤ₊ as 𝜉^𝑛_𝐶=min𝑖≥𝑛∶Φ_𝒯𝑖∈𝐶; (26) then 𝐸[𝑀_𝜉^𝑛_𝐶]≤𝑀₀ for any 𝑛∈ℤ₊, and 𝐸^{𝜉𝑛𝐶−1}_𝑘=0𝛿Φ_𝒯𝑖∣ℱ₀≤𝑀₀+𝑏<∞ (27) because 𝑉 is bounded on 𝐶.
It is assumed that 𝜏_𝐶≤𝜉^𝑛_𝐶,∀𝐶∈ℬ⁺(𝒳) (28) so that 𝐸_𝑥^𝜏𝐶−1_{𝑘=𝒯𝑖}𝑓Φ_𝑘𝑟(𝑘)≤𝐸_𝑥^{𝜉𝑛𝐶−1}_𝑘=0𝛿Φ_𝒯𝑖<𝑀₀+𝑏<∞ (29) which implies (𝑓,𝑟)-ergodicity of the chain Φ_𝑛.

Theorem 8 below is similar to Proposition 5.1.4 in [4] and Theorem 2.8 in [9]. Its importance is that it generalizes Theorem 7. By employing condition 𝐷(𝜙,𝑉,𝐶), Theorems 1 and 2 we are a able to get quite a wide range of functions of the forms 𝑓 and 𝑟.

Theorem 8. Let the chain Φ_𝑛 be aperiodic and irreducible, that is, both 𝜙∘𝑉-ergodic and 𝑟_𝜙-ergodic for some 𝑉∶𝒳→[1,∞), 𝜙∶[1,∞]→(1,∞] and subgeometric rate function 𝑟∈Λ. Assume that condition 𝐷(𝜙,𝑉,𝐶) holds for 𝜙 such that lim_𝑡→∞𝜙^′(𝑡)=0 and a petite set 𝐶 such that sup_𝐶<+∞. Further let Ψ_𝑖,Ψ_𝑗∶𝒳→[1,∞) be pairs of ultimately nondecreasing functions, which satisfy Ψ_𝑖(𝑥)Ψ_𝑗(𝑥)≤𝑥+𝑦 and Ψ_𝑖(𝑥),Ψ_𝑗(𝑥)→∞, where 𝑖=1 (resp., 𝑖=2) whenever 𝑗=2 (resp., 𝑗=1). Then the chain (Φ_𝑛 is Ψ_𝑗(𝜙∘𝑉),Ψ_𝑖∘𝑟_𝜙)-ergodic.

Proof. By Theorem 2 we know that the chain is (1,𝑟_𝜙)-ergodic and also that for a large enough 𝑥, it holds that Ψ_𝑖(𝑥),Ψ_𝑗(𝑥)≤2𝑥. Then by Lemma 2.7 in [9] for some constant 𝑎 such that Ψ_𝑖(𝑥)≤𝑎𝑥 we have that Ψ_𝑖∘𝑟_𝜙∈Λ, where 𝑖/𝑗=1,2. Hence the chain is (1,Ψ_𝑖∘𝑟_𝜙)-ergodic. Further Theorem 1 states that the chain is (𝜙∘𝑉,1)-ergodic; then, it follows that there exists a class of sets where the chain is (Ψ_𝑗(𝜙∘𝑉),1)-ergodic 𝑗/𝑖=1,2. Its worth noting that there is a balance between the rate of convergence Ψ_𝑖∘𝑟_𝜙 and the norm Ψ_𝑗(𝜙∘𝑉); that is, the slower the former the larger the latter.
Because of the 𝜓-irreducibility and aperiodicity of the chain Φ_𝑛, there is only one full absorbing set 𝑆, which can be partitioned into two partitions say, 𝐺_𝑟 of (1,Ψ_𝑖∘𝑟_𝜙)-regular sets and say, 𝐺_𝑛 of (Ψ_𝑗(𝜙∘𝑉),1)-regular sets. The intersection of these two partitions yields the set 𝐺_𝑟,𝑛 of (Ψ_𝑗(𝜙∘𝑉),Ψ_𝑖∘𝑟_𝜙)-regular sets, and by Proposition 2.6 in [9] we have sup_{𝑥∈𝐺𝑟,𝑛}𝐸^𝜏𝐶−1_𝑘=0Ψ_𝑖𝑟_𝜙(𝑘)Ψ_𝑗𝜙∘𝑉Φ_𝑘<∞ (30) for any accessible set 𝐶∈ℬ⁺(𝒳). Indeed (30) implies that the chain is (Ψ_𝑗(𝜙∘𝑉),Ψ_𝑖∘𝑟_𝜙)-ergodic.

4. Conclusion

The refinement of Theorem 2.1 in Yüksel and Meyn [3] and Proposition 5.1.2 in Zurkowski [4] has been accomplished in this study and presented in the form of Theorem 7. Theorems 4 and 8 follow as a consequence of Theorem 7. This study like a multitude of previous studies before it may pave way for more research on random-time state-dependent Foster-Lyapunov drift conditions especially that interest on the same has developed from researchers on control and optimization theory, information theory, and so forth.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.