Research Article | Open Access

# Some Limit Properties of the Harmonic Mean of Transition Probabilities for Markov Chains in Markovian Environments Indexed by Cayley's Trees

**Academic Editor:**Onesimo Hernandez Lerma

#### Abstract

We prove some limit properties of the harmonic mean of a random transition probability for finite Markov chains indexed by a homogeneous tree in a nonhomogeneous Markovian environment with finite state space. In particular, we extend the method to study the tree-indexed processes in deterministic environments to the case of random enviroments.

#### 1. Introduction

A treeis a graph which is connected and doesn't contain any circuits. Given any two vertices, letbe the unique path connectingand. Define the graph distanceto be the number of edges contained in the path.

Letbe an infinite tree with root. The set of all vertices with distancefrom the root is called theth generation of, which is denoted by. We denote bythe union of the firstgenerations of. For each vertex, there is a unique path fromto andfor the number of edges on this path. We denote the first predecessor ofby . The degree of a vertex is defined to be the number of neighbors of it. If every vertex of the tree has degree, we say it is Cayley’s tree, which is denoted by. Thus, the root vertex hasneighbors in the first generation and every other vertex hasneighbors in the next generation. For any two vertices and of tree, writeifis on the unique path from the rootto. We denote bythe farthest vertex fromsatisfyingand. We use the notationand denote bythe number of vertices of.

In the following, we always letdenote the Cayley tree.

A tree-indexed Markov chain is the particular case of a Markov random field on a tree. Kemeny et al. [1] and Spitzer [2] introduced two special finite tree-indexed Markov chains with finite transition matrix which is assumed to be positive and reversible to its stationary distribution, and these tree-indexed Markov chains ensure that the cylinder probabilities are independent of the direction we travel along a path. In this paper, we omit such assumption and adopt another version of the definition of tree-indexed Markov chains which is put forward by Benjamini and Peres [3]. Yang and Ye[4] extended it to the case of nonhomogeneous Markov chains indexed by infinite Cayley’s tree and we restate it here as follows.

*Definition 1 ( T-indexed nonhomogeneous Markov chains (see [4])). *Letbe an infinite Cayley tree, a finite state space, anda stochastic process defined on probability space, which takes values in the finite set. Let
be a distribution on and
a transition probability matrix on. If, for any vertex,
thenwill be called -valued nonhomogeneous Markov chains indexed by infinite Cayley’s tree with initial distribution (1) and transition probability matrices.

The subject of tree-indexed processes has been deeply studied and made abundant achievements. Benjamini and Peres [3] have given the notion of the tree-indexed Markov chains and studied the recurrence and ray-recurrence for them. Berger and Ye [5] have studied the existence of entropy rate for some stationary random fields on a homogeneous tree. Ye and Berger [6, 7], by using Pemantle's result [8] and a combinatorial approach, have studied the Shannon-McMillan theorem with convergence in probability for a PPG-invariant and ergodic random field on a homogeneous tree. Yang and Liu [9] and Yang [10] have studied a strong law of large numbers for Markov chains fields on a homogeneous tree (a particular case of tree-indexed Markov chains and PPG-invariant random fields). Yang and Ye [4] have established the Shannon-McMillan theorem for nonhomogeneous Markov chains on a homogeneous tree. Huang and Yang [11] has studied the strong law of large numbers for finite homogeneous Markov chains indexed by a uniformly bounded infinite tree.

The previous results are all about tree-indexed processes in deterministic environments. Recently, we are interested in random fields indexed by trees in random environments. In the rest of this paper we formulate a model of Markov chain indexed by trees in random environment especially in Markovian environment and study some limit properties of the harmonic mean of random transition probability for finite Markov chains indexed by a homogeneous tree in a nonhomogeneous Markovian environment. We are also interested in the strong law of large numbers of Markov chains indexed by trees in random environments which we will prepare for in another paper.

*Definition 2. *Letbe an infinite Cayley tree,andtwo finite state spaces. Suppose thatis a-valued random field indexed by, if, for any vertex,
for each, where , is a family of stochastic matrices. Then we calla Markov chain indexed by treein a random environment. Theare called the environmental process or control process indexed by tree . Moreover, ifis a *T*-indexed Markov chain with initial distributionand one-step transition probability matrices, we calla Markov chain indexed by treein a nonhomogeneous Markovian environment.

*Remark 3. *If every vertex has degree two, then our model of Markov chain indexed by homogeneous tree with Markovian environments is reduced to the model of Markov chain in Markovian environments which was introduced by Cogburn [12] in spirit.

*Remark 4. *We also point out that our model is different from the tree-valued random walk in random environment (RWRE) that is studied by Pemantle and Peres [13] and Hu and Shi [14]. For the model of RWRE on the trees that they studied, the random environment processis a family of i.i.d nondegenerate random vectors and the processis a nearest-neighbor walk satisfying some conditions. But in our model,our environmental processcan be any Markov chain indexed by trees. Given, the processis another Markov chain indexed by trees with the law.

In this paper we assume thatis a nonhomogeneous-indexed Markov chain on state space. The probability of going fromtoin one step in theth environment is denoted by. We also suppose that the one-step transition probability of going fromtofor nonhomogeneous-indexed Markov chainis. In this case,is a Markov chain indexed bywith initial distributionand one-step transition on determined by where. Thenwill be called the bichain indexed by tree. Obviously, we have

#### 2. Main Results

For every finite, letbe a Markov chain indexed by an infinite Cayley treein Markovian environment, which is defined as in Definition 2. Now we suppose thatare functions defined on. Letbe a real number, ,; now we define a stochastic sequence as follows: At first we come to prove the following fact.

Lemma 5. *is a nonnegative martingale.*

*Proof of Lemma 5. *Obviously, we have
Here, the second equation holds because of the fact thatis a bichain indexed by treeand (8) is being used. Furthermore, we have
On the other hand, we also have
Combining (11) and (12), we get
Thus, we complete the proof of Lemma 5.

Theorem 6. *Letbe a Markov chain indexed by an infinite Cayley treein a nonhomogeneous Markovian environment. Suppose that the initial distribution and the transition probability functions satisfy
**
if there exist two positive constantsandsuch that
**
Denote, , and
**
then we have
*

*Proof. *By Lemma 5, we have known thatis a nonnegative martingale. According to Doob martingale convergence theorem, we have
so that
which implies that
We arrive at
Combining (20) with the inequalitiesand and taking, it follows that
Note that the following elementary fact holds:
Let. It follows from (15), (22), and (23) that
Here, the second equation holds because by lettingin inequality (24), we get
If, similar to the analysis of inequality (24), by using (15), (22), and (23) again, we can arrive at
Lettingin inequality (26), we get
Combining (25) and (27), we obtain that our assertion (17) is true.

Corollary 7. *Letbe a nonhomogeneous Markov chain indexed by an infinite Cayley tree. Suppose that the initial distribution and the transition probability functions satisfy
**
if there exist two positive constantsandsuch that
**
Denote; then we have
*

*Proof. *If we take, that is,, then the model of Markov chain indexed by treein Markovian environment reduces to the formulation of a nonhomogeneous Markov chain indexed by tree. Then we arrive at our conclusion (30) directly from Theorem 6.

Corollary 8 (see [15]). *Letbe a Markov chain in a nonhomogeneous Markovian environment. Suppose that the initial distribution and the transition probability functions satisfy
**
if there exist two positive constantsandsuch that
**
Denote, , and
**
then we have
*

*Proof. *If every vertex of the treehas degree, then the nonhomogeneous Markov chain indexed by treedegenerates into the nonhomogeneous Markov chain on line; thus, this corollary can be obtained from Theorem 6 directly.

#### Acknowledgment

This work was supported by the National Natural Science Foundation of China no. 11201344. The author declares that there is no conflict of interests regarding the publication of this paper.

#### References

- J. G. Kemeny, J. L. Snell, and A. W. Knapp,
*Denumerable Markov Chains*, Springer, New York, NY, USA, 2nd edition, 1976. View at: MathSciNet - F. Spitzer, “Markov random fields on an infinite tree,”
*The Annals of Probability*, vol. 3, no. 3, pp. 387–398, 1975. View at: Google Scholar | Zentralblatt MATH | MathSciNet - I. Benjamini and Y. Peres, “Markov chains indexed by trees,”
*The Annals of Probability*, vol. 22, no. 1, pp. 219–243, 1994. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - W. Yang and Z. Ye, “The asymptotic equipartition property for nonhomogeneous Markov chains indexed by a homogeneous tree,”
*IEEE Transactions on Information Theory*, vol. 53, no. 9, pp. 3275–3280, 2007. View at: Publisher Site | Google Scholar | MathSciNet - T. Berger and Z. X. Ye, “Entropic aspects of random fields on trees,”
*IEEE Transactions on Information Theory*, vol. 36, no. 5, pp. 1006–1018, 1990. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - Z. Ye and T. Berger, “Ergodicity, regularity and asymptotic equipartition property of random fields on trees,”
*Journal of Combinatorics, Information & System Sciences*, vol. 21, no. 2, pp. 157–184, 1996. View at: Google Scholar | Zentralblatt MATH | MathSciNet - Z. Ye and T. Berger,
*Information Measures for Discrete Random Fields*, Science Press, Beijing, China, 1998. View at: MathSciNet - R. Pemantle, “Automorphism invariant measures on trees,”
*The Annals of Probability*, vol. 20, no. 3, pp. 1549–1566, 1992. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - W. Yang and W. Liu, “Strong law of large numbers for Markov chains field on a Bethe tree,”
*Statistics & Probability Letters*, vol. 49, no. 3, pp. 245–250, 2000. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - W. Yang, “Some limit properties for Markov chains indexed by a homogeneous tree,”
*Statistics & Probability Letters*, vol. 65, no. 3, pp. 241–250, 2003. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - H. Huang and W. Yang, “Strong law of large numbers for Markov chains indexed by an infinite tree with uniformly bounded degree,”
*Science in China A*, vol. 51, no. 2, pp. 195–202, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - R. Cogburn, “Markov chains in random environments: the case of Markovian environments,”
*The Annals of Probability*, vol. 8, no. 5, pp. 908–916, 1980. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - R. Pemantle and Y. Peres, “Critical random walk in random environment on trees,”
*The Annals of Probability*, vol. 23, no. 1, pp. 105–140, 1995. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - Y. Hu and Z. Shi, “Slow movement of random walk in random environment on a regular tree,”
*The Annals of Probability*, vol. 35, no. 5, pp. 1978–1997, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet - Y. Q. Li, H. S. Wang, and Z. Wang, “Some strong limit theorems of geometric average of transition probabilities and their function for Markov chains in Markovian environments,”
*Acta Mathematica Scientia A*, vol. 31, no. 2, pp. 508–517, 2011 (Chinese). View at: Google Scholar | MathSciNet

#### Copyright

Copyright © 2013 Huilin Huang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.