Journal of Probability and Statistics

Volume 2013 (2013), Article ID 424601, 5 pages

http://dx.doi.org/10.1155/2013/424601

## On the Study of Transience and Recurrence of the Markov Chain Defined by Directed Weighted Circuits Associated with a Random Walk in Fixed Environment

Department of Civil Engineering, Section of Mathematics and Statistics, School of Technological Sciences, University of Thessaly, Pedion Areos, 38334 Volos, Greece

Received 23 October 2012; Revised 2 March 2013; Accepted 27 March 2013

Academic Editor: Man Lai Tang

Copyright © 2013 Chrysoula Ganatsiou. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

By using the cycle representation theory of Markov processes, we investigate proper criterions regarding transience and recurrence of the corresponding Markov chain represented uniquely by directed cycles (especially by directed circuits) and weights of a random walk with jumps in a fixed environment.

#### 1. Introduction

A systematic research has been developed (Kalpazidou [1], MacQueen [2], Minping and Min [3], Zemanian [4], and others) in order to investigate representations of the finite-dimensional distributions of Markov processes (with discrete or continuous parameter) having an invariant measure, as decompositions in terms of the cycle (or circuit) passage functions:
for any directed sequence (or ) of states called a *cycle (or a circuit)*, , of the corresponding Markov process. The representations are called *cycle (or circuit) representations *while the corresponding discrete parameter Markov processes generated by directed circuits , , are called *circuit chains*.

Following the context of the theory of Markov processes’ cycle-circuit representation, the present work arises as an attempt to investigate proper criterions regarding the properties of transience and recurrence of the corresponding Markov chain represented uniquely by directed cycles (especially by directed circuits) and weights of a random walk with jumps (having one elastic left barrier) in a fixed ergodic environment (Kalpazidou [1], Derriennic [5]).

The paper is organized as follows. In Section 2, we give a brief account of certain concepts of cycle representation theory of Markov processes that we will need throughout the paper. In Section 3, we present some auxiliary results in order to make the presentation of the paper more comprehensible. In particular, in Section 3, a random walk with jumps (having one elastic left barrier) in a fixed ergodic environment is considered, and the unique representations by directed cycles (especially by directed circuits) and weights of the corresponding Markov chain are investigated. These representations will give us the possibility to study proper criterions regarding transience and recurrence of the abovementioned Markov chain, as it is described in Section 4.

Throughout the paper, we will need the following notations:

#### 2. Preliminaries

Let us consider a denumerable set S. Then the directed sequence modulo the cyclic permutations, where , , completely defines a *directed circuit* in . The ordered sequence associated with the given directed is called a *directed cycle* in .

A directed circuit may be considered as , if there exists an , such that , , , where is a periodic function from to .

The corresponding directed cycle is defined by the ordered sequence . The values are the *points* of , while the directed pairs , , are the *directed edges* of .

The smallest integer satisfying the equation , for all , is the *period *of . A directed circuit such that is called a *loop*. (In the present work we will use directed circuits with distinct point elements.)

Let us also consider a directed circuit (or a directed cycle ) with period . Then we may define by
the *-step passage function* associated with the directed circuit , for any , .

Furthermore we may define by
the *passage function* associated with the directed circuit , for any . The above definitions are due to MacQueen [2] and Kalpazidou [1].

Given a denumerable set and an infinite denumerable class of overlapping directed circuits (or directed cycles) with distinct points (except for the terminals) in such that all the points of can be reached from one another following paths of circuit edges; that is, for each two distinct points and of there exists a finite sequence , , of circuits (or cycles) of such that lies on and lies on , and any pair of consecutive circuits have at least one point in common. Generally we may assume that the class contains, among its elements, circuits (or cycles) with period greater or equal to 2.

With each directed circuit (or directed cycle) let us associate a strictly* positive weight * which must be independent of the choice of the representative of , that is, it must satisfy the consistency condition , , where is the translation of length (that is, , , for any fixed ).

For a given class of overlapping directed circuits (or cycles) and for a given sequence of weights we may define by the elements of a Markov transition matrix on , if and only if , for any . This means that a given Markov transition matrix , , can be represented by directed circuits (or cycles) and weights if and only if there exists a class of overlapping directed circuits (or cycles) and a sequence of positive weights such that the formula (5) holds. In this case, the Markov transition matrix has a unique stationary distribution which is a solution of and is defined by

It is known that the following classes of Markov chains may be represented uniquely by circuits (or cycles) and weights:(i)the recurrent Markov chains (Minping and Min [3]),(ii)the reversible Markov chains.

#### 3. Auxiliary Results

Let us consider a Markov chain on with transitions , , and whose the elements of the corresponding Markov transition matrix are defined by such that , , , for every , as it is shown in (Figure 1).

Assume that and are arbitrary fixed sequences with , , , for every . If we consider the directed circuits , ,, and the collections of weights and , respectively, then we may obtain that with

Here the class contains the directed circuits , , and . Furthermore we may define and , such that , for every , with .

The transition matrix with where , if , are consecutive points of the circuit , , if is a point of the circuit , and , if is a point of the circuit , expresses the representation of the Markov chain by directed cycles (especially by directed circuits) and weights.

Furthermore we consider also the “*adjoint*” Markov chain on whose elements of the corresponding Markov transition matrix are defined by
such that , , for every , as it is shown in (Figure 2).

Assume that and are arbitrary fixed sequences with , , , for every . If we consider the directed circuits , , , and the collections of weights , and respectively, then we may have with

Here the class contains the directed circuits , , and . Furthermore we may define such that , for every , with

So the transition matrix with elements equivalent to that given by the above-mentioned formulas (11), (12) expresses also the representation of the “adjoint” Markov chain by directed cycles (especially by directed circuits) and weights.

Consequently we have the following.

Proposition 1. *The Markov chain defined as above has a unique representation by directed cycles (especially by directed circuits) and weights.*

*Proof. * Let us consider the set of directed circuits and , for every , since only the transitions from to , to , and to are possible. There are three circuits through each point , , and and two circuits through 0: , .

The problem we have to manage is the definition of the weights. We may symbolize by the weight of the circuit c_{k} and by the weight of the circuit , for any . The sequences , must be a solution of

Let us take by , , .

As a consequence we may have

Given the sequences and , it is clear that the above sequences and exist and are unique. This means that the sequences and are defined uniquely, up to multiplicative constant factors, by

(The unicity is understood up to the constant factors , .)

Proposition 2. * The “adjoint” Markov chain defined as above has a unique representation by directed cycles (especially by directed circuits) and weights.*

*Proof. * Following an analogous way of that given in the proof of Proposition 1 the problem we have also to manage here is the definition of the weights. To this direction we may symbolize by the weight of the circuit and by the weight of the circuit , for every . The sequences and must be solutions of

By considering the sequences and where , , , we may obtain that

For given sequences , it is obvious that , exist and are unique for those sequences, that is, the sequences , are defined uniquely, up to multiplicative constant factors, by

(The unicity is based to the constant factors , .)

#### 4. Recurrence and Transience of the Markov Chains and

We have that for the Markov chain , there is a unique invariant measure up to a multiplicative constant factor , , while for the Markov chain , , with . In the case that an irreducible chain is recurrent there is only one invariant measure (finite or not), so we may obtain the following.

Proposition 3. *(i) The Markov chain defined as above is positive recurrent if and only if
**(ii) The Markov chain defined as above is positive recurrent if and only if
*

In order to obtain recurrence and transience criterions for the Markov chains and we shall need the following proposition (Karlin and Taylor [6]).

Proposition 4. * Let us consider a Markov chain on which is irreducible. Then if there exists a strictly increasing function that is harmonic on the complement of a finite interval and that is bounded, then the chain is transient. In the case that there exists such a function which is unbounded then the chain is recurrent.*

Following this direction, we shall use a well-known method-theorem based on the Foster-Kendall theorem (Karlin and Taylor [6]) by considering the harmonic function ). For the Markov chain this is a solution of

Since , for every , we obtain that

If we put , we get (with ), , which is the equation of definition of the sequences and (as a multiplicative factor of the ) for the Markov chain such that , , for every . This means that the strictly increasing harmonic functions of the Markov chain are in correspondence with the weight representations of the Markov chain such that

To express this kind of duality we will call the Markov chain , the *adjoint* of the Markov chain and reciprocally in the case that the relation (28) is satisfied.

Equivalently for the Markov chain , the harmonic function , ) satisfies the equation

Since , for every , we have

If we put , we get (with ), , which is the equation of the definition of the sequences and (as a multiplicative factor of the ) for the Markov chain such that , , for every . By considering a similar approximation of that given before for the Markov chain , we may say that the strictly increasing harmonic functions of the Markov chain are in correspondence with the weight representations of the Markov chain such that equivalent equations of (28) are satisfied.

So we may have the following.

Proposition 5. * The Markov chain defined as above is transient if and only if the adjoint Markov chain is positive recurrent and reciprocally. Moreover the adjoint Markov chains and are null recurrent simultaneously. **In particular *(i)*the Markov chain ** defined as above is transient if and only if ** and *;(ii)*the Markov chain ** defined as above is transient if and only if ** and *;(iii)*the adjoint Markov chains ** and ** are null recurrent if*

*Proof. * The proof of Proposition 5 is an application mainly of Proposition 4 as well as of Proposition 3.

#### Acknowledgment

C. Ganatsiou is indebted to the referee for the valuable comments, which led to a significant change of the first version.

#### References

- S. Kalpazidou,
*Cycle Representations of Markov Processes*, Springer, New York, NY, USA, 1995. - J. MacQueen, “Circuit processes,”
*Annals of Probability*, vol. 9, pp. 604–610, 1981. View at Publisher · View at Google Scholar - Q. Minping and Q. Min, “Circulation for recurrent markov chains,”
*Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete*, vol. 59, no. 2, pp. 203–210, 1982. View at Publisher · View at Google Scholar · View at Scopus - A. H. Zemanian,
*Infinite Electrical Networks*, Cambridge University Press, Cambridge, UK, 1991. - Yv. Derriennic, “Random walks with jumps in random environments (examples of cycle and weight representations),” in
*Probability Theory and Mathematical Statistics: Proceedings of The 7th Vilnius Conference 1998*, B. Grigelionis, J. Kubilius, V. Paulauskas, V. A. Statulevicius, and H. Pragarauskas, Eds., pp. 199–212, VSP, Vilnius, Lithuania, 1999. View at Google Scholar - S. Karlin and H. Taylor,
*A First Course in Stochastic Processes*, Academic Press, New York, NY, USA, 1975.