An Approach of Randomness of a Sample Based on Its Weak Ergodic Limit

Abstract

For a Polish Sample Space with a Borel -field with a surjective measurable transformation, we define an equivalence relation on sample points according to their ergodic limiting averages. We show that this equivalence relation partitions the subset of sample points on measurable invariant subsets, where each limiting distribution is the unique ergodic probability measure defined on each set. The results obtained suggest some natural objects for the model of a probabilistic time-invariant phenomenon are uniquely ergodic probability spaces. As a consequence of the results gained in this paper, we propose a notion of randomness that is weaker than recent approaches to Schnorr randomness.

1. Introduction

To understand the meaning of randomness several approaches have been undertaken. Some of them include Kolmogorov [1], Martin-Löf [2], Zvonkin and Levin [3], and Schnorr [4] among many others.

On the initial attempts, researchers were trying to identify when a sequence of numbers was random. The sequence of digits of or sequences of computer-generated numbers produced by an algorithm that seems to exhibit a “random behavior” motivated the original problem. In any case of interest, the sequences are elements of or and researchers would like to give a criterion that decides for a sequence to be “random.” In general, sample spaces that allow a Polish Space structure and a surjective transformation (the shifting transformation) embrace the interesting cases.

In this paper, we show that a natural approach for a sample space is to consider all samples that describe the same physical time-invariant phenomenon. We would demonstrate that this method allows for a richer probability structure. The latter introduces a different view for randomness, in a way that two sample objects are equivalent if they describe the same probability phenomenon (in a way that we determine precisely later), and, as a consequence, we give an alternative answer to the problem of trying to define randomness.

The Ergodic Theorem and the Ergodic Decomposition Theorem motivate our definition. Indeed, our approach goes hand in hand with Gray [5] and extends some results of Shields [6]. Some recent works which study the relation between Ergodic Theory and randomness are V’yugin [7], Gács et al. [8], and Galatolo et al. [9].

The Birkhoff-Kinchin Ergodic Theorem (See Billingsley [10]) states that if is a measure preserving transformation defined on a probability space and is an integrable random vector, then where is the -field of invariant sets (under ). For another interpretation of this fact, we define for each It follows that defines a (finite) kernel, where is an ergodic measure. Moreover for any measurable random vector The Birkhoff Ergodic Theorem implies that whenever is an ergodic measure preserving transformation and is an integrable random vector there exists a set with such that for

Using the terminology of Gács et al. [8], if satisfies (4), is said to be typical with respect to the observable . Moreover, if the last equation holds for any bounded continuous function , then is said to be typical. The latter is equivalent to the weak convergence of . For a review on weak convergence see Billingsley [11]. Moreover, if is typical for any effective ergodic surjective transformation, then it is said that is typical. For a definition of effective ergodic transformation see Gács et al. [8].

For any computable measure , defined on , with Borel -field , V’yugin [7] showed that any Martin-Löf random sequence is typical. For a review on Martin-Löf randomness see Martin-Löf [2]. Galatolo et al. [9] generalize the last result to computable probability spaces and -computable observables. For a discussion of the definition and consequences of computable probability, spaces see Gács et al. [8].

Moreover Gács et al. [8] extend the concept of Schnorr randomness to computable probability spaces, and they show that the idea of Schnorr randomness is weaker than the notion of Martin-Löf randomness in this context. For a definition of Schnorr randomness see Schnorr [4]. Gács et al. [8] also showed that on an arbitrary computable probability space space with no atoms is Schnorr random if and only if is typical for every mixing endomorphism .

In this paper, we obtain another approach to randomness. Assume that is a Polish Space, with metric , and assume the Borel -field on . Moreover assume a measure preserving transformation defined on . For most problems related to finding a reasonable definition for randomness it is sufficient to assume that is the sample space of real valued sequences with the metric , the shifting transformation , and the sequence of processes and with a probability that makes independent with the same distribution defined as for all Borel set.

With the help of the Ergodic Decomposition Theorem (see Proposition 1) it is possible to show that there exists a set with where, for any , are probability measures that converge weakly, and the limiting distribution is an ergodic invariant probability measure on .

The latter suggests, in a manner to be precise by Theorem 3, that only samples that describe a physical phenomenon (those whose ergodic limit is a “reasonable” probability) should be considered as samples for any physical “reasonable probability space.” It is also apparent, as shown by Theorem 3, that a stronger theory can be established if other samples are not considered.

We propose to partition the sample space according to their weak ergodic limit: assume a measurable space , as above, and assume a surjective measurable function . We define an equivalent relation on , by iff for any bounded continuous function If we say that has the same stochastic type of . Clearly is an equivalence relation on elements on . Moreover if weakly and then weakly, and if is a null invariant probability (see Definition 2) then it is a invariant probability (see Theorem 3).

We notice that the requirement of weak convergence of as to a probability measure is quite natural. The latter is because for any probability space defined on a Polish Space with a measure preserving transformation for almost all the mentioned weak convergence holds (see Proposition 1). In the case where we say that stochastic type of is .

The main contribution of this paper is Theorem 3. Roughly speaking, Theorem 3 states that any reasonable space is divided into nonoverlapping subspaces, where on each subspace the statistics behavior of all the points is identical, and where these spaces have a simple statistical structure (they are uniquely ergodic). In fact Shields [6] [Section I.4.c] suggested in the case of finite alphabets ( for some ) that, as a consequence of the Ergodic Decomposition Theorem, such decomposition is natural.

As a result of Theorem 3, for any physical “reasonable probability space” only the samples that describe a physical phenomenon (those whose ergodic limit is a “reasonable” probability) are sufficient as realizations of the sample space to study the physical phenomenon (the other realizations are unnecessary).

In the current stay of affairs, for any sample realization of a probability space and any statistical application, only a finite set of values are observed, due to our technical limitations (the amount of time required to gather data is finite). Therefore, the assumption that any   belongs to the sample space (for instance in the case that ) is a technical assumption that it is motivated as long as it provides a rich and consistent theory. However we are showing in Theorem 3 that a richer theory is obtained if we restrict the sample space to those whose weak ergodic limit is a null invariant probability. The gain achieved in the modeling structure arises from the fact that the limiting probabilities describing the physical phenomenon are uniquely ergodic.

Therefore, our conclusions suggest further studies could point out to consider the uniquely ergodic probability spaces as natural spaces for the modeling of any physical time-invariant phenomenon.

Also, as a consequence of Theorem 3, and results obtained by Gács et al. [8], a natural definition for a to be random is that converges weakly to a null invariant ergodic measure (see Definition 2). In fact, it follows that our definition of randomness extends to noncomputable probability spaces, and by Gács et al. [8] on computable probability spaces our notion of randomness is weaker than Schnorr randomness (and therefore weaker than Martin-Löf randomness).

In our approach, we do not consider any possible mixing endomorphism. Contrary to existing methods we fix a surjective transformation . Our approach works on any Polish Space, and it works for any probability measure (instead of computable probability spaces).

Issues that remain open include studying the consequences of modeling probability spaces as uniquely ergodic spaces. On the side of randomness, it remains open to examining definitions of randomness for discrete sequences that model non-ergodic Dependent Processes and a definition of randomness for functions defined on an interval (in the ergodic and non-ergodic cases). Finally, it remains open to provide specific sequences that are random (or whose limiting distribution is a particular distribution) and to provide examples of sequences that are random in the sense we give in this paper and are not random in the Schnorr sense (and therefore not random in the Martin-Löf sense).

2. Randomness

The first proposition says that on “reasonable spaces,” for any reasonable physical realization, always the ergodic averages converge weakly to ergodic invariant measures.

Proposition 1. Assume a measure preserving transformation defined on a probability space , where is a Polish Space, and is the Borel -field. Let be the -field of invariant sets (with respect to ). Define by (2).
There exist a set with where for any , are probability measures that converge weakly to the ergodic invariant probability measure where is the regular conditional probability kernel with respect to of the Ergodic Decomposition Theorem.

Proof. Assume that is a countable dense subset of . We notice that the class of sets that are finite intersections of balls with a rational radius centered at points in is a countable -system with . Let be the countable field generated by . By the Ergodic Decomposition Theorem there exists a set , with such that, for ,where is the ergodic invariant probability measure corresponding to the regular conditional probability kernel with respect to of the Ergodic Decomposition Theorem.
Assume an open set , then there exist , with , where can be assumed disjoint sets with for each . It follows that for where the second equality follows by absolute convergence of , and the third equality follows Fatou’s Lemma. Finally the result follows by the Portmanteau Theorem (see Billingsley [11]).

The latter proposition suggests that, for any reasonable physical realization (), the ergodic average converges weakly to an ergodic invariant probability measure. It turns out that a weaker concept for invariability is sufficient to describe the limiting ergodic behavior of any reasonable “physical realization.”

Definition 2. Given a measurable space and a surjective transformation , we say that a probability measure defined on is a null invariant probability if implies that .

Under conditions of Proposition 1 we obtain a necessary requirement for the limit probabilities of to be null invariant. In Theorem 3 we prove that the latter condition is sufficient demand for the limiting probabilities to converge weakly to an invariant probability measure.

In the next theorem, we prove that for any physical “reasonable probability space” only the realizations (or samples) that describe a physical phenomenon (those whose ergodic limit is a “reasonable.” probability) are sufficient as realizations of the sample space to study the physical phenomenon (the other samples are unnecessary) and that those realizations (or samples) partition the whole space according to the physical phenomenon that they describe. Moreover, we show that when we restrict ourselves to those samples that describe the same physical phenomenon (those whose limiting ergodic behavior is the same), the mathematical structure of the space defined is richer (in the sense that the probability spaces are uniquely invariant). The above conclusions make a strong point to consider the unique invariant probability spaces as natural spaces for the modeling of a physical time-invariant phenomenon.

Theorem 3. Assume a measurable surjective function defined on a measurable space , where is a Polish Space and is the Borel -field, where is a metric for which is a complete separable space. Let and assume that weakly, where is a null invariant probability. Then is a invariant probability measure defined on . Moreover ifthen is a measurable invariant set with where either or and is ergodic iff . Moreover, there exist a countable field where any open can be written as a countable union of elements of , such that, for any , and , (and in particular ).
Moreover, if is an ergodic probability measure there is only one ergodic invariant measure on (namely, ).

Proof. Let be a countable dense subset of . For each , define to be a countable set of balls with center at , with radius when such that , for , where , , and denotes the boundary of given set . The set with the desired properties exists by a standard argument. Let be the field generated by . Clearly any open set can be written as a countable union of elements in . By the Portmanteau Theorem for any .
For any set , it is clear that , and since the class of sets where is a -system that contains since is a null invariant probability, it follows by the Dynkin’s theorem that for any , proving that is -invariant.
Clearly , for any , and therefore using the Dinkyn theorem the limiting measure associated with and is the same, proving that . We also notice that, for , if is an element such that , then , for any , and then by the Dynkin’s theorem proving that . Next we prove that is a measurable set. For the latter, let be an enumeration of . Define byThen by the Portmanteau Theorem , where is defined by , and therefore is measurable.
By the Ergodic Decomposition Theorem, there exist a kernel and set , with with the property that for all    for all , is an ergodic measure, andwhere is the restriction to of and is the -field of invariant sets (under ). It follows that for Since the class of sets where is a system, by the Dynkin’s theorem it follows that for all , . In particular , proving that or .
Assume that , then by the Ergodic Decomposition Theorem, there exist a set where and , , for where is ergodic. It follows that is ergodic.
Finally assume that , and assume an ergodic probability measure on . It is clear that can be extended to , in a natural way as . We also denote the extension as . Moreover the extension to is also ergodic. By the Ergodic Decomposition Theorem, it follows thatIt follows that, on , and by the Dinkyn’s Theorem on , and therefore on .

Probably the most interesting consequence of the previous theorem is the uniform convergence of ergodic averages for all sample points. If is a compact set then, for any continuous bounded function defined on (for the induced topology), for all uniformly on . The latter is the content of the Oxtoby Theorem [12] [Theorem  6.19].

The consequence of Theorem 3 that, for any , is ergodic iff has been observed in the case of finite alphabets by Shields [6] [Section I.9.b]. In fact Shields [6] [Sections I.4.c and I.9.b] suggests a similar approach in the case of finite alphabets, although uniquely ergodicity is not obtained.

Finally we observe some elementary facts of our definition.

Proposition 4. Assume a measurable space and a surjective transformation ; then: (i)If is periodic on in the sense that there exist with , with period (namely, , then ), where is a probability measure supported on a finite set.(ii)If there exist such that , then . In particular it is impossible to tell from a finite set of values of any information about the distribution of .(iii)For any , .

Conflicts of Interest

The author declares that there are no conflicts of interest regarding the publication of this paper.