Research Article | Open Access
A Note on the Large Deviation Principle for Discrete Associated Random Variables
We present sufficient conditions under which the sequence of arithmetic means , where , is the partial sum built on a stationary sequence of associated integer-valued and uniformly bounded random variables, which satisfy the large deviation principle.
1. Introduction and the Notation
Let be a strictly stationary sequence of square-integrable random variables (r.v.’s). Denote , , and . Consider a sequence satisfying the strong law of large numbers (SLLN) and the central limit theorem (CLT), that is, . SLLN asserts that the arithmetic mean converges to almost surely when , whereas the CLT specifies the probability that differs from by the quantity of order . Such deviation is called normal. However, in the case of and being distant by the quantity of order , we deal with so-called large deviation events.
It turns out that under certain condition on the tail distribution function of , for , the probability tends to zero with exponential rate; that is, where is called the rate function.
In the i.i.d. case, the well-known Cramer theorem gives the explicit formula for the rate function (see, for example, ). Let us recall that if the moment generating function is finite for all , then for all , where which means that is the Fenchel-Legendre transform of the function .
In general setting, we follow  to give the definition of the rate function as the function defined on a Polish space , taking values in closed half line and satisfying three conditions:(i) is not identically infinite;(ii) is lower semicontinuous;(iii) is compact for all .
Let us further recall that it is said that the sequence of probability measures satisfies the large deviation principle (LDP) with rate and the rate function , if(D1) is the rate function in the sense of the invoked definition;(D2), for all closed ;(D3), for all open .It is also well known that the rate function corresponding to the sequence is uniquely determined.
The role of open and closed sets in the above definition is similar to the one they play in Portmanteau theorem stating equivalent conditions for the weak convergence of probability measures. In fact, LDP may be thought of as the analogue of the weak convergence but in the exponential scale. Bryc  proved that if the sequence of probability measures is exponentially tight (see Appendix) and for all continuous and bounded functions defined on the Polish space (denoted by ) the limit exists, then satisfies LDP with rate and the rate function
This result explains the relation between the LDP and the concept of weak convergence of probability measures. Moreover, it appears very useful in proving the LDP, since it suffices to show existence of for sufficiently large subfamily of continuous and bounded functions.
There is only a few results on LDP for dependent r.v.’s other than Markov processes. Bryc  and Bryc and Dembo  studied LDP for strongly mixing sequences of r.v.’s; Henriques and Oliveira [5, 6] dealt with stationary sequences of associated absolutely continuous r.v.’s. They assumed that the probability density function of the random variable satisfies the following condition:
Our goal is to extend the results of Henriques and Oliveira [5, 6] by proving the LDP for a stationary sequence of integer-valued r.v.’s satisfying the following additional conditions:(A1), , are associated;(A2), , are uniformly bounded; that is, there exists such that ;(A3) for some .
The paper is organized as follows. Section 2 presents the LDP in question, its proof, and an example demonstrating the applicability of the result. For convenience while reading, we place the technical lemmas used in the proof in Section 3. At the end, in the Appendix, we recall the Gärtner-Ellis theorem and essential results of Varadhan  and Bryc  as well as two lemmas on convergence of real sequences. In the proofs we will follow the ideas of Henriques and Oliveira [5, 6].
2. Main Result
Theorem 1. Let be a strictly stationary sequence of integer-valued r.v.’s satisfying conditions (A1), (A2), and (A3). Then the sequence of probability measures satisfies the large deviation principle with rate and the rate function being Fenchel-Legendre transform of the function which means that
Proof. The main tool in proving LDP for dependent r.v.’s is the Gärtner-Ellis theorem (see Appendix) which states that it is enough to verify the existence of limit (6) together with the differentiability of to have LDP.
To be more precise, in order to obtain the upper bound (A.1), we need to prove that limit (6) exists, which is shown in Lemma 3 (see Section 3). Actually, we prove even more—the finiteness of (6) for all .
With a view to getting the lower bound (A.2), we first need to verify the existence of a more general limit for any real continuous, concave, and bounded from above function (the class of such functions will be denoted by ). It is presented in Lemma 4, Section 3. In fact, these are continuous and bounded functions which is required in limit (A.5). Nevertheless, in order to claim the sole existence of this limit, it suffices to consider the subfamily , since it is well separated (see Definition B.7 and Theorem B.8 in ).
Further, from (A2), it is easy to see that the distributions of are exponentially tight (see Appendix).
As a result, by Lemma A.4, we can claim that the LDP holds with rate and the rate function However, to make sure that is the Fenchel-Legendre transform of defined by (6), we are still in need of showing convexity of (see Lemma A.2). To this end, somewhat unobvious implication is inevitable. If are such that, for all , for , then It is shown in Lemma 6.
We are already in a position to prove the convexity of , proceeding exactly like in the proof of Theorem 3.20 in . According to Theorem B.2 in , the rate function may be presented in the following form: Since for there exists such that is immersed in , it is enough to write Let us now take such that . As the assumption of Lemma 6 is satisfied, we have Hence, which means that is midconvex (called by some authors Jensen-convex or J-convex). The function is measurable; thus according to Sierpiński Theorem (see , Theorem 9.4.2. and Theorem 5.3.5), it is convex and the proof is completed.
Finally, let us present an example of a sequence satisfying the assumptions of our theorem; for this sequence obviously the results of  do not apply.
Example 2. Let be a Gaussian sequence with the squared exponential covariance function; that is, This sequence is stationary and associated (positively correlated Gaussian). Define a sequence as follows: where is an arbitrary number. The r.v.’s inherit the properties of association and stationarity from the sequence (we applied the same nonincreasing indicator function to the r.v.’s which are associated). Furthermore, for , from the covariance inequality for Gaussian r.v.’s (see ) we have Therefore and the binary sequence is stationary and fulfills assumptions (A1), (A2), and (A3).
3. Auxiliary Results
Proof. The proof is nearly rewritten from Theorem 3.16 in . For , , by assumption (A1), we have . Since the sequence of r.v.’s , , is stationary, we obtain . Denoting , by Lemma A.5 we get the existence of (6). It remains to verify its finiteness for all . By assumption (A2), we have Thus, for arbitrarily chosen , is finite.
Lemma 4. Let be the sequence as in Theorem 1. Then, for every real continuous, concave, and bounded from above function , the limit exists.
Proof. Except for the steps where discreteness of r.v.’s in question is involved, the proof goes along exactly like in Theorem 3.18 in .
Without loss of generality, we may and do assume that for and for some . Since is continuous and concave, it is a Lipschitz function with constant , for example. By assumption (A2), we get that for Thus, since is Lipschitz, By assumed concavity of , this implies that which is equivalent to When we denote , the above inequality takes the following form: We will now aim at finding the bound for the second term at the r.h.s. of inequality (24).
Let us now define, for and , the sequence of continuous functions in the following way: Each is thus a polyline with vertices having first coordinate of the form , , and second coordinate in the interval . Now, since is nonpositive and Lipschitz with constant , we have so is absolutely continuous (since it is Lipschitz continuous) and almost everywhere differentiable with .
We will now make use of the Newman identity (see ) which allows us to express the covariance of two absolutely continuous functions of arbitrary r.v.’s via the covariance of the indicators of these r.v.’s. Let us recall that if and are absolutely continuous functions and , are random variables, such that and are square-integrable, then In light of the above identity, we can write where the last inequality is a consequence of nonpositivity of and application of the well-known Hoeffding identity stating that By the assumption of stationarity of , we can bound the above expression in the following way: Hence, we get which, on the basis of uniform boundedness of , , yields Let us now define the following quantity: , and restate (32): Going back to inequality (24), we obtain Choose , where is the constant taken from assumption (A3). By Lemma A.6, Therefore it is easy to show that putting , for sufficiently large , we have As a result, from inequality (34), we arrive at and by Lemma A.5 we conclude that the limit exists and the proof is completed.
Lemma 5. Let and be integer-valued, associated, square-integrable r.v.’s; then for any and .
Proof. The proof follows from the covariance inequality obtained in  for integer-valued associated r.v.’s , .
Lemma 6. Let be the sequence as in Theorem 1. Let also be such that, for all , Then,
Proof. Again, apart from the calculations we conduct with remark that the r.v.’s are integer-valued, the proof goes like in Theorem 3.19 in .
From assumption (40) we know that there exist , , such that, for sufficiently large , ; thus, for some , Now, for , by Lemma 5 and by stationarity of r.v’s, we can write
Hence and in light of inequality (42), we get Next, on the basis of (A2) we see that which yields that By triangle inequality, this implies Putting in the above inequality and plugging it in inequality (44), we arrive at If we finally put , where , then by Lemma A.6 we get the thesis.
We recall the Gärtner-Ellis theorem invoked throughout the proof of the main result.
Theorem A.1 (Gärtner-Ellis [14, 15]). Assume that, for every , defined by (6) exists. Then, its Fenchel-Legendre transform satisfies the following:(a)for every closed (b)if is differentiable, then for every open
The next two theorems we lean against in the proof are Varadhan’s Lemma (necessary condition for LDP) and Bryc’s result (sufficient conditions for LDP).
Lemma A.2 (Varadhan ). Assume that the large deviation principle is satisfied with a rate function . Moreover, assume that for every Then(i)for every the limit above exists and is the Fenchel-Legendre transform of ; that is, ;(ii)if the rate function is convex, then it is the Fenchel-Legendre transform of ; that is, .
In order to formulate the next lemma let us recall the notion of exponential tightness of a sequence of probability measures.
Definition A.3. It is said that the sequence of probability measures is exponentially tight, if for every there exists a compact set such that
Lemma A.4 (Bryc ). Assume that , the distributions of , are exponentially tight and that, for every continuous and bounded function , the following limit exists: Then the probability measures , , satisfy the large deviation principle with rate and the rate function , where the is taken over the family of bounded and continuous functions. Moreover, for every such ,
Finally, we invoke two lemmas concerning the existence of the limit of specific real sequences. Both of them are taken from .
Lemma A.5. Let and be real sequences such that , and for some , . Then the limit exists.
Lemma A.6. Let be a real sequence of nonnegative numbers such that there exist satisfying Then, for all and ,
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
- F. Hollander, Large Deviations, Fields Institute Monographs, 2000.
- W. Bryc, “Large deviations by the asymptotic value method,” in Diffusion Processes and Related Problems in Analysis, vol. 1 of Progress in Probability no. 22, pp. 447–472, Birkhäuser, Boston, Mass, USA, 1990.
- W. Bryc, “On large deviations for uniformly strong mixing sequences,” Stochastic Processes and their Applications, vol. 41, no. 2, pp. 191–202, 1992.
- W. Bryc and A. Dembo, “Large deviations and strong mixing,” Annales de l'Institut Henri Poincaré Probabilité et Statistiques, vol. 32, no. 4, pp. 549–569, 1996.
- C. Henriques and P. E. Oliveira, “Large deviations for the empirical mean of associated random variables,” Statistics & Probability Letters, vol. 78, no. 6, pp. 594–598, 2008.
- P. E. Oliveira, Asymptotics for Associated Random Variables, Springer, New York, NY, USA, 2012.
- A. Bulinski and A. Shashkin, Limit Theorems for Associated Random Fields and Related Systems, vol. 10 of Advanced Series on Statistical Science and Applied Probability, World Scientific, Singapore, 2007.
- B. L. S. P. Rao, Associated Sequences, Demimartingales and Nonparametric Inference, Birkhäuser, 2012.
- S. R. S. Varadhan, “Asymptotic probabilities and differential equations,” Communications on Pure and Applied Mathematics, vol. 19, no. 3, pp. 261–286, 1966.
- M. Kuczma, An Introduction to the Theory of Functional Equations and Inequalities, Birkhäuser, Basel, Switzerland, 2nd edition, 2009.
- P. Matula, “A note some inequalities for certain classes of positively dependent random variables,” Probability and Mathematical Statistics, vol. 24, no. 1, pp. 17–26, 2004.
- C. M. Newman, “Asymptotic independence and limit theorems for positively and negatively dependent random variables,” in Inequalities in Statistics and Probability, Y. L. Tong, Ed., Institute of Mathematical Statistics, Hayward, Calif, USA, 1984.
- P. Matula, “On some inequalities for positively and negatively dependent random variables with applications,” Publicationes Mathematicae Debrecen, vol. 63, no. 4, pp. 511–521, 2003.
- J. Gärtner, “On large deviations from the invariant measure,” Theory of Probability & Its Applications, vol. 22, no. 1, pp. 24–39, 1977.
- R. S. Ellis, “Large deviations for a general class of random vectors,” The Annals of Probability, vol. 12, no. 1, pp. 1–12, 1984.
Copyright © 2015 Przemysław Matuła and Maciej Ziemba. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.