Journal of Applied Mathematics and Stochastic Analysis

Volume 2008 (2008), Article ID 735436, 18 pages

http://dx.doi.org/10.1155/2008/735436

## Central Limit Theorem of the Smoothed Empirical Distribution Functions for Asymptotically Stationary Absolutely Regular Stochastic Processes

^{1}Laboratoire de Statistique et Probabilités, CNRS, (UMR 5219), Université Paul Sabatier, Toulouse Cedex 931062, France^{2}IUFM du Limousin, 209 Boulevard de Vanteaux, Limoges Cedex 87036, France

Received 9 March 2007; Revised 27 September 2007; Accepted 24 October 2007

Academic Editor: Andrew Rosalsky

Copyright © 2008 Echarif Elharfaoui and Michel Harel. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Let be an estimator obtained by integrating a kernel type density estimator based on a random sample of size . A central limit theorem is established for the target statistic , where the underlying random vector forms an asymptotically stationary absolutely regular stochastic process, and is an estimator of a multivariate parameter by using a vector of U-statistics. The results obtained extend or generalize previous results from the stationary univariate case to the asymptotically stationary multivariate case. An example of asymptotically stationary absolutely regular multivariate ARMA process and an example of a useful estimation of are given in the applications.

#### 1. Introduction

The purpose of this paper is to estimate the value of a multivariate distribution function, called the target distribution function, at a given point, when observing a nonstationary process. Clearly, there must be a connection between the process and the target distribution. We will assume that as time goes, the marginal distribution of the process gets closer and closer to the target in a suitable sense. The point at which we want to estimate the target distribution is not any bona fide vector, for we will assume that it can be estimated by a vector of -statistics. Such a problem is clearly out of reach with that generality, and we will assume that, though nonstationary, the process exhibits an asymptotic form of stationarity and has a suitable mixing property. Those will be defined formally after this general introduction.

Let be a stochastic process indexed by the positive integers, taking value in a finite dimensional Euclidean space . Identifying with a product of a finite number copies or the real line, we write for the distribution function of . We will assume that the process has some form of asymptotic stationarity, implying that the sequence converges in a sense to be made precise to a limiting distribution function .

For , let denote the -algebra of events generated by .

We will say that the nonstationary stochastic process is absolutely regular if where

Assume that for some positive less than ,

We consider a parameter in whose components can be naturally estimated by -statistics. To be more formal and precise, we assume that is defined as follows. Let be an integer to be the degree of the -statistics. Let be a function from into , invariant by permutation of its arguments. We are interested in parameters of the form and the function is called the kernel of the parameter

*Example 1.1. *Take to be . The mean vector corresponds to taking , and is the
identity.

*Example 1.2. *Take to be . Consider to be the 2-dimensional vector whose components are the marginal variances. We take and is going to be a function defined on . It has two arguments, each being in , and it is defined by

Such a parameter can be estimated naturally by -statistics, essentially replacing in (1.3) by an empirical counterpart. By using the invariance of the estimator of is then of the form

Now, we have described the parameter and its estimator, going back to our problem, we need to define an estimator of the distribution function .

A natural one would be the empirical distribution function calculated on the observed values of the process. Even if the empirical distribution function is optimal with respect to the speed of convergence of the mean square error, it is not appropriate for not taking care of the fact that is smooth, and particularly of the existence of a density .

It is, therefore, natural to seek an estimator of the target distribution which is smooth. A good candidate is to smooth the empirical distribution function with a kernel. Another way to introduce this estimator is to say that we integrate a standard kernel estimator of the density.

Such an estimator estimates the mean distribution function But since the sequence has a limit , it estimates the limit as well. To be explicit, we consider a sequence , of distribution function converging, in the usual sense of convergence in distribution, to that of the point mass at the origin. We write for the empirical distribution function pertaining to the measure having mass at each sample point . Our nonparametric estimator of is where denotes the convolution operator. Finally, our estimator of is

Our method is the adaption of some of the ideas of Puri and Ralescu [1] who proved a central limit theorem of for the i.i.d. case which was generalized by Sun [2] for the stationary absolutely regular case, then Sun [3] proved the asymptotic normality of and the perturbed sample quantiles for the nonstationary strong mixing condition. We also have to mention Harel and Puri [4, 5] who proved central limit theorems of -statistics for nonstationary (not necessarily bounded) strong mixing double array of random variables, Ducharme and El Mouvid [6] who proved limit theorems for the conditional cumulative distribution function by using the convergence of the rapport of two -statistics, and Oodaira and Yoshihara [7] who obtained the law of the iterated logarithm for the sum of random variables satisfying the absolute regularity, then Harel and Puri [8] proved the law of the iterated logarithm for perturbed empirical distribution function when the random variables are nonstationary absolutely regular; later, this result was generalized for the strong mixing condition by Sun and Chiang [9]. In addition, some of the ideas of Billingsley [10] and Yoshihara [11] have been used to study our problem. For the study of some limit theorems dealing with -statistic for processes which are uniformly mixing in both directions of time, the reader is also referred to Denker and Keller [12].

#### 2. Preliminaries

To specify our assumption on the process, it is convenient to introduce copies of . Hence we write , an infinite sequence of copies of . The basic idea is to think of the process at time as taking value in and we think of each as the th component of We then agree on the following definition.

*Definition 2.1. *A canonical -subspace of is any subspace of the form with . We write for a generic canonical -subspace.

*Remark 2.2. *For if we note and we have with and

The origin of this terminology is that when is the real
line, then a canonical -subspace is a
subspace spanned by exactly distinct vectors of the canonical basis of . We write for a sum over all canonical -subspaces
included in .

To such a canonical subspace , we can associate the distribution function of as well as the
distribution function with the same marginals Clearly, the marginals of are independent, while these of are not.

Consider two nested canonical subspaces and , where For a function symmetric in its argument and defined on , we can define its projection onto the functions defined on by
Identifying with and with allows to project functions defined on onto functions on However, with this identification, the projection depends on the particular choice of in To remove the dependence in we sum over all choices of in by
Given -statistics of degree , we can then define an analogue of Hoeffding decomposition (e.g., Hoeffding
[13]) when the random variables come from a nonstationary process. For this purpose, consider,
firstly, an expectation of if the process had no dependence, namely,
Then for any , we define where is the Dirac function.

Finally, for , we set The analogue of Hoeffding decomposition is the equality
When we have a vector of -statistics defined by a function as in (1.3), we
can write the decomposition componentwise. This is a little cumbersome to write
explicitly. Identifying with says, we write and each -statistics
has a Hoeffding type decomposition where
and
is defined by (2.3) for the component of .

We can construct the vector Now, writing for the largest
of the 's, we can write a vector version of the Hoeffding decomposition Note that this decomposition makes an explicit use of convention (2.7), and this
is why this convention was introduced.

We now need to specify exactly what we mean by asymptotic stationary of a process. For this, recall the following notion of distance
between probability measures.

*Definition 2.3. *The distance in total variation
between two probability measures and defined on the same -algebra
is If is a canonical subspace of , we write as the -algebra generated by the 's with . We write as the probability measure pertaining to the process , which is a probability measure on .

*Definition 2.4. *The process
with probability measure on is geometrically asymptotically (pairwise) stationary if there exists a strictly stationary process with distribution on , and a positive less than 1, such that for , Since is strictly
stationary in this definition, its restriction to depends in fact
only on . Hence, this definition asserts that the process is very close
of being stationary when is large. It
also implies that
This asserts that the marginal distribution of the process converges
geometrically fast to a fixed distribution.

We suppose that there exists a strictly stationary
process with
probability measure on , which is absolutely regular with the same rate as
the process . is the
distribution function of .

We define the function on by
Next, we denote Identifying with says, the
vector of -statistics being defined by a vector function , we can write We can construct the vector
Let
where for , otherwise, and is the
differential operator.

We have We also define

#### 3. Weak Convergence of the Smoothed Empirical Distribution Function

In this section, we identify with and we have, of course, a vector of -statistics defined obviously by a vector function , where the degree of is .

Let be a
probability density function on and let be a sequence
of positive *window-width*, tending to zero as Denote , and consider the perturbed empirical distribution
function defined by (1.7)
corresponding to the sequence

Consider the smoothed empirical distribution defined in (1.7) and using the kernel density estimator , where , and define Note that this is of the form (1.7), with where

For a better understanding of the use of the integral type estimators , it is of interest to study the asymptotic behavior of the distribution of (defined by (3.1)) evaluated at a random point (defined by (1.5)). Such a statistic is useful in estimating a functional if is unknown.

Supposing that the conditions introduced in Section 2 are satisfied, our main result establishes that is an estimator which converges to , and the asymptotic normality will allow us to obtain confidence intervals for . Finally, by using the notations introduced in Section 2, we can write the following result.

Theorem 3.1. * We suppose that *

(i)*there exists a finite positive constant such that
where is the number
introduced in (1.2);*(ii)*the mixing rate of absolute regularity verifies
Condition (1.2);*(iii)*condition (2.14) is verified;*(iv)*
where is a
probability density function;
*(v)*the sequence and are twice
differentiable on with uniformly
bounded first and second partial derivatives.**
Then converges in
law to a normal distribution as , where is defined in (2.22).*

We are then faced with a difficulty as the variance defined in (2.22) is unknown. In order to overcome this difficulty, we can proceed to an estimation of the variance by truncating the expansion of , keeping only the first more informative terms and estimating by its empirical counterpart defined by where From condition (1.2), we have where is some finite positive constant.

To obtain a suitable value for , a simple criterion consists of computing the smallest integer for which where would be the needed level of precision.

From the empirical construction of the estimator , we deduce easily the convergence in distribution of to

#### 4. Applications

##### 4.1. Application to an ARMA Process

First, we give an example for which the the stochastic process satisfies our general condition. It means that is a multivariate asymptotical stationary absolute regular stochastic process.

*Example 4.1. * ARMA
process.

Consider a -variate
ARMA(1,1) process defined by
where the initial vector has a measure
which is not necessarily the invariant measure and admits a strictly positive
density, and are square
matrices and is a -variate white
noise with strictly positive density and geometrical absolute regularity. If
the eigenvalues of the square matrix have modulus
strictly less than 1, then the process satisfying
Condition (2.14) (for a proof, see (5.4) in [8]), the process is asymptotically stationary and
geometrically absolutely regular.

Consider the strictly stationary process satisfying
(4.1), associated to the process where has a measure
which is the invariant measure. Some parameters of the model (4.1) can be
estimated by estimators of the form (1.5) and we can apply Theorem 3.1.

For example, take and denote by the mean and by the covariance
function of the process

Consider to be the
2-dimensional vector which is the column matrix , and suppose that we want to estimate , then one possibility should be to use the estimator , and its associated kernel is, of course,
the identity.

For estimating the two parameters and , where is the first
column and is the second
column of , we could also use the estimator , where the associated kernel of is defined by
and the associated kernel of is defined by

##### 4.2. Application to Estimation of the Median

We give a very simple example for which it is useful to estimate For simplicity, we suppose

Let be a random sample for which the sequence of distribution functions of converges to the limiting distribution function with median A well-known estimator of is the Hodges-Lehmann estimator defined by The estimator is a weighed -statistic with kernel

The theorems of convergence for -statistics remain true for weighted -statistics. We can easily conclude that convergence in law to , and also converges in law to We can confront these two results to evaluate the validity of the estimation of the parameter by

#### 5. Proof of Theorem 3.1

We are going to use the following lemma proved by Harel and Puri [4, Lemma 2.2], which is a generalization of a lemma of Yoshihara [11, Lemma 2].

Lemma 5.1. * Suppose that (3.2), (3.3), and Condition (ii) of Theorem 3.1 are satisfied, then ** where is .*

Writing as

will allow us to determine the contributions of the stochastic behavior of and that of to the limiting distribution.

First, we use smoothness of our nonparametric estimator to linearize the term , approximating it by the differential minus a centralization term . The second term , plus the centralization term defined above, is analyzed using an empirical process technique for dependent random variables. Finally, the last term satisfies , using the exponential asymptotic stationarity.

Setting we can rewrite the first and second terms as

Lemma 5.2. * Under the conditions
of Theorem 3.1,
converges in
law to the normal distribution as , where is defined in (2.22).
*

*Proof. *From the decomposition we can write
where and is defined by
(2.3) for the component of

From the exponential asymptotic stationarity, converges to zero, and from Lemma 5.1 and Markov inequality, we deduce that converges to zero in probability.

It remains to show that converges to a
normal distribution, noting that is a
nonstationary absolutely regular unbounded sequence of random variables which
verifies the mixing rate (1.2). To prove the asymptotic normality of , we use the following lemma, obtained by Harel and
Puri [4, Lemma
2.3].

Lemma 5.3. * Let
be a
nonstationary absolutely regular unbounded sequence of random variables, which
verifies the mixing rate (1.2). Suppose that for any positive , there exists a sequence
of random
variables satisfying (1.2) such that ** where is a positive
constant; ** where is a positive
constant; ** where is a positive
constant; ** Then converges in
law to a normal distribution with mean zero and variance .*

First, we prove (5.10) and (5.11) for the sequence .

Put where From condition (v),?we have where and (5.10) is proved.

Now, by using the inequality let such that , we obtain

From (3.2), we have where is constant positive, which implies and (5.11) is proved.

We now show (5.12).

We first denote that where

It results that where Thus to prove (5.12), we have to show that From (2.14) and the convergence of to the function , as , and by using [4, Lemma 3.1] of Harel and Puri, as .

From the well-known inequality on moments of absolute regular processes, we have, for , where From (3.3), we have where is a finite positive constant which implies so that then We have We deduce that Consequently, (5.12) is verified. Analogously, we show (5.13).

We put where and is defined by (2.16) for the component of .

By using the Lebesgue dominated convergence theorem, we obtain which implies that which proves (5.14). Thus, assumptions (5.10) and (5.14) are satisfied, and converges in law to the normal distribution . Consequently, converges in law to the normal distribution . Therefore, Lemma 5.2 is proved.

Lemma 5.4. * Under the
conditions of Theorem 3.1,
converges to
zero in probability. *

*Proof. *We write
where
with Of course is a
multidimensional empirical process.

From [11, Theorem 3]by Yoshihara, we have where
Then, we deduce that where is a positive
constant.

To prove that it suffices to show that
Since is
differentiable, there exists in such that The differential being bounded,
there exists a positive constant such that which implies that
We have
where is the copula
of and denotes the
indicator function.

We have
with where is the module
of continuity for any bounded function defined by We generalize [2, Relation (3.11)] by Sun from the univariate case to the multivariate case by
using similar methods as in [14, Lemmas (6.3) and (6.5)] by Harel and Puri. Therefore, we get It results from the inequalities (5.51) and (5.54) that converges to
zero in probability as .

For the second term of and using the
Lagrange form of Taylor theorem, applied to on the points and until the
second order, there exists such that with

In the same way, there exists such that with ,
From Harel and Puri [5], we deduce that converges to a
multinormal distribution as .

From Condition (iv), we have
which implies
Consequently, we deduce that converges to zero in probability, and Lemma 5.4 is proved.

Therefore, the proof of Theorem 3.1 follows from Lemmas
5.2 and 5.4.

#### References

- M. L. Puri and S. S. Ralescu, “Central limit theorem for perturbed empirical distribution functions evaluated at a random point,”
*Journal of Multivariate Analysis*, vol. 19, no. 2, pp. 273–279, 1986. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. Sun, “Asymptotic behavior of the perturbed empirical distribution functions evaluated at a random point for absolutely regular sequences,”
*Journal of Multivariate Analysis*, vol. 47, no. 2, pp. 230–249, 1993. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. Sun, “Perturbed empirical distribution functions and quantiles under dependence,”
*Journal of Theoretical Probability*, vol. 8, no. 4, pp. 763–777, 1995. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - M. Harel and M. L. Puri, “Limiting behavior of $U$-statistics, $V$-statistics, and one sample rank order statistics for nonstationary absolutely regular processes,”
*Journal of Multivariate Analysis*, vol. 30, no. 2, pp. 181–204, 1989. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - M. Harel and M. L. Puri, “Weak invariance of generalized $U$-statistics for nonstationary absolutely regular processes,”
*Stochastic Processes and Their Applications*, vol. 34, no. 2, pp. 341–360, 1990. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - G. R. Ducharme and M. Mint El Mouvid, “Almost sure convergence of the local linear estimator of the conditional cumulative distribution function,”
*Comptes Rendus de l'Académie des Sciences*, vol. 333, no. 9, pp. 873–876, 2001. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - H. Oodaira and K.-I. Yoshihara, “The law of the iterated logarithm for stationary processes satisfying mixing conditions,”
*Kōdai Mathematical Seminar Reports*, vol. 23, pp. 311–334, 1971. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - M. Harel and M. L. Puri, “Nonparametric density estimators based on nonstationary absolutely regular random sequences,”
*Journal of Applied Mathematics and Stochastic Analysis*, vol. 9, no. 3, pp. 233–254, 1996. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - S. Sun and C.-Y. Chiang, “Limiting behaviour of the perturbed empirical distribution functions evaluated at $U$-statistics for strongly mixing sequences of random variables,”
*Journal of Applied Mathematics and Stochastic Analysis*, vol. 10, no. 1, pp. 3–20, 1997. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - P. Billingsley,
*Convergence of Probability Measures*, John Wiley & Sons, New York, NY, USA, 1968. View at Zentralblatt MATH · View at MathSciNet - K.-I. Yoshihara, “Limiting behavior of $U$-statistics for stationary, absolutely regular processes,”
*Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete*, vol. 35, no. 3, pp. 237–252, 1976. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - M. Denker and G. Keller, “On $U$-statistics and V. Mises' statistics for weakly dependent processes,”
*Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete*, vol. 64, no. 4, pp. 505–522, 1983. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - W. Hoeffding, “A class of statistics with asymptotically normal distribution,”
*Annals of Mathematical Statistics*, vol. 19, pp. 293–325, 1948. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet - M. Harel and M. L. Puri, “Conditional empirical processes defined by nonstationary absolutely regular sequences,”
*Journal of Multivariate Analysis*, vol. 70, no. 2, pp. 250–285, 1999. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet