Research Article | Open Access
A Note on the Adaptive Estimation of a Multiplicative Separable Regression Function
We investigate the estimation of a multiplicative separable regression function from a bidimensional nonparametric regression model with random design. We present a general estimator for this problem and study its mean integrated squared error (MISE) properties. A wavelet version of this estimator is developed. In some situations, we prove that it attains the standard unidimensional rate of convergence under the MISE over Besov balls.
We consider the bidimensional nonparametric regression model with random design described as follows. Let be a stochastic process defined on a probability space , where is a strictly stationary stochastic process, is a strictly stationary stochastic process with support in , and is an unknown bivariate regression function. It is assumed that , exists, are independent, are independent, and, for any , and are independent. In this study, we focus our attention on the case where is a multiplicative separable regression function: there exist two functions and such that We aim to estimate from the random variables: . This problem is plausible in many practical situations as in utility, production, and cost function applications (see, e.g., Linton and Nielsen , Yatchew and Bos , Pinske , Lewbel and Linton , and Jacho-Chávez ).
In this note, we provide a theoretical contribution to the subject by introducing a new general estimation method for . A sharp upper bound for its mean integrated squared error (MISE) is proved. Then we adapt our methodology to propose an efficient and adaptive procedure. It is based on two wavelet thresholding estimators following the construction studied in Chaubey et al. . It has the features to be adaptive for a wide class of unknown functions and enjoy nice MISE properties. Further details on wavelet estimators can be found in, for example, Antoniadis , Vidakovic , and Härdle et al. . Despite the so-called “curse of dimensionality” coming from the bidimensionality of (1), we prove that our wavelet estimator attains the standard unidimensional rate of convergence under the MISE over Besov balls (for both the homogeneous and inhomogeneous zones). It completes asymptotic results proved by Linton and Nielsen  via nonadaptive kernel methods for the structured nonparametric regression model.
The paper is organized as follows. Assumptions on (1) and some notations are introduced in Section 2. Section 3 presents our general MISE result. Section 4 is devoted to our wavelet estimator and its performances in terms of rate of convergence under the MISE over Besov balls. Technical proofs are collected in Section 5.
2. Assumptions and Notations
For any , we set We set provided that they exist.
We formulate the following assumptions.(H1)There exists a known constant such that (H2)There exists a known constant such that (H3)The density of , denoted by , is known and there exists a constant such that (H4)There exists a known constant such that The assumptions (H1) and (H2), involving the boundedness of , are standard in nonparametric regression models. The knowledge of discussed in (H3) is restrictive but plausible in some situations, the most common case being ~ (the uniform distribution on ). Finally, mention that (H4) is just a technical assumption more realistic to the knowledge of and (depending on and , resp.).
3. MISE Result
Theorem 1 presents an estimator for and shows an upper bound for its MISE.
Theorem 1. One considers (1) under (H1)–(H4). One introduces the following estimator for (2):
where denotes an arbitrary estimator for in , denotes an arbitrary estimator for in , denotes the indicator function,
and refers to (H4).
Then there exists a constant such that
The form of (9) is derived to the multiplicative separable structure of (2) and a ratio-type normalization. Other results about such ratio-type estimators in a general statistical context can be found in Vasiliev .
Based on Theorem 1, is efficient for if and only if is efficient for and is efficient for in terms of MISE. Even if several methods are possible, we focus our attention on wavelet methods enjoying adaptivity for a wide class of unknown functions and having optimal properties under the MISE. For details on the interests of wavelet methods in nonparametric statistics, we refer to Antoniadis , Vidakovic , and Härdle et al. .
4. Adaptive Wavelet Estimation
Before introducing our wavelet estimators, let us present some basics on wavelets.
4.1. Wavelet Basis on [0, 1]
Let us briefly recall the construction of wavelet basis on the interval introduced by Cohen et al. . Let be a positive integer, and let and be the initial wavelets of the Daubechies orthogonal wavelets . We set With appropriate treatments at the boundaries, there exists an integer satisfying such that the collection , is an orthonormal basis of .
Any can be expanded on as where and are the wavelet coefficients of defined by
4.2. Besov Balls
For the sake of simplicity, we consider the sequential version of Besov balls defined as follows. Let , , and . A function belongs to if and only if there exists a constant (depending on ) such that the associated wavelet coefficients (14) satisfy In this expression, is a smoothness parameter and and are norm parameters. For a particular choice of , , and , contains the Hölder and Sobolev balls (see, e.g., DeVore and Popov , Meyer , and Härdle et al. ).
4.3. Hard Thresholding Estimators
In the sequel, we consider (1) under (H1)–(H4).
We consider hard thresholding wavelet estimators for and in (9). They are based on a term-by-term selection of estimators of the wavelet coefficients of the unknown function. Those which are greater to a threshold are kept; the others are removed. This selection is the key to the adaptivity and the good performances of the hard thresholding wavelet estimators (see, e.g., Donoho et al. , Delyon and Juditsky , and Härdle et al. ).
To be more specific, we use the “double thresholding” wavelet technique, introduced by Delyon and Juditsky  then recently improved by Chaubey et al. . The role of the second thresholding (appearing in the definition of the wavelet estimator for ) is to relax assumption on the model (see Remark 6).
Estimator for . We define the hard thresholding wavelet estimator by where where is the integer part of , where is the integer satisfying , , , and Estimator for . We define the hard thresholding wavelet estimator by where Where is the integer part of , , Where is the integer satisfying , , , and Estimator for h. From (16) and (20), we consider the following estimator for (2): where and refers to (H4).
Let us mention that is adaptive in the sense that it does not depend on or in its construction.
Remark 2. Since is defined with and is defined with , thanks to the independence of , and are independent.
Remark 3. The calibration of the parameters in and is based on theoretical considerations; thus defined, and can attain a fast rate of convergence under the MISE over Besov balls (see , Theorem 6.1]). Further details are given in the proof of Theorem 4.
4.4. Rate of Convergence
Theorem 4 investigates the rate of convergence attains by under the MISE over Besov balls.
The rate of convergence is the near optimal one in the minimax sense for the unidimensional regression model with random design under the MISE over Besov balls (see, e.g., Tsybakov , and Härdle et al. ). Thus Theorem 4 proves that our estimator escapes to the so-called “curse of dimensionality.” Such a result is not possible with the standard bidimensional hard thresholding wavelet estimator attaining the rate of convergence with under the MISE over bidimensional Besov balls defined with as smoothness parameter (see Delyon and Juditsky ).
Theorem 4 completes asymptotic results proved by Linton and Nielsen  investigating this problem for the structured nonparametric regression model via another estimation method based on nonadaptive kernels.
Remark 5. In Theorem 4, we take into account both the homogeneous zone of Besov balls, that is, and , and the inhomogeneous zone, that is, and , for the case and the same for . This has the advantage to cover a very rich class of unknown regression functions .
Remark 6. Note that Theorem 4 does not require the knowledge of the distribution of ; and the existence of is enough.
Remark 7. Let us mention that the phenomenon of curse of dimensionality has also been studied via wavelet methods by Neumann  but for the multidimensional Gaussian white noise model and with different approaches based on anysotropic frameworks.
Remark 8. Our study can be extended to the multidimensional case considered by Yatchew and Bos , that is, and ; and denoting two positive integers. In this case, adapting our framework to the multidimensional case ( dimensional Besov balls, dimensional (tensorial) wavelet basis, dimensional hard thresholding wavelet estimator, see, e.g, Delyon and Juditsky ), one can prove that (9) attains the rate of convergence , where and .
In this section, for the sake of simplicity, denotes a generic constant; its value may change from one term to another.
Proof of Theorem 1. Observe that Therefore, using the triangular inequality, the Markov inequality, (H1), (H2), (H4), , and again the Markov inequality, we get On the other hand, we have the decomposition Owing to the triangular inequality, (H1) and (H2), we have Putting (28) and (30) together, we obtain Therefore, by the elementary inequality:, , an integration over and taking the expectation, it comes Now observe that, owing to the independence of , the independence between and , and , we obtain Then, using similar arguments to (33), , , (H1), (H2), (H3), and , we have Equations (32) and (34) yield the desired inequality:
Proof of Theorem 4. We aim to apply Theorem 1 by investigating the rate of convergence attained by and under the MISE over Besov balls.
First of all, remark that, for , any integer and any .(i)Using similar arguments to (33), we obtain (ii)Using similar arguments to (34) and , we have with .
Applying [6, Theorem 6.1] (see the Appendix) with , , , , and (so ) with , , either and or and , we prove the existence of a constant such that when is large enough.
The MISE of can be investigated in a similar way: for , any integer and any . (i)We show that (ii)We show that with always .
Applying again [6, Theorem 6.1] (see the Appendix) with , , , , and with , , either and or and ; we prove the existence of a constant such that when is large enough.
Using the independence between and (see Remark 2), it follows from (39) and (43) that Owing to Theorem 1, (39), (43) and (44), we get with .
Theorem 4 is proved.
Appendixwhere and is the integer satisfying Here, we suppose that there exist (i) functions with for any ,(ii)two sequences of real numbers and satisfying and , such that, for , (A1)any integer and any , (A2)there exist two constants, and , such that, for any integer and any , Let be (A.1) under (A1) and (A2). Suppose that with , and or and . Then there exists a constant such that
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
- O. B. Linton and J. P. Nielsen, “A kernel method of estimating structured nonparametric regression based on marginal integration,” Biometrika, vol. 82, no. 1, pp. 93–100, 1995.
- A. Yatchew and L. Bos, “Nonparametric least squares estimation and testing of economic models,” Journal of Quantitative Economics, vol. 13, pp. 81–131, 1997.
- J. Pinske, Feasible Multivariate Nonparametric Regression Estimation Using Weak Separability, University of British Columbia, Vancouver, Canada, 2000.
- A. Lewbel and O. Linton, “Nonparametric matching and efficient estimators of homothetically separable functions,” Econometrica, vol. 75, no. 4, pp. 1209–1227, 2007.
- D. Jacho-Chávez, A. Lewbel, and O. Linton, “Identification and nonparametric estimation of a transformed additively separable model,” Journal of Econometrics, vol. 156, no. 2, pp. 392–407, 2010.
- Y. P. Chaubey, C. Chesneau, and H. Doosti, “Adaptive wavelet estimation of a density from mixtures under multiplicative censoring,” 2014, http://hal.archives-ouvertes.fr/hal-00918069.
- A. Antoniadis, “Wavelets in statistics: a review (with discussion),” Journal of the Italian Statistical Society B, vol. 6, no. 2, pp. 97–144, 1997.
- B. Vidakovic, Statistical Modeling by Wavelets, John Wiley & Sons, New York, NY, USA, 1999.
- W. Härdle, G. Kerkyacharian, D. Picard, and A. Tsybakov, Wavelet, Approximation and Statistical Applications, vol. 129 of Lectures Notes in Statistics, Springer, New York, NY, USA, 1998.
- V. A. Vasiliev, “One investigation method of a ratios type estimators,” in Proceedings of the 16th IFAC Symposium on System Identification, pp. 1–6, Brussels, Belgium, July 2012.
- A. Cohen, I. Daubechies, and P. Vial, “Wavelets on the interval and fast wavelet transforms,” Applied and Computational Harmonic Analysis, vol. 1, no. 1, pp. 54–81, 1993.
- R. DeVore and V. Popov, “Interpolation of Besov spaces,” Transactions of the American Mathematical Society, vol. 305, pp. 397–414, 1988.
- Y. Meyer, Wavelets and Operators, Cambridge University Press, Cambridge, UK, 1992.
- D. L. Donoho, I. M. Johnstone, G. Kerkyacharian, and D. Picard, “Density estimation by wavelet thresholding,” Annals of Statistics, vol. 24, no. 2, pp. 508–539, 1996.
- B. Delyon and A. Juditsky, “On minimax wavelet estimators,” Applied and Computational Harmonic Analysis, vol. 3, no. 3, pp. 215–228, 1996.
- A. B. Tsybakov, Introduction à L'Estimation Non-Paramétrique, Springer, New York, NY, USA, 2004.
- M. H. Neumann, “Multivariate wavelet thresholding in anisotropic function spaces,” Statistica Sinica, vol. 10, no. 2, pp. 399–431, 2000.
Copyright © 2014 Christophe Chesneau. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.