- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Mathematical Problems in Engineering
Volume 2010 (2010), Article ID 956907, 30 pages
QML Estimators in Linear Regression Models with Functional Coefficient Autoregressive Processes
School of Mathematics and Statistics, Hubei Normal University, Huangshi 435002, China
Received 30 December 2009; Revised 19 March 2010; Accepted 6 April 2010
Academic Editor: Massimo Scalia
Copyright © 2010 Hongchang Hu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
This paper studies a linear regression model, whose errors are functional coefficient autoregressive processes. Firstly, the quasi-maximum likelihood (QML) estimators of some unknown parameters are given. Secondly, under general conditions, the asymptotic properties (existence, consistency, and asymptotic distributions) of the QML estimators are investigated. These results extend those of Maller (2003), White (1959), Brockwell and Davis (1987), and so on. Lastly, the validity and feasibility of the method are illuminated by a simulation example and a real example.
Consider the following linear regression model: where 's are scalar response variables, 's are explanatory variables, is a -dimensional unknown parameter, and the 's are functional coefficient autoregressive processes given as where 's are independent and identically distributed random errors with zero mean and finite variance , is a one-dimensional unknown parameter and is a real valued function defined on a compact set which contains the true value as an inner point and is a subset of . The values of and are unknown.
Model (1.1) includes many special cases, such as an ordinary linear regression model when see [1–11]. In the sequel, we always assume that for some , is a linear regression model with constant coefficient autoregressive processes (when , see Maller , Pere , and Fuller ), time-dependent and functional coefficient autoregressive processes (when , see Kwoun and Yajima ), constant coefficient autoregressive processes (when , see White [16, 17], Hamilton , Brockwell and Davis , and Abadir and Lucas ), time-dependent or time-varying autoregressive processes (when , see Carsoule and Franses , Azrak and Mélard , and Dahlhaus ), and so forth.
Regression analysis is one of the most mature and widely applied branches of statistics. Linear regression analysis is one of the most widely used statistical techniques. Its applications occur in almost every field, including engineering, economics, the physical sciences, management, life and biological sciences, and the social sciences. Linear regression model is the most important and popular model in the statistical literature, which attracts many statisticians to estimate the coefficients of the regression model. For the ordinary linear regression model (when the errors are independent and identically distributed random variables), Bai and Guo , Chen , Anderson and Taylor , Drygas , González-Rodríguez et al. , Hampel et al. , He , Cui , Durbin , Hoerl and Kennard , Li and Yang , and Zhang et al.  used various estimation methods (Least squares estimate method, robust estimation, biased estimation, and Bayes estimation) to obtain estimators of the unknown parameters in (1.1) and discussed some large or small sample properties of these estimators.
However, the independence assumption for the errors is not always appropriate in applications, especially for sequentially collected economic and physical data, which often exhibit evident dependence on the errors. Recently, linear regression with serially correlated errors has attracted increasing attention from statisticians. One case of considerable interest is that the errors are autoregressive processes and the asymptotic theory of this estimator was developed by Hannan and Kavalieris . Fox and Taqqu  established its asymptotic normality in the case of long-memory stationary Gaussian observations errors. Giraitis and Surgailis  extended this result to non-Gaussian linear sequences. The asymptotic distribution of the maximum likelihood estimator was studied by Giraitis and Koul in  and Koul in  when the errors are nonlinear instantaneous functions of a Gaussian long-memory sequence. Koul and Surgailis  established the asymptotic normality of the Whittle estimator in linear regression models with non-Gaussian long-memory moving average errors. When the errors are Gaussian, or a function of Gaussian random variables that are strictly stationary and long range dependent, Koul and Mukherjee  investigated the linear model. Shiohama and Taniguchi  estimated the regression parameters in a linear regression model with autoregressive process.
In addition to (constant or functional or random coefficient) autoregressive model, it has gained much attention and has been applied to many fields, such as economics, physics, geography, geology, biology, and agriculture. Fan and Yao , Berk , Hannan and Kavalieris , Goldenshluger and Zeevi , Liebscher , An et al. , Elsebach , Carsoule and Franses , Baran et al. , Distaso , and Harvill and Ray  used various estimation methods (the least squares method, the Yule-Walker method, the method of stochastic approximation, and robust estimation method) to obtain some estimators and discussed their asymptotic properties, or investigated hypotheses testing.
This paper discusses the model (1.1)-(1.2) including stationary and explosive processes. The organization of the paper is as follows. In Section 2 some estimators of and are given by the quasi-maximum likelihood method. Under general conditions, the existence and consistency the quasi-maximum likelihood estimators are investigated, and asymptotic normality as well, in Section 3. Some preliminary lemmas are presented in Section 4. The main proofs are presented in Section 5, with some examples in Section 6.
2. Estimation Method
Write the “true” model as where , and 's are i.i.d errors with zero mean and finite variance . Define , and by (2.2) we have Thus is measurable with respect to the -field generated by , and
Assume at first that the ’s are i.i.d. . Using similar arguments to those of Fuller  or Maller , we get the log-likelihood of conditional on : At this stage we drop the normality assumption, but still maximize (2.5) to obtain QML estimators, denoted by (when they exist): Thus satisfy the following estimation equations: where
Remark 2.1. If then the above equations become the same as Maller's . Therefore, we extend the QML estimators of Maller .
To calculate the values of the QML estimators, we may use the grid search method, steepest ascent method, Newton-Raphson method, and modified Newton-Raphson method. In order to calculate in Section 6, we introduce the most popular modified Newton-Raphson method proposed by Davidon-Fletcher-Powell (see Hamilton ).
Let vector denote an estimator of that has been calculated at the th iteration, and let denote an estimation of . The new estimator is given by for the positive scalar that maximizes where vector and symmetric matrix where
Once and the gradient at have been calculated, a new estimation is found from where
It is well known that least squares estimators in ordinary linear regression model are very good estimators, so a recursive algorithms procedure is to start the iteration with which are least squares estimators of and , respectively. Take such that . Iterations are stopped if some termination criterion is reached, for example, if for some prechosen small number .
Up to this point, we obtain the values of QML estimators when the function is known. However, the function is never the case in practice; we have to estimate it. By (2.12) and (1.2), we obtain Based on the dataset , we may obtain the estimation function of by some smoothing methods (see Simonff , Fan and Yao , Green and Silverman , Fan and Gijbels , etc.)
To obtain our results, the following conditions are sufficient.
(A1) is positive definite for sufficiently large and where and denotes the maximum in absolute value of the eigenvalues of a symmetric matrix.
(A2) There is a constant such that for any and .
(A3) The derivatives exist and are bounded for any and .
Remark 2.2. Maller  applied the condition (A1), and Kwoun and Yajima  used the conditions (A2) and (A3). Thus our conditions are general. (A1) delineates the class of for which our results hold in the sense required. It is further discussed by Maller in . Kwoun and Yajima  call stable if is bounded. Thus (A2) implies that is stable. However, is not stationary. In fact, by (2.3), we obtain that
which is dependent of .
For ease of exposition, we will introduce the following notations which will be used later in the paper.
Define -vector , and By (2.7) and (2.8), we get where and the * indicates that the element is filled in by symmetry. Thus, where
3. Statement of Main Results
Theorem 3.1. Suppose that conditions (A1)–(A3) hold. Then there is a sequence such that, for each , as , the probability Furthermore, where, for each and ; define neighborhoods
Theorem 3.2. Suppose that conditions (A1)–(A3) hold. Then
Remark 3.3. For , our results still hold.
In the following, we will investigate some special cases in the model (1.1)-(1.2). Although the following results are directly obtained from Theorems 3.1 and 3.2, we discuss these results in order to compare with the corresponding results.
Remark 3.5. These results are the same as the corresponding results of Maller .
Corollary 3.6. If and , then, for , where
Remark 3.7. These estimators are the same as the least squares estimators (see White ). For , are explosive processes. In the case, the corollary is the same as the results of White . While , notice that and , and by Corollary 3.6 we obtain The result was discussed by many authors, such as Fujikoshi and Ochi  and Brockwell and Davis .
Corollary 3.8. Let . If conditions (A2) and (A3) hold, then where
Corollary 3.9. Let . If condition (A1) holds, then
Remark 3.10. Let . Note that and we easily obtain asymptotic normality of the (quasi-)maximum likelihood or least squares estimator in ordinary linear regression models from the corollary.
4. Some Lemmas
Lemma 4.1. The matrix is positive definite for large enough with and .
Proof. It is easy to show that the matrix is positive definite for large enough . By (2.8), we have Note that and are independent of each other; thus by (2.7) and , we have Hence, from (4.1) and (4.2), By (2.8) and (2.17), we have Note that is a martingale difference sequence with so By (2.7) and (2.8) and noting that and are independent of each other, we have From (4.4)–(4.7), it follows that .
Lemma 4.2. If condition (A1) holds, then, for any , the matrix is positive definite for large enough , and
Proof. Let and be the smallest and largest roots of . Then from the study of Rao in [47, Ex 22.1], for unit vectors . Thus by (2.24), there are some and such that implies that By (4.10), we have By the study of Rao in [47, page 60] and (2.23), we have From (4.12) and ,
Lemma 4.3 (see ). Let be a symmetric random matrix with eigenvalues . Then
Lemma 4.4. For each , and also where
Proof. Let be a square root decomposition of . Then
Let . Then
From (2.28), (2.29), and (4.18),
In the first step, we will show that, for each , In fact, note that where
Let , and let . By Cauchy-Schwartz inequality, Lemma 4.2, condition (A3), and noting that , we have that Here for some . Similar to the proof of , we easily obtain that By Cauchy-Schwartz inequality, Lemma 4.2, condition (A3), and noting that , we have that Hence, (4.24) follows from (4.25)–(4.29).
In the second step, we will show that Note that Consider where For and each , we have By (4.34) and Lemma 4.2, we have Using Cauchy-Schwartz inequality, condition (A3), and (4.35), we obtain Let Then from Lemma 4.2, By condition (A2) and (4.38), we have Thus by Chebychev inequality and (4.39), Using the similar argument as , we obtain that Using the similar argument as , we obtain that By Cauchy-Schwartz inequality, (4.35), and (4.27), we get Thus (4.30) follows immediately from (4.32), (4.36), and (4.40)–(4.43).
In the third step, we will show that Let Then By (4.34), it is easy to show that From condition (A3), (2.30), and (4.23), we obtain that Hence, by Markov inequality, Using the similar argument as (4.40), we easily obtain that By Markov inequality and noting that we have that Using the similar argument as (4.6), we easily obtain that Hence, (4.44) follows immediately from (4.46), (4.47), and (4.49)–(4.53).
This completes the proof of (4.15) from (4.21), (4.24), (4.30), and (4.44). To prove (4.16), we need to show that This follows immediately from (2.27) and Markov inequality.
Finally, we will prove (4.17). By (4.15) and (4.16), we have uniformly in for each . Thus, by Lemma 4.3, This implies (4.17).
Lemma 4.5 (see ). Let be a zero-mean, square-integrable martingale array with differences , and let be an a.s. finite random variable. Suppose that , for all , and . Then where the r.v. has characteristic function .
5. Proof of Theorems
5.1. Proof of Theorem 3.1
Take , let be the boundary of , and let . Using (2.27) and Taylor expansion, for each , we have where for some .
Let and . Take and , and by (5.2) we obtain that By Lemma 4.1 and Chebychev inequality, we obtain Let , then , and using (4.17), we have By (5.3)–(5.5), we have By Lemma 4.3, as . Hence . Moreover, from (4.17), we have This implies that is concave on . Noting this fact and (5.6), we get On the event in the brackets, the continuous function has a unique maximum in over the compact neighborhood . Hence Moreover, there is a sequence such that