Mathematical Problems in Engineering

Mathematical Problems in Engineering / 2010 / Article
Special Issue

Nonlinear Time Series: Computations and Applications

View this Special Issue

Research Article | Open Access

Volume 2010 |Article ID 956907 |

Hongchang Hu, "QML Estimators in Linear Regression Models with Functional Coefficient Autoregressive Processes", Mathematical Problems in Engineering, vol. 2010, Article ID 956907, 30 pages, 2010.

QML Estimators in Linear Regression Models with Functional Coefficient Autoregressive Processes

Academic Editor: Massimo Scalia
Received30 Dec 2009
Revised19 Mar 2010
Accepted06 Apr 2010
Published28 Jun 2010


This paper studies a linear regression model, whose errors are functional coefficient autoregressive processes. Firstly, the quasi-maximum likelihood (QML) estimators of some unknown parameters are given. Secondly, under general conditions, the asymptotic properties (existence, consistency, and asymptotic distributions) of the QML estimators are investigated. These results extend those of Maller (2003), White (1959), Brockwell and Davis (1987), and so on. Lastly, the validity and feasibility of the method are illuminated by a simulation example and a real example.

1. Introduction

Consider the following linear regression model: where 's are scalar response variables, 's are explanatory variables, is a -dimensional unknown parameter, and the 's are functional coefficient autoregressive processes given as where 's are independent and identically distributed random errors with zero mean and finite variance , is a one-dimensional unknown parameter and is a real valued function defined on a compact set which contains the true value as an inner point and is a subset of . The values of and are unknown.

Model (1.1) includes many special cases, such as an ordinary linear regression model when see [111]. In the sequel, we always assume that for some , is a linear regression model with constant coefficient autoregressive processes (when , see Maller [12], Pere [13], and Fuller [14]), time-dependent and functional coefficient autoregressive processes (when , see Kwoun and Yajima [15]), constant coefficient autoregressive processes (when , see White [16, 17], Hamilton [18], Brockwell and Davis [19], and Abadir and Lucas [20]), time-dependent or time-varying autoregressive processes (when , see Carsoule and Franses [21], Azrak and Mélard [22], and Dahlhaus [23]), and so forth.

Regression analysis is one of the most mature and widely applied branches of statistics. Linear regression analysis is one of the most widely used statistical techniques. Its applications occur in almost every field, including engineering, economics, the physical sciences, management, life and biological sciences, and the social sciences. Linear regression model is the most important and popular model in the statistical literature, which attracts many statisticians to estimate the coefficients of the regression model. For the ordinary linear regression model (when the errors are independent and identically distributed random variables), Bai and Guo [1], Chen [2], Anderson and Taylor [3], Drygas [4], González-Rodríguez et al. [5], Hampel et al. [6], He [7], Cui [8], Durbin [9], Hoerl and Kennard [10], Li and Yang [11], and Zhang et al. [24] used various estimation methods (Least squares estimate method, robust estimation, biased estimation, and Bayes estimation) to obtain estimators of the unknown parameters in (1.1) and discussed some large or small sample properties of these estimators.

However, the independence assumption for the errors is not always appropriate in applications, especially for sequentially collected economic and physical data, which often exhibit evident dependence on the errors. Recently, linear regression with serially correlated errors has attracted increasing attention from statisticians. One case of considerable interest is that the errors are autoregressive processes and the asymptotic theory of this estimator was developed by Hannan and Kavalieris [25]. Fox and Taqqu [26] established its asymptotic normality in the case of long-memory stationary Gaussian observations errors. Giraitis and Surgailis [27] extended this result to non-Gaussian linear sequences. The asymptotic distribution of the maximum likelihood estimator was studied by Giraitis and Koul in [28] and Koul in [29] when the errors are nonlinear instantaneous functions of a Gaussian long-memory sequence. Koul and Surgailis [30] established the asymptotic normality of the Whittle estimator in linear regression models with non-Gaussian long-memory moving average errors. When the errors are Gaussian, or a function of Gaussian random variables that are strictly stationary and long range dependent, Koul and Mukherjee [31] investigated the linear model. Shiohama and Taniguchi [32] estimated the regression parameters in a linear regression model with autoregressive process.

In addition to (constant or functional or random coefficient) autoregressive model, it has gained much attention and has been applied to many fields, such as economics, physics, geography, geology, biology, and agriculture. Fan and Yao [33], Berk [34], Hannan and Kavalieris [35], Goldenshluger and Zeevi [36], Liebscher [37], An et al. [38], Elsebach [39], Carsoule and Franses [21], Baran et al. [40], Distaso [41], and Harvill and Ray [42] used various estimation methods (the least squares method, the Yule-Walker method, the method of stochastic approximation, and robust estimation method) to obtain some estimators and discussed their asymptotic properties, or investigated hypotheses testing.

This paper discusses the model (1.1)-(1.2) including stationary and explosive processes. The organization of the paper is as follows. In Section 2 some estimators of and are given by the quasi-maximum likelihood method. Under general conditions, the existence and consistency the quasi-maximum likelihood estimators are investigated, and asymptotic normality as well, in Section 3. Some preliminary lemmas are presented in Section 4. The main proofs are presented in Section 5, with some examples in Section 6.

2. Estimation Method

Write the “true” model as where , and 's are i.i.d errors with zero mean and finite variance . Define , and by (2.2) we have Thus is measurable with respect to the -field generated by , and

Assume at first that the ’s are i.i.d. . Using similar arguments to those of Fuller [14] or Maller [12], we get the log-likelihood of conditional on : At this stage we drop the normality assumption, but still maximize (2.5) to obtain QML estimators, denoted by (when they exist): Thus satisfy the following estimation equations: where

Remark 2.1. If then the above equations become the same as Maller's [12]. Therefore, we extend the QML estimators of Maller [12].
To calculate the values of the QML estimators, we may use the grid search method, steepest ascent method, Newton-Raphson method, and modified Newton-Raphson method. In order to calculate in Section 6, we introduce the most popular modified Newton-Raphson method proposed by Davidon-Fletcher-Powell (see Hamilton [18]).
Let vector denote an estimator of that has been calculated at the th iteration, and let denote an estimation of . The new estimator is given by for the positive scalar that maximizes where vector and symmetric matrix where
Once and the gradient at have been calculated, a new estimation is found from where
It is well known that least squares estimators in ordinary linear regression model are very good estimators, so a recursive algorithms procedure is to start the iteration with which are least squares estimators of and , respectively. Take such that . Iterations are stopped if some termination criterion is reached, for example, if for some prechosen small number .
Up to this point, we obtain the values of QML estimators when the function is known. However, the function is never the case in practice; we have to estimate it. By (2.12) and (1.2), we obtain Based on the dataset , we may obtain the estimation function of by some smoothing methods (see Simonff [43], Fan and Yao [33], Green and Silverman [44], Fan and Gijbels [45], etc.)
To obtain our results, the following conditions are sufficient.
(A1) is positive definite for sufficiently large and where and denotes the maximum in absolute value of the eigenvalues of a symmetric matrix.
(A2) There is a constant such that for any and .
(A3) The derivatives exist and are bounded for any and .

Remark 2.2. Maller [12] applied the condition (A1), and Kwoun and Yajima [15] used the conditions (A2) and (A3). Thus our conditions are general. (A1) delineates the class of for which our results hold in the sense required. It is further discussed by Maller in [12]. Kwoun and Yajima [15] call stable if is bounded. Thus (A2) implies that is stable. However, is not stationary. In fact, by (2.3), we obtain that which is dependent of .
For ease of exposition, we will introduce the following notations which will be used later in the paper.
Define -vector , and By (2.7) and (2.8), we get where and the * indicates that the element is filled in by symmetry. Thus, where

3. Statement of Main Results

Theorem 3.1. Suppose that conditions (A1)–(A3) hold. Then there is a sequence such that, for each , as , the probability Furthermore, where, for each and ; define neighborhoods

Theorem 3.2. Suppose that conditions (A1)–(A3) hold. Then

Remark 3.3. For , our results still hold.
In the following, we will investigate some special cases in the model (1.1)-(1.2). Although the following results are directly obtained from Theorems 3.1 and 3.2, we discuss these results in order to compare with the corresponding results.

Corollary 3.4. Let . If condition (A1) holds, then, for , (3.1), (3.2), and (3.4) hold.

Remark 3.5. These results are the same as the corresponding results of Maller [12].

Corollary 3.6. If and , then, for , where

Remark 3.7. These estimators are the same as the least squares estimators (see White [16]). For , are explosive processes. In the case, the corollary is the same as the results of White [17]. While , notice that and , and by Corollary 3.6 we obtain The result was discussed by many authors, such as Fujikoshi and Ochi [46] and Brockwell and Davis [19].

Corollary 3.8. Let . If conditions (A2) and (A3) hold, then where

Corollary 3.9. Let . If condition (A1) holds, then

Remark 3.10. Let . Note that and we easily obtain asymptotic normality of the (quasi-)maximum likelihood or least squares estimator in ordinary linear regression models from the corollary.

4. Some Lemmas

To prove Theorems 3.1 and 3.2, we first introduce the following lemmas.

Lemma 4.1. The matrix is positive definite for large enough with and .

Proof. It is easy to show that the matrix is positive definite for large enough . By (2.8), we have Note that and are independent of each other; thus by (2.7) and , we have Hence, from (4.1) and (4.2), By (2.8) and (2.17), we have Note that is a martingale difference sequence with so By (2.7) and (2.8) and noting that and are independent of each other, we have From (4.4)–(4.7), it follows that .

Lemma 4.2. If condition (A1) holds, then, for any , the matrix is positive definite for large enough , and

Proof. Let and be the smallest and largest roots of . Then from the study of Rao in [47, Ex 22.1], for unit vectors . Thus by (2.24), there are some and such that implies that By (4.10), we have By the study of Rao in [47, page 60] and (2.23), we have From (4.12) and ,

Lemma 4.3 (see [48]). Let be a symmetric random matrix with eigenvalues . Then

Lemma 4.4. For each , and also where

Proof. Let be a square root decomposition of . Then Let . Then From (2.28), (2.29), and (4.18), Let
In the first step, we will show that, for each , In fact, note that where
Let , and let . By Cauchy-Schwartz inequality, Lemma 4.2, condition (A3), and noting that , we have that Here for some . Similar to the proof of , we easily obtain that By Cauchy-Schwartz inequality, Lemma 4.2, condition (A3), and noting that , we have that Hence, (4.24) follows from (4.25)–(4.29).
In the second step, we will show that Note that Consider where For and each , we have By (4.34) and Lemma 4.2, we have Using Cauchy-Schwartz inequality, condition (A3), and (4.35), we obtain Let Then from Lemma 4.2, By condition (A2) and (4.38), we have Thus by Chebychev inequality and (4.39), Using the similar argument as , we obtain that Using the similar argument as , we obtain that By Cauchy-Schwartz inequality, (4.35), and (4.27), we get Thus (4.30) follows immediately from (4.32), (4.36), and (4.40)–(4.43).
In the third step, we will show that Let Then By (4.34), it is easy to show that From condition (A3), (2.30), and (4.23), we obtain that Hence, by Markov inequality, Using the similar argument as (4.40), we easily obtain that By Markov inequality and noting that we have that Using the similar argument as (4.6), we easily obtain that Hence, (4.44) follows immediately from (4.46), (4.47), and (4.49)–(4.53).
This completes the proof of (4.15) from (4.21), (4.24), (4.30), and (4.44). To prove (4.16), we need to show that