QML Estimators in Linear Regression Models with Functional Coefficient Autoregressive Processes

Abstract

This paper studies a linear regression model, whose errors are functional coefficient autoregressive processes. Firstly, the quasi-maximum likelihood (QML) estimators of some unknown parameters are given. Secondly, under general conditions, the asymptotic properties (existence, consistency, and asymptotic distributions) of the QML estimators are investigated. These results extend those of Maller (2003), White (1959), Brockwell and Davis (1987), and so on. Lastly, the validity and feasibility of the method are illuminated by a simulation example and a real example.

1. Introduction

Consider the following linear regression model:𝑦_𝑡=𝑥^𝑇_𝑡𝛽+𝜀_𝑡,𝑡=1,2,…,𝑛, (1.1) where 𝑦_𝑡's are scalar response variables, 𝑥_𝑡's are explanatory variables, 𝛽 is a 𝑑-dimensional unknown parameter, and the 𝜀_𝑡's are functional coefficient autoregressive processes given as𝜀₁=𝜂₁,𝜀_𝑡=𝑓_𝑡(𝜃)𝜀_𝑡−1+𝜂_𝑡,𝑡=2,3,…,𝑛, (1.2) where 𝜂_𝑡's are independent and identically distributed random errors with zero mean and finite variance 𝜎², 𝜃 is a one-dimensional unknown parameter and 𝑓_𝑡(𝜃) is a real valued function defined on a compact set Θ which contains the true value 𝜃₀ as an inner point and is a subset of 𝑅¹. The values of 𝜃₀ and 𝜎² are unknown.

Model (1.1) includes many special cases, such as an ordinary linear regression model when 𝑓_𝑡(𝜃)≡0; see [1–11]. In the sequel, we always assume that 𝑓_𝑡(𝜃)≠0, for some 𝜃∈Θ, is a linear regression model with constant coefficient autoregressive processes (when 𝑓_𝑡(𝜃)=𝜃, see Maller [12], Pere [13], and Fuller [14]), time-dependent and functional coefficient autoregressive processes (when 𝛽=0, see Kwoun and Yajima [15]), constant coefficient autoregressive processes (when 𝑓_𝑡(𝜃)=𝜃and𝛽=0, see White [16, 17], Hamilton [18], Brockwell and Davis [19], and Abadir and Lucas [20]), time-dependent or time-varying autoregressive processes (when 𝑓_𝑡(𝜃)=𝑎_𝑡and𝛽=0, see Carsoule and Franses [21], Azrak and Mélard [22], and Dahlhaus [23]), and so forth.

Regression analysis is one of the most mature and widely applied branches of statistics. Linear regression analysis is one of the most widely used statistical techniques. Its applications occur in almost every field, including engineering, economics, the physical sciences, management, life and biological sciences, and the social sciences. Linear regression model is the most important and popular model in the statistical literature, which attracts many statisticians to estimate the coefficients of the regression model. For the ordinary linear regression model (when the errors are independent and identically distributed random variables), Bai and Guo [1], Chen [2], Anderson and Taylor [3], Drygas [4], González-Rodríguez et al. [5], Hampel et al. [6], He [7], Cui [8], Durbin [9], Hoerl and Kennard [10], Li and Yang [11], and Zhang et al. [24] used various estimation methods (Least squares estimate method, robust estimation, biased estimation, and Bayes estimation) to obtain estimators of the unknown parameters in (1.1) and discussed some large or small sample properties of these estimators.

However, the independence assumption for the errors is not always appropriate in applications, especially for sequentially collected economic and physical data, which often exhibit evident dependence on the errors. Recently, linear regression with serially correlated errors has attracted increasing attention from statisticians. One case of considerable interest is that the errors are autoregressive processes and the asymptotic theory of this estimator was developed by Hannan and Kavalieris [25]. Fox and Taqqu [26] established its asymptotic normality in the case of long-memory stationary Gaussian observations errors. Giraitis and Surgailis [27] extended this result to non-Gaussian linear sequences. The asymptotic distribution of the maximum likelihood estimator was studied by Giraitis and Koul in [28] and Koul in [29] when the errors are nonlinear instantaneous functions of a Gaussian long-memory sequence. Koul and Surgailis [30] established the asymptotic normality of the Whittle estimator in linear regression models with non-Gaussian long-memory moving average errors. When the errors are Gaussian, or a function of Gaussian random variables that are strictly stationary and long range dependent, Koul and Mukherjee [31] investigated the linear model. Shiohama and Taniguchi [32] estimated the regression parameters in a linear regression model with autoregressive process.

In addition to (constant or functional or random coefficient) autoregressive model, it has gained much attention and has been applied to many fields, such as economics, physics, geography, geology, biology, and agriculture. Fan and Yao [33], Berk [34], Hannan and Kavalieris [35], Goldenshluger and Zeevi [36], Liebscher [37], An et al. [38], Elsebach [39], Carsoule and Franses [21], Baran et al. [40], Distaso [41], and Harvill and Ray [42] used various estimation methods (the least squares method, the Yule-Walker method, the method of stochastic approximation, and robust estimation method) to obtain some estimators and discussed their asymptotic properties, or investigated hypotheses testing.

This paper discusses the model (1.1)-(1.2) including stationary and explosive processes. The organization of the paper is as follows. In Section 2 some estimators of 𝛽,𝜃, and 𝜎² are given by the quasi-maximum likelihood method. Under general conditions, the existence and consistency the quasi-maximum likelihood estimators are investigated, and asymptotic normality as well, in Section 3. Some preliminary lemmas are presented in Section 4. The main proofs are presented in Section 5, with some examples in Section 6.

2. Estimation Method

Write the “true” model as𝑦_𝑡=𝑥^𝑇_𝑡𝛽₀+𝑒_𝑡𝑒,𝑡=1,2,…,𝑛,₁=𝜂₁,𝑒_𝑡=𝑓_𝑡(𝜃₀)𝑒_𝑡−1+𝜂_𝑡,𝑡=2,3,…,𝑛, (2.1)(2.2) where 𝑓^′_𝑡(𝜃₀)=(𝑑𝑓_𝑡(𝜃)/𝑑𝜃)∣_𝜃=𝜃0≠0, and 𝜂_𝑡's are i.i.d errors with zero mean and finite variance 𝜎²₀. Define ∏⁻¹_𝑖=0𝑓_𝑡−𝑖(𝜃₀)=1, and by (2.2) we have𝑒_𝑡=^𝑡−1_𝑗=0(^𝑗−1_𝑖=0𝑓_𝑡−𝑖(𝜃₀))𝜂_𝑡−𝑗. (2.3) Thus 𝑒_𝑡 is measurable with respect to the 𝜎-field 𝐻 generated by 𝜂₁,𝜂₂,…,𝜂_𝑡, and𝐸𝑒_𝑡=0,Var(𝑒_𝑡)=𝜎²₀^𝑡−1_𝑗=0(^𝑗−1_𝑖=0𝑓²_𝑡−𝑖(𝜃₀)). (2.4)

Assume at first that the 𝜂_𝑡’s are i.i.d. 𝑁(0,𝜎²). Using similar arguments to those of Fuller [14] or Maller [12], we get the log-likelihood of 𝑦₂,𝑦₃,…,𝑦_𝑛 conditional on 𝑦₁:Ψ_𝑛(𝛽,𝜃,𝜎²)=log𝐿_𝑛1=−2(𝑛−1)log𝜎²−12𝜎^2𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)²−12(𝑛−1)log(2𝜋). (2.5) At this stage we drop the normality assumption, but still maximize (2.5) to obtain QML estimators, denoted by 𝜎²_𝑛,𝛽_𝑛,𝜃_𝑛 (when they exist):𝜕Ψ_𝑛𝜕𝜎²=−𝑛−12𝜎²+12𝜎^4𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)²,𝜕Ψ_𝑛=1𝜕𝜃𝜎^2𝑛_𝑡=2𝑓^′_𝑡(𝜃)(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)𝜀_𝑡−1,𝜕Ψ_𝑛=1𝜕𝛽𝜎^2𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃)𝑥_𝑡−1). (2.6)(2.7)(2.8) Thus 𝜎²_𝑛,𝛽_𝑛,𝜃_𝑛 satisfy the following estimation equations:𝜎²_𝑛=1𝑛−1^𝑛_𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)²,_𝑛𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)𝑓^′_𝑡(𝜃_𝑛)𝜀̂_𝑡−1=0,_𝑛𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)=0, (2.9)(2.10)(2.11) where𝜀_𝑡̂=𝑦_𝑡−𝑥^𝑇_𝑡𝛽_𝑛. (2.12)

Remark 2.1. If 𝑓_𝑡(𝜃)=𝜃, then the above equations become the same as Maller's [12]. Therefore, we extend the QML estimators of Maller [12].
To calculate the values of the QML estimators, we may use the grid search method, steepest ascent method, Newton-Raphson method, and modified Newton-Raphson method. In order to calculate in Section 6, we introduce the most popular modified Newton-Raphson method proposed by Davidon-Fletcher-Powell (see Hamilton [18]).
Let (𝑑+2)×1 vector 𝜃^(𝑚)=(𝜎^(𝑚)2,𝛽^(𝑚),𝜃^(𝑚)) denote an estimator of 𝜃=(𝜎²,𝛽,𝜃) that has been calculated at the 𝑚th iteration, and let 𝐴^(𝑚) denote an estimation of [𝐇(𝜃^(𝑚))]⁻¹. The new estimator 𝜃^(𝑚+1) is given by 𝜃^(𝑚+1)=𝜃^(𝑚)+𝑠𝐴^(𝑚)𝑔(𝜃^(𝑚)) (2.13) for 𝑠 the positive scalar that maximizes Ψ_𝑛{𝜃^(𝑚)+𝑠𝐴^(𝑚)𝑔(𝜃^(𝑚))}, where (𝑑+2)×1 vector 𝑔(𝜃^(𝑚))=𝜕Ψ_𝑛(𝜃)𝜕𝜃∣_𝜃𝜃=^(𝑚)=𝜕Ψ_𝑛𝜕𝜎²∣_{𝜎2=𝜎(𝑚)2}𝜕Ψ_𝑛∣𝜕𝛽^{𝛽=𝛽(𝑚)}𝜕Ψ_𝑛∣𝜕𝜃^{𝜃=𝜃(𝑚)} (2.14) and (𝑑+2)×(𝑑+2) symmetric matrix 𝐇(𝜃^(𝑚)𝜕)=−²Ψ_𝑛(𝜃)𝜕𝜃𝜕𝜃^𝑇∣_𝜃𝜃=^(𝑚)=𝜕²Ψ_𝑛𝜕(𝜎²)²𝜕²Ψ_𝑛𝜕𝜎²𝜕𝜕𝛽²Ψ_𝑛𝜕𝜎²∗𝜕𝜕𝜃₂Ψ_𝑛𝜕𝛽𝜕𝛽^𝑇𝜕²Ψ_𝑛𝜕𝜕𝛽𝜕𝜃∗∗²Ψ_𝑛𝜕𝜃²∣_𝜃𝜃=^(𝑚), (2.15) where 𝜕²Ψ_𝑛𝜕(𝜎²)²=𝑛−12𝜎⁴−1𝜎^6𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)²,𝜕²Ψ_𝑛𝜕𝜎²1𝜕𝛽=−𝜎^4𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃)𝑥_𝑡−1)^𝑇,𝜕²Ψ_𝑛𝜕𝜎²1𝜕𝜃=−𝜎^4𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃)𝜀_𝑡−1)𝑓^′_𝑡𝜕(𝜃),₂Ψ_𝑛𝜕𝛽𝜕𝛽^𝑇1=−𝜎^2𝑛_𝑡=2(𝑥_𝑡−𝑓_𝑡(𝜃)𝑥_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃)𝑥_𝑡−1)^𝑇𝜕₂,Ψ_𝑛1𝜕𝛽𝜕𝜃=−𝜎^2𝑛_𝑡=2(𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡+𝑓^′_𝑡(𝜃)𝜀_𝑡𝑥_𝑡−1−2𝑓_𝑡(𝜃)𝑓^′_𝑡(𝜃)𝑥_𝑡−1𝜀_𝑡−1𝜕),₂Ψ_𝑛𝜕𝜃²1=−𝜎^2𝑛_𝑡=2((𝑓^′_𝑡²(𝜃)+𝑓_𝑡(𝜃)𝑓_𝑡^′′(𝜃))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)𝜀_𝑡𝜀_𝑡−1). (2.16)(2.17)(2.18)
Once 𝜃^(𝑚+1) and the gradient at 𝜃^(𝑚+1) have been calculated, a new estimation 𝐴^(𝑚+1) is found from 𝐴^(𝑚+1)=𝐴^(𝑚)−𝐴^(𝑚)(Δ𝑔^(𝑚+1))(Δ𝑔^(𝑚+1))^𝑇𝐴^(𝑚)(Δ𝑔^(𝑚+1))^𝑇𝐴^(𝑚)(Δ𝑔^(𝑚+1))−(Δ𝜃^(𝑚+1))(Δ𝜃^(𝑚+1))^𝑇(Δ𝑔^(𝑚+1))^𝑇(Δ𝜃^(𝑚+1)), (2.19) where Δ𝜃^(𝑚+1)=𝜃^(𝑚+1)−𝜃^(𝑚),Δ𝑔^(𝑚+1)=𝑔(𝜃^(𝑚+1))−𝑔(𝜃^(𝑚)). (2.20)
It is well known that least squares estimators in ordinary linear regression model are very good estimators, so a recursive algorithms procedure is to start the iteration with 𝛽⁽⁰⁾,𝜎⁽⁰⁾² which are least squares estimators of 𝛽 and 𝜎², respectively. Take 𝜃⁽⁰⁾ such that 𝑓_𝑡(𝜃⁽⁰⁾)=0. Iterations are stopped if some termination criterion is reached, for example, if ‖𝜃^(𝑚+1)−𝜃^(𝑚)‖‖𝜃^(𝑚)‖<𝛿, (2.21) for some prechosen small number 𝛿>0.
Up to this point, we obtain the values of QML estimators when the function 𝑓_𝑡(𝜃)=𝑓(𝑡,𝜃) is known. However, the function 𝑓_𝑡(𝜃) is never the case in practice; we have to estimate it. By (2.12) and (1.2), we obtain 𝑓(𝑡,𝜃_𝑛𝜀̂)=^𝑡𝜀̂_𝑡−1,𝑡=2,3,…,𝑛. (2.22) Based on the dataset {𝑓(𝑡,𝜃_𝑛),𝑡=2,3,…,𝑛}, we may obtain the estimation function 𝑓(𝑡,𝜃_𝑛) of 𝑓(𝑡,𝜃) by some smoothing methods (see Simonff [43], Fan and Yao [33], Green and Silverman [44], Fan and Gijbels [45], etc.)
To obtain our results, the following conditions are sufficient.
(A1) 𝑋_𝑛=∑^𝑛_𝑡=2𝑥_𝑡𝑥^𝑇_𝑡 is positive definite for sufficiently large 𝑛 and lim_𝑛→∞max_{1≤𝑡≤𝑛}𝑥^𝑇_𝑡𝑋_𝑛⁻¹𝑥_𝑡=0,limsup_𝑛→∞|𝜆|_max(𝑋_𝑛^−1/2𝑍_𝑛𝑋_𝑛^−𝑇/2)<1, (2.23)(2.24) where 𝑍_𝑛=(1/2)∑^𝑛_𝑡=2(𝑥_𝑡𝑥^𝑇_𝑡−1+𝑥_𝑡−1𝑥^𝑇_𝑡) and |𝜆|_max(⋅) denotes the maximum in absolute value of the eigenvalues of a symmetric matrix.
(A2) There is a constant 𝛼>0 such that ^𝑡_𝑗=1(^𝑗−1_𝑖=0𝑓²_𝑡−𝑖(𝜃))≤𝛼 (2.25) for any 𝑡∈{1,2,…,𝑛} and 𝜃∈Θ.
(A3) The derivatives 𝑓^′_𝑡(𝜃)=𝑑𝑓_𝑡(𝜃)/𝑑𝜃,𝑓_𝑡^′′(𝜃)=𝑑𝑓^′_𝑡(𝜃)/𝑑𝜃 exist and are bounded for any 𝑡 and 𝜃∈Θ.

Remark 2.2. Maller [12] applied the condition (A1), and Kwoun and Yajima [15] used the conditions (A2) and (A3). Thus our conditions are general. (A1) delineates the class of 𝑥_𝑡 for which our results hold in the sense required. It is further discussed by Maller in [12]. Kwoun and Yajima [15] call {𝑒_𝑡} stable if Var(𝑒_𝑡) is bounded. Thus (A2) implies that {𝑒_𝑡} is stable. However, {𝑒_𝑡} is not stationary. In fact, by (2.3), we obtain that Cov(𝑒_𝑡,𝑒_𝑡+𝑘)=𝜎²₀{^𝑘−1_𝑖=0𝑓_{𝑡+𝑘−𝑖}(𝜃₀)+𝑓_𝑡(𝜃₀)^𝑘_𝑖=0𝑓_{𝑡+𝑘−𝑖}(𝜃₀)+⋯+^𝑡−2_𝑙=0𝑓_𝑡−𝑙(𝜃₀)^{𝑡+𝑘−2}_𝑖=0𝑓_{𝑡+𝑘−𝑖}(𝜃₀)}, (2.26) which is dependent of 𝑡.
For ease of exposition, we will introduce the following notations which will be used later in the paper.
Define (𝑑+1)-vector 𝜑=(𝛽,𝜃), and 𝑆_𝑛(𝜑)=𝜎²𝜕Ψ_𝑛𝜕𝜑=𝜎²(𝜕Ψ_𝑛,𝜕𝛽𝜕Ψ_𝑛𝜕𝜃),𝐹_𝑛(𝜑)=−𝜎²𝜕²Ψ_𝑛𝜕𝜑𝜕𝜑^𝑇. (2.27) By (2.7) and (2.8), we get F_𝑛𝑋(𝜑)=^𝑛(𝜃)^𝑛_𝑡=2(𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡+𝑓^′_𝑡(𝜃)𝜀_𝑡𝑥_𝑡−1−2𝑓_𝑡(𝜃)𝑓^′_𝑡(𝜃)𝑥_𝑡−1𝜀_𝑡−1)∗^𝑛_𝑡=2((𝑓^′_𝑡²(𝜃)+𝑓_𝑡(𝜃)𝑓_𝑡^′′(𝜃))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)𝜀_𝑡𝜀_𝑡−1), (2.28) where 𝑋_𝑛(𝜃)=−𝜎²(𝜕²Ψ_𝑛/𝜕𝛽𝜕𝛽^𝑇) and the * indicates that the element is filled in by symmetry. Thus, 𝐷_𝑛=𝐸(𝐹_𝑛(𝜑₀=𝑋))_𝑛(𝜃₀∗)0_𝑛𝑡=2((𝑓^′_𝑡²(𝜃₀)+𝑓_𝑡(𝜃₀)𝑓_𝑡^′′(𝜃₀))𝐸𝑒²_𝑡−1−𝑓_𝑡^′′(𝜃₀)𝐸(𝑒_𝑡𝑒_𝑡−1=𝑋))_𝑛(𝜃₀∗)0_𝑛𝑡=2𝑓^′_𝑡²(𝜃₀)𝐸𝑒²_𝑡−1=𝑋_𝑛(𝜃₀)0∗Δ_𝑛(𝜃₀,𝜎₀), (2.29) where Δ_𝑛(𝜃₀,𝜎₀)=^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝐸𝑒²_𝑡−1=𝜎²₀^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)^𝑡−2_𝑗=0(^𝑗−1_𝑖=0𝑓²_𝑡−𝑖(𝜃))=𝑂(𝑛). (2.30)

3. Statement of Main Results

Theorem 3.1. Suppose that conditions (A1)–(A3) hold. Then there is a sequence 𝐴_𝑛↓0 such that, for each 𝐴>0, as 𝑛→∞, the probability 𝑃{thereareestimators𝜑_𝑛,𝜎²_𝑛with𝑆_𝑛(𝜑_𝑛)=0,and(𝜑_𝑛,𝜎²_𝑛)∈𝑁^′_𝑛(𝐴)}→1. (3.1) Furthermore, (𝜑_𝑛,𝜎²_𝑛)→_𝑝(𝜑₀,𝜎²₀),𝑛→∞, (3.2) where, for each 𝑛=1,2,…,𝐴>0 and 𝐴_𝑛∈(0,𝜎²₀); define neighborhoods 𝑁_𝑛(𝐴)={𝜑∈𝑅^𝑑+1∶(𝜑−𝜑₀)^𝑇𝐷_𝑛(𝜑−𝜑₀)≤𝐴²𝑁},_′𝑛(𝐴)=𝑁_𝑛(𝐴)∩{𝜎²∈[𝜎²₀−𝐴_𝑛,𝜎²₀+𝐴_𝑛]}. (3.3)

Theorem 3.2. Suppose that conditions (A1)–(A3) hold. Then 1𝜎_𝑛𝐹_𝑛^𝑇/2(𝜑_𝑛)(𝜑_𝑛−𝜑₀)→_𝐷𝑁(0,𝐼_𝑑+1),𝑛→∞. (3.4)

Remark 3.3. For 𝜃∈𝑅^𝑚,𝑚∈𝑁, our results still hold.
In the following, we will investigate some special cases in the model (1.1)-(1.2). Although the following results are directly obtained from Theorems 3.1 and 3.2, we discuss these results in order to compare with the corresponding results.

Corollary 3.4. Let 𝑓_𝑡(𝜃)=𝜃. If condition (A1) holds, then, for |𝜃|≠1, (3.1), (3.2), and (3.4) hold.

Remark 3.5. These results are the same as the corresponding results of Maller [12].

Corollary 3.6. If 𝛽=0 and 𝑓_𝑡(𝜃)=𝜃, then, for ∣𝜃∣≠1, ∑^𝑛_𝑡=2𝜀²_𝑡−1𝜎_𝑛(𝜃_𝑛−𝜃₀)→_𝐷𝑁(0,1),𝑛→∞, (3.5) where 𝜎²_𝑛=1𝑛−1^𝑛_𝑡=2(𝜀_𝑡−𝜃_𝑛𝜀_𝑡−1)²,𝜃_𝑛=∑^𝑛_𝑡=2(𝜀_𝑡𝜀_𝑡−1)∑^𝑛_𝑡=2𝜀²_𝑡−1. (3.6)

Remark 3.7. These estimators are the same as the least squares estimators (see White [16]). For |𝜃|>1, {𝜀_𝑡} are explosive processes. In the case, the corollary is the same as the results of White [17]. While |𝜃|<1, notice that 𝜎²_𝑛→_𝑝𝜎²₀ and (1/(𝑛−1))∑^𝑛_𝑡=2𝜀²_𝑡−1→_𝑝𝐸𝜀²_𝑡=𝜎²₀/(1−𝜃²₀), and by Corollary 3.6 we obtain √𝑛(𝜃_𝑛−𝜃₀)→_𝐷𝑁(0,1−𝜃²₀). (3.7) The result was discussed by many authors, such as Fujikoshi and Ochi [46] and Brockwell and Davis [19].

Corollary 3.8. Let 𝛽=0. If conditions (A2) and (A3) hold, then 𝐹_𝑛^1/2(𝜃_𝑛)𝜎_𝑛(𝜃_𝑛−𝜃₀)→_𝐷𝑁(0,1),𝑛→∞, (3.8) where 𝐹_𝑛(𝜃_𝑛)=^𝑛_𝑡=2((𝑓^′_𝑡²(𝜃_𝑛)+𝑓_𝑡(𝜃_𝑛)𝑓_𝑡^′′(𝜃_𝑛))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃_𝑛)𝜀_𝑡𝜀_𝑡−1𝜎),_2𝑛=1𝑛−1^𝑛_𝑡=2(𝜀_𝑡−𝑓_𝑡(𝜃_𝑛)𝜀_𝑡−1)². (3.9)

Corollary 3.9. Let 𝑓_𝑡(𝜃)=𝑎_𝑡. If condition (A1) holds, then 1𝜎_𝑛{^𝑛_𝑡=2(𝑥_𝑡−𝑎_𝑡𝑥_𝑡−1)(𝑥_𝑡−𝑎_𝑡𝑥_𝑡−1)^𝑇}^𝑇/2(𝛽_𝑛−𝛽₀)→_𝐷𝑁(0,𝐼_𝑑),𝑛→∞. (3.10)

Remark 3.10. Let 𝑎_𝑡=0. Note that ∑^𝑛_𝑡=2𝑥_𝑡𝑥^𝑇_𝑡=𝑂(√𝑛) and 𝜎²_𝑛→_𝑝𝜎²₀; we easily obtain asymptotic normality of the (quasi-)maximum likelihood or least squares estimator in ordinary linear regression models from the corollary.

4. Some Lemmas

To prove Theorems 3.1 and 3.2, we first introduce the following lemmas.

Lemma 4.1. The matrix 𝐷_𝑛 is positive definite for large enough 𝑛 with 𝐸(𝑆_𝑛(𝜑₀))=0 and Var(S_𝑛(𝜑₀))=𝜎²₀𝐷_𝑛.

Proof. It is easy to show that the matrix 𝐷_𝑛 is positive definite for large enough 𝑛. By (2.8), we have 𝜎²₀𝐸(𝜕Ψ_𝑛∣𝜕𝛽^𝛽=𝛽0)=^𝑛_𝑡=2𝐸(𝑒_𝑡−𝑓_𝑡(𝜃₀)𝑒_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)=^𝑛_𝑡=2(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)𝐸𝜂_𝑡=0. (4.1) Note that 𝑒_𝑡−1 and 𝜂_𝑡 are independent of each other; thus by (2.7) and 𝐸𝜂_𝑡=0, we have 𝜎²₀𝐸(𝜕Ψ_𝑛∣𝜕𝜃^𝜃=𝜃0)=^𝑛_𝑡=2𝐸((𝑒_𝑡−𝑓_𝑡(𝜃₀)𝑒_𝑡−1)𝑓^′_𝑡(𝜃₀)𝑒_𝑡−1)=^𝑛_𝑡=2𝑓^′_𝑡(𝜃₀)𝐸(𝜂_𝑡𝑒_𝑡−1)=0. (4.2) Hence, from (4.1) and (4.2), 𝐸(𝑆_𝑛(𝜑₀))=𝜎²₀𝐸(𝜕Ψ_𝑛∣𝜕𝛽^𝛽=𝛽0,𝜕Ψ_𝑛∣𝜕𝜃^𝜃=𝜃0)=0. (4.3) By (2.8) and (2.17), we have Var(𝜎²₀𝜕Ψ_𝑛∣𝜕𝛽^𝛽=𝛽0)=Var{^𝑛_𝑡=2(𝑒_𝑡−𝑓_𝑡(𝜃₀)𝑒_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)}=Var{^𝑛_𝑡=2𝜂_𝑡(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)}=𝜎²₀𝑋_𝑛(𝜃₀). (4.4) Note that {𝑓^′_𝑡(𝜃₀)𝜂_𝑡𝑒_𝑡−1,𝐻_𝑡} is a martingale difference sequence with Var(𝑓^′_𝑡(𝜃₀)𝜂_𝑡𝑒_𝑡−1)=𝑓^′_𝑡²(𝜃₀)𝐸𝜂²_𝑡𝐸𝑒²_𝑡−1=𝜎²₀𝑓^′_𝑡²(𝜃₀)𝐸𝑒²_𝑡−1, (4.5) so Var(𝜎²₀𝜕Ψ_𝑛∣𝜕𝜃^𝜃=𝜃0)=Var{^𝑛_𝑡=2𝜂_𝑡𝑓^′_𝑡(𝜃₀)𝑒_𝑡−1}=^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝐸𝑒²_𝑡−1=𝜎²₀Δ_𝑛(𝜃₀,𝜎₀). (4.6) By (2.7) and (2.8) and noting that 𝑒_𝑡−1 and 𝜂_𝑡 are independent of each other, we have Cov(𝜎²₀𝜕Ψ_𝑛∣𝜕𝛽^𝛽=𝛽0,𝜎²₀𝜕Ψ_𝑛∣𝜕𝜃^𝜃=𝜃0)=𝐸(𝜎²₀𝜕Ψ_𝑛∣𝜕𝛽^𝛽=𝛽0,𝜎²₀𝜕Ψ_𝑛∣𝜕𝜃^𝜃=𝜃0)=𝐸(^𝑛_𝑡=2𝜂²_𝑡(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)𝑓^′_𝑡(𝜃₀)𝑒_𝑡−1)+𝐸(^𝑛_𝑡=3𝜂_𝑡(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑡−1_𝑠=2𝜂_𝑠𝑓^′_𝑠(𝜃₀)𝑒_𝑠−1)+𝐸(^𝑛_𝑠=3𝜂_𝑠𝑓^′_𝑠(𝜃₀)𝑒_𝑠−1^𝑠−1_𝑡=2𝜂_𝑡(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1))=0. (4.7) From (4.4)–(4.7), it follows that Var(𝑆_𝑛(𝜑₀))=𝜎²₀𝐷_𝑛.

Lemma 4.2. If condition (A1) holds, then, for any 𝜃∈Θ, the matrix 𝑋_𝑛(𝜃) is positive definite for large enough 𝑛, and lim_𝑛→∞max_{1≤𝑡≤𝑛}𝑥^𝑇_𝑡𝑋_𝑛⁻¹(𝜃)𝑥_𝑡=0. (4.8)

Proof. Let 𝜆₁ and 𝜆_𝑑 be the smallest and largest roots of |𝑍_𝑛−𝜆𝑋_𝑛|=0. Then from the study of Rao in [47, Ex 22.1], 𝜆₁≤𝑢^𝑇𝑍_𝑛𝑢𝑢^𝑇𝑋_𝑛𝑢≤𝜆_𝑑 (4.9) for unit vectors 𝑢. Thus by (2.24), there are some 𝛿∈(max{0,1−(1+min_{2≤𝑡≤𝑛}|𝑓²_𝑡(𝜃)|)/max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃)|},1) and 𝑛₀(𝛿) such that 𝑛≥𝑁₀ implies that |𝑢^𝑇𝑍_𝑛𝑢|≤(1−𝛿)𝑢^𝑇𝑋_𝑛𝑢. (4.10) By (4.10), we have 𝑢^𝑇𝑋_𝑛𝑢=^𝑛_𝑡=2(𝑢^𝑇(𝑥_𝑡−𝑓_𝑡(𝜃)𝑥_𝑡−1))²=^𝑛_𝑡=2((𝑢^𝑇𝑥_𝑡)²+𝑓²_𝑡(𝜃)(𝑢^𝑇𝑥_𝑡−1)²−𝑓_𝑡(𝜃)𝑢^𝑇𝑥_𝑡−1𝑥^𝑇_𝑡𝑢−𝑓_𝑡(𝜃)𝑢^𝑇𝑥_𝑡𝑥^𝑇_𝑡−1≥𝑢)_𝑛𝑡=2(𝑢^𝑇𝑥_𝑡)²+min_{2≤𝑡≤𝑛}|𝑓²_𝑡(𝜃)|^𝑛_𝑡=2(𝑢^𝑇𝑥_𝑡−1)²−max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃)|𝑢^𝑇𝑍_𝑛𝑢≥𝑢^𝑇𝑋_𝑛𝑢+min_{2≤𝑡≤𝑛}|𝑓²_𝑡(𝜃)|𝑢^𝑇𝑋_𝑛𝑢−max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃)|𝑢^𝑇𝑍_𝑛𝑢≥(1+min_{2≤𝑡≤𝑛}|𝑓²_𝑡(𝜃)|−max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃)|(1−𝛿))𝑢^𝑇𝑋_𝑛𝑢=𝐶(𝜃,𝛿)𝑢^𝑇𝑋_𝑛𝑢. (4.11) By the study of Rao in [47, page 60] and (2.23), we have (𝑢^𝑇𝑥_𝑡)²𝑢^𝑇𝑋_𝑛𝑢→0. (4.12) From (4.12) and 𝐶(𝜃,𝛿)>0, 𝑥^𝑇_𝑡𝑋_𝑛⁻¹(𝜃)=sup_𝑢((𝑢^𝑇𝑥_𝑡)²𝑢^𝑇𝑋_𝑛(𝜃)𝑢)≤sup_𝑢((𝑢^𝑇𝑥_𝑡)²𝐶(𝜃,𝛿)𝑢^𝑇𝑋_𝑛𝑢)→0. (4.13)

Lemma 4.3 (see [48]). Let 𝑊_𝑛 be a symmetric random matrix with eigenvalues 𝜆_𝑗(𝑛),1≤𝑗≤𝑑. Then 𝑊_𝑛→_𝑝𝐼⇔𝜆_𝑗(𝑛)→_𝑝1,𝑛→∞. (4.14)

Lemma 4.4. For each 𝐴>0, sup_{𝜑∈𝑁𝑛(𝐴)}‖𝐷_𝑛^−1/2F_𝑛(𝜑)𝐷_𝑛^−𝑇/2−Φ_𝑛‖→_𝑝0,𝑛→∞, (4.15) and also Φ_𝑛→_𝐷Φ,lim_𝑐→0limsup_𝐴→∞limsup_𝑛→∞𝑃{inf_{𝜑∈𝑁𝑛(𝐴)}𝜆_min(𝐷_𝑛^−1/2F_𝑛(𝜑)𝐷_𝑛^−𝑇/2)≤𝑐}=0, (4.16)(4.17) where Φ_𝑛=𝐼_𝑑00∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1Δ_𝑛(𝜃₀,𝜎₀),Φ=𝐼_𝑑+1. (4.18)

Proof. Let 𝑋_𝑛(𝜃₀)=𝑋_𝑛^1/2(𝜃₀)𝑋_𝑛^𝑇/2(𝜃₀) be a square root decomposition of 𝑋_𝑛(𝜃₀). Then 𝐷_𝑛=𝑋_𝑛^1/2(𝜃₀∗)0Δ_𝑛(𝜃₀,𝜎₀)𝑋_𝑛^𝑇/2(𝜃₀∗)0Δ_𝑛(𝜃₀,𝜎₀)=𝐷_𝑛^1/2𝐷_𝑛^𝑇/2. (4.19) Let 𝜑∈𝑁_𝑛(𝐴). Then (𝜑−𝜑₀)^𝑇𝐷_𝑛(𝜑−𝜑₀)=(𝛽−𝛽₀)^𝑇𝑋_𝑛(𝜃₀)(𝛽−𝛽₀)+(𝜃−𝜃₀)²Δ_𝑛(𝜃₀,𝜎₀)≤𝐴². (4.20) From (2.28), (2.29), and (4.18), 𝐷_𝑛^−1/2𝐹_𝑛(𝜑)𝐷_𝑛^−𝑇/2−Φ_𝑛=𝑋_𝑛^−1/2(𝜃₀)𝑋_𝑛(𝜃)𝑋_𝑛^−𝑇/2(𝜃₀)−𝐼_𝑑𝑋_𝑛^−1/2(𝜃₀)∑^𝑛_𝑡=2(𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡+𝑓^′_𝑡(𝜃)𝜀_𝑡𝑥_𝑡−1−2𝑓_𝑡(𝜃)𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡−1)Δ_𝑛(𝜃₀,𝜎₀)∗∑^𝑛_𝑡=2((𝑓^′_𝑡²(𝜃)+𝑓_𝑡(𝜃)𝑓_𝑡^′′(𝜃))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)𝜀_𝑡𝜀_𝑡−1)−∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1Δ_𝑛(𝜃₀,𝜎₀). (4.21) Let 𝑁^𝛽_𝑛(𝐴)={𝛽∶|(𝛽−𝛽₀)^𝑇𝑋_𝑛^1/2(𝜃₀)|²≤𝐴²𝑁},_𝜃𝑛(𝐴)={𝜃∶|𝜃−𝜃₀𝐴|≤Δ_𝑛(𝜃₀,𝜎₀)}. (4.22)(4.23)
In the first step, we will show that, for each 𝐴>0, sup_{𝜃∈𝑁𝜃𝑛(𝐴)}‖𝑋_𝑛^−1/2(𝜃₀)𝑋_𝑛(𝜃)𝑋_𝑛^−𝑇/2(𝜃₀)−𝐼_𝑑‖→0,𝑛→∞. (4.24) In fact, note that 𝑋_𝑛^−1/2(𝜃₀)𝑋_𝑛(𝜃)𝑋_𝑛^−𝑇/2(𝜃₀)−𝐼_𝑑=𝑋_𝑛^−1/2(𝜃₀)(𝑋_𝑛(𝜃)−𝑋_𝑛(𝜃₀))𝑋_𝑛^−𝑇/2(𝜃₀)=𝑋_𝑛^−1/2(𝜃₀)(𝑇₁+𝑇₂−𝑇₃)𝑋_𝑛^−𝑇/2(𝜃₀), (4.25) where 𝑇₁=^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))𝑥_𝑡−1(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇,𝑇₂=^𝑛_𝑡=2(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)𝑥^𝑇_𝑡−1,𝑇₃=^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))²𝑥_𝑡−1𝑥^𝑇_𝑡−1. (4.26)
Let 𝑢,𝑣∈𝑅^𝑑,|𝑢|=|𝑣|=1, and let 𝑢^𝑇_𝑛=𝑢^𝑇𝑋_𝑛^−1/2(𝜃₀),𝑣^𝑇_𝑛=𝑋_𝑛^−𝑇/2(𝜃₀)𝑣. By Cauchy-Schwartz inequality, Lemma 4.2, condition (A3), and noting that 𝜃∈𝑁^𝜃_𝑛(𝐴), we have that |𝑢^𝑇_𝑛𝑇₁𝑣_𝑛|=|^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))𝑢^𝑇_𝑛𝑥_𝑡−1(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇𝑣_𝑛|≤max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃)||^𝑛_𝑡=2𝑢^𝑇_𝑛𝑥_𝑡−1(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇𝑣_𝑛|≤max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃)|(^𝑛_𝑡=2𝑢^𝑇_𝑛𝑥_𝑡−1𝑥^𝑇_𝑡−1𝑢_𝑛)^1/2⋅(^𝑛_𝑡=2𝑣^𝑇_𝑛(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇𝑣_𝑛)^1/2≤max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃)|(^𝑛_𝑡=2𝑢^𝑇_𝑛𝑥_𝑡𝑥^𝑇_𝑡𝑢_𝑛)^1/2≤max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)||𝜃₀−𝜃|⋅√𝑛max_{1≤𝑡≤𝑛}(𝑥^𝑇_𝑡𝑋_𝑛⁻¹(𝜃₀)𝑥_𝑡)≤𝐶𝑛Δ_𝑛(𝜃₀,𝜎₀)𝑜(1)→0. (4.27) Here 𝜃=𝑎𝜃+(1−𝑎)𝜃₀ for some 0≤𝑎≤1. Similar to the proof of 𝑇₁, we easily obtain that |𝑢^𝑇_𝑛𝑇₂𝑣_𝑛|→0. (4.28) By Cauchy-Schwartz inequality, Lemma 4.2, condition (A3), and noting that 𝑁^𝜃_𝑛(𝐴), we have that |𝑢^𝑇_𝑛𝑇₃𝑣_𝑛|=|𝑢^𝑇_𝑛^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))²𝑥_𝑡−1𝑥^𝑇_𝑡−1𝑣_𝑛|≤max_{2≤𝑡≤𝑛}|𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃)|²(^𝑛_𝑡=2𝑢^𝑇_𝑛𝑥_𝑡𝑥^𝑇_𝑡𝑢_𝑛^𝑛_𝑡=2𝑣^𝑇_𝑛𝑥_𝑡𝑥^𝑇_𝑡𝑣_𝑛)^1/2≤𝑛max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)|²|𝜃₀−𝜃|²max_{1≤𝑡≤𝑛}(𝑥^𝑇_𝑡𝑋_𝑛⁻¹(𝜃₀)𝑥_𝑡)≤𝑛𝐴²Δ_𝑛(𝜃₀,𝜎₀)𝑜(1)→0. (4.29) Hence, (4.24) follows from (4.25)–(4.29).
In the second step, we will show that 𝑋_𝑛^−1/2(𝜃₀)∑^𝑛_𝑡=2(𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡+𝑓^′_𝑡(𝜃)𝜀_𝑡𝑥_𝑡−1−2𝑓_𝑡(𝜃)𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡−1)Δ_𝑛(𝜃₀,𝜎₀)→_𝑝0. (4.30) Note that 𝜀_𝑡=𝑦_𝑡−𝑥^𝑇_𝑡𝛽=𝑥^𝑇_𝑡(𝛽₀−𝛽)+𝑒_𝑡,𝜀_𝑡−𝑓_𝑡(𝜃₀)𝜀_𝑡−1=(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽)+𝜂_𝑡. (4.31) Consider 𝐽=^𝑛_𝑡=2(𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡+𝑓^′_𝑡(𝜃)𝜀_𝑡𝑥_𝑡−1−2𝑓_𝑡(𝜃)𝑓^′_𝑡(𝜃)𝜀_𝑡−1𝑥_𝑡−1)=^𝑛_𝑡=2(𝜀_𝑡−1𝑓^′_𝑡(𝜃)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)+𝑓^′_𝑡(𝜃)(𝜀_𝑡−𝑓_𝑡(𝜃₀)𝜀_𝑡−1)𝑥_𝑡−1)+2𝑓_𝑡(𝜃)(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))𝜀_𝑡−1𝑥_𝑡−1=𝑇₁+𝑇₂+𝑇₃+𝑇₄+2𝑇₅+2𝑇₆, (4.32) where 𝑇₁=^𝑛_𝑡=2𝑥^𝑇_𝑡−1𝑓^′_𝑡(𝜃)(𝛽₀−𝛽)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1),𝑇₂=^𝑛_𝑡=2𝑓^′_𝑡(𝜃)𝑒_𝑡−1(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1𝑇),₃=^𝑛_𝑡=2𝑓^′_𝑡(𝜃)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽)𝑥_𝑡−1,𝑇₄=^𝑛_𝑡=2𝑓^′_𝑡(𝜃)𝜂_𝑡𝑥_𝑡−1,𝑇₅=^𝑛_𝑡=2𝑓^′_𝑡(𝜃)(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))𝑥^𝑇_𝑡−1(𝛽₀−𝛽)𝑥_𝑡−1,𝑇₆=^𝑛_𝑡=2𝑓^′_𝑡(𝜃)(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))𝑒_𝑡−1𝑥_𝑡−1. (4.33) For 𝛽∈𝑁^𝛽_𝑛(𝐴) and each 𝐴>0, we have |(𝛽₀−𝛽)^𝑇𝑥_𝑡|²=(𝛽₀−𝛽)^𝑇𝑋_𝑛^1/2(𝜃₀)𝑋_𝑛^−1/2(𝜃₀)𝑥_𝑡𝑥^𝑇_𝑡𝑋_𝑛^−𝑇/2(𝜃₀)𝑋_𝑛^𝑇/2(𝜃₀)(𝛽₀−𝛽)≤max_{1≤𝑡≤𝑛}(𝑥^𝑇_𝑡𝑋_𝑛⁻¹(𝜃₀)𝑥_𝑡)(𝛽₀−𝛽)^𝑇𝑋_𝑛(𝜃₀)(𝛽₀−𝛽)≤𝐴²max_{1≤𝑡≤𝑛}(𝑥^𝑇_𝑡𝑋_𝑛⁻¹(𝜃₀)𝑥_𝑡). (4.34) By (4.34) and Lemma 4.2, we have sup_{𝛽∈𝑁𝛽𝑛(𝐴)}max_{1≤𝑡≤𝑛}|(𝛽₀−𝛽)^𝑇𝑥_𝑡|→0,𝑛→∞,𝐴>0. (4.35) Using Cauchy-Schwartz inequality, condition (A3), and (4.35), we obtain 𝑢^𝑇_𝑛𝑇₁=^𝑛_𝑡=2𝑢^𝑇_𝑛𝑥^𝑇_𝑡−1(𝛽₀−𝛽)𝑓^′_𝑡(𝜃)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)≤{^𝑛_𝑡=2(𝑥^𝑇_𝑡−1(𝛽₀−𝛽))²}^1/2×{^𝑛_𝑡=2(𝑓^′_𝑡²(𝜃)𝑢^𝑇_𝑛(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇𝑢_𝑛)²}^1/2≤√𝑛max_{1≤𝑡≤𝑛}|(𝛽₀−𝛽)^𝑇𝑥_𝑡|max_{1≤𝑡≤𝑛}|𝑓^′_𝑡√(𝜃)|=𝑜(𝑛). (4.36) Let 𝑎_𝑡𝑛=𝑢^𝑇_𝑛𝑓^′_𝑡(𝜃)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1). (4.37) Then from Lemma 4.2, max_{2≤𝑡≤𝑛}𝑎²_𝑡𝑛=max_{2≤𝑡≤𝑛}|𝑓^′_𝑡²(𝜃)|×max_{2≤𝑡≤𝑛}{𝑢^𝑇(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)𝑋_𝑛⁻¹(𝜃₀)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇𝑢}=𝑜(1). (4.38) By condition (A2) and (4.38), we have Var(𝑢^𝑇_𝑛𝑇₂)=Var(^𝑛_𝑡=2𝑎_𝑡𝑛𝑒_𝑡−1)=Var(^𝑛_𝑡=2𝑎_𝑡𝑛𝑒_𝑡−1)=Var{^𝑛−1_𝑗=1𝜂_𝑗(^𝑛_𝑡=𝑗+1𝑎_𝑡𝑛^{𝑡−𝑗−1}_𝑖=0𝑓_𝑡−𝑖(𝜃₀))}=𝜎²₀^𝑛−1_𝑗=1(^𝑛_𝑡=𝑗+1𝑎_𝑡𝑛^{𝑡−𝑗−1}_𝑖=0𝑓_𝑡−𝑖(𝜃₀))≤𝜎²₀max_{2≤𝑡≤𝑛}|𝑎_𝑡𝑛|^𝑛−1_𝑗=1(^{𝑡−𝑗−1}_𝑖=0𝑓_𝑡−𝑖(𝜃₀))≤𝛼𝜎²₀max_{2≤𝑡≤𝑛}|𝑎_𝑡𝑛|𝑛=𝑜(𝑛). (4.39) Thus by Chebychev inequality and (4.39), 𝑢^𝑇_𝑛𝑇₂=𝑜_𝑝(√𝑛). (4.40) Using the similar argument as 𝑇₁, we obtain that 𝑢^𝑇_𝑛𝑇₃=𝑜_𝑝(√𝑛). (4.41) Using the similar argument as 𝑇₂, we obtain that 𝑢^𝑇_𝑛𝑇₄=𝑜_𝑝(√𝑛),𝑢^𝑇_𝑛𝑇₆=𝑜_𝑝(√𝑛). (4.42) By Cauchy-Schwartz inequality, (4.35), and (4.27), we get 𝑢^𝑇_𝑛𝑇₅=^𝑛_𝑡=2𝑓^′_𝑡(𝜃)(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))𝑥^𝑇_𝑡−1(𝛽₀−𝛽)𝑢^𝑇_𝑡𝑥_𝑡−1≤{^𝑛_𝑡=2𝑓^′_𝑡²(𝜃)(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃))²(𝑥^𝑇_𝑡−1(𝛽₀−𝛽))^2𝑛_𝑡=2(𝑢^𝑇_𝑡𝑥_𝑡−1)²}^1/2≤max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)|max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)||𝜃₀−𝜃|⋅{^𝑛_𝑡=2(𝑥^𝑇_𝑡−1(𝛽₀−𝛽))^2𝑛_𝑡=2(𝑢^𝑇_𝑡𝑥_𝑡−1)²}^1/2𝐴≤𝐶Δ_𝑛(𝜃₀,𝜎₀)√𝑛𝑜(1)=𝑜(√𝑛). (4.43) Thus (4.30) follows immediately from (4.32), (4.36), and (4.40)–(4.43).
In the third step, we will show that ∑^𝑛_𝑡=2((𝑓^′_𝑡²(𝜃)+𝑓_𝑡(𝜃)𝑓_𝑡^′′(𝜃))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)𝜀_𝑡𝜀_𝑡−1)−∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1Δ_𝑛(𝜃₀,𝜎₀)→_𝑝0. (4.44) Let 𝐽=^𝑛_𝑡=2((𝑓^′_𝑡²(𝜃)+𝑓_𝑡(𝜃)𝑓_𝑡^′′(𝜃))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)𝜀_𝑡𝜀_𝑡−1)−^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1. (4.45) Then 𝐽=^𝑛_𝑡=2((𝑓^′_𝑡²(𝜃)+𝑓_𝑡(𝜃)𝑓_𝑡^′′(𝜃))𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)(𝑓_𝑡(𝜃)𝜀_𝑡−1+𝜂_𝑡)𝜀_𝑡−1)−^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1=^𝑛_𝑡=2{𝑓^′_𝑡²(𝜃)𝜀²_𝑡−1−𝑓_𝑡^′′(𝜃)𝜂_𝑡𝜀_𝑡−1−𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1}=^𝑛_𝑡=2{𝑓^′_𝑡²(𝑥^𝑇_𝑡−1(𝛽₀−𝛽))²+(𝑓^′_𝑡²(𝜃)−𝑓^′_𝑡²(𝜃₀))²𝑒²_𝑡−1}+^𝑛_𝑡=2{2𝑓^′_𝑡²(𝜃₀)𝑥^𝑇_𝑡−1(𝛽₀−𝛽)𝑒_𝑡−1−𝑓_𝑡^′′(𝜃)𝑥^𝑇_𝑡−1(𝛽₀−𝛽)𝜂_𝑡−1−𝑓_𝑡^′′(𝜃₀)𝜂_𝑡𝑒_𝑡−1}=𝑇₁+𝑇₂+2𝑇₃−𝑇₄−𝑇₅. (4.46) By (4.34), it is easy to show that 𝑇₁=𝑜(𝑛). (4.47) From condition (A3), (2.30), and (4.23), we obtain that |𝐸𝑇₂|=|(𝑓^′_𝑡²(𝜃)−𝑓^′_𝑡²(𝜃₀))²𝐸𝑒²_𝑡−1|=|𝑓_𝑡^′′(𝜃)(𝜃−𝜃₀𝑓)(^′𝑡(𝜃)+𝑓^′_𝑡(𝜃₀)𝑓^′_𝑡²(𝜃₀))𝑓^′_𝑡²(𝜃₀)𝐸𝑒²_𝑡−1|≤max_{2≤𝑡≤𝑛}|𝑓_𝑡^′′(𝑓𝜃)(^′𝑡(𝜃)+𝑓^′_𝑡(𝜃₀)𝑓^′_𝑡²(𝜃₀))||𝜃−𝜃₀|Δ_𝑛(𝜃₀,𝜎₀)≤max_{2≤𝑡≤𝑛}|𝑓_𝑡^′′(𝑓𝜃)(^′𝑡(𝜃)+𝑓^′_𝑡(𝜃₀)𝑓^′_𝑡²(𝜃₀)𝐴)|Δ_𝑛(𝜃₀,𝜎₀)Δ_𝑛(𝜃₀,𝜎₀)=𝑜(Δ_𝑛(𝜃₀,𝜎₀)). (4.48) Hence, by Markov inequality, 𝑇₂=𝑂_𝑝(Δ_𝑛(𝜃₀,𝜎₀)). (4.49) Using the similar argument as (4.40), we easily obtain that 𝑇₃=𝑜_𝑝(√𝑛). (4.50) By Markov inequality and noting that Var(𝑇₄)≤𝜎²₀𝑛max_{2≤𝑡≤𝑛}|𝑓_𝑡^′′(𝜃)|²max_{2≤𝑡≤𝑛}(𝑥^𝑇_𝑡(𝛽₀−𝛽))=𝑜(𝑛), (4.51) we have that 𝑇₄=𝑜_𝑝(√𝑛). (4.52) Using the similar argument as (4.6), we easily obtain that 𝑇₅=𝑂_𝑝(Δ_𝑛(𝜃₀,𝜎₀)). (4.53) Hence, (4.44) follows immediately from (4.46), (4.47), and (4.49)–(4.53).
This completes the proof of (4.15) from (4.21), (4.24), (4.30), and (4.44). To prove (4.16), we need to show that ∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1Δ_𝑛(𝜃₀,𝜎₀)→_𝑝1,𝑛→∞. (4.54) This follows immediately from (2.27) and Markov inequality.
Finally, we will prove (4.17). By (4.15) and (4.16), we have 𝐷_𝑛^−1/2𝐹(𝜑)𝐷_𝑛^−𝑇/2→_𝑝𝐼,𝑛→∞, (4.55) uniformly in 𝜑∈𝑁_𝑛(𝐴) for each 𝐴>0. Thus, by Lemma 4.3, 𝜆_min(𝐷_𝑛^−1/2𝐹(𝜑)𝐷_𝑛^−𝑇/2)→_𝑝1,𝑛→∞. (4.56) This implies (4.17).

Lemma 4.5 (see [49]). Let {𝑆_ni,F_ni,1≤𝑖≤𝑘_𝑛,𝑛≥1} be a zero-mean, square-integrable martingale array with differences 𝑋_ni, and let 𝜂² be an a.s. finite random variable. Suppose that ∑_𝑖𝐸{𝑋²_ni𝐼(|𝑋_ni|>𝜀)∣F_{𝑛,𝑖−1}}→_𝑝0, for all 𝜀→0, and ∑_𝑖𝐸{𝑋²_ni∣F_{𝑛,𝑖−1}}→_𝑝𝜂². Then 𝑆_𝑛𝑘𝑛=_𝑖𝑋_ni→_𝐷𝑍, (4.57) where the r.v. 𝑍 has characteristic function 𝐸{exp(−(1/2)𝜂²𝑡²)}.

5. Proof of Theorems

5.1. Proof of Theorem 3.1

Take 𝐴>0, let𝑀_𝑛(𝐴)={𝜑∈𝑅^𝑑+1∶(𝜑−𝜑₀)^𝑇𝐷_𝑛(𝜑−𝜑₀)=𝐴²} (5.1) be the boundary of 𝑁_𝑛(𝐴), and let 𝜑∈𝑀_𝑛(𝐴). Using (2.27) and Taylor expansion, for each 𝜎²>0, we haveΨ_𝑛(𝜑,𝜎²)=Ψ_𝑛(𝜑₀,𝜎²)+(𝜑−𝜑₀)^𝑇𝜕Ψ_𝑛(𝜑₀,𝜎²)+1𝜕𝜑2(𝜑−𝜑₀)^𝑇𝜕²Ψ_𝑛(𝜑₀,𝜎²)𝜕𝜑𝜕𝜑^𝑇(𝜑−𝜑₀)=1𝜎²Ψ_𝑛(𝜑₀,𝜎²)+(𝜑−𝜑₀)^𝑇𝑆_𝑛(𝜑₀1)−2𝜎²(𝜑−𝜑₀)^𝑇𝐹_𝑛(𝜑)(𝜑−𝜑₀), (5.2) where 𝜑=𝑎𝜑+(1−𝑎)𝜑₀ for some 0≤𝑎≤1.

Let 𝑄_𝑛(𝜑)=(1/2)(𝜑−𝜑₀)^𝑇𝐹_𝑛(𝜑)(𝜑−𝜑₀) and 𝑣_𝑛(𝜑)=(1/𝐴)𝐷_𝑛^𝑇/2(𝜑−𝜑₀). Take 𝑐>0 and 𝜑∈𝑀_𝑛(𝐴), and by (5.2) we obtain that𝑃{Ψ_𝑛(𝜑,𝜎²)≥Ψ_𝑛(𝜑₀,𝜎²)forsome𝜑∈𝑀_𝑛(𝐴)}≤𝑃{(𝜑−𝜑₀)^𝑇𝑆_𝑛(𝜑₀)≥𝑄_𝑛(𝜑),𝑄_𝑛(𝜑)>𝑐𝐴²forsome𝜑∈𝑀_𝑛(𝐴)}+𝑃{𝑄_𝑛(𝜑)≤𝑐𝐴²forsome𝜑∈𝑀_𝑛(𝐴)}≤𝑃{𝑣^𝑇_𝑛(𝜑)𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)>𝑐𝐴forsome𝜑∈𝑀_𝑛(𝐴)}+𝑃{𝑣^𝑇_𝑛(𝜑)𝐷_𝑛^−1/2𝐹_𝑛(𝜑)𝐷_𝑛^−𝑇/2𝑣_𝑛(𝜑)≤𝑐forsome𝜑∈𝑀_𝑛(𝐴)}≤𝑃{|𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)|>𝑐𝐴}+𝑃{inf_{𝜑∈𝑁𝑛(𝐴)}𝜆_min(𝐷_𝑛^−1/2𝐹_𝑛(𝜑)𝐷_𝑛^−𝑇/2)≤𝑐}. (5.3) By Lemma 4.1 and Chebychev inequality, we obtain𝑃{|𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)|>𝑐𝐴}≤Var(𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀))𝑐²𝐴²=𝜎²₀𝑐²𝐴². (5.4) Let 𝐴→∞, then 𝑐↓0, and using (4.17), we have𝑃{inf_{𝜑∈𝑁𝑛(𝐴)}𝜆_min(𝐷_𝑛^−1/2𝐹_𝑛(𝜑)𝐷_𝑛^−𝑇/2)≤𝑐}→0. (5.5) By (5.3)–(5.5), we havelim_𝐴→∞liminf_𝑛→∞𝑃{Ψ_𝑛(𝜑,𝜎²)<Ψ_𝑛(𝜑₀,𝜎²)∀𝜑∈𝑀_𝑛(𝐴)}=1. (5.6) By Lemma 4.3, 𝜆_min(𝑋_𝑛(𝜃₀))→∞ as 𝑛→∞. Hence 𝜆_min(𝐷_𝑛)→∞. Moreover, from (4.17), we have inf_{𝜑∈𝑁𝑛(𝐴)}𝜆_min(𝐹_𝑛(𝜑))→_𝑝∞. (5.7) This implies that Ψ_𝑛(𝜑,𝜎²) is concave on 𝑁_𝑛(𝐴). Noting this fact and (5.6), we getlim_𝐴→∞liminf_𝑛→∞𝑃{sup_{𝜑∈𝑀𝑛(𝐴)}Ψ_𝑛(𝜑,𝜎²)<Ψ_𝑛(𝜑₀,𝜎²),Ψ_𝑛(𝜑,𝜎²)isconcaveon𝑁_𝑛(𝐴)}=1. (5.8) On the event in the brackets, the continuous function Ψ_𝑛(𝜑,𝜎²) has a unique maximum in 𝜑 over the compact neighborhood 𝑁_𝑛(𝐴). Hence lim_𝐴→∞liminf_𝑛→∞𝑃{𝑆_𝑛(𝜑_𝑛(𝐴))=0foraunique𝜑_𝑛(𝐴)∈𝑁_𝑛(𝐴)}=1. (5.9) Moreover, there is a sequence 𝐴_𝑛→∞ such that 𝜑_𝑛=𝜑(𝐴_𝑛) satisfies liminf_𝑛→∞𝑃{𝑆_𝑛(𝜑_𝑛)=0and𝜑_𝑛maximizesΨ_𝑛(𝜑,𝜎²)uniquelyin𝑁_𝑛(𝐴)}=1. (5.10) Thus the 𝜑_𝑛=(𝛽_𝑛,𝜃_𝑛) is a QML estimator for 𝜑₀. It is clearly consistent, and lim_𝐴→∞liminf_𝑛→∞𝑃{𝜑_𝑛∈𝑁_𝑛(𝐴)}=1. (5.11) Since 𝜑_𝑛=(𝛽_𝑛,𝜃_𝑛) is a QML estimator for 𝜑₀, 𝜎²_𝑛 is a QML estimator for 𝜎²₀ from (2.9).

To complete the proof, we will show that 𝜎²_𝑛→𝜎²₀ as 𝑛→∞. If 𝜑_𝑛∈𝑁_𝑛(𝐴), then 𝛽_𝑛∈𝑁^𝛽_𝑛(𝐴) and 𝜃_𝑛∈𝑁^𝜃_𝑛(𝐴). By (2.12) and (2.1), we have𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1=(𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)+(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛𝑒_𝑡−1)). (5.12) By (2.9), (2.11), and (5.12), we have(𝑛−1)𝜎²_𝑛=^𝑛_𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)²=^𝑛_𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1){(𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)+(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1=)}_𝑛𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)(𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)+^𝑛_𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1)=^𝑛_𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1). (5.13) From (5.12), it follows that^𝑛_𝑡=2{(𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)}²=^𝑛_𝑡=2(𝜀_𝑡̂−𝑓_𝑡(𝜃_𝑛)𝜀̂_𝑡−1)²−2^𝑛_𝑡=2𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1)+^𝑛_𝑡=2(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1)². (5.14) From (2.2),^𝑛_𝑡=2(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1)²=^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)𝑒_𝑡−1−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1)²=^𝑛_𝑡=2𝜂²_𝑡+2^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝜂_𝑡𝑒_𝑡−1+^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))²𝑒²_𝑡−1. (5.15) By (5.13)–(5.15), we have(𝑛−1)𝜎²_𝑛=^𝑛_𝑡=2(𝑒_𝑡−𝑓_𝑡(𝜃_𝑛)𝑒_𝑡−1)²−^𝑛_𝑡=2((𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛))²=^𝑛_𝑡=2𝜂²_𝑡+2^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝜂_𝑡𝑒_𝑡−1+^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))²𝑒²_𝑡−1−^𝑛_𝑡=2((𝑥_𝑡−𝑓_𝑡(𝜃_𝑛)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛))²=𝑇₁+2𝑇₂+𝑇₃−𝑇₄. (5.16) By the law of large numbers,1𝑇𝑛−1¹=1𝑛−1^𝑛_𝑡=2𝜂²_𝑡→𝜎²₀,a.s.(𝑛→∞). (5.17) Since {(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝜂_𝑡𝑒_𝑡−1,𝐻_𝑡−1} is a martingale difference sequence with Var{(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝜂_𝑡𝑒_𝑡−1}=(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))²𝜎²₀𝐸𝑒²_𝑡−1,Var(𝑇₂)=^𝑛_𝑡=2𝐸((𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝜂_𝑡𝑒_𝑡−1)²=𝜎²₀^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))²𝐸𝑒²_𝑡−1=𝜎²₀^𝑛_𝑡=2𝑓^′_𝑡²(𝜃)𝑓^′_𝑡²(𝜃₀)(𝜃₀−𝜃_𝑛)²𝑓^′_𝑡²(𝜃₀)𝐸𝑒²_𝑡−1≤𝜎²₀max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)𝑓^′_𝑡(𝜃₀)|²|(𝜃₀−𝜃_𝑛)|²Δ_𝑛(𝜃₀,𝜎²₀)≤𝐶𝐴². (5.18) By Chebychev inequality, we have1𝑇𝑛−1²→_𝑝0,(𝑛→∞). (5.19) By Markov inequality and noting that 𝐸𝑇₃≤𝐶𝐴², we obtain that1𝑇𝑛−1³→_𝑝0,(𝑛→∞). (5.20) Write𝑇₄=^𝑛_𝑡=2((𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)+(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝑥^𝑇_𝑡−1(𝛽₀−𝛽_𝑛))²=(𝛽₀−𝛽_𝑛)^𝑇𝑋_𝑛(𝜃₀)(𝛽₀−𝛽_𝑛)+^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))²(𝑥^𝑇_𝑡−1(𝛽₀−𝛽_𝑛))²+2^𝑛_𝑡=2(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇(𝛽₀−𝛽_𝑛)(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))𝑥^𝑇_𝑡−1(𝛽₀−𝛽_𝑛)=𝐼₁+𝐼₂+2𝐼₃. (5.21) Noting that 𝛽∈𝑁^𝛽_𝑛(𝐴), we have𝐼₁=𝑂_𝑝(1),(𝑛→∞). (5.22) By (4.34) and condition (A3), we have|𝐼₂|≤max_{2≤𝑡≤𝑛}|𝑥^𝑇_𝑡−1(𝛽₀−𝛽_𝑛)|²𝑛max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)|²|𝜃₀−𝜃_𝑛|²=𝑜(1),(𝑛→∞). (5.23) By (4.34), condition (A3), and Cauchy-Schwartz inequality, we have|𝐼₃|²≤^𝑛_𝑡=2(𝛽₀−𝛽_𝑛)^𝑇(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)^𝑇(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1)(𝛽₀−𝛽_𝑛)⋅^𝑛_𝑡=2(𝑓_𝑡(𝜃₀)−𝑓_𝑡(𝜃_𝑛))²(𝑥^𝑇_𝑡−1(𝛽₀−𝛽_𝑛))²≤(𝛽₀−𝛽_𝑛)^𝑇𝑋_𝑛(𝜃₀)(𝛽₀−𝛽_𝑛)𝑛max_{2≤𝑡≤𝑛}|𝑓^′_𝑡(𝜃)|²|𝜃₀−𝜃_𝑛|²⋅max_{2≤𝑡≤𝑛}|𝑥^𝑇_𝑡−1(𝛽₀−𝛽_𝑛)|²≤𝑂_𝑝𝐴(1)𝑛²Δ_𝑛(𝜃₀,𝜎²₀)𝑜(1)=𝑜_𝑝(1). (5.24) By (5.21)–(5.24), we obtain1𝑇𝑛−1⁴→_𝑝0,(𝑛→∞). (5.25) From (5.16), (5.17), (5.19), (5.20), and (5.25), we have 𝜎²_𝑛→𝜎²₀. We therefore complete the proof of Theorem 3.1.

5.2. Proof of Theorem 3.2

It is easy to know that 𝑆_𝑛(𝜑_𝑛)=0 and 𝐹_𝑛(𝜑_𝑛) is nonsingular from Theorem 3.1. By Taylor's expansion, we have0=𝑆_𝑛(𝜑_𝑛)=𝑆_𝑛(𝜑₀)−𝐹_𝑛(𝜑_𝑛)(𝜑_𝑛−𝜑₀). (5.26) Since 𝜑_𝑛∈𝑁_𝑛(𝐴), also 𝜑_𝑛∈𝑁_𝑛(𝐴). By (4.15), we have𝐹_𝑛(𝜑_𝑛)=𝐷_𝑛^1/2(Φ_𝑛+𝐴_𝑛)𝐷_𝑛^𝑇/2, (5.27) where 𝐴_𝑛 is a symmetric matrix with 𝐴_𝑛→_𝑝0. By (5.26) and (5.27), we have𝐷_𝑛^𝑇/2(𝜑_𝑛−𝜑₀)=𝐷_𝑛^𝑇/2𝐹_𝑛⁻¹(𝜑_𝑛)𝑆_𝑛(𝜑₀)=(Φ_𝑛+𝐴_𝑛)⁻¹𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀). (5.28) Similar to (5.27), we have𝐹_𝑛(𝜑_𝑛)=𝐷_𝑛^1/2(Φ_𝑛+𝐴_𝑛)𝐷_𝑛^𝑇/2=(𝐷_𝑛^1/2(Φ_𝑛+𝐴_𝑛)^1/2)((Φ_𝑛+𝐴_𝑛)^𝑇/2𝐷_𝑛^𝑇/2)=𝐹_𝑛^1/2(𝜑_𝑛)𝐹_𝑛^𝑇/2(𝜑_𝑛). (5.29) Here 𝐴_𝑛→_𝑝0. By (5.28), (5.29), and noting that 𝜎²_𝑛→_𝑝𝜎²₀ and 𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)=𝑂_𝑝(1), we obtain that𝐹_𝑛^𝑇/2(𝜑_𝑛)(𝜑_𝑛−𝜑₀)𝜎_𝑛=(Φ_𝑛+𝐴_𝑛)^1/2(Φ_𝑛+𝐴_𝑛)⁻¹𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)𝜎_𝑛=Φ_𝑛^−1/2𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)𝜎₀+𝑜_𝑝(1). (5.30) From (2.7) and (2.8), we have𝑆_𝑛(𝜑₀)𝜎₀=1𝜎₀(^𝑛_𝑡=2𝜂_𝑡(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1),^𝑛_𝑡=2𝑓^′_𝑡(𝜃₀)𝜂_𝑡𝑒_𝑡−1). (5.31) From (2.29) and (4.18), we haveΦ_𝑛^−1/2𝐷_𝑛^−1/2=𝐼_𝑑00Δ_𝑛(𝜃₀,𝜎₀)∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1𝑋_𝑛^−1/2(𝜃₀01)0Δ_𝑛(𝜃₀,𝜎₀)=𝑋_𝑛^−1/2(𝜃₀01)0∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1. (5.32) By (5.30)–(5.32), we haveΦ_𝑛^−1/2𝐷_𝑛^−1/2𝑆_𝑛(𝜑₀)𝜎₀=1𝜎₀(^𝑛_𝑡=2𝜂_𝑡𝑋_𝑛^−1/2(𝜃₀)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1∑),^𝑛𝑡=2𝑓^′_𝑡(𝜃₀)𝜂_𝑡𝑒_𝑡−1∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1). (5.33)

Let 𝑢∈𝑅^𝑑 with |𝑢|=1, and 𝑎_𝑡𝑛=𝑢𝑋_𝑛^−1/2(𝜃₀)(𝑥_𝑡−𝑓_𝑡(𝜃₀)𝑥_𝑡−1). Then max_{2≤𝑡≤𝑛}𝑎_𝑡𝑛=𝑜(1), and we will consider the limiting distribution of the following 2-vector:1𝜎₀(^𝑛_𝑡=2𝑎_𝑡𝑛𝜂_𝑡,∑^𝑛_𝑡=2𝑓^′_𝑡(𝜃₀)𝜂_𝑡𝑒_𝑡−1∑^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1). (5.34) By Cramer-Wold device, it will suffice to find the asymptotic distribution of the following random:^𝑛_𝑡=2𝜂_𝑡(𝑢₁𝑎_𝑡𝑛𝜎₀+𝑢₂𝑓^′_𝑡(𝜃₀)𝑒_𝑡−1𝜎₀Δ_𝑛(𝜃₀,𝜎₀))=^𝑛_𝑡=2𝜂_𝑡𝑚_𝑡(𝑛), (5.35) where (𝑢₁,𝑢₂)∈𝑅² with 𝑢²₁+𝑢²₂=1. Note that 𝐸{𝜂_𝑡𝑚_𝑡(𝑛)∣𝐻_𝑡−1}=0, so the sums in (5.35) are partial sums of a martingale triangular array with respect to {𝐻_𝑡}, and we will verify the Lindeberg conditions for their convergence to normality as follows: ^𝑛_𝑡=2𝐸(𝜂²_𝑡𝑚²_𝑡(𝑛)∣𝐻_𝑡−1)=𝜎²₀^𝑛_𝑡=2𝑚²_𝑡(𝑛)=𝑢²₁^𝑛_𝑡=2𝑎²_𝑡𝑛+2𝑢₁𝑢₂Δ_𝑛(𝜃₀,𝜎₀)^𝑛𝑡=2𝑓^′_𝑡(𝜃₀)𝑎_𝑡𝑛𝑒_t−1+𝑢²₂Δ_𝑛(𝜃₀,𝜎₀)^𝑛_𝑡=2𝑓^′_𝑡²(𝜃₀)𝑒²_𝑡−1=𝑢²₁+𝑜_𝑝(1)+𝑢²₂=1+𝑜_𝑝(1). (5.36) Noting that max_{1≤𝑡≤𝑛}(𝑒²_𝑡/𝑛)=𝑜_𝑝(1) and max_{2≤𝑡≤𝑛}𝑎_𝑡𝑛=𝑜(1), we obtain thatmax_{2≤𝑡≤𝑛}|𝑚_𝑡(𝑛)|≤max_{2≤𝑡≤𝑛}|𝑢₁𝑎_𝑡𝑛𝜎₀|+max_{2≤𝑡≤𝑛}|𝑢₂𝑓^′_𝑡(𝜃₀)𝑒_𝑡−1𝜎₀Δ_𝑛(𝜃₀,𝜎₀)|=𝑜_𝑝(1). (5.37) Hence, for given 𝛿>0, there is a set whose probability approaches 1 as 𝑛→∞ on which max_{2≤𝑡≤𝑛}|𝑚_𝑡(𝑛)|≤𝛿. On this event, for any 𝑐>0,^𝑛_𝑡=2𝐸{𝜂²_𝑡𝑚²_𝑡(𝑛)𝐼(|𝜂_𝑡𝑚_𝑡(𝑛)|>𝑐)∣𝐻_𝑡−1}=^∞_𝑡=2^∞_𝑐𝑦²𝑑𝑝{|𝜂_𝑡𝑚_𝑡(𝑛)|≤𝑦∣𝐻_𝑡−1}=^𝑛_𝑡=2𝑚²_𝑡(𝑛)^∞_{𝑐/𝑚𝑡(𝑛)}𝑦²𝑑𝑝{|𝜂₁|≤𝑦∣𝐻_𝑡−1}≤^𝑛_𝑡=2𝑚²_𝑡(𝑛)^∞_𝑐/𝛿𝑦²𝑑𝑝{|𝜂₁|≤𝑦∣𝐻_𝑡−1}=𝑜_𝛿^𝑛_𝑡=2𝑚²_𝑡(𝑛)=𝑜_𝛿𝑂_𝑝(1)→0,𝑛→∞. (5.38) Here 𝑜_𝛿→0 as 𝛿→0. This verifies the Lindeberg conditions, and by Lemma 4.5^𝑛_𝑡=2𝜂_𝑡𝑚_𝑡(𝑛)→_𝐷𝑁(0,1). (5.39) Thus we complete the proof of Theorem 3.2.

6. Numerical Examples

6.1. Simulation Example

We will simulate a regression model (1.1), where 𝑥_𝑡=𝑡/20,𝛽=3.5,𝑡=1,2,…,100, and the random errors𝜀_𝑡=12𝜋sin(2𝑡+𝜃)𝜀_𝑡−1+𝜂_𝑡, (6.1) where 𝜃=2,𝜂_𝑡∼𝑁(0,1).

By the ordinary least squares method, we obtain the least squares estimators 𝛽_𝐿𝑆=3.5136, and 𝜎²_𝐿𝑆=1.0347. So we take 𝛽⁽⁰⁾=3.5136,𝜎⁽⁰⁾²=1.0347, 𝜃⁽⁰⁾=𝜋/2, and 𝛿=0.01. Therefore, using the iterative computing method, we obtain 𝜃⁽¹⁾=(1.0159,3.5076,1.6573)^𝑇,𝜃⁽²⁾=(1.0283,3.5076,1.7599)^𝑇. (6.2) Since ‖𝜃⁽²⁾−𝜃⁽¹⁾‖/‖𝜃⁽¹⁾‖<0.01, the QML estimators of 𝛽,𝜃, and 𝜎² are given by 𝜎²_𝑛=1.0283,𝛽_𝑛=3.5076,𝜃_𝑛=1.7599. (6.3)

These values closely approximate their true values, so our method is successful, especially in estimating the parameters 𝛽 and 𝜎².

6.2. Empirical Example

We will use the data studied by Fuller in [14]. The data pertain to the consumption of spirits in the United Kingdom from 1870 to 1983. The dependent variable 𝑦_𝑡 is the annual per capita consumption of spirits in the United Kingdom. The explanatory variables 𝑥_𝑡1 and 𝑥_𝑡2 are per capita income and price of spirits, respectively, both deflated by a general price index. All data are in logarithms. The model suggested by Prest can be written as𝑦_𝑡=𝛽₀+𝛽₁𝑥_𝑡1+𝛽₂𝑥_𝑡2+𝛽₃𝑥_𝑡3+𝛽₄𝑥_𝑡4+𝜀_𝑡, (6.4) where 1869 is the origin for 𝑡, 𝑥_𝑡3=𝑡/100,and𝑥_𝑡4=(𝑡−35)²/10⁴, and assuming that 𝜀_𝑡 is a stationary time series.

Fuller [14] obtained the estimated generalized least squares equation𝑦_𝑡=2.36+0.72𝑥_𝑡1−0.80𝑥_𝑡2−0.81𝑥_𝑡3−0.92𝑥_𝑡4,𝜀_𝑡=0.7633𝜀_𝑡−1+𝜂_𝑡, (6.5) where 𝜂_𝑡 is a sequence of uncorrelated (0,0.000417) random variables.

Using our method, we obtain the following models:𝑦_𝑡=2.3607+0.7437𝑥_𝑡1−0.8210𝑥_𝑡2−0.7857𝑥_𝑡3−0.9178𝑥_𝑡4,𝜀_𝑡=(0.70054+0.00424𝑡)𝜀_𝑡−1+𝜂_𝑡, (6.6) where 𝜂_𝑡 is a sequence of uncorrelated (0,0.000413) random variables.

By the models (6.6), the residual mean square is 0.000413, which is smaller than 0.000417 calculated by the models (6.5).

From the above examples, it can be seen that our method is successful and valid. However, a further discussion of fitting the function 𝑓_𝑡(𝜃) is needed so that we can find a good method to use in practical applications.

Acknowledgments

The author would like to thank an anonymous referee for useful comments and suggestions. The paper was supported by the Key Project of Chinese Ministry of Education (no. 209078) and Scientific Research Item of Department of Education, Hubei (no. D20092207).