Moment Conditions Selection Based on Adaptive Penalized Empirical Likelihood

Abstract

Empirical likelihood is a very popular method and has been widely used in the fields of artificial intelligence (AI) and data mining as tablets and mobile application and social media dominate the technology landscape. This paper proposes an empirical likelihood shrinkage method to efficiently estimate unknown parameters and select correct moment conditions simultaneously, when the model is defined by moment restrictions in which some are possibly misspecified. We show that our method enjoys oracle-like properties; that is, it consistently selects the correct moment conditions and at the same time its estimator is as efficient as the empirical likelihood estimator obtained by all correct moment conditions. Moreover, unlike the GMM, our proposed method allows us to carry out confidence regions for the parameters included in the model without estimating the covariances of the estimators. For empirical implementation, we provide some data-driven procedures for selecting the tuning parameter of the penalty function. The simulation results show that the method works remarkably well in terms of correct moment selection and the finite sample properties of the estimators. Also, a real-life example is carried out to illustrate the new methodology.

1. Introduction

As Xie et al. [1] show, growing attention is being paid to the fields of artificial intelligence (AI) and data mining as tablets and mobile application and social media dominate the technology landscape. Moment conditions often appear in the study of artificial intelligence (AI) and data mining. We all know that empirical likelihood is a very practical tool for the study of moment conditions. When a parametric likelihood function is not specified for a model, estimating equations may provide an alternative instrument for statistical inference. For example, let 𝑍₁,…,𝑍_𝑛 be independent and identically distributed random vectors from a distribution, and let 𝜃∈ℝ^𝑝 be a vector of unknown parameters. Suppose that the information of the distribution is available in the form of an unbiased estimating function 𝑔(𝑧;𝜃)={𝑔₁(𝑧;𝜃),…,𝑔_𝑟(𝑧;𝜃)}^𝑇 satisfying 𝐸{𝑔(𝑍;𝜃)}=0 and 𝑟≥𝑝. When 𝑟=𝑝, 𝜃 can be estimated by solving the estimating equations 0=𝑛⁻¹∑^𝑟_𝑖=1𝑔(𝑍_𝑖;𝜃). Allowing 𝑟>𝑝 provides a useful device to combine available information for improving estimation efficiency, but directly solving 0=𝑛⁻¹∑^𝑟_𝑖=1𝑔(𝑍_𝑖;𝜃) may not be feasible.

The generalized method of moments (GMM) and empirical likelihood (EL) are two popular methodologies for estimating the parameters in the structural equations. As was introduced by Hansen [2], the GMM estimator 𝜃_𝑛 is defined as 𝜃_𝐺=argmin_𝜃∈Θ∑^𝑛_𝑖=1𝑔(𝑍_𝑖,𝜃)√𝑛^𝑇𝑊_𝑛∑^𝑛_𝑖=1𝑔𝑍_𝑖,𝜃√𝑛, (1) where Θ is the parameter space where 𝜃₀ lies and 𝑊_𝑛 is a given 𝑞×𝑞 weight matrix. Unlike the GMM, the EL uses likelihood to optimally combine information given in the estimating equations. More specifically, the estimator 𝜃_𝐸 is defined by maximizing the following empirical likelihood: 𝐿(𝜃)=sup^𝑛_𝑖=1𝑛𝜔_𝑖∶𝜔_𝑖≥0,^𝑛_𝑖=1𝜔_𝑖=1,^𝑛_𝑖=1𝜔_𝑖𝑔𝑍_𝑖;𝜃=0. (2) The estimator 𝜃_𝐸 is optimal in the sense of Godambe and Heyde [3]. It is known that maximizing (2) is equivalent to minimizing the empirical likelihood ratio 𝑙(𝜃)=−log𝐿(𝜃)−𝑛log(𝑛)=^𝑛_𝑖=1log1+𝜆^𝑇_𝜃𝑔𝑍_𝑖;𝜃, (3) where 𝜆_𝜃 satisfies 𝑛⁻¹∑^𝑛_𝑖=1𝑔(𝑍_𝑖;𝜃){1+𝜆^𝑇_𝜃𝑔(𝑍_𝑖;𝜃)}⁻¹=0.

Both the GMM and the EL have been successfully used for parameter estimation and variable selection in general estimating equations. The statistical properties of the GMM and the EL estimators rely heavily on the quality of these moment conditions. The strong and valid moment conditions can help to reduce finite-sample bias and improve efficiency of the GMM and the EL estimators. However, when some moment conditions are misspecified, the GMM and the EL estimators may be inconsistent. In this paper, we are interested in estimating some unknown parameter 𝜃₀ identified by a set, set-1, of some moment restrictions which can be used to estimate 𝜃₀ consistently. Meanwhile, it is supposed that there is another set, set-2, of possibly misspecified moment conditions. When the moment conditions in set-2 (or some of them) are correctly specified, including them into estimation equations can improve the asymptotic efficiency of the estimator for 𝜃₀. However, if they are misspecified, then using these moment conditions will lead to inconsistent estimation. Hence, whenever an empirical researcher has a set of moment conditions and there is no prior information about their validity, it is important to have some procedures to select the correctly specified moment conditions in that set and include them in the estimation equations.

Note that both the GMM estimators and the EL estimators are defined through moment restrictions. They generally have the same asymptotic distributions, but possibly different higher order asymptotic properties; see Newey and Smith [4] and Schennach [5]. As discussed in Newey and Smith [4], the small sample performance of the GMM is poor in some applications and the EL has advantages over the GMM estimators. First, unlike GMM, the asymptotic bias of the EL estimator does not grow with the number of moment restrictions. Consequently, with many moment conditions, the bias of EL will be less than the bias of GMM. The relatively low asymptotic bias of the EL indicates that it is an important alternative to the GMM. Second, unlike the GMM, the EL does not require weight matrix estimation and is invariant to nonsingular linear transformations of the moment conditions. The third theoretical advantage of EL is that after it is bias corrected, it is higher efficient relative to the GMM bias corrected estimators. The reason is that the biased corrected EL estimators inherit the higher order property of maximum likelihood estimators.

Inspired by the idea of Liao [6] and considering the above advantages of the EL estimators relative to the GMM estimators, we propose a novel method for moment selection and parameter estimation simultaneously. The new method attaches a penalty function to the EL criterion and the resulting estimator of 𝜃₀ is then called the EL shrinkage estimator. Our method embeds the moment selection in EL estimation and once a certain moment condition is selected, it will be automatically included into estimating 𝜃₀. Hence, our method not only selects the correct moment conditions in the set-2 in one step but also deals with the moment selection and efficient estimation simultaneously. Under some regularity conditions, we show that the EL shrinkage estimator of 𝜃₀ is root-𝑛 consistent and asymptotically normal. Moreover, we show that consistent moment selection is automatically achieved in the penalized EL estimation and the EL shrinkage estimator of 𝜃₀ is asymptotically oracle-efficient (i.e., as efficient as the oracle EL estimator based on all valid moment conditions). Unlike the GMM, our proposed method allows us to carry out confidence regions for parameters included in the model without estimating the covariances of the estimators.

The rest of the paper is organized as follows. In Section 2, based on the EL and penalty method, the parameter estimation and moment condition selection are introduced. The theoretical properties of the EL shrinkage estimators and the empirical likelihood ratio are presented in Section 3. Section 4 provides simple and data-driven procedures of selecting the tuning parameters. Simulation studies and a real-life example are given in Section 5. Proofs and the technical derivations are included in the appendix.

2. Methodology

Suppose that we are interested in estimating some unknown parameter 𝜃₀ identified by the following moment restrictions: 𝐸𝑔_𝑞𝑍,𝜃₀=0, (4) where 𝑍 is a 𝑑_𝑧-dimensional random vector, 𝜃₀ is a 𝑑_𝜃-dimensional parameter vector, the subscript 𝑞 of 𝑔_𝑞(⋅,⋅) denotes the number of moment conditions, and 𝑔_𝑞(⋅,⋅)∶𝑅^𝑑𝑧×𝑅^𝑑𝜃→𝑅^𝑞. The moment conditions in (4) can be used to estimate 𝜃₀ consistently. Suppose there is another set of possibly misspecified moment conditions as 𝐸𝑔_𝑘𝑍,𝜃=₀^?0, (5) where “=^?” signifies that equality may hold for some elements but not others, the subscript 𝑘 of 𝑔_𝑘(⋅,⋅) denotes the number of moment conditions, and 𝑔_𝑘(⋅,⋅)∶𝑅^𝑑𝑧×𝑅^𝑑𝜃→𝑅^𝑘. The goal of this paper is to consistently select the correct moment conditions in the set-2 and automatically include them into the empirical likelihood estimation to improve the efficiency of estimating 𝜃₀.

To incorporate moment selection into the estimation procedure, we first introduce a set of auxiliary unknown parameters 𝛽₀ and reparametrize the moment conditions in the set-2 as 𝐸𝑔_𝑘𝑍,𝜃₀−𝛽₀=0. (6) From (6), we see that if the 𝑗th (𝑗=1,…,𝑘) moment condition in (5) is correctly specified (or misspecified), then 𝛽_0,𝑗=0(or𝛽_0,𝑗≠0). Hence, the zero/nonzero components in 𝛽₀ can be used to identify the correctly specified/misspecified moment conditions in the set-2 and consistent moment selection is equivalent to consistent selection of the zero components in 𝛽₀. We thus stack the moment conditions in (4) and (6) to get 𝐸𝜌𝑍,𝜃₀,𝛽₀𝑔≡𝐸^𝑞𝑧,𝜃₀𝑔_𝑘𝑧,𝜃₀−𝛽₀=0. (7)

Let {𝑍_𝑖}_𝑖≤𝑛 be a sample of 𝑍. The EL shrinkage estimator (𝜃^𝒮_𝑛,𝛽^𝒮_𝑛) of (𝜃₀,𝛽₀) is defined as 𝜃^𝒮_𝑛,𝛽^𝒮_𝑛=argmin^𝑛_{𝜃,𝛽∈Θ×ℬ𝑖=1}log1+𝜆^𝑇_𝜃𝜌𝑍_𝑖,𝜃,𝛽+𝑛^𝑘_𝑗=1𝑃_𝜏𝑛𝛽_𝑗, (8) where 𝜆_𝜃 satisfies 𝑛⁻¹∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽){1+𝜆^𝑇_𝜃𝜌(𝑍_𝑖,𝜃,𝛽)}⁻¹=0, Θ×ℬ is the parameter space where (𝜃₀,𝛽₀) lies, and 𝜏_𝑛 is the tuning parameter in some general penalty function 𝑃_𝜏𝑛(⋅). The success of our method in simultaneous moment selection and efficient estimation relies on the “oracle properties” of the shrinkage techniques. That is to say, if 𝛽_0,𝑗=0, for some 𝑗∈{1,…,𝑘}, our method will estimate 𝛽_0,𝑗 as zero with probability approaching 1 (w.p.a.1.). When 𝛽_0,𝑗 is estimated as zero w.p.a.1., the information contained in the 𝑗th moment condition of (5) is automatically used in estimating 𝜃₀ w.p.a.1. On the other hand, the nonzero components in 𝛽₀ are consistently estimated and their estimators are nonzero w.p.a.1. Hence, our method can consistently distinguish the zero and nonzero components in 𝛽₀ and is consistent in moment selection. Moreover, it estimates 𝜃₀ as if we knew all potentially correct moment conditions in the set-2.

There are many popular choices for the penalty function 𝑃_𝜏𝑛(⋅). For example, the bridge penalty is defined as 𝑃_𝜏𝑛(𝛽)=𝜏_𝑛|𝛽|^𝛾, where 𝛾∈(0,1); the adaptive Lasso penalty is defined as 𝑃_𝜏𝑛(𝛽)=𝜏_𝑛𝜔_𝛽|𝛽|, where 𝜔_𝛽=|𝛽_𝑛|^−𝜔  (𝜔>0) and 𝛽_𝑛 is some first-step consistent estimator of 𝛽₀; and the smoothly clipped absolute deviation (SCAD) penalty is defined as 𝑃_𝜏𝑛𝛽=𝜏_𝑛𝛽𝐼0≤𝛽≤𝜏_𝑛+𝑎𝜏_𝑛𝛽−𝛽²+𝜏²_𝑛/2𝜏𝑎−1×𝐼_𝑛≤𝛽≤𝑎𝜏_𝑛+(𝑎+1)𝜏²_𝑛2𝐼𝛽≥𝑎𝜏_𝑛, (9) where 𝑎 is some positive real number strictly larger than 2. The above penalty functions differ in their empirical implementations, although the related EL shrinkage estimators may have the same asymptotic properties (see the results in Section 3). We focus on the SCAD penalty in this paper.

3. Asymptotic Theory

This section establishes the oracle property of the adaptive empirical likelihood (EL) shrinkage estimator. We state our theorems here, but their proofs are relegated to the appendix.

Let 𝒮_𝛽≡{𝑗∶𝛽_0,𝑗≠0,𝑗=1,…,𝑘} and 𝒮_𝛽,𝑛≡{𝑗∶𝛽_𝑛,𝑗≠0,𝑗=1,…,𝑘} be the index set of the nonzero components in 𝛽₀ and 𝛽_𝑛, respectively. For ease of notation, we sort the elements in 𝛽₀ in the following way: 𝛽^′₀=(𝛽^′_0,−,𝛽^′_0,+), where 𝛽_0,−=0 and 𝛽_0,+≠0. Let 𝑘₀ denote the number of valid moment conditions in the set-2. By definition, we know that 𝛽_0,− and 𝛽_0,+ are 𝑘₀ and 𝑘−𝑘₀ dimensional vectors, respectively. We define 𝛼^′=(𝜃^′,𝛽^′) and 𝑚(𝛼)𝜌𝑔≡𝐸𝑍,𝜃,𝛽≡𝐸^𝑞()𝑔𝑍,𝜃_𝑘()≡𝐺𝑍,𝜃−𝛽^𝑞(𝜃)𝐺_𝑘(𝜃)−𝛽 (10) for any (𝜃,𝛽)∈Θ×ℬ. We use ‖⋅‖ to denote the Euclidean norm in the Euclidean space.

We first present and discuss the sufficient conditions for consistency of 𝛼_𝑛.

Assumption 1. (i) 𝐸[𝑔_𝑙(𝑧,𝜃₀)𝑔^𝑇_𝑙(𝑧,𝜃₀)] is positive definite for 𝑙=𝑞,𝑘;
(ii) 𝜕𝑔_𝑙(𝑧,𝜃)/𝜕𝜃 and 𝜕²𝑔_𝑙(𝑧,𝜃)/𝜕𝜃𝜕𝜃^𝑇 are continuous in a neighborhood of the true value 𝜃₀ for 𝑙=𝑞,𝑘;
(iii) ‖𝜕𝑔_𝑙(𝑧,𝜃)/𝜕𝜃‖, ‖𝜕²𝑔_𝑙(𝑧,𝜃)/𝜕𝜃𝜕𝜃^𝑇‖, and ‖𝑔_𝑙(𝑧,𝜃)‖³ are bounded by some integrable function 𝐻(𝑧) in this neighborhood of the true value 𝜃₀ for 𝑙=𝑞,𝑘, and the rank of 𝐸[𝜕𝑔_𝑞+𝑘(𝑧,𝜃)/𝜕𝜃] is 𝑝;
(iv) 𝐸{sup_{(𝜃,𝛽)∈Θ×ℬ}(‖𝜌(𝑍_𝑖,𝜃,𝛽)‖(𝑞+𝑘)^−1/2)^𝜅}<∞ for some 𝜅>10/3 when 𝑛 is large.

Assumption 1 is similar to those of Qin and Lawless [7]. We emphasize that the dimensionality 𝑞+𝑘 cannot exceed 𝑛 because the convex hull of {𝑔_𝑞+𝑘(𝑍_𝑖,𝜃₀)}^𝑛_𝑖=1 is at most at a subset in ℝ^𝑛.

Assumption 2. (i) 𝐺_𝑘(𝜃) is continuous in 𝜃 and for any 𝜀>0 there exists some 𝛿_𝜀 such that inf_{{𝜃∈Θ∶||𝜃−𝜃0||≥𝜀}}𝐺_𝑞(𝑍,𝜃)>𝛿_𝜀; (11)
(ii) the following uniform law of large numbers (ULLN) holds: sup_𝜃∈Θ𝑛^𝑛−1_𝑖=1𝑔_𝑙𝑍_𝑖,𝜃−𝐸𝑔_𝑙𝑍_𝑖,𝜃=𝑜_𝑝(1) (12) for 𝑙=𝑞,𝑘;
(iii) 𝑊₀=(1/2)𝐸[𝜌(𝑍,𝜃,𝛽)𝜌^𝑇(𝑍,𝜃,𝛽)] is positive definite; 𝑊_𝑛 is a symmetric and real matrix and its eigenvalues are bounded from below and above by some fixed finite positive constants for all 𝑛;
(iv) the penalty function 𝑃_𝜏𝑛(⋅) is nonnegative and 𝑃_𝜏𝑛(𝛽_0,𝑗)=𝑜_𝑝(1) for 𝑗=1,…,𝑘.

Condition (11) in Assumption 2(i) is the identifiable uniqueness condition for 𝜃₀. By definition, 𝛽₀=𝐺_𝑘(𝜃₀); thus, 𝛽₀ is locally uniquely identified under (11) and the continuity of 𝐺_𝑘(𝜃). Assumptions 2(ii) and (iii) are two conditions whose application range is very wide because it does not specify the data structure, the properties of the moment functions, and the form of the weight matrix 𝑊_𝑛. It is clear that when 𝑊_𝑛 is an identity matrix this assumption holds automatically. We choose 𝑊_𝑛=(1/2𝑛)∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)𝜌^𝑇(𝑍_𝑖,𝜃,𝛽) usually. Assumption 1(iv) implies that the shrinkage effect of the penalty function on the moment selection coefficients (i.e. 𝛽_0,+) converges in probability to zero as 𝑛→∞. It states that the nonzero parameters cannot converge to zero too fast. This is reasonable because otherwise the noise is too strong. This condition includes the case that 𝑃_𝜏𝑛(𝛽_0,𝑗)=0 for 𝑗=1,…,𝑘 as a special example.

Assumption 3. (i) The following functional central limit theorem (FCLT) holds: sup_𝜃∈Θ𝑛^𝑛−1/2_𝑖=1𝑔_𝑙𝑍_𝑖,𝜃−𝐸𝑔_𝑙𝑍_𝑖,𝜃=𝑂_𝑃(1), (13) for 𝑙=𝑞,𝑘;
(ii) 𝐺_𝑙(𝜃) is continuously differentiable in some neighborhood of 𝜃₀ for 𝑙=𝑞,𝑘;
(iii) 𝜕𝐺_𝑙(𝜃₀)/𝜕𝜃^′ has full column rank;
(iv) the penalty function 𝑃_𝜏𝑛(⋅) satisfies 𝑃_𝜏𝑛(0)=0 and is continuously twice differentiable at 𝛽_0,𝑗 for any 𝑗∈𝒮_𝛽 with max_{𝑗∈𝒮𝛽}𝑃_𝜏^′′_𝑛𝛽_0,𝑗=𝑜_𝑝(1). (14)

Assumption 3(i) can be verified by applying Donsker’s theorem in specific models. Assumption 3(ii) imposes a local differentiability condition on the expectation of the moment function 𝑔_𝑙(𝑍,𝜃),  𝑙=𝑞,𝑘. Assumption 3(iii) is a local identification condition for 𝜃₀. If this assumption fails, the resulting estimator 𝛼_𝑛 may not be √𝑛-consistent. Assumption 3(iv) imposes some local smoothness conditions on the penalty function 𝑃_𝜏𝑛(⋅). Intuitively, this condition implies that attaching a penalty function to the empirical likelihood criterion function does not cause any local identification problem for the unknown parameter (𝜃₀,𝛽₀). It can be verified that the bridge, adaptive Lasso, and SCAD penalty functions satisfy Assumption 3(iv).

Theorem 4. Under Assumptions 1, 2, and 3, as 𝑛→∞ and with probability tending to 1, the EL shrinkage estimator defined in (8) satisfies(a)𝛼_𝑛→𝛼₀ and(b)‖𝛼_𝑛−𝛼₀‖=𝑂_𝑝(𝛿_𝑛),where 𝛿_𝑛=max{𝑏_𝑛,𝑛^−1/2} and 𝑏_𝑛=max_{𝑗∈𝒮𝛽}|𝑃^′_𝜏𝑛(𝛽_0,𝑗)|.

It is clear from Theorem 4 that, by choosing a proper 𝜏_𝑛, there exists a consistent EL shrinkage estimator 𝛼_𝑛 whose convergence rate is of the order 𝛿_𝑛. We now show that this estimator must possess the sparsity property 𝛽^𝒮_𝑛,𝑗=0 for all 𝑗∈𝒮^𝑐_𝛽, which is stated in Theorem 6.

Assumption 5. (i) The tuning parameter 𝜏_𝑛 satisfies √𝑛max_{𝑗∈𝒮𝛽}𝑃^′_𝜏𝑛𝛽_0,𝑗=𝑜_𝑝(1); (15)
(ii) for any 𝑗∈𝒮^𝑐_𝛽 and any random sequence {𝛽_𝑗,𝑛}_𝑛 with 𝛽_𝑗,𝑛≠0 a.e. for all 𝑛 and 𝛽_𝑗,𝑛=𝑂_𝑝(𝑛^−1/2), there is liminf_𝑛→∞𝑃^′_𝜏𝑛𝛽_𝑗,𝑛𝑟_𝑛𝜏_𝑛>0a.e., (16) where 𝑟_𝑛 is some nonnegative sequence such that 𝑛^1/2𝜏_𝑛𝑟_𝑛→∞.

Assumption 5(i) indicates that the convergence rate of |𝑃^′_𝜏𝑛(𝛽_0,𝑗)| for all 𝑗∈𝒮_𝛽 is faster than √𝑛. Under this assumption, Theorem 4 implies that √𝑛𝛼_𝑛−𝛼₀=𝑂_𝑝(1); (17) that is, the convergence rate of 𝛼_𝑛 is √𝑛. Assumption 5(ii) is a generalized version of condition (3.5) in Fan and Li [8]. Intuitively, Assumption 5(ii) implies that the shrinkage estimator 𝛽_𝑛,𝑗 of 𝛽_0,𝑗𝑗∈𝒮^𝑐_𝛽 is the minimizer of 𝑃_𝜏𝑛(⋅) w.p.a.1. From Assumptions 2(iv) and 3(iv), we know that 𝑃_𝜏𝑛(⋅) is locally minimized at 0. Hence, Assumption 5(ii) is the key condition needed for showing consistent moment selection. It can be verified that the bridge, adaptive Lasso, and SCAD penalty functions satisfy Assumption 5.

Theorem 6. Under Assumptions 1, 2, 3, and 5, one has lim_𝑛→∞𝑃𝑟𝛽_𝑛,𝑗=0=1𝑓𝑜𝑟𝑎𝑛𝑦𝑗∈𝒮^𝑐_𝛽. (18)

From the consistency of 𝛽_𝑛 and Theorem 6, we can immediately get lim_𝑛→∞Pr𝒮_𝛽=𝒮_𝛽,𝑛=1, (19) that is, the consistent moment selection. We next provide the conditions needed for deriving the limiting distribution of the EL shrinkage estimator.

Assumption 7. (i) There exists a symmetric, nonrandom, and positive definite matrix 𝑊₀ such that 𝑊_𝑛⟶_𝑝𝑊₀; (20)
(ii) the following central limit theorem (CLT) holds: 𝑛^𝑛−1/2_𝑖=1𝑔_𝑞+𝑘𝑍_𝑖,𝜃₀−𝐸𝑔_𝑞+𝑘𝑍_𝑖,𝜃₀⟶_𝑑Ψ𝜃₀, (21) where Ψ(𝜃₀) is some Gaussian random vector.

Assumption 7(i) is a regularity condition. Assumption 7(ii) can be verified by applying CLTs in models with specific moment functions and data structure.

Next we will consider the oracle properties of the EL shrinkage estimation. The oracle properties state that the EL shrinkage estimation can consistently identify all potentially valid moment conditions in set-2 and its estimator of 𝜃₀ is as efficient as the oracle EL estimator based on all valid moment conditions. As the consistent moment selection is directly implied by Theorems 4 and 6, the oracle properties follow if we can show that the asymptotic variance-covariance matrix of 𝛽_𝑛 coincides with that of the oracle EL estimator.

Let 𝑔_𝑑𝛽−(𝑍,𝜃) and 𝑔_𝑑𝛽+(𝑍,𝜃) denote the potentially valid and misspecified moment functions in set-2, respectively. We define 𝑔_{𝑞+𝑑𝛽−}()≡𝑔𝑍,𝜃^𝑞()𝑔𝑍,𝜃_𝑑𝛽−().𝑍,𝜃 (22) If we had prior information about the validity of the moment conditions in set-2, then there would be 𝑞+𝑘₀ moment conditions to estimate 𝜃₀. We can stack these moment conditions as 𝑚_𝑒𝜃₀=𝐸𝑔_{𝑞+𝑑𝛽−}𝑍,𝜃₀=0. (23) From the moment conditions in (23), we can compute the asymptotic variance-covariance matrix of the optimally oracle EL estimator as Σ^∗=𝜕𝑚_𝑒(𝜃)𝜕𝜃^′₀^𝑇𝑉⁻¹_𝑒,𝑜𝜕𝑚_𝑒(𝜃)𝜕𝜃^′₀⁻¹, (24) where 𝑉_𝑒,𝑜 is the leading (𝑞+𝑘₀)×(𝑞+𝑘₀) submatrix of 𝐸[Ψ(𝜃₀)Ψ^𝑇(𝜃₀)].

In the EL shrinkage estimation, if we choose a weight matrix 𝑊^∗_𝑛 such that 𝑊^∗_𝑛⟶_𝑝𝑊₀=𝐸Ψ𝜃₀Ψ^𝑇𝜃₀⁻¹, (25) then an interesting question is whether the resulting empirical likelihood (EL) shrinkage estimator 𝜃_𝑛 of 𝜃₀ could be as efficient as the optimally weighted oracle EL estimator. The answer to the above question is affirmative, as illustrated in the following theorem.

Theorem 8 (Oracle Property). Under Assumptions 1–5, one has lim_𝑛→∞𝑃𝑟𝒮_𝛽=𝒮_𝛽,𝑛=1. (26) Furthermore, if the weight matrix 𝑊_𝑛 satisfies (25) and Assumption 7 holds, then one has √𝑛𝜃^𝒮_𝑛−𝜃₀⟶_𝑑𝑁0,Σ^∗, (27) where Σ^∗ is defined in (24).

The empirical likelihood method is capable of finding estimators, constructing confidence regions, and testing hypotheses. The following theorem, a generalization of the Wilks theorem, allows us to carry out inference for parameters included in the model without estimating their estimators’ covariance for our proposed method.

Theorem 9. Suppose Assumptions 1–7 hold. The empirical likelihood ratio statistic for testing 𝐻₀∶𝜃=𝜃₀ is 𝑊_𝐸𝜃₀=2_𝐸𝜃₀−2_𝐸𝜃^𝒮_𝑛, (28) where _𝐸(𝜃) is given by _𝐸(𝜃)=∑^𝑛_𝑖=1log[1+𝜆^𝑇𝑔_𝑞+𝑘0(𝑍_𝑖,𝜃)]. Under Assumptions 1–7, 𝑊_𝐸(𝜃₀)→𝜒²_𝑝 as 𝑛→∞, where 𝐻₀ is true.

Theorem 9 allows us to use the EL ratio statistic for testing or obtaining confidence limits for parameters in a completely analogous way to that for parametric likelihood. The asymptotic confidence region of level 1−𝛿 for 𝜃₀ is 𝜃₀∣𝑊_𝐸𝜃₀≤𝜒²_𝑝(𝛿), (29) where 𝜒²_𝑝(𝛿) is the (1−𝛿) quantile of the chi-square distribution with 𝑝 degrees of freedom.

4. Adaptive Selection of Tuning Parameter

From the results of the previous sections, we see that the tuning parameter 𝜆_𝑛 plays an important role in deriving the oracle properties of the EL shrinkage estimator. Assumptions 2(iv), 3(iv), and 5(i)-(ii) are sufficient conditions imposed on 𝜆_𝑛 for the oracle properties to hold. However, these conditions do not provide a straightforward mechanism for choosing the tuning parameter 𝜆_𝑛 in finite samples. For practical implementation of the shrinkage techniques, it is important to have some procedures of selecting 𝜆_𝑛 such that the EL shrinkage estimator not only enjoys the oracle properties asymptotically but also has good finite-sample properties.

To choose the penalty parameter 𝜆, some data-driven approaches for selecting tuning parameters need to be proposed. In the following, we will propose empirical likelihood based AIC-type criterion (EmAIC), BIC-type criterion (EmBIC), and Hannan-Quinn information criterion- (HQIC-) type criterion (EmHQIC). They are defined, respectively, as EmAIC(𝜆)=GEL_𝑛(𝜆)−2𝒮^𝑐_𝛽,𝜆,EmBIC(𝜆)=GEL_𝑛(𝜆)−log(𝑛)𝒮^𝑐_𝛽,𝜆,EmHQIC(𝜆)=GEL_𝑛(𝜆)−𝑄loglog(𝑛)𝒮^𝑐_𝛽,𝜆, (30)(31)(32) where 𝒮_𝛽,𝜆 is the index set of nonzero elements in 𝛽_𝜆,𝑛, 𝒮^𝑐_𝛽,𝜆 is the complement set of 𝒮_𝛽,𝜆, and |𝒮^𝑐_𝛽,𝜆| denotes the cardinality of the index set of 𝒮^𝑐_𝛽,𝜆 and it stands for the number of moment conditions selected by the EL shrinkage method given 𝜆, 𝑄>2. GEL_𝑛(𝜆) is the generalized empirical likelihood (GEL) statistic proposed in Hong et al. [9], which is defined as GEL_𝑛(𝜆)=−2min_{𝛼𝒮𝛽,𝜆}max_𝜋∑^𝑛_𝑖=1𝜈[𝜋^′𝜌(𝑍_𝑖,𝛼_{𝒮𝛽,𝜆})], where 𝜈(⋅) is some concave function and its domain contains 0, 𝜋 is some 𝑞+𝑘 dimensional vector, and 𝛼_{𝒮𝛽,𝜆}=(𝜃,𝛽_{𝒮𝛽,𝜆},0) is transferred from 𝛼 by setting the elements of 𝛽 whose index belongs to 𝒮^𝑐_𝛽,𝜆 to be zero.

5. Numerical Studies

In this section, we first carry out simulations to demonstrate the performance of our method for finite data sets. We then apply our method to one real dataset. We compare our proposed method with the adaptive EL shrinkage method and the GMM shrinkage method. We find that both the adaptive EL and GMM shrinkage methods can consistently select the correct moment conditions in set-2 and automatically include them into the estimation to improve the efficiency of estimating 𝜃₀. However, the adaptive EL shrinkage is more efficient relative to the adaptive GMM estimators because of the advantages of EL relative to GMM.

5.1. Simulation Example

Example 1. In this simulation study, the data are generated from the following linear model: 𝑌_𝑖=𝜃₁₀+𝜃₂₀𝑋_𝑖+𝑢_𝑖, (33) where 𝑢_𝑖∼𝑁0,𝜎²_𝑢,𝑋_𝑖∼𝑁0,𝜎²_𝑥,𝐸𝑋_𝑖𝑢_𝑖≠0, (34) for all 𝑖. The available IVs are (𝑍_1,𝑖,𝑍_2,𝑖), where 𝑍_1,𝑖 is a scale random variable and 𝑍_2,𝑖=(𝑍_21,𝑖,𝑍_22,𝑖) is a random vector. There are two elements in 𝑍_21,𝑖 which denote the potentially valid IVs and there are eight elements in 𝑍_22,𝑖 which are misspecified IVs.
In (33), we take (𝜃₁₀,𝜃₂₀)=(0.8,0.8). The random variables 𝑋_𝑖, 𝑍_1,𝑖, 𝑍_21,𝑖, 𝑍^∗_22,𝑖, and 𝑢_𝑖 are generated from the following joint normal distribution: (𝑋_𝑖,𝑍_1,𝑖,𝑍_21,𝑖,𝑢_𝑖,𝑍^∗_22,𝑖)^′∼𝑁(0,Σ), (35) where the diagonal elements of Σ are 1, 𝐸(𝑋_𝑖𝑍_1,𝑖)=𝜎_𝑧1𝑥,𝐸(𝑋_𝑖𝑍_21,𝑖)=(𝜎_𝑧2𝑥,𝜎_𝑧2𝑥), 𝐸(𝑋_𝑖𝑢_𝑖)=0.4, and all other elements in Σ are zero. 𝑍_22,𝑖 is generated by the following equation: 𝑍_22,𝑖=𝑍^∗_22,𝑖+0.5𝑢_𝑖∗𝑙, (36) where 𝑙 is a 1×8 vector of ones. The correlation 𝜎_𝑧𝑗𝑥 of 𝑋_𝑖 and 𝑍_𝑗,𝑖(𝑗=1,2) measures the signal strength of the IV 𝑍_𝑗,𝑖 about the endogenous variable 𝑋_𝑖. There is one specification of (𝜎_𝑧1𝑥,𝜎_𝑧2𝑥) used in the simulation; that is, (𝜎_𝑧1𝑥,𝜎_𝑧2𝑥)=(0.4,0.4).
We assume the econometrician knows that 𝑍_1,𝑖 is a valid IV, while being unsure about validity of the IVs in 𝑍_2,𝑖. Hence, the moment conditions in set-1 are 𝐸𝑌_𝑖−𝜃₁₀−𝜃₂₀𝑋_𝑖𝐸𝑌=0,_𝑖−𝜃₁₀−𝜃₂₀𝑋_𝑖𝑍_1,𝑖=0, (37) while the moment conditions in set-2 are 𝐸𝑌_𝑖−𝜃₁₀−𝜃₂₀𝑋_𝑖𝑍=^′_2,𝑖^?0. (38) The SCAD penalty is used in the empirical likelihood shrinkage estimation, where the first-step estimators of the moment selection coefficients are from the empirical likelihood estimation using the moment conditions in (37) and the reparametrized moment conditions in (38).

For the specification of (𝜎_𝑧1𝑥,𝜎_𝑧2𝑥), we use the simulated samples with sample sizes 𝑛=100 and 𝑛=500, respectively, in our simulation study, and for each sample size, 2000 simulated samples are drawn from the data generating mechanism. With each simulated sample, we calculate four different types of estimators, which include the oracle estimator, empirical likelihood estimator, empirical likelihood shrinkage estimator using 𝜆_𝑛 selected by EmAIC, and EL shrinkage estimator using 𝜆_𝑛 selected by EmBIC. The oracle estimator is an EL estimator based on the moment conditions in set-1 and all valid moment conditions in set-2. The EL estimator is an EL estimator based only on the moment conditions in the set-1. Given the specification of (𝜎_𝑧1𝑥,𝜎_𝑧2𝑥) and the sample size 𝑛, we can get 2000 estimators of (𝜃_1,0,𝜃_2,0) for each type of estimator using the 2000 simulated samples. Hence, we can estimate the finite sample marginal densities of different estimators for (𝜃_1,0,𝜃_2,0) and the simulation results are presented in Figures 1 and 2. Tables 1 and 2 contain the selection probabilities of the correct, underselected, and overselected sets of moment conditions in EL (GMM) shrinkage estimation using the tuning parameters from EmAIC (GMM-AIC) and EmBIC (GMM-BIC), respectively.

Table 1 

The selection probabilities of adaptive EL shrinkage estimation.

Table 2 

The selection probabilities of adaptive GMM shrinkage estimation.

(a)

(b)

Figure 1 

Finite sample densities of EL and EL shrinkage estimators for 𝜃2,0 in the cases of example (1). The SCAD penalty is used in the EL shrinkage estimation and the first-step estimators of moment selection coefficients are the EL estimators; (2) oracle estimators are the EL estimators using the moment conditions in set-1 and all correct moment conditions in set-2; (3) EL estimators only use the moment conditions in set-1; (4) SEL-A refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmAIC; (5) SEL-B refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmBIC.

(a)

(b)

Figure 2 

Finite sample densities of EL and EL shrinkage estimators for 𝜃1,0 in the cases of example (1). The SCAD penalty is used in the EL shrinkage estimation and the first-step estimators of moment selection coefficients are the EL estimators; (2) oracle estimators are the EL estimators using the moment conditions in set-1 and all correct moment conditions in set-2; (3) EL estimators only use the moment conditions in set-1; (4) SEL-A refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmAIC; (5) SEL-B refers to the EL shrinkage estimators using tuning parameters selected by minimizing EmBIC.

There are several remarks we can make based on the simulation results presented in Figures 1 and 2. First, when the signal strength of the moment conditions in set-1 is 𝐸[𝑋_𝑖𝑍_1,𝑖]=0.4, the EL shrinkage method selects all valid moment conditions in set-2 with high probability and selects the overselected sets of moment conditions with low probability. Second, when the sample size 𝑛 is increased from 100 to 500, the probability of selecting the set of valid moment conditions in set-2 increases greatly and the probability of selecting the overselected or underselected sets of moment conditions decreases sharply. Third, if we compare the EL shrinkage estimators based on different data-driven procedures of selecting the tuning parameter, we see that the EL shrinkage estimation using the tuning parameters from EmAIC has lower probability of selecting inconsistent sets of moment conditions. But it has nontrivial probability of selecting underselected sets of moment conditions, even when the sample size is increased from 100 to 500. On the other hand, the EL shrinkage estimation using the tuning parameters from EmBIC has lower probability of selecting the overselected sets of moment conditions and higher probability of selecting the set of correct moment conditions, but its probability of selecting the overselected sets of moment conditions is higher. Fourth, the finite sample densities of the EL shrinkage estimators behave much better than those of the EL estimators in all scenarios of this simulation study. Comparing the EL shrinkage estimator with the EL estimator, the most obvious improvement is the reduction of the variance, as we can see from the finite sample densities depicted in Figures 1 and 2. Also note that when the sample size is increased, the finite sample densities of the EL shrinkage estimators are approaching those of the oracle EL estimators. Finally, when the moment conditions in (37) are 𝐸[𝑋_𝑖𝑍_1,𝑖]=0.4, the densities of the EL shrinkage estimators 𝜃_1,𝑛 of 𝜃_1,0 are almost the same as those of the oracle estimators and the EL estimators. This is because the moment conditions in set-2 only contain the information about 𝜃_2,0. Hence, when 𝜃_2,0 could be reliably estimated using the set-1 moment conditions, the extra valid moment conditions in set-2 do not help to reduce the variances of the estimators of 𝜃_1,0.

5.2. Real Data Example

Example 2. We apply the EL shrinkage method to study the following labor supply equation in the life-cycle labor supply model [6, 10, 11]: Δlogℎ_𝑖,𝑡=𝛼_𝑡+Δlog𝜔_𝑖,𝑡𝛿₀+𝜀_𝑖,𝑡, (39) where ℎ_𝑖,𝑡 is the annual hours working for money, 𝜔_𝑖,𝑡 is the hourly wage rate of individual 𝑖 at period 𝑡, 𝛼_𝑡 is a time varying constant, 𝜀_𝑖,𝑡 is the time varying error term, and 𝛿₀ measures the intertemporal substitution elasticity of labor supply with respect to the evolutionary wage changes and the theoretical prediction for its sign is positive.
Due to the measurement errors in 𝜔_𝑖,𝑡, the OLS estimator of (39) may be inconsistent. MaCurdy [10] proposes to use a set of family background variables to construct the set-1 moment conditions; we only use the parents’ economic status as the credibly valid IV and include the rest of them into set-2. We also include the alternative measure of wage 𝜔^∗_𝑖,𝑡 and the wage 𝜔_𝑖,𝑡 itself into set-2. Our sample is constructed from the Michigan Panel Study of Income Dynamics (PSID) dataset from year 1970 to year 1981.
We next apply the EL shrinkage estimation to the labor supply equation (39). The estimators of the moment selection coefficients are included in Table 3. As a comparison, we also include the EL estimators of the moment selection coefficients in different specifications of 𝛼_𝑡 in Table 3. In the first two rows of Table 3, the constant term 𝛼_𝑡 in (39) is treated to be time variant, while in its last two rows, 𝛼_𝑡 is taken to be a time invariant constant. From Table 3, we see that the EL estimators of the moment selection coefficients are nonzero and it is hard to determine which moment conditions are valid (misspecified) based on these estimators. On the other hand, the EL shrinkage estimation gives the same moment selection result in the different specifications of 𝛼_𝑡. The moment conditions constructed from the IVs by MaCurdy [10] and Altonji [11] are picked up by our shrinkage method, while the moment condition constructed using the imputed wage 𝜔_𝑖,𝑡 is not selected, which implies that Δlog(𝜔_𝑖,𝑡) is an endogenous variable in the labor supply equation (39).
The results of the EL shrinkage estimation of the labor supply equation (39) are contained in Table 4. As a comparison, we also include the EL estimators of 𝛿₀ based on the moment condition in set-1 and the postmoment selection EL (PEL) estimators of 𝛿₀ in Table 4. Columns (1)-(2) of Table 4 present the EL estimators of 𝛿₀ based on the following IV: parent’s economic status when individual was young, which provides the moment condition in set-1. Compared with other estimators in Table 4, the EL estimators in columns (1)-(2) are larger in magnitude and have larger standard errors. On the other hand, the EL shrinkage estimators in columns (3)-(4) have much smaller standard errors, because some moment conditions in set-2 are selected and automatically included into estimation by the EL shrinkage method.
From Table 4, we see that, compared with the EL estimators, the EL shrinkage estimators of 𝛽_7,0 are closer to zero, which implies that part of the information in the moment condition constructed by 𝜔_𝑖,𝑡 is indeed used in the EL shrinkage estimation. Based on the above reasoning, we can deduce that the shrinkage effect of the penalty function on the estimators of 𝛽_7,0 may introduce some bias to the estimator of 𝛿₀. To get rid of this bias, we conduct another EL estimation based on the moment condition in set-1 and the moment conditions in set-2 selected by our method. These PEL estimators are included in columns (5)-(6) of Table 4. We can see that the PEL estimators are slightly larger in magnitude than the EL shrinkage estimators and their standard errors are almost the same.

Table 3 

EL and GMM shrinkage estimators of the moment selection coefficients¹.

Table 4 

EL and GMM shrinkage estimation of the labor supply equation¹.

Remark 3. We now give detailed instructions about some marks in Table 3.(1)Standard errors are in parentheses and 𝑛=3487.(2)EL (GMM) estimation with the time dummy variables.(3)EL (GMM) estimation without time dummy variables.(4)EL (GMM) shrinkage estimation with time dummy variables, where the penalty function is the SCAD and the tuning parameter equals 0.000374 (selected by EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ)).(5)EL (GMM) shrinkage estimation without time dummy variables, where the penalty function is the SCAD and the tuning parameters equals 0.000948 (selected by EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ)).

Remark 4. We now give detailed instructions about some marks in Table 4.(1)Standard errors are in parentheses and sample size 𝑛=3487.(2)𝑑_𝑡 refers to the set of time dummy variables for the years from 1971 to 1981.(3)EL (GMM) is the EL (GMM) estimation only using the moment conditions in set-1.(4)SEL (SGMM) denotes the EL (GMM) shrinkage estimation based on the SCAD penalty. In column (3) the tuning parameter equals 0.000949 and in column (4) the tuning parameter equals 0.000374. EmAIC (GMM-AIC), EmBIC (GMM-BIC), and EmHQIC (GMM-HQ) produce the same number of the tuning parameter in each case.(5)PEL (P-GMM) denotes the EL (GMM) estimation based on the moment conditions selected by the EL (GMM) shrinkage estimation. The results in columns (5) and (6) are based on the moment conditions selected in (3) and (4), respectively.

Appendix

Proofs. We first introduce some notations and definitions. Let (𝜃,𝛽,𝜆)=𝑛⁻¹∑^𝑛_𝑖=1log{1+𝜆^𝑇𝜌(𝑍_𝑖,𝜃,𝛽)} and 𝜌(𝜃)=𝑛⁻¹∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽). Let 𝑣_𝑛(𝑔)=(1/√𝑛)∑^𝑛_𝑖=1[𝑔(𝑍_𝑖)−𝐸[𝑔(𝑍_𝑖)]] denote the empirical process indexed by some function 𝑔. Suppose that {𝑋_𝑛} is a sequence of random vectors; then for a given sequence of nonnegative constants 𝛿_𝑛, we write 𝑋_𝑛=𝑂_𝑝(𝛿_𝑛) to mean that, for any constant 𝜀>0, there is a finite constant 𝐶_𝜀 such that Pr(‖𝑋_𝑛‖>𝐶_𝜀𝛿_𝑛)<𝜀 eventually; we write 𝑋_𝑛=𝑜_𝑝(𝛿_𝑛) to mean that, for any constants 𝜀₁,𝜀₂>0, there is Pr(‖𝑋_𝑛‖>𝜀₁𝛿_𝑛)<𝜀₂ eventually.

In this appendix, we prove two lemmas which are useful for deriving the asymptotic properties of the EL shrinkage estimator. Define 𝑉_𝑛𝜃,𝛽≡𝑛^𝑛−1_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽^𝑇𝑊_𝑛𝑛^𝑛−1_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽,𝑉_0,𝑛𝜃,𝛽≡𝐸𝜌𝑍,𝜃,𝛽^𝑇𝑊_𝑛𝐸𝜌𝑍,𝜃,𝛽, (A.1)(A.2) where 𝑊_𝑛=(1/2𝑛)∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)𝜌^𝑇(𝑍_𝑖,𝜃,𝛽).

Lemma 5. Under Assumption 1(iii), one has 12𝑉_0,𝑛𝜃,𝛽−𝑅_𝑛≤𝑉_𝑛𝜃,𝛽≤2𝑉_0,𝑛𝜃,𝛽+2𝑅_𝑛 (A.3) for all (𝜃,𝛽)∈Θ×ℬ, where 𝑅_𝑛≡sup_{𝜃,𝛽∈Θ×ℬ}𝑛⁻¹𝑣_𝑛𝜌𝑍,𝜃,𝛽^𝑇𝑊_𝑛𝑣_𝑛𝜌𝑍,𝜃,𝛽. (A.4)

Proof. By Assumption 2(iii), we deduce that 2∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)𝑛−𝐸𝜌(𝑍,𝜃,𝛽)^𝑇×𝑊_𝑛2∑^𝑛_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽𝑛−𝐸𝜌𝑍,𝜃,𝛽≥0, (A.5) for all (𝜃,𝛽)∈Θ×ℬ, which implies that 𝑛^𝑛−1_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)^𝑇𝑊_𝑛𝑛^𝑛−1_𝑖=1𝜌𝑍_𝑖≥1,𝜃,𝛽2𝑉_0,𝑛𝜃,𝛽−𝑅_𝑛. (A.6) Note that Assumption 2(iii) also implies ∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)𝑛−2𝐸𝜌(𝑍,𝜃,𝛽)^𝑇×𝑊_𝑛∑^𝑛_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽𝑛−2𝐸𝜌𝑍,𝜃,𝛽≥0 (A.7) for all (𝜃,𝛽)∈Θ×ℬ, which implies that 𝑛^𝑛−1_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)^𝑇𝑊_𝑛𝑛^𝑛−1_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽≥2𝑉_0,𝑛𝜃,𝛽+2𝑅_𝑛. (A.8) From the inequalities in (A.6) and (A.8), we immediately get the claimed results in (A.3).

Lemma 6. Under Assumptions 2(iii) and 3(ii)-(iii), one has 𝑐₁+𝑜(1)𝛼−𝛼₀²≤𝑉_0,𝑛≤𝑐𝜃,𝛽₂+𝑜(1)𝛼−𝛼₀², (A.9) for all 𝛼 in shrinking neighborhoods of 𝛼₀, where 𝑐₁, 𝑐₂ are generic positive finite constants.

Proof. Denote 𝑔_𝑞(𝑍,𝜃)=𝑔_𝑞,1(𝑍,𝜃),…,𝑔_𝑞,𝑞(,𝑔𝑍,𝜃)_𝑘(𝑍,𝜃)=𝑔_𝑘,1(𝑍,𝜃),…,𝑔_𝑘,𝑘(𝑍,𝜃). (A.10) First note that by Assumption 3(ii) 𝑚(𝛼)=𝐺_𝑞(𝜃)𝐺_𝑘(𝜃)=−𝛽𝜕𝐺_𝑞𝜃𝜕𝜃^𝑇0𝜕𝐺_𝑘𝜃𝜕𝜃^𝑇−𝐼_𝑘𝜃−𝜃₀𝛽−𝛽₀≡𝜕𝑚_𝑒𝜃𝜕𝛼^𝑇𝜃−𝜃₀𝛽−𝛽₀, (A.11) where 𝜕𝐺_𝑞𝜃𝜕𝜃^𝑇=𝜕𝐸[𝑔_𝑞,1(𝑍,𝜃₁)]𝜕𝜃^𝑇𝑇𝑔,…,𝜕𝐸_𝑞,𝑞𝑍,𝜃₁𝜕𝜃^𝑇𝑇,𝜕𝐺_𝑘𝜃𝜕𝜃^𝑇=𝜕𝐸[𝑔_𝑘,1(𝑍,𝜃₁)]𝜕𝜃^𝑇𝑇,…,𝜕𝐸[𝑔_𝑘,𝑘(𝑍,𝜃₁)]𝜕𝜃^𝑇𝑇, (A.12)𝜃(𝑗=1,…,𝑞+𝑘) lies between 𝜃 and 𝜃₀ and 𝐼_𝑘 is 𝑘×𝑘 identity matrix. As 𝜃 is in the shrinking neighborhood of 𝜃₀ and 𝜕𝐺_𝑙(𝜃)/𝜕𝜃^𝑇,  (𝑙=𝑞,𝑘) is continuous in 𝜃; we deduce that 𝜕𝐺_𝑙𝜃𝜕𝜃^𝑇=𝜕𝐺_𝑙𝜃₀𝜕𝜃^𝑇+𝑜(1)for𝑙=𝑞,𝑘. (A.13) By (A.11), (A.13), and Cauchy-Schwarz inequality, we have 𝑚(𝛼)=𝑚𝛼₀𝜕𝛼^𝑇𝛼−𝛼₀+𝑜𝛼−𝛼₀. (A.14) Using Assumption 2(iii), the result in (A.14), and Cauchy-Schwarz inequality, we get 𝑉_0,𝑛𝜃,𝛽=𝛼−𝛼₀^𝑇𝑚(𝛼₀)𝜕𝛼^𝑇𝑇𝑊_𝑛𝑚𝛼₀𝜕𝛼^𝑇𝛼−𝛼₀+𝑜𝛼−𝛼₀². (A.15) As 𝜕𝐺_𝑞(𝜃₀)/𝜕𝜃^𝑇 has full column rank and is strictly positive definite, 𝑚(𝛼₀)/𝜕𝛼^𝑇 has full rank and [𝑚(𝛼₀)/𝜕𝛼^𝑇]^𝑇𝑊_𝑛[𝑚(𝜃₀)/𝜕𝛼^𝑇] is strictly positive definite. Let 𝛾_1,𝑛 and 𝛾_2,𝑛   (𝛾_1,𝑛,𝛾_2,𝑛>0) denote the smallest and largest eigenvalues of [𝑚(𝛼₀)/𝜕𝛼^𝑇]^𝑇𝑊_𝑛[𝑚(𝛼₀)/𝜕𝛼^𝑇]; then by Assumptions 2(iii) and 3(iii), we have 0<𝑐₁≤𝛾_1,𝑛≤𝛾_2,𝑛≤𝑐₂<∞ (A.16) which together with (A.15) implies that 𝛼−𝛼₀^𝑇𝑚𝛼₀𝜕𝛼^𝑇𝑇𝑊_𝑛𝑚𝛼₀𝜕𝛼^𝑇×𝛼−𝛼₀+𝑜𝛼−𝛼₀²≤𝛾_2,𝑛𝛼−𝛼₀²+𝑜𝛼−𝛼₀²≤𝑐₂+𝑜(1)𝛼−𝛼₀². (A.17) The right inequality in (A.9) is implied by (A.17). The left inequality in (A.9) can be similarly derived. This finishes the proof.

Lemma 7. Under Assumption 1, for any 𝜉 with (1/𝜅+1/10)≤𝜉≤2/5 and as 𝑛→∞, max_{1≤𝑖≤𝑛}sup_{(𝜃,𝛽)∈Θ×ℬ}|𝜆^𝑇𝜌(𝑍_𝑖,𝜃,𝛽)|=𝑜_𝑝(1) for all 𝜆∈Λ_𝑛={𝜆∶‖𝜆‖≤𝑛^−𝜉}, and Λ_𝑛⊆Λ_𝑛(𝜃) = {𝜆∶𝜆^𝑇𝜌(𝑍_𝑖,𝜃,𝛽)>−1,𝑖=1,…,𝑛} for all (𝜃,𝛽)∈Θ×ℬ.

Proof. Following Owen [12], Assumption 2(vi) implies that max_{1≤𝑖≤𝑛}sup_{(𝜃,𝛽)∈Θ×ℬ}𝜌𝑍_𝑖,𝜃,𝛽=𝑜_𝑝𝑛^1/5,1𝜅+110≤𝜉,max_{1≤𝑖≤𝑛}sup_{(𝜃,𝛽)∈Θ×ℬ}𝜆^𝑇𝜌𝑍_𝑖,𝜃,𝛽≤𝑛^−𝜉max_{1≤𝑖≤𝑛}sup_{(𝜃,𝛽)∈Θ×ℬ}𝜌𝑍_𝑖,𝜃,𝛽=𝑜_𝑝𝑛^{−𝜉+1/𝜅}𝑞+𝑘^1/2. (A.18)

Lemma 8. Let (𝜃,𝛽,𝜆)=𝑛⁻¹∑^𝑛_𝑖=1log{1+𝜆^𝑇𝜌(𝑍_𝑖,𝜃,𝛽)}; one has =1𝜃,𝛽,𝜆√𝑛^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)^𝑇×𝑊_𝑛1√𝑛^𝑛_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽+𝑜_𝑝1𝑛, (A.19) where 𝑊_𝑛=(1/2𝑛)∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)𝜌^𝑇(𝑍_𝑖,𝜃,𝛽).

The detailed theoretical reasons of this lemma can be found in Following Hjort et al. [13], so we neglect it here.

Proof of Theorem 4. (a) By the definition of 𝛼_𝑛, one has 𝜃_𝑛,𝛽_𝑛,𝜆+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_𝑛≤𝜃₀,𝛽₀,𝜆+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_0,𝑗. (A.20) Applying Lemma 7, one can deduce that 𝑉_𝑛𝜃_𝑛,𝛽_𝑛+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_𝑛+𝑜_𝑝1𝑛≤𝑉_𝑛𝜃₀,𝛽₀+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_0,𝑗+𝑜_𝑝1𝑛; (A.21) that is, 𝑉_𝑛𝜃_𝑛,𝛽_𝑛+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_𝑛≤𝑉_𝑛𝜃₀,𝛽₀+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_0,𝑗. (A.22) Applying Lemma 5 and Assumption 2(iv), one can deduce from (A.1) that 𝑉_0,𝑛𝜃_𝑛,𝛽_𝑛≤2^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_0,𝑗+4𝑅_𝑛, (A.23) where 𝑅_𝑛 is defined in Lemma 5.
From Assumption 2(ii) and the definition of 𝜌(𝑍,𝜃,𝛽), one gets sup_{(𝜃,𝛽)∈Θ×ℬ}𝑣_𝑛𝜌𝑍,𝜃,𝛽√𝑛=sup_𝜃∈Θ𝑛^𝑛−1_𝑖=1𝑔_𝑞+𝑘𝑍_𝑖,𝜃−𝐸𝑔_𝑞+𝑘𝑍_𝑖,𝜃=𝑜_𝑝(1). (A.24) By the triangle inequality, ULLN in (A.24), and Assumptions 2(iii)-(iv), one has 𝑅_𝑛=𝑜_𝑝(1),^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_0,𝑗=𝑜_𝑝(1). (A.25) From Assumption 2(iii) and results in (A.23) and (A.25), one can deduce that 𝐺_𝑞𝜃_𝑛=𝑜_𝑝(1),𝐺_𝑘𝜃_𝑛−𝛽_𝑛=𝑜_𝑝(1). (A.26) Now, the first result in (A.26) and Assumption 1(i) imply that 𝜃_𝑛→_𝑝𝜃₀. From the second result in (A.26), the triangle inequality, consistency of 𝜃_𝑛, and Assumption 2(i), one has 𝑜_𝑝(1)=𝐺_𝑘𝜃_𝑛−𝛽_𝑛≥𝐺_𝑘𝜃_𝑛−𝐺_𝑘𝜃₀−𝛽_𝑛−𝛽₀=𝛽_𝑛−𝛽₀+𝑜_𝑝(1), (A.27) which implies that 𝛽_𝑛→_𝑝𝛽₀. So one gets the result that as 𝑛→∞ and with probability tending to 1, the EL shrinkage estimator defined in (6) satisfies 𝛼_𝑛→𝛼₀.
(b) Using the inequalities in (A.3) and (A.22) and Lemma 7, one gets 12𝑉_0,𝑛𝜃_𝑛,𝛽_𝑛+^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_𝑛,𝑗≤^𝑘_𝑗=1𝑝_𝜏𝑛𝛽_0,𝑗+2𝑅_𝑛, (A.28) where 𝑅_𝑛 is defined in Lemma 5. By Assumptions 2(iv) and 3(iv) and the inequality in (A.28), one has 𝑉_0,𝑛𝜃_𝑛,𝛽_𝑛+2_{𝑗∈𝒮𝛽}𝑝_𝜏𝑛𝛽_𝑛,𝑗−𝑝_𝜏𝑛𝛽_0,𝑗≤4𝑅_𝑛. (A.29) Next, by Assumption 3(iv), Taylor expansion, the triangle inequality, and Cauchy-Schwarz inequality, one gets 𝑝_𝜏𝑛𝛽_0,𝑗−𝑝_𝜏𝑛𝛽_𝑛,𝑗=_{𝑗∈𝒮𝛽}𝑝^′_𝜏𝑛𝛽_0,𝑗𝛽_𝑛,𝑗−𝛽_0,𝑗+12𝑝_𝜏^′′_𝑛𝛽_𝑗𝛽_𝑛,𝑗−𝛽_0,𝑗²≤max_{𝑗∈𝒮𝛽}𝑝^′_𝜏𝑛𝛽_0,𝑗𝛼_𝑛−𝛼₀+max_{𝑗∈𝒮𝛽}𝑝_𝜏^′′_𝑛𝛽_0,𝑗2+𝑜_𝑝(1)𝛼_𝑛−𝛼₀², (A.30) where 𝛽_𝑗 lies between 𝛽_0,𝑗 and 𝛽_𝑛,𝑗 for 𝑗∈𝒮_𝛽. From Lemma 6, one obtains 𝑉_0,𝑛𝜃_𝑛,𝛽_𝑛≥𝑐₁+𝑜_𝑝(1)𝛼_𝑛−𝛼₀², (A.31) where 𝑐₁>0 is a finite constant. The inequality in (A.31), together with Assumption 3(iv), and the inequalities in (A.29) and (A.30) imply that 𝑐₁+𝑜_𝑝(1)𝛼_𝑛−𝛼₀²−2max_{𝑗∈𝒮𝛽}𝑝^′_𝜏𝑛𝛽_0,𝑗𝛼_𝑛−𝛼₀≤4𝑅_𝑛. (A.32) By Assumption 3(i), one has 𝑅_𝑛=𝑂_𝑝(𝑛⁻¹). When the sample size 𝑛 is large enough, by definition, the probability that 𝑐₁+𝑜_𝑝(1)≤𝑐₁/2 is strictly smaller than any given small number 𝜔/2, together with the inequality in (A.32), implies that 𝛼Pr^𝑛−𝛼₀𝛿_𝑛>𝑀≤inf_{𝛼∈𝒜∶𝛼−𝛼0>𝑀𝛿𝑛}𝛼−𝛼₀≤2𝑐₁𝑏_𝑛+𝑏²_𝑛+2𝑅_𝑛𝑐₁+𝜔2≤Pr𝑀<2𝑐₁𝑏_𝑛+𝑏²_𝑛+2𝑅_𝑛/𝑐₁𝛿_𝑛+𝜔2, (A.33) where 𝛿_𝑛=max{𝑏_𝑛,𝑛^−1/2}. By definition, 2𝑐₁(𝑏_𝑛+𝑏²_𝑛+2𝑅_𝑛/𝑐₁)/𝛿_𝑛=𝑂_𝑝(1). Hence, one can choose some large enough number 𝑀_𝜔 such that Pr𝑀<2𝑐₁𝑏_𝑛+𝑏²_𝑛+2𝑅_𝑛/𝑐₁𝛿_𝑛<𝜔2. (A.34) This and the results in (A.33) immediately imply that 𝛼Pr^𝑛−𝛼₀𝛿_𝑛>𝑀<𝜔 (A.35) eventually, which gives us ‖𝛼_𝑛−𝛼₀‖=𝑂_𝑝(𝛿_𝑛).

Proof of Theorem 6. We know that 𝜃,𝛽,𝜆=𝑛^𝑛−1_𝑖=1𝜌(𝑍_𝑖,𝜃,𝛽)^𝑇×𝑊_𝑛𝑛^𝑛−1_𝑖=1𝜌𝑍_𝑖,𝜃,𝛽+𝑜_𝑝1𝑛. (A.36) On the event {𝛽_𝑛,𝑗≠0} for some 𝑗∈𝒮^𝑐_𝛽, we have the following KKT optimality condition: 2𝑛^𝑛−1/2_𝑖=1𝜌𝑍_𝑖,𝜃_𝑛,𝛽_𝑛^𝑇𝑊_𝑛𝑛^𝑛−1/2_𝑖=1𝜕𝜌𝑍_𝑖,𝜃_𝑛,𝛽_𝑛𝜕𝛽_𝑗,+𝑛𝑝^′_𝜏𝑛𝛽_𝑛,𝑗=0 (A.37) which implies 𝑊_𝑛𝑞+𝑗𝑛^𝑛−1/2_𝑖=1𝜌𝑍_𝑖,𝜃_𝑛,𝛽_𝑛=√𝑛𝑝^′_𝜏𝑛𝛽_𝑛,𝑗2, (A.38) where 𝑊_𝑛(𝑞+𝑗) denotes the (𝑞+𝑗)th row of the weight matrix 𝑊_𝑛.
By Assumption 3(ii) and the consistency of 𝜃_𝑛, there is 𝑛^𝑛−1/2_𝑖=1𝜌𝑍_𝑖,𝛼_𝑛=𝑣_𝑛𝜌𝑍,𝛼_𝑛+𝑛^1/2𝑚𝛼_𝑛=𝑣_𝑛𝜌𝑍,𝛼_𝑛+𝜕𝑚𝜃_𝑛𝜕𝛼^𝑇𝑛^1/2𝛼_𝑛−𝛼₀, (A.39) where 𝜕𝑚(𝜃)/𝜕𝛼^𝑇 is defined in (A.11) and 𝜃_𝑛=(𝜃_1,𝑛,…,𝜃_{𝑞+𝑘,𝑛}) and 𝜃_𝑗,𝑛  (𝑗=1,…,𝑞+𝑘) lie between 𝜃₀ and 𝜃_𝑛. From Assumption 3(i), we have 𝑣_𝑛[𝜌(𝑍,𝛼_𝑛)]=𝑂_𝑝(1). By Theorem 4 and Assumption 5(i), we have 𝑛^1/2(𝛼_𝑛−𝛼₀)=𝑂_𝑝(1). By the triangle inequality, Assumption 3(ii), and the consistency of 𝜃_𝑛, we deduce that 𝜕𝑚𝜃_𝑛𝜕𝛼^𝑇≤𝜕𝑚𝜃_𝑛𝜕𝛼^𝑇−𝜕𝑚𝜃₀𝜕𝛼^𝑇+𝜕𝑚𝜃₀𝜕𝛼^𝑇=𝑂_𝑝(1). (A.40) Hence, we have 𝑛^−1/2∑^𝑛_𝑖=1𝜌(𝑍_𝑖,𝛼_𝑛)=𝑂_𝑝(1) which combined with Assumption 2(iii) implies that 𝑊_𝑛𝑞+𝑗𝑛^𝑛−1/2_𝑖=1𝜌𝑍_𝑖,𝛼_𝑛=𝑂_𝑝(1). (A.41) By Theorem 6 and Assumption 5(i), we have 𝑛^1/2𝛽_𝑛,𝑗=𝑂_𝑝(1) for all 𝑗∈𝒮^𝑐_𝛽. Hence, conditional on the event {𝛽_𝑛,𝑗≠0} for some 𝑗∈𝒮^𝑐_𝛽, we can invoke Assumption 5(ii) to deduce that 𝑛^1/2𝑝^′_𝜏𝑛𝛽_𝑛,𝑗2=𝑛^1/2𝑟_𝑛𝜏_𝑛2𝑝^′_𝜏𝑛𝛽_𝑛,𝑗𝑟_𝑛𝜆_𝑛⟶_𝑝∞, (A.42) for 𝑗∈𝒮^𝑐_𝛽. Now, using (A.38), (A.41), and (A.42), we deduce that Pr(𝛽_𝑛,𝑗=0)→1 as 𝑛→∞ for any 𝑗∈𝒮^𝑐_𝛽.

Proof of Theorem 8. Let 𝑔_𝑑𝛽−(𝑍,𝜃) and 𝑔_𝑑𝛽+(𝑍,𝜃) denote the potentially valid and misspecified moment functions in set-2, respectively. We define 𝑔_{𝑞+𝑑𝛽−}()≡𝑔𝑍,𝜃^𝑞()𝑔𝑍,𝜃_𝑑𝛽−(),𝜃𝑍,𝜃𝜕𝑚₀𝜕𝛼^𝑇_𝒮=𝜕𝐸𝑔_{𝑞+𝑑𝛽−}𝑍,𝜃₀𝜕𝜃^𝑇0𝜕𝐸𝑔_𝑑𝛽+𝑍,𝜃₀𝜕𝜃^𝑇−𝐼_𝑘−𝑘0, (A.43) where 𝐼_𝑘−𝑘0 denotes a (𝑘−𝑘₀)×(𝑘−𝑘₀) identity matrix. If we define 𝑀_𝒮=𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮^𝑇𝑊₀𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮, (A.44) then under Assumptions 2(iii) and 3(iii), we know that 𝑀_𝒮 is nonsingular matrix.
Recall that 𝛼^𝑇_0,𝒮=(𝜃^𝑇₀,𝛽^𝑇_0,+) and accordingly 𝛼^𝑇_𝑛,𝒮=(𝜃^𝑇_𝑛,𝛽^𝑇_𝑛,+). For any compact subset 𝐾 in 𝑅^{𝑑𝜃+𝑑0𝑘−𝑘}, we denote any element 𝑢_𝒮∈𝐾 as 𝑢^𝑇_𝒮=(𝑢^𝑇_𝜃,𝑢^𝑇_𝛽,+), where 𝑢_𝜃 are the first 𝑑_𝜃 elements in 𝑢_𝒮 and 𝑢_𝛽,+ are the last 𝑑_𝑘−𝑘0 elements in 𝑢_𝒮. Denote 𝑉_2,𝑛𝑢_𝒮=𝑛^𝑛−1/2_𝑖=1𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛^𝑇×𝑊_𝑛𝑛^𝑛−1/2_𝑖=1𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛−𝑛^𝑛−1/2_𝑖=1𝜌𝑍,𝛼₀^𝑇×𝑊_𝑛𝑛^𝑛−1/2_𝑖=1𝜌𝑍,𝛼₀+𝑛_𝑗∈𝒮𝑝_𝜏𝑛𝛽_0,𝑗+𝑢_𝛽+,𝑗√𝑛−𝑝_𝜏𝑛𝛽_0,𝑗≡𝑉^∗_2,𝑛𝑢_𝒮+𝑛_𝑗∈𝒮𝑝_𝜏𝑛𝛽_0,𝑗+𝑢_𝛽+,𝑗√𝑛−𝑝_𝜏𝑛𝛽_0,𝑗, (A.45) where 𝜌^𝒮(𝑍,𝛼_0,𝒮+𝑢_𝒮/√𝑛)≡𝜌(𝑍_𝑖,𝜃₀+𝑢_𝜃/√𝑛,𝛽_0,++𝑢_𝜃0,+/√𝑛,𝛽_0,−). From Theorem 6, we know that 𝛽_0,−=0 w.p.a.1. Thus, √𝑛(𝛼_𝑛,𝒮−𝛼_0,𝒮) is the minimizer of 𝑉_2,𝑛(𝑢_𝒮) w.p.a.1.
If we define ℱ_𝑛≡𝑓^𝑛_𝑢𝒮=𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛−𝜌𝑍,𝛼₀∶𝑢_𝒮∈𝐾, (A.46) then by Assumptions 3(i) we know that ℱ_𝑛 is a Donsker class. As 𝐾 is compact, so there exists some constant 𝐶_𝑘, such that sup_{𝑢𝒮∈𝐾}‖𝑛^−1/2𝑢_𝒮‖≤𝑛^−1/2𝐶_𝑘=𝑜(1). Now we can use Lemma 2.17 in Pakes and Pollard [14] to deduce that 𝑣_𝑛𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛−𝜌𝑍,𝛼₀=𝑜_𝑝(1), (A.47) uniformly over 𝑢_𝒮∈𝐾.
By Assumption 3(ii) and the compactness of 𝐾, we have √𝑛𝐸𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛−𝐸𝜌𝑍,𝛼₀=𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮𝑢_𝒮+𝑜(1), (A.48) uniformly over 𝑢_𝒮∈𝐾. Thus, (A.47) and (A.48) imply that, uniformly over 𝑢_𝒮∈𝐾, there is 𝑛^𝑛−1/2_𝑖=1𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛=𝑣_𝑛𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛−𝜌𝑍,𝛼₀+𝑣_𝑛𝜌𝑍,𝛼₀+√𝑛𝐸𝜌^𝒮𝑍,𝛼_0,𝒮+𝑢_𝒮√𝑛−𝐸𝜌𝑍,𝛼₀=𝑣_𝑛𝜌𝑍,𝛼₀+𝜕𝑚𝜃₀𝜕𝛼^′_𝒮𝑢_𝒮+𝑜_𝑝(1). (A.49) Now, we can use the result in (A.49), Assumptions 3(i) and 7(i), and the compactness of 𝐾 to deduce that 𝑉^∗_2,𝑛𝑢_𝒮=𝑢^𝑇_𝒮𝜕𝑚𝜃₀𝜕𝛼^′_𝒮^𝑇×𝑊₀𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮𝑢_𝒮+2𝑢^𝑇_𝒮𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮^𝑇×𝑊₀𝑣_𝑛𝜌𝑍,𝛼₀+𝑜_𝑝(1), (A.50) uniformly over 𝑢_𝒮∈𝐾. If 𝑗∈𝒮, then by Assumptions 3(iv) and 5(i) and the compactness of 𝐾 we have 𝑛𝑝_𝜏𝑛𝛽_0,𝑗+𝑢_𝛽+,𝑗√𝑛−𝑝_𝜏𝑛𝛽_0,𝑗=√𝑛𝑝^′_𝜏𝑛𝛽_0,𝑗𝑢_𝛽+,𝑗+𝑝_𝜏^′′_𝑛𝛽_0,𝑗+𝑜_𝑝(1)𝑢²_𝛽+,𝑗⟶0 (A.51) uniformly over 𝑢_𝛽+,𝑗.
Using Assumption 7(ii), the results in (A.50), and (A.51), we get 𝑉_2,𝑛𝑢_𝒮⟶_𝑑𝑉₂𝑢_𝒮=𝑢^𝑇_𝒮𝑀_𝒮𝑢_𝒮+2𝑢^𝑇_𝒮𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮^𝑇𝑊₀Ψ𝜃₀, (A.52) uniformly over 𝑢_𝒮∈𝐾. It is clear that 𝑉₂(𝑢_𝒮) is uniquely minimized at 𝑢^∗_𝒮=−𝑀_𝒮⁻¹𝜕𝑚𝜃₀𝜕𝛼^𝑇_𝒮^𝑇𝑊₀Ψ𝜃₀. (A.53) By Theorem 4 and Assumption 5(i), there is √𝑛𝛼_𝑛,𝒮−𝛼_0,𝒮=𝑂_𝑝(1). (A.54) Now, the uniform weak convergence in (A.52), the unique minimization in (A.53), and the asymptotic tightness of 𝛼_𝑛,𝒮 in (A.54) enable us to invoke the ACMT to deduce that √𝑛𝛼_𝑛,𝒮−𝛼_0,𝒮⟶_𝑑𝑁0,𝑀_𝒮⁻¹Σ_𝒮𝑀_𝒮⁻¹. (A.55)
The first result is implied by Theorems 4 and 6, so we only need to show the second claim. First note that if 𝑊_𝑛⟶_𝑝𝑊₀=𝐸Ψ𝜃₀Ψ^𝑇𝜃₀⁻¹, (A.56) then the centered limiting distribution in (A.55) will be simplified to √𝑛𝛼_𝑛,𝒮−𝛼_0,𝒮⟶_𝑑𝑁0,𝑀⁻¹₁₁. (A.57) Denote Ω_𝜃0 to be the first 𝑑_𝜃0×𝑑_𝜃0 submatrix of 𝑀_𝒮⁻¹ and 𝜕𝐺_𝛽+(𝜃₀)/𝜕𝜃^𝑇=𝜕𝐸[𝑔_𝑑𝛽+(𝑍,𝜃₀)]/𝜕𝜃^𝑇. Note that 𝑀_𝒮=𝜕𝐸𝑔_{𝑞+𝑑𝛽−}𝑍,𝜃₀𝜕𝜃^𝑇0𝜕𝐸𝑔_𝑑𝛽+𝑍,𝜃₀𝜕𝜃^𝑇−𝐼_{𝑑𝛽+0×𝑑𝛽+0}^𝑇×𝑊₀𝜕𝐸𝑔_{𝑞+𝑑𝛽−}𝑍,𝜃₀𝜕𝜃^𝑇0𝜕𝐸𝑔_𝑑𝛽+𝑍,𝜃₀𝜕𝜃^𝑇−𝐼_{𝑑𝛽+0×𝑑𝛽+0}=Σ_𝒮¹¹Σ_𝒮¹²Σ_𝒮²¹𝑊₂₂, (A.58) where Σ_𝒮¹¹=𝜕𝑚_𝑒𝜃₀𝜕𝜃^𝑇𝑇𝑊₁₁𝜕𝑚_𝑒𝜃₀𝜕𝜃^𝑇+𝜕𝑚_𝑒𝜃₀𝜕𝜃^𝑇𝑇𝑊₁₂𝜕𝐺_𝛽+𝜃₀𝜕𝜃^𝑇+𝜕𝐺_𝛽+𝜃₀𝜕𝜃^𝑇𝑇𝑊₂₁𝜕𝑚_𝑒𝜃₀𝜕𝜃^𝑇+𝜕𝐺_𝛽+𝜃₀𝜕𝜃^𝑇𝑇𝑊₂₂𝜕𝐺_𝛽+𝜃₀𝜕𝜃^𝑇,Σ_𝒮¹²=−𝜕𝑚_𝑒𝜃₀𝜕𝜃^𝑇𝑇𝑊₁₂−𝜕𝐺_𝛽+𝜃₀𝜕𝜃^𝑇𝑇𝑊₂₂=Σ_𝒮^21𝑇. (A.59) From (A.37), it is easy to get Ω_𝜃⁻¹₀=Σ_𝒮¹¹𝑊⁻¹₂₂Σ_𝒮²¹=𝜕𝑚_𝑒(𝜃₀)𝜕𝜃^𝑇𝑇𝑉⁻¹_𝑒,0𝜕𝑚_𝑒𝜃₀𝜕𝜃^𝑇=Σ^∗−1, (A.60) where the last equality is due to the fact that (𝑊₁₁−𝑊₁₂𝑊⁻¹₂₂𝑊₂₁)⁻¹=𝑉_𝑒,0. Now, using results in (A.34) and (A.38) and the Continuous Mapping Theorem (CMT), we can deduce that √𝑛𝜃^𝒮_𝑛−𝜃₀⟶_𝑑𝑁0,Σ^∗, (A.61) which establishes the semiparametric efficiency of the GMM shrinkage estimator 𝜃^𝒮_𝑛.

Proof of Theorem 9. We denote 𝑆⁻¹_22.1=(𝐸(𝜕𝑔_𝑞+𝑘0/𝜕𝜃)^𝑇𝐸(𝑔𝑔^𝑇)𝐸(𝜕𝑔_𝑞+𝑘0/𝜕𝜃))⁻¹, 𝑆₁₁=−𝐸(𝑔_𝑞+𝑘0𝑔^𝑇_𝑞+𝑘0), 𝑆₂₂=𝐸(𝜕𝑔_𝑞+𝑘0/𝜕𝜃), 𝑆₂₁=𝐸(𝜕𝑔_𝑞+𝑘0/𝜕𝜃)^𝑇, and _𝐸(𝜃)=∑^𝑛_𝑖=1log[1+𝜆^𝑇𝑔_𝑞+𝑘0(𝑍_𝑖,𝜃)]. The log-empirical likelihood ratio test statistic is 𝑊_𝐸𝜃₀=2_𝑖log1+𝜆^𝑇𝑔_𝑞+𝑘0𝑍_𝑖,𝜃₀−_𝑖log1+𝜆^𝑇𝑔_𝑞+𝑘0𝑍_𝑖,𝜃^𝒮_𝑛. (A.62) Note that _𝐸𝜃^𝒮_𝑛=^𝑛_𝑖=1log1+𝜆^𝑇𝑔_𝑞+𝑘0𝑍_𝑖,𝜃^𝒮_𝑛𝑛=−2𝑄^𝑇_1𝑛𝜃₀,0𝐴𝑄_1𝑛𝜃₀,0+𝑜_𝑝(1), (A.63) where 𝑄_1𝑛(𝜃)=1𝑛_𝑖11+𝜆^𝑇𝑔_𝑞+𝑘0𝑍_𝑖𝑔,𝜃^𝑞+𝑘0𝑍_𝑖,𝑄,𝜃_2𝑛(𝜃)=1𝑛_𝑖11+𝜆^𝑇𝑔_𝑞+𝑘0𝑍_𝑖,𝜃𝜕𝑔_𝑞+𝑘0𝑍_𝑖,𝜃𝜕𝜃^𝑇𝜆,𝐴=𝑆⁻¹₁₁𝐼+𝑆₁₂𝑆⁻¹_22.1𝑆₂₁𝑆⁻¹₁₁. (A.64)
Also under 𝐻₀, 1𝑛_𝑖11+𝜆^𝑇𝑔_𝑞+𝑘0𝑍_𝑖,𝜃₀𝑔_𝑞+𝑘0𝑍_𝑖,𝜃₀=0⟹𝜆₀=−𝑆⁻¹₁₁𝑄_1𝑛𝜃₀,0+𝑜_𝑝(1),_𝑖log1+𝜆^𝑇₀𝑔_𝑞+𝑘0𝑍_𝑖,𝜃₀𝑛=−2𝑄^𝑇_1𝑛𝜃₀,0𝑆⁻¹₁₁𝑄_1𝑛𝜃₀,0+𝑂_𝑃(1). (A.65) Thus, 𝑊_𝐸𝜃₀=𝑛𝑄^𝑇_1𝑛𝜃₀,0𝐴−𝑆⁻¹₁₁𝑄_1𝑛𝜃₀,0+𝑜_𝑝(1)=(−𝑆^−1/2₁₁)√𝑛𝑄_1𝑛(𝜃₀,0)^𝑇×−𝑆^−1/2₁₁𝑆₁₂𝑆⁻¹_22.1𝑆₂₁−𝑆^−1/2₁₁×−𝑆^−1/2₁₁√𝑛𝑄_1𝑛𝜃₀,0+𝑜_𝑝(1). (A.66)
Note that (−𝑆^−1/2₁₁)√𝑛𝑄_1𝑛(𝜃₀,0) converges to a standard multivariate normal distribution and that (−𝑆^−1/2₁₁)𝑆₁₂𝑆⁻¹_22.1𝑆₂₁(−𝑆^−1/2₁₁) is symmetric and idempotent, with trace equal to 𝑝. Hence, the empirical likelihood ratio statistic 𝑊_𝐸(𝜃₀) converges to 𝜒²_𝑝.

Conflict of Interests

The author declares that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research was supported by the NSF Project (ZR2011AQ007) of Shandong Province of China, NNSF Projects (11201499 and 11301309) of China, and the Fundamental Research Funds for the Central Universities (27R1310008A).