Modelling and Simulation in Engineering

Volume 2019, Article ID 6342702, 10 pages

https://doi.org/10.1155/2019/6342702

## A Modified New Two-Parameter Estimator in a Linear Regression Model

^{1}Department of Physical Sciences, Landmark University, Omu-Aran, Nigeria^{2}Department of Statistics, Federal University of Technology, Akure, Nigeria^{3}School of Mathematical Sciences, Universiti Sains Malaysia, Malaysia

Correspondence should be addressed to Adewale F. Lukman; gn.ude.uml@imnaralof.elaweda

Received 31 December 2018; Revised 25 February 2019; Accepted 5 March 2019; Published 26 May 2019

Academic Editor: Zhiping Qiu

Copyright © 2019 Adewale F. Lukman et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The literature has shown that ordinary least squares estimator (OLSE) is not best when the explanatory variables are related, that is, when multicollinearity is present. This estimator becomes unstable and gives a misleading conclusion. In this study, a modified new two-parameter estimator based on prior information for the vector of parameters is proposed to circumvent the problem of multicollinearity. This new estimator includes the special cases of the ordinary least squares estimator (OLSE), the ridge estimator (RRE), the Liu estimator (LE), the modified ridge estimator (MRE), and the modified Liu estimator (MLE). Furthermore, the superiority of the new estimator over OLSE, RRE, LE, MRE, MLE, and the two-parameter estimator proposed by Ozkale and Kaciranlar (2007) was obtained by using the mean squared error matrix criterion. In conclusion, a numerical example and a simulation study were conducted to illustrate the theoretical results.

#### 1. Introduction

The general linear regression model in matrix form is defined aswhere *y* is a vector of the dependent variable, is a known full-rank matrix of explanatory variables, is a vector of regression coefficients, and is vector of disturbance such that and . The ordinary least squares estimator (OLSE) of in model (1) is defined as

According to the Gauss–Markov theorem, the OLS estimator is considered best, linear, and unbiased, possessing minimum variance in the class of all linear unbiased estimators. However, different studies have shown that the OLS estimator is not best when the explanatory variables are related, that is, when multicollinearity is present [1]. This estimator becomes unstable and gives a misleading conclusion. Many biased estimators have been proposed as an alternative to OLSE to circumvent this problem. These include Stein estimator [2], principal components estimator [3], ridge estimator (RRE) estimator [1], contraction estimator [4], modified ridge regression estimator (MRRE) [5], and Liu estimator [6].

Hoerl and Kennard [1] proposed a ridge estimator (RRE)where . was obtained by augmenting the equation to the original equation (1) and then applying the OLS estimator. Mayer and Willke [4] defined the contraction estimator

Liu [6] combined the Stein estimator with a ridge estimator to combat the problem of multicollinearity. was obtained by augmenting the equation to the original equation (1) and then applying OLS. This is defined as follows:where .

Swindel [5] modified the ridge estimator (MRRE) by adding a prior information. The estimator is defined as follows:where represent the prior information on . MRRE tends to as *k* tends to infinity. Also, MRRE returns the estimates of the OLS estimator when *k* = 0.

Based on prior information, Li and Yang [7] proposed a modified Liu estimator (MLE):

MLE includes OLS and Liu as special cases. In recent times, different researchers have suggested the use of two-parameter estimators to handle multicollinearity. Ozkale and Kaciranlar [8] proposed the two-parameter estimator (TPE), which is defined aswhere . TPE includes OLS, RRE, LE, and the contraction estimators as special cases.

The primary focus of this study is to provide an alternative method in a linear regression model to circumvent the problem of multicollinearity. A modified two-parameter (MTP) estimator is proposed based on prior information and is compared with OLS, LE, RRE, MRRE, MLE, and TPE, respectively, using the mean squared error matrix (MSEM) criterion. The article is structured as follows: We introduce the new estimator in Section 2. In Section 3, we discuss the superiority of the new estimator. Section 4 consists of a numerical example and a simulation study. Concluding remarks are provided in Section 5.

#### 2. Modified Two-Parameter Estimator

Let and MRRE in equation (6) can be re-expressed as

Similarly, , and then the modified Liu estimator in equation (7) can be written as

MRRE and MLE are the convex combination of the prior information and the OLS estimator. From equation (8), ; therefore, the modified two-parameter based on the prior information can be defined as follows:

Also, MTPE is a convex combination of the prior information and OLSE. It includes the special cases of OLSE, RRE, MRE, LE, and MLE. The following cases are possible: ; ordinary least squares estimator ; Liu estimator ; modified ridge estimator ; ridge estimator ; modified Liu estimator

Suppose there exist an orthogonal matrix *T* such that , where is the eigenvalue of . and are the matrices of eigenvalues and eigenvectors of , respectively. Substituting , in model (1), then the equivalent model can be rewritten as

The following representations of the estimators are as follows:

The following notations and lemmas are needful to prove the statistical property of .

Lemma 1. *Let M be an positive definite matrix, that is, M > 0, and be some vector, then if and only if [9].*

Lemma 2. *Let be two linear estimators of . Suppose that , where denotes the covariance matrix of and . Consequently,if and only if , where [10].*

#### 3. Establishing Superiority of Modified Two-Parameter Estimator Using MSEM Criterion

In this section, MTPE is compared with the following estimators: OLS, RRE, LE, MRRE, MLE, and TPE.

##### 3.1. Comparison between the MTPE and OLS Using MSEM Criterion

From the representation , the bias vector and covariance matrix of MTPE are obtained as follows:where .

Recall that and let . Therefore,

Hence,

From the representation, , the MSEM of OLS is given as

Let *k* > 0 and 0 < *d* < 1. Thus, the following theorem holds.

Theorem 3. *Consider two biased competing homogenous linear estimators and . If k > 0 and , the estimator is superior to estimator using the MSEM criterion, that is, if and only if*

*Proof. *Using (17) and (19), the following was obtained: will be positive definite (pd) if and only if or . It was observed that for and *k* > 0. Therefore, is pd. By Lemma 2, the proof is completed.

##### 3.2. Comparison between the MTPE and RRE Using MSEM Criterion

From the representation, , the bias vector and covariance matrix of RRE is given as follows:

Hence,where . The difference between and in the MSEM sense is as follows:

Let *k* > 0 and 0 < *d* < 1. Thus, the following theorem holds.

Theorem 4. *Consider two biased competing homogenous linear estimators and . If k > 0 and , the estimator is superior to estimator using the MSEM criterion, that is, if and only if*

##### 3.3. Comparison between the MTPE and LE Using MSEM Criterion

From the representation, , the bias vector and covariance matrix of RRE are provided as follows:

Hence,where . Considering the difference between (18) and (29),where , , and .

Theorem 5. *Consider two biased competing homogenous linear estimators and . If k > 0 and , the estimator is superior to estimator using the MSEM criterion, that is, if and only if*

*Proof. *Using (17) and (28), the following was obtained:

By computation, will be positive definite (pd) if and only if, . For and *k* > 1, it was observed that . Therefore, is pd. By Lemma 2, the proof is completed.

##### 3.4. Comparison between the MTPE and MRRE Using MSEM Criterion

From the representation, , the bias vector and covariance matrix of MRRE are provided as follows:

Hence,where . Considering the difference between (18) and (35),

Theorem 6. *Consider two biased competing homogenous linear estimators and . If k > 0 and , the estimator is superior to the estimator using the MSEM criterion, that is, if and only if*

*Proof. *Using (17) and (34), the following was obtained:Evidently, for and *k* > 0, will be positive definite (pd). By Lemma 2, the proof is completed.

##### 3.5. Comparison between the MTPE and MLE Using MSEM Criterion

From the representation, , the bias vector and covariance matrix of MLE are provided as follows:

Hence,where The mean square error difference between (18) and (41) is given aswhere , , .

Theorem 7. *Consider two biased competing homogenous linear estimators and . If k > 0 and , the estimator is superior to the estimator using the MSEM criterion, that is, if and only if , where , , and .*

*Proof. *Using (17) and (40), the following was obtained:

By computation,By computation, will be positive definite if and only if .

##### 3.6. Comparison between the MTPE and TPE Using MSEM Criterion

From the representation , the bias vector and covariance matrix of TPE are provided as follows:

Hence,

Considering the matrix difference between (18) and (47)

Obviously, if and only if thus, the following results hold.

Theorem 8. *The modified two-parameter estimator is superior to the two-parameter estimator in the MSEM sense if and only if .*

#### 4. Selection of Bias Parameters

Selecting an appropriate parameter is crucial in this study. The use of the Ridge estimator largely depends on the ridge parameter, *k*. Several methods for estimating this ridge parameter have been proposed. This includes Hoerl and Kennard [1], Kibria [11], Muniz and Kibria [12], Aslam [13], Dorugade [14], Kibria and Banik [15], Lukman and Ayinde [16], Lukman et al. [17], and others. For the purpose of practical application of this new estimator, the optimum values of *k* and *d* are obtained. In order to obtain an optimum value of *k*, we assume the value of *d* is fixed.

Recall from equation (18),

Differentiating equation (49) with respect to *k* gives the following result:

Let , the value of *k* is as follows: and are replaced by their unbiased estimators and . The harmonic mean version is defined aswhere .

Recall that considering this special case implies that in equation (51) will becomewhich is the estimated value of *k* introduced by Hoerl and Kennard [1]. Hoerl et al. [18] defined the harmonic version of the ridge parameter, *k*, as follows:

The optimum value of *d* is obtained by differentiating equation (49) with respect to *d* with fixed *k*. The result is as follows:

Let , the value of *d* is as follows: and are replaced by their unbiased estimators and . Recall that considering this special case implies that in equation (54) will become

Equation (57) is the same as the optimum value of *d* proposed by Liu [6], which is defined as follows:

Theorem 9. *Iffor all i, then are always positive.*

*Proof. *The values of *k* in (51) are always positive if Since must be positive for all *i*, it is observed that for all *i*. This inequality depends on the unknown parameters and which is replaced by their unbiased estimators and .

The selection of the estimator of the parameters *d* and *k* in can be obtained iteratively as follows: Step 1: calculate from (59). Step 2: estimate by using in step 1. Step 3: estimate from (56) by using the estimator in step 2. Step 4: if is negative use , can take negative value. However, takes value between 0 and 1.

#### 5. Numerical Example and Monte-Carlo Simulation

Hussain dataset which was originally adopted by Eledum and Zahri [19] is used in this study to illustrate the performance of the new estimator. The dataset was also adopted in the study of Lukman et al. [20]. This is provided in Table 1. The regression model is defined as follows:where represents the product value in the manufacturing sector, the values of the imported intermediate commodities, imported capital commodities, represents the value of imported raw materials. The variance inflation factors are and and the condition number of is approximately 5660049. The variance inflation factor and the condition number both indicate the presence of severe multicollinearity.