Modelling and Simulation in Engineering

Modelling and Simulation in Engineering / 2020 / Article

Research Article | Open Access

Volume 2020 |Article ID 9574304 | https://doi.org/10.1155/2020/9574304

Adewale F. Lukman, B. M. Golam Kibria, Kayode Ayinde, Segun L. Jegede, "Modified One-Parameter Liu Estimator for the Linear Regression Model", Modelling and Simulation in Engineering, vol. 2020, Article ID 9574304, 17 pages, 2020. https://doi.org/10.1155/2020/9574304

Modified One-Parameter Liu Estimator for the Linear Regression Model

Academic Editor: Mohamed B. Trabia
Received18 Dec 2019
Revised06 Jul 2020
Accepted11 Jul 2020
Published19 Aug 2020

Abstract

Motivated by the ridge regression (Hoerl and Kennard, 1970) and Liu (1993) estimators, this paper proposes a modified Liu estimator to solve the multicollinearity problem for the linear regression model. This modification places this estimator in the class of the ridge and Liu estimators with a single biasing parameter. Theoretical comparisons, real-life application, and simulation results show that it consistently dominates the usual Liu estimator. Under some conditions, it performs better than the ridge regression estimators in the smaller MSE sense. Two real-life data are analyzed to illustrate the findings of the paper and the performances of the estimators assessed by MSE and the mean squared prediction error. The application result agrees with the theoretical and simulation results.

1. Introduction

The linear regression model (LRM) is where is a vector of the predictand, is a known matrix of predictor variables, is a vector of unknown regression parameters, is a vector of errors such that and , and is an identity matrix. The parameters in (1) are mostly estimated by the ordinary least square (OLS) estimator defined in (2)

The performance of the estimator is conditional on the non-violation of the assumption of model (1) that the predictor variables are independent. However, in most real-life applications, we observed that the predictor variables grow together, which result in the problem termed multicollinearity. The consequence of this on the OLS estimator is that it reduces its efficiency and it became unstable (for examples, [1, 2]). Many methods exist in literature to combat the multicollinearity problem. Biased estimators with one biasing parameter include the ridge regression estimator by Hoerl and Kennard [1] and the Liu estimator by Kejian [3], among others.

The objective of this paper is to propose a new one-parameter Liu-type estimator for the regression parameter when the predictor variables of the model are linearly related. Since we want to compare the performance of the proposed estimator with the usual Liu and ridge regression estimators, we will give a brief description of each of them as follows.

1.1. Ridge Regression Estimator (RRE)

Hoerl and Kennard [1] proposed by augmenting to the linear regression model (1). The ridge regression estimator is defined as

1.2. Liu Estimator

Since the ridge regression estimator is a complicated function of , Liu [3] derived by augmenting to the linear regression model (1) [4]. The Liu estimator of is

This estimator is a linear function of the shrinkage parameter .

1.3. Proposed One-Parameter Liu Estimator

One of the limitations of the shrinkage parameter by Kejian [3] is that it returns a negative value most of the time which affects the performance of the estimator [4, 5]. In this study, we augment to the LRM. This is done by minimizing subject to where is a constant. The modified Liu estimator of is

This modification provides a substantial improvement in the performance of the modified Liu estimator and will give a positive value of the shrinkage parameter . The estimator will always produce a smaller mean square error compared to the OLS estimator.

The bias, variance, and MSE of the proposed estimator are, respectively, given as follows:

To compare the performance of the estimators, we will consider the linear regression model in canonical form, which is given as follows: where , , . and are the eigenvalues and eigenvectors of . The ordinary, ridge, Liu, and modified Liu estimators of are

The following notations and lemmas will be used to prove the statistical property of .

Lemma 1. Given that matrix and is a vector, if and only if [6].

Lemma 2. Given two estimators of , and . Suppose that , where and represent the covariance matrix of and , respectively. Therefore, if and only if where where and represent the mean squared error matrix and bias vector of [7].

According to Özkale and Kaçiranlar [4], if , then where is the scalar mean square error.

The rest of the paper is as follows. The theoretical comparison among the estimators and the estimation of the biasing parameter of the proposed estimator are given in Section 2. A simulation study and numerical examples are conducted in Sections 3 and 4, respectively. This paper ends up with some concluding remarks in Section 5.

2. Comparison among Estimators

In this section, we will show theoretical comparisons among the estimators. First, we will compare between the proposed estimator and OLSE.

2.1. The Proposed Estimator and OLSE

Theorem 3. Given two estimators of , , and , if , then the estimator is better than , that is, if and only if, where .

Proof. Recall that Then, We observed from equation (16) that which shows that.

2.2. The Proposed Estimator and RRE

Theorem 4. Given two estimators of , , and , if , then the estimator is better than , that is, if and only if, where and .

Proof. Recall that Therefore, From equation (18), we observed that . Hence, this shows that .

2.3. The Proposed Estimator and Liu Estimator

Theorem 5. Given two estimators of , , and , if , then the estimator is better than , that is, if and only if, where and .

Proof. Note that Therefore, From equation (20), we observed that . Hence, this shows that.

2.4. Determination of

The shrinkage parameter is selected according to Kejian [3], Kibria [8], Kibria and Banik [9], Lukman and Ayinde [10], and Qasim et al. [11].

The partial derivative of (21) with respect to and setting it to zero, we obtained

In eqn. (22), we replace and with its unbiased estimate and obtained:

Taking a critical look at equation (23), the estimate of the shrinkage parameter will often return a positive value since and will always be greater than zero. For practical purposes, we obtained the minimum value of (24) as

3. Simulation Study

As theoretical comparison among the estimators in Section 2 gives the conditional dominance among the estimators, a simulation study conducted using the R 3.4.1 programming languages is considered in this section to grasp the better picture about the performance of the estimators.

3.1. Simulation Technique

We generated the explanatory variables by the following references of Gibbons [12] and Qasim et al. [11]: where are independent standard normal pseudorandom numbers, and represents the correlation between any two predictor variables. The number of predictor variables is three and seven. The predictands for the regression models are generated as follows: where . is constrained to unity, according to Newhouse and Oman [13], Lukman et al. [14], and Lukman et al. [15]. We examined the performances of the estimators, using mean square error criteria. The simulation is performed using the following condition: (1)Sample sizes: , 50, 100, and 200(2)Number of replications: 1000(3)The variances: are 1, 25, and 100(4)The multicollinearity levels: , 0.8, 0.9, and 0.99(5), 0.2,…, 0.9, and 1

The simulated results for and , 0.80, 0.90, and 0.99 are presented in Tables 13 for , 50, and 100, respectively. and , 0.80, 0.90, and 0.99 are presented in Tables 46 for , 50, and 100, respectively. For a better picture, we have plotted MSE vs. for , , and , 0.90, and 0.99 in Figures 1 and 2, respectively. We also plotted MSE vs. and MSE vs. in Figure 3.


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.36210.35170.29090.27820.54730.51860.37500.34611.15401.01170.53230.445812.7744.15540.85670.3894
0.20.36210.34200.29770.27220.54730.49290.39080.33291.15400.89950.58260.409612.7742.33951.38770.4531
0.30.36210.33290.30470.26660.54730.46990.40730.32061.15400.80940.63760.378112.7741.60332.11700.7151
0.40.36210.32450.31210.26120.54730.44930.42480.30921.15400.7360.69730.351312.7741.21433.04461.1753
0.50.36210.31670.31970.25610.54730.43060.44310.29851.15400.67540.76170.329212.7740.97794.17051.8339
0.60.36210.30930.32760.25130.54730.41370.46220.28881.15400.62470.83070.311712.7740.82145.49462.6906
0.70.36210.30250.33580.24680.54730.39910.48220.27991.15400.58190.90450.299012.7740.71177.01693.7457
0.80.36210.29610.34430.24250.54730.38450.50310.27181.15400.54540.98300.291012.7740.63148.73754.9990
0.90.36210.290.3530.23850.54730.37180.52480.26461.15400.51391.06610.287612.7740.570810.6566.4505
1.00.36210.28440.36210.23480.54730.36010.54730.25831.15400.48671.15400.289012.7740.523912.7748.1003

50.18.02097.75856.13725.760012.96712.2328.36427.502928.46224.84012.0679.5146319.35102.3917.4514.1230
0.28.02097.51096.33145.577112.96711.5678.81697.094328.46221.94513.4928.3869319.3556.00831.4454.7897
0.38.02097.27706.52945.398012.96710.9629.28436.700328.46219.58815.0177.3584319.3536.97850.32710.344
0.48.02097.05576.73115.222612.96710.4119.76646.32128.46217.64116.6406.4290319.3526.81674.09520.784
0.58.02096.84616.93675.051012.9679.907110.2635.956428.46216.01018.3625.5988319.3520.580102.7536.112
0.68.02096.64747.1464.883212.9679.444510.7755.606528.46214.62720.1844.8677319.3516.415136.2956.327
0.78.02096.45887.35914.719212.9679.018911.3015.271328.46213.44222.1054.2357319.3513.467174.7281.429
0.88.02096.27957.57594.558912.9678.626211.8424.950828.46212.41824.1243.7028319.3511.293218.04111.42
0.98.02096.10907.79534.402412.9678.263012.3974.645028.46211.52626.2433.2691319.359.6369266.24146.30
1.08.02095.94678.02094.249712.9677.926212.9674.353828.46210.74128.4622.9345319.358.3433319.33186.06

100.131.99330.93924.42122.90151.81948.87133.33329.865113.8499.33148.08837.8221277.4409.2569.14915.713
0.231.99329.94525.20322.16351.81946.20135.15528.218113.8487.72653.81433.2821277.4223.57125.2018.323
0.331.99329.00526.00021.44051.81943.77537.03426.629113.8478.27859.93529.1371277.4147.37200.7940.484
0.431.99328.11626.81220.73151.81941.56138.97225.098113.8470.46666.45025.3871277.4106.67295.9482.197
0.531.99327.27427.63920.03851.81939.53640.96823.626113.8463.91973.36122.0311277.481.687410.64143.46
0.631.99326.47428.48019.35951.81937.67743.02222.211113.8458.36880.66719.0711277.464.998544.90224.28
0.731.99325.71529.33618.69551.81935.96645.13420.855113.8453.61288.36816.5061277.453.189698.70324.65
0.831.99324.99430.20718.04551.81934.38747.30419.557113.8449.49896.46414.3361277.444.476872.06444.57
0.931.99324.30731.09217.41151.81932.92649.53318.316113.8445.910104.9612.5621277.437.8391065.0584.04
1.031.99323.65431.99316.79151.81931.57051.81917.134113.8442.758113.8411.1821277.432.6551277.4743.07


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.13360.13250.12490.12320.18060.17850.16460.16140.32530.31720.26850.25772.82062.13490.74530.5194
0.20.13360.13140.12580.12240.18060.17650.16620.15990.32530.30970.27410.25262.82061.69770.89030.4386
0.30.13360.13040.12670.12160.18060.17460.16790.15840.32530.30250.27990.24772.82061.39961.05680.3791
0.40.13360.12940.12770.12080.18060.17270.16960.15700.32530.29580.28590.24292.82061.18611.24460.3410
0.50.13360.12850.12860.12000.18060.17090.17130.15560.32530.28950.29200.23832.82061.02741.45380.3243
0.60.13360.12750.12960.11920.18060.16920.17310.15420.32530.28360.29830.23392.82060.90581.68440.3289
0.70.13360.12660.13050.11850.18060.16760.17490.15290.32530.27800.30480.22972.82060.81051.93630.3550
0.80.13360.12570.13150.11770.18060.16600.17670.15160.32530.27270.31150.22562.82060.73442.20970.4025
0.90.13360.12490.13250.11700.18060.16440.17860.15040.32530.26770.31830.22172.82060.67262.50440.4713
10.13360.12410.13360.11630.18060.16300.18060.14910.32530.26300.32530.21802.82060.62172.82060.5615

50.13.04923.01742.78312.72594.17074.10673.65373.54437.59647.36625.90575.563369.76652.39115.9449.5391
0.23.04922.98602.81192.69764.17074.04433.70923.49047.59647.14776.08155.396669.76641.18119.9047.0945
0.33.04922.95532.84102.66954.17073.98363.76513.43707.59646.93996.26035.232969.76633.44624.3695.1550
0.43.04922.92512.87022.64154.17073.92443.82163.38407.59646.74216.44215.072369.76627.84029.3393.7205
0.53.04922.89542.89962.61384.17073.86673.87853.33157.59646.55386.62694.914669.76623.62034.8142.7912
0.63.04922.86622.92912.58624.17073.81043.93603.27967.59646.37426.81484.760169.76620.35040.7942.3669
0.73.04922.83752.95892.55884.17073.75553.99393.22817.59646.20287.00564.608569.76617.75547.2802.4478
0.83.04922.80942.98882.53154.17073.70204.05233.17717.59646.03917.19954.459969.76615.65554.2703.0337
0.93.04922.78173.01892.50454.17073.64984.11133.12677.59645.88267.39654.314469.76613.92961.7664.1247
13.04922.75443.04922.47764.17073.59884.17073.07677.59645.73307.59644.171969.76612.48969.7675.7209

100.112.17712.04911.10810.87716.66116.40414.58214.14130.35129.42723.56122.182279.07209.5663.55837.824
0.212.17711.92311.22410.76316.66116.15414.80513.92330.35128.55024.26721.511279.07164.7079.44727.978
0.312.17711.80011.34010.65016.66115.90915.03013.70730.35127.71624.98620.852279.07133.7297.35020.146
0.412.17711.67811.45810.53716.66115.67115.25713.49430.35126.92225.71720.205279.07111.26117.2714.328
0.512.17711.55911.57610.42516.66115.43915.48713.28230.35126.16626.45919.569279.0794.352139.2010.525
0.612.17711.44211.69510.31416.66115.21315.71813.07230.35125.44427.21418.945279.0781.240163.148.7356
0.712.17711.32711.81410.20416.66114.99215.95112.86430.35124.75527.98018.334279.0770.829189.108.9606
0.812.17711.21311.93410.09416.66114.77616.18612.65830.35124.09728.75817.734279.0762.401217.0811.200
0.912.17711.10212.0559.984016.66114.56616.42312.45430.35123.46829.54817.146279.0755.468247.0615.453
112.17710.99212.1779.876016.66114.36016.66112.25230.35122.86630.35116.570279.0749.684279.0721.721


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.11240.11210.11050.11030.14920.14780.13960.13800.28670.27810.23040.22093.07212.14090.68830.4808
0.20.11240.11180.11070.11020.14920.14650.14040.13720.28670.27020.23560.21653.07211.62060.83600.4210
0.30.11240.11160.11080.11010.14920.14530.14140.13650.28670.26290.24100.21243.07211.29791.01300.3905
0.40.11240.11140.11100.11000.14920.14420.14230.13580.28670.25620.24680.20863.07211.08251.21920.3893
0.50.11240.11120.11120.11000.14920.14320.14340.13520.28670.25010.25270.20503.07210.93051.45480.4174
0.60.11240.1110.11140.11000.14920.14220.14440.13460.28670.24440.25900.20173.07210.81861.71970.4748
0.70.11240.11080.11160.10990.14920.14120.14550.13410.28670.23910.26550.19873.07210.73332.01380.5615
0.80.11240.11060.11190.10990.14920.14030.14670.13360.28670.23420.27230.19593.07210.66672.33730.6775
0.90.11240.11050.11210.10990.14920.13950.14790.13320.28670.22970.27930.19343.07210.61342.69000.8228
1.00.11240.11040.11240.10990.14920.13870.14920.13280.28670.22550.28670.19123.07210.56993.07210.9973

50.12.06312.04521.91261.88023.24403.19542.85232.76976.95766.71915.25644.917276.77253.31414.7478.2979
0.22.06312.02761.92891.86423.24403.14802.89422.72906.95766.49455.43134.752976.77239.90618.9716.0738
0.32.06312.01021.94541.84833.24403.10192.93652.68876.95766.28275.60974.592176.77231.41223.8624.5166
0.42.06311.99321.96191.83253.24403.05702.97932.64876.95766.08265.79174.434976.77225.62629.4203.6261
0.52.06311.97641.97851.81673.24403.01333.02242.60926.95765.89345.97724.281176.77221.46635.6453.4025
0.62.06311.95991.99521.80113.24402.97073.06592.57016.95765.71436.16624.131076.77218.35042.5373.8457
0.72.06311.94362.01211.78563.24402.92913.10982.53146.95765.54456.35873.984376.77215.93950.0964.9557
0.82.06311.92762.0291.77023.24402.88873.15422.49316.95765.38346.55483.841276.77214.02458.3216.7325
0.92.06311.91192.04601.75483.24402.84923.19892.45536.95765.23056.75443.701676.77212.47167.2139.1762
1.02.06311.89642.06311.73963.24402.81083.24402.41786.95765.08506.95763.565676.77211.18976.77212.287

100.18.16328.09017.54817.415412.92012.72311.33410.99927.80926.85320.97019.601307.09213.2658.71732.755
0.28.16328.01827.6157.349612.92012.53211.50510.83327.80925.95121.67518.937307.09159.5875.68423.760
0.38.16327.94747.68227.284212.92012.34611.67610.66927.80925.10022.39418.287307.09125.5695.30817.423
0.48.16327.87777.74987.219112.92012.16411.84910.50727.80924.29623.12617.650307.09102.37117.5913.743
0.58.16327.80917.81787.154412.92011.98712.02410.34627.80923.53523.87217.028307.0985.681142.5312.721
0.68.16327.74157.88627.090112.92011.81412.20010.18627.80922.81524.63216.419307.0973.175170.1314.356
0.78.16327.6757.95497.026112.92011.64612.37810.02827.80922.13125.40615.823307.0963.493200.3818.648
0.88.16327.60968.02406.962512.92011.482112.5579.871727.80921.48226.19315.242307.0955.802233.2925.598
0.98.16327.54518.09346.899212.92011.32212.7389.716827.80920.86526.99414.674307.0949.561268.8635.206
1.08.16327.48168.16326.836412.92011.16612.9209.563427.80920.27927.80914.120307.0944.407307.0947.470


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.83750.80190.60730.56451.30381.21370.80300.71662.76262.36891.18680.962230.13410.7202.22630.9170
0.20.83750.76940.62990.54421.30381.13560.84960.67672.76262.07581.31620.867030.1346.48303.54810.9295
0.30.83750.73950.65320.52461.30381.06730.89850.63922.76261.84921..45710.783330.1344.56305.31471.3867
0.40.83750.71210.67730.50581.30381.00720.94960.60392.76261.66911.60930.710930.1343.47207.52602.2886
0.50.83750.68680.70210.48771.30380.95391.00300.57082.76261.52241.77300.650030.1342.776210.1823.6354
0.60.83750.66350.72770.47051.30380.90631.05860.54002.76261.40061.94810.600530.1342.299613.2835.4268
0.70.83750.64180.75400.45391.30380.86361.11660.51152.76261.29802.13460.562430.1341.956516.8287.6631
0.80.83750.62170.78110.43811.30380.82521.17670.48532.76261.21042.33250.535730.1341.700420.81910.344
0.90.83750.60300.80890.42311.30380.79031.23910.46132.76261.13482.54180.520430.1341.503825.25413.470
1.00.83750.58550.83750.40881.30380.75871.30380.43952.76261.06882.76260.516630.1341.349330.13417.040

50.120.93620.04015.02413.88532.59630.32919.80317.52569.06559.18229.02223.128753.34266.9850.69115.905
0.220.93619.21615.61813.33932.59628.35721.01916.46369.06551.79432.37820.591753.34160.1484.63315.061
0.320.93618.45716.22712.80932.59626.62422.28615.45269.06546.06036.00818.328753.34111.36129.6125.249
0.420.93617.75516.85212.29532.59625.09123.60514.49369.06541.47839.91116.337753.3483.415185.6146.470
0.520..93617.10417.49311.79732.59623.72424.97513.58669.06537.72744.08714.620753.3465.445252.6578.724
0.620.93616.49818.15011.31532.59622.49826.39612.72969.06534.59748.53614.176753.3453.028330.73122.01
0.720.93615.93318.82310.84832.59621.39227.86911.92469.06531.94353.25812.005753.3444.012419.83176.33
0.820.93615.40519.51210.39832.59620.38929.39311.17169.06529.66258.25411.107753.3437.220519.97241.68
0.920.93614.91120.2169.963032.59619.47430.96910.46969.06527.67963.52310.482753.3431.956631.14318.06
1.020.93614.44720.9369.544132.59618.63832.5969.817969.06525.93869.06510.131753.3427.782753.34405.48

100.183.74680.15760.06455.497130.38121.3179.15670.028276.26236.71115.9792.3583013.41067.6201.9762.631
0.283.74676.85962.44353.307130.38113.4284.02865.771276.26207.15129.4182.1903013.4640.15337.8359.156
0.383.74673.81964.88551.181130.38106.4889.10561.719276.26184.20143.9573.1133013.4444.94517.8299.809
0.483.74671.00667.39049.118130.38100.3494.38757.872276.26165.85159.5765.1293013.4333.07741.94184.59
0.583.74668.39769.95847.119130.3894.86899.87454.230276.26150.84176.2958.2363013.4261.121010.2313.50
0.683.74665.97072.58945.182130.3889.957105.5750.793276.26138.30194.1052.4343013.4211.391322.6486.54
0.783.74663.70775.28443.309130.3885.526111.4647.561276.26127.67213.0047.7253013.4175.281679.1703.71
0.883.74661.59278.04141.498130.3881.506117.5644.534276.26118.54233.0044.1073013.4148.062079.7965.01
0.983.74659.61080.86239.751130.3877.842123.8741.712276.26110.59254.0841.5813013.4126.972524.51270.4
1.083.74657.74983.74638.067130.3874.490130.3839.095276.26103.62276.2640.1473013.4110.243013.41620.0


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.48760.47460.39420.37560.77040.73500.54730.50571.64171.47490.86620.743317.6737.98461.99101.0357
0.20.48760.46240.40380.36660.77040.70290.56930.48591.64171.34110.93430.688617.6735.34412.81350.9031
0.30.48760.45080.41360.35780.77040.67380.59190.46691.64171.23141.00700.638417.6734.02523.86611.0004
0.40.48760.43990.42360.34920.77040.64710.61520.44851.64171.13991.08420.592717.6733.21475.14861.3276
0.50.48760.42950.43380.34080.77040.62270.63930.43091.64171.06231.16590.551517.6732.66366.66101.8849
0.60.48760.41960.44410.33260.77040.60030.66410.41401.64170.99571.25200.514817.6732.26538.40352.6720
0.70.48760.41030.45470.32460.77040.57960.68960.39791.64170.93791.34270.482617.6731.965410.3763.6892
0.80.48760.40140.46550.31670.77040.56040.71580.38241.64170.88731.43790.454917.6731.732712.5784.9363
0.90.48760.39290.47640.30910.77040.54270.74280.36771.64170.84241.53750.431717.6731.547915.0116.4134
1.00.48760.38490.48760.30160.77040.52610.77040.35371.64170.80251.64170.413017.6731.398417.6738.1205

50.112.19011.8669.83939.364819.26118.37613.65012.58941.04236.87421.53918.403441.82199.5147.22522.001
0.212.19011.56010.0839.134119.26117.57314.20612.08441.04233.52523.27016.999441.82133.2068.35517.906
0.312.19011.27010.3318.907719.26116.84214.77911.59541.04230.77525.11015.704441.8299.85795.16419.490
0.412.19010.99410.5848.685619.26116.17415.36911.12441.04228.47527.05914.517441.8279.213127.6526.753
0.512.19010.73310.8418.468019.26115.56015.97510.66941.04226.52229.11713.440441.8265.056165.8239.694
0.612.19010.73310.8418.467919.26114.99416.59810.23241.04224.84031.28412.471441.8254.734209.6658.313
0.712.19010.73310.8418.468019.26114.47017.2399.810841.04223.37533.56011.612441.8246.888259.1882.611
0.812.19010.48411.1028.254519.26113.98517.8969.406841.04222.08635.94510.861441.8240.740314.38112.59
0.912.19010.24811.3678.045519.26113.53318.5709.019741.04220.94138.43910.220441.8235.807375.26148.24
1.012.1909.599912.1907.444619.26113.11119.2618.649541.04219.91741.0429.6868441.8231.774441.82189.58

100.148.75947.46439.35737.45877.04473.50354.60050.354164.17147.5086.14873.5991767.3798.06188.6487.571
0.248.75946.23940.33336.53577.04470.29456.82448.331164.17134.1093.07667.9781767.3532.77273.2371.092
0.348.75945.07941.32535.62977.04467.37059.11646.377164.17123.10100.4462.7911767.3399.37380.5377.317
0.448.75943.97842.33534.74077.04464.69661.47544.489164.17113.90108.2458.0411767.3316.75510.52106.245
0.548.75942.93343.36333.86877.04462.24063.90142.669164.17106.09116.4753.7251767.3260.08663.22157.88
0.648.75941.93844.40733.01477.04459.97666.39540.917164.1799.359125.1449.8451767.3218.75838.63232.21
0.748.75940.99145.46932.17777.04457.88268.95639.231164.1793.497134.2446.4001767.3187.321036.7329.25
0.848.75940.08746.54831.35777.04455.93971.58437.613164.1788.338143.7843.3901767.3162.691257.6449.00
0.948.75939.22447.64530.55577.04454.13174.28036.063164.1783.757153.7640.8161767.3142.921501.1591.44
1.048.75938.39948.75929.77077.04452.44377.04434.580164.1779.656164.1738.6771767.3126.751767.3756.59


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.22250.21950.19890.19420.34460.33660.28650.27510.71530.67830.49380.45437.40014.66471.63351.1431
0.20.22250.21650.20140.19190.34460.32900.29240.26970.71530.64530.51490.43587.40013.49611.98661.0057
0.30.22250.21370.20380.18960.34460.32180.29850.26430.71530.61580.53690.41827.40012.84692.41160.9403
0.40.22250.21100.20640.18740.34460.31500.30460.25910.71530.58920.55970.40157.40012.42682.90850.9468
0.50.22250.20840.20900.18520.34460.30850.31100.25400.71530.56520.58350.38577.40012.12783.47731.0252
0.60.22250.20580.21160.18310.34460.30240.31740.24910.71530.54340.60810.37077.40011.90144.1181.1755
0.70.22250.20340.21420.18100.34460.29660.32400.24430.71530.52360.63360.35667.40011.72244.83071.3977
0.80.22250.20100.21690.17890.34460.29100.33080.23970.71530.50550.65990.34347.40011.57675.61521.6918
0.90.22250.19870.21970.17690.34460.28580.33760.23520.71530.48890.68720.33117.40011.45526.47172.0578
1.00.22250.19650.22250.17500.34460.28080.34460.23080.71530.47360.71530.31977.40011.35237.40012.4958

50.15.56205.48694.96184.83548.61618.41637.13806.837217.88216.96012.26011.220185.00116.5739.23526.109
0.25.56205.41395.02604.77318.61618.22637.29216.690617.88216.13412.80910.728185.0087.15948.42522.173
0.35.56205.34305.09084.71158.61618.04537.44886.546617.88215.39013.37710.256185.0070.70759.36719.988
0.45.56205.27415.15624.65058.61617.87287.69806.405017.88214.71713.9639.8020185.0059.97772.06019.556
0.55.56205.20715.22234.59018.61617.70817.76976.266017.88214.10515.6879.3674185.0052.27586.50520.874
0.65.56205.14195.28904.53048.61617.55087.93396.129417.88213.54615.1938.9517185.0046.388102.7023.944
0.75.56205.07855.35634.47128.61617.40048.10075.995417.88213.03516.0008.5551185.0041.691120.6528.766
0.85.56205.01695.42424.41288.61617.25658.27005.864017.88212.56516.5008.1774185.0037.827140.3535.339
0.95.56204.95685.49284.35498.61617.11868.44185.735017.88212.13117.1817.8188185.0034.577161.8043.664
1.05.56204.89835.56204.29778.61616.98638.61615.608617.88211.73117.8827.4791185.0031.796185.0053.740

100.122.24821.94819.84919.34334.46433.66628.55327.34971.52867.84349.04044.873740.01466.29156.77104.14
0.222.24821.65620.10619.09334.46432.90729.17026.76271.52864.54051.23642.903740.01348.64193.5888.328
0.322.24821.37320.36518.84634.46432.18329.79726.18571.52861.56453.50841.009740.01282.80237.3979.512
0.422.24821.09820.62618.60234.4643149330.43425.61771.52858.87055.85639.190740.01239.86288.2077.693
0.522.24820.83020.89118.36034.46430.83531.08125.06071.52856.42258.27937.447740.01209.02346.0082.874
0.622.24820.56921.15718.12034.46430.20631.73824.51371.52854.18760.77735.779740.01185.44410.8195.052
0.722.24820.31621.42617.88334.46429.60432.40523.97571.52852.14163.35234.187740.01166.62482.61114.23
0.822.24820.06921.69817.64934.46429.02833.08123.44871.52850.25966.00132.671740.01151.14561.41140.41
0.922.24819.82921.97217.41634.46428.47633.76822.93071.52848.52468.72731.230740.01138.11647.21173.58
1.022.24819.59422.24817.18734.46427.94634.46422.42271.52846.91971.52829.864740.01126.96740.01213.75


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.02600.02590.02530.02520.03550.03540.03430.03410.06500.06450.06120.06040.58740.55320.37470.3374
0.20.02600.02580.02540.02510.03550.03520.03440.03390.06500.06410.06160.06000.58740.52210.39470.3201
0.30.02600.02570.02540.02500.03550.03510.03460.03380.06500.06360.06200.05970.58740.49400.41560.3038
0.40.02600.02570.02550.02500.03550.03500.03470.03370.06500.06320.06240.05930.58740.46850.43740.2883
0.50.02600.02560.02560.02490.03550.03480.03480.03360.06500.06280.06280.05890.58740.44520.46020.2737
0.60.02600.02550.02570.02480.03550.03470.03500.03350.06500.06240.06320.05860.58740.42400.48380.2601
0.70.02600.02540.02570.02480.03550.03460.03510.03340.06500.06200.06360.05820.58740.40460.50840.2473
0.80.02600.02540.02580.02470.03550.03440.03520.03330.06500.06160.06410.05790.58740.38680.53380.2355
0.90.02600.02530.02590.02470.03550.03430.03540.03310.06500.06120.06450.05760.58740.37050.56020.2246
1.00.02600.02520.02600.02460.03550.03420.03550.03300.06500.06080.06500.05720.58740.35560.58740.2145

50.10.62080.61950.60890.60630.85600.85330.83220.82701.57961.56991.49571.477414.615113.77629.19028.1593
0.20.62080.61810.61020.60500.85600.85060.83480.82441.57961.56031.50491.468414.615113.00829.72947.6677
0.30.62080.61680.61150.60370.85600.84790.83740.82181.57961.55071.51411.459314.615112.303310.28457.1920
0.40.62080.61550.61280.60240.85600.84530.84010.81921.57961.54131.52341.450314.615111.654910.85566.7321
0.50.62080.61410.61420.60110.85600.84260.84270.81661.57961.53191.53271.441314.615111.057011.44246.2882
0.60.62080.61280.61550.59980.85600.84000.84540.81401.57961.52271.54201.432414.615110.504512.04525.8601
0.70.62080.61150.61680.59850.85600.83730.84800.81141.57961.51351.55131.423414.61519.992912.66395.4479
0.80.62080.61020.61810.59720.85600.83470.85070.80881.57961.50451.56071.414614.61519.518413.29845.0515
0.90.62080.60890.61950.59590.85600.83210.85330.80631.57961.49551.57021.405714.61519.077413.94884.6711
1.00.62080.60760.62080.59460.85600.82960.85600.80371.57961.48661.57961.396914.61518.666814.61514.3065

100.12.47842.47312.43152.42113.41933.40863.32513.30446.31276.27415.97915.906358.449955.098836.749932.6162
0.22.47842.46782.43672.41593.41933.39803.33553.29406.31276.23596.01575.870158.449952.029738.910730.6432
0.32.47842.46262.44182.41083.41933.38743.34593.28376.31276.19806.05245.834058.449949.211841.134028.7328
0.42.47842.45732.44702.40563.41933.37683.35633.27346.31276.16056.08935.798058.449946.618343.419926.8850
0.52.47842.45212.45232.40053.41933.36643.36683.26316.31276.12336.12625.762158.449944.226045.768525.0998
0.62.47842.44692.45752.39533.41933.35593.37733.25296.31276.08656.16335.726358.449942.014648.179623.3772
0.72.47842.44172.46272.39023.41933.34563.38773.24266.31276.05006.20055.690758.449939.966250.653321.7172
0.82.47842.43662.46792.38513.41933.33523.39823.23246.31276.01396.23785.655258.449938.065253.189620.1197
0.92.47842.43142.47322.38003.41933.32503.40883.22226.31275.97816.27525.619858.449936.297855.788418.5849
1.02.47842.42632.47842.37483.41933.31473.41933.21206.31275.94276.31275.584558.449934.651858.449917.1126

3.2. Simulation Result Discussion

From Tables 18 and Figures 14, we observed that the modified Liu estimator consistently performs better than the Liu estimator and other existing estimators in this study.


0.70.80.90.99
OLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiuOLSRidgeLiuMLiu

10.10.06080.06070.05980.05950.08560.08540.08350.08300.16200.16110.15430.15271.56041.47821.03280.9358
0.20.06080.06060.05990.05940.08560.08510.08370.08280.16200.16020.15520.15191.56041.40311.08400.8901
0.30.06080.06050.06000.05930.08560.08490.08390.08250.16200.15930.15600.15111.56041.33431.13710.8463
0.40.06080.06030.06010.05920.08560.08460.08420.08230.16200.15850.15680.15041.56041.27121.19210.8042
0.50.06080.06020.06020.05910.08560.08440.08440.08210.16200.15760.15770.14961.56041.21321.24880.7641
0.60.06080.06010.06030.05900.08560.08410.08460.08190.16200.15680.15850.14881.56041.15981.30750.7257
0.70.06080.06000.06050.05890.08560.08390.08490.08170.16200.15590.15940.14811.56041.11051.36790.6892
0.80.06080.05990.06060.05880.08560.08370.08510.08150.16200.15510.16020.14731.56041.06491.43020.6546
0.90.06080.05980.06070.05870.08560.08340.08540.08120.16200.15430.16110.14661.56041.02261.49440.6218
1.00.06080.05960.06080.05860.08560.08320.08560.08100.16200.15350.16200.14591.56040.98341.56040.5908

50.11.52091.51771.49311.48692.14022.13382.08442.07224.05044.02743.85183.808539.010036.944625.449322.8472
0.21.52091.51461.49611.48392.14022.12752.09062.06604.05044.00473.87363.786939.010035.046526.806521.6023
0.31.52091.51151.49921.48082.14022.12132.09672.05994.05043.98223.89543.765439.010033.297228.201020.3948
0.41.52091.50841.50231.47782.14022.11502.10292.05394.05043.95983.91743.744039.010031.681329.632919.2247
0.51.52091.50531.50541.47472.14022.10882.10912.04784.05043.93773.93933.722639.010030.185031.102318.0919
0.61.52091.50221.50851.47172.14022.10272.11532.04174.05043.91583.96143.701339.010028.796432.609016.9966
0.71.52091.49911.51161.46862.14022.09652.12152.03574.05043.89413.98353.680139.010027.505234.153115.9387
0.81.52091.49611.51471.46562.14022.09042.12772.02964.05043.87264.00573.659039.010026.302235.734714.9181
0.91.52091.49301.51781.46262.14022.08432.13392.02364.05043.85124.02803.637939.010025.179437.353613.9350
1.01.52091.49001.52091.45952.14022.07832.14022.01754.05043.83014.05043.616939.010024.129639.010012.9893

100.16.08356.07095.97215.94758.56068.53538.33738.288216.201516.109615.406115.2323156.0399147.7764101.746291.3151
0.26.08356.05845.98445.93528.56068.51018.36208.263616.201516.018615.493415.1457156.0399140.1802107.184586.3224
0.36.08356.04595.99675.92308.56068.48508.38678.239216.201515.928415.581015.0595156.0399133.1789112.771581.4783
0.46.08356.03356.00915.91078.56068.46008.41148.214716.201515.839015.668814.9735156.0399126.7096118.507076.7827
0.56.08356.02116.02145.89858.56068.43528.43628.190316.201515.750415.756914.8878156.0399120.7180124.391072.2358
0.66.08356.00886.03385.88638.56068.41058.46108.165916.201515.662615.845314.8024156.0399115.1568130.423767.8373
0.76.08355.99656.04625.87418.56068.38598.48588.141616.201515.575615.933914.7172156.0399109.9844136.604963.5875
0.86.08355.98426.05865.86198.56068.36148.51078.117316.201515.489316.022914.6323156.0399105.1643142.934659.4862
0.96.08355.97206.07105.84978.56068.33708.53568.093116.201515.403816.112014.5476156.0399100.6645149.413055.5334
1.06.08355.95986.08355.83758.56068.31278.56068.068916.201515.319016.201514.4633156.039996.4563156.039951.7293

Results from Tables 18 show that increasing the sample size results in a decrease in the MSE values for each of the estimator. It is evident that MSE values of the estimators increase as the degree of correlation and the number of explanatory variables increase. The simulation results show that the proposed estimator performed best at most levels of the multicollinearity, sample size (), and number of explanatory variables with few exceptions. The only exceptions to its performance are when, and they are defined as follows: (i), , , and for , 5, and 10(ii), , and for and 10(iii), , and and 0.9 when (iv), , , and for , 5, and 10(v), , , and for , 5, and 10(vi) and for at , 5, and 10 including and 0.9 at

The instances mentioned above are the only times that ridge regression dominates the proposed estimator. The new estimator consistently dominates the OLS and the Liu estimator. Also, from Tables 18, we observed that the values of OLS and Liu are the same when .

Consistently when , 0.8, and 0.9, the proposed estimator performs better than other estimators at the different sample sizes irrespective of the values of the biasing parameter and . The fact that the ridge estimator dominates the proposed estimator in some of the exceptions mentioned earlier does not show that it performs better. It only shows that at those intervals, the performance of the new estimator drops. Thus, this necessitates the use of real-life data in the next session because the values of and will be estimated rather than choosing it arbitrarily.

4. Applications

We adopt two datasets to illustrate the theoretical findings of the paper. These include the Portland cement data and the French economy data.

4.1. Portland Dataset

The first user of this dataset was Woods et al. [16] and later adopted by Kaciranlar et al. [17] and Li and Yang [18]. It consists of one predictand, , which is the heat evolved after 180 days of curing measured in calories per gram of cement, and four predictors: X1 is the tricalcium aluminate, X2 is the tricalcium silicate, X3 is the tetracalcium aluminoferrite, and X4 is the β-dicalcium silicate. Variance inflation factors (VIFs) and a condition number are adopted to diagnose multicollinearity in the model [19]. The VIFs are 38.50, 254.42, 46.87, and 282.51 while the condition number is approximately 424. Both tests are evidence that the model possesses severe multicollinearity. The regression output is available in Table 9. According to Kejian [3], the optimum biasing parameter is expressed as


Coef.

2.19302.14782.16952.19302.1425
1.15331.16381.15871.15331.1650
0.75850.72610.74160.75830.7221
0.48630.49310.48990.48640.4939
MSE0.06380.06360.06290.06170.0608
MSPE2.89662.26652.88052.13421.2789
3.36361.72560.97822.6817

Following Özkale and Kaçiranlar [4], we replaced with if .

The ridge biasing parameters are computed by

We also adopt the leave-one-out crossvalidation to validate the performance of the estimators (see [20]). The performance of the estimator is assessed through the mean squared prediction error (MSPE). The result is presented in Table 9.

The theoretical results are computed as follows:

We observed that the theoretical comparisons stated in Sections 2.1, 2.2, and 2.3 are valid since each of the estimates are less than 1. From Table 9, the regression coefficients and MSE of the OLS and Liu estimators are approximately the same because is close to unity. Recall that the Liu estimator becomes OLS when . The proposed estimator possesses the smallest mean square error and average MSE of the validation error. Also, the performances of the estimators largely depend on their biasing parameters.

4.2. French Economy Dataset

The detail about this dataset is initially described in Chatterjee and Price [21] and is later available in the following references Malinvard [22] and Kejian [3]. It comprises one predictand, imports, and three predictor variables (domestic production, stock information, and domestic consumption) with eighteen observations. The variance inflation factors are , , and and the condition number 32612. Both results indicate the presence of severe multicollinearity. We analyzed the data using the biasing parameters for each of the estimators and present the results in Table 10. The proposed estimator performed the best in the sense of smaller MSE and MSPE. As mentioned earlier, the estimators’ performance is a function of the biasing parameter. The proposed estimator with the biasing parameter performs best. Also, the theoretical results agree with Section 2 and support the simulation and real-life findings.


Coef.

-19.7127-16.7613-18.8782-18.8410-19.2613
0.03270.14190.06360.06480.0493
0.40590.35760.39220.39140.3984
0.24210.07090.19370.19180.2161
MSE17.332621.3051916.601716.60294.9448
MSPE0.82670.43710.14110.13020.0871
0.05270.01320.94240.9701

The theoretical results are computed as follows:

5. Some Concluding Remarks

Both ridge regression and Liu estimators are widely accepted in the linear regression model as an alternative to the OLS estimator to circumvent the problem of multicollinearity. In this study, we proposed a modified Liu estimator, which possesses a single parameter which places it in the class of the ridge and Liu estimators. Theoretical comparisons, simulation study, and real-life applications evidently show that the proposed estimator consistently dominates the existing Liu estimator and ridge regression estimator under some conditions. We recommend the use of this estimator for the linear regression model with multicollinearity problem. We note that the proposed estimator can be extended to other regression models, for example, logistic regression, Poisson, ZIP, gamma, and related models, and these possibilities are under current investigation.

Data Availability

Data will be made available on request.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

References

  1. A. E. Hoerl and R. W. Kennard, “Ridge regression: biased estimation for nonorthogonal problems,” Technometrics, vol. 12, no. 1, pp. 55–67, 1970. View at: Publisher Site | Google Scholar
  2. A. F. Lukman, K. Ayinde, S. Binuomote, and O. A. Clement, “Modified ridge-type estimator to combat multicollinearity: application to chemical data,” Journal of Chemometrics, vol. 33, no. 5, article e3125, 2019. View at: Publisher Site | Google Scholar
  3. L. Kejian, “A new class of blased estimate in linear regression,” Communications in Statistics - Theory and Methods, vol. 22, no. 2, pp. 393–402, 1993. View at: Publisher Site | Google Scholar
  4. M. R. Özkale and S. Kaçiranlar, “The restricted and unrestricted two-parameter estimators,” Communications in Statistics - Theory and Methods, vol. 36, no. 15, pp. 2707–2725, 2007. View at: Publisher Site | Google Scholar
  5. A. F. Lukman, K. Ayinde, S. Siok Kun, and E. T. Adewuyi, “A modified new two-parameter estimator in a linear regression model,” Modelling and Simulation in Engineering, vol. 2019, Article ID 6342702, 10 pages, 2019. View at: Publisher Site | Google Scholar
  6. R. W. Farebrother, “Further results on the mean square error of ridge regression,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 38, no. 3, pp. 248–250, 1976. View at: Publisher Site | Google Scholar
  7. G. Trenkler and H. Toutenburg, “Mean squared error matrix comparisons between biased estimators—an overview of recent results,” Statistical Papers, vol. 31, no. 1, pp. 165–179, 1990. View at: Publisher Site | Google Scholar
  8. B. M. G. Kibria, “Performance of some new ridge regression estimators,” Communications in Statistics: Simulation and Computation, vol. 32, no. 2, pp. 419–435, 2003. View at: Publisher Site | Google Scholar
  9. B. M. G. Kibria and S. Banik, “Some ridge regression estimators and their performances,” Journal of Modern Applied Statistical Methods, vol. 15, no. 1, pp. 206–238, 2016. View at: Publisher Site | Google Scholar
  10. A. F. Lukman and K. Ayinde, “Review and classifications of the ridge parameter estimation techniques,” Hacettepe Journal of Mathematics and Statistics, vol. 46, no. 5, pp. 953–967, 2017. View at: Google Scholar
  11. M. Qasim, M. Amin, and T. Omer, “Performance of some new Liu parameters for the linear regression model,” Communications in Statistics - Theory and Methods, vol. 49, no. 17, pp. 4178–4196, 2019. View at: Publisher Site | Google Scholar
  12. D. G. Gibbons, “A simulation study of some ridge estimators,” Journal of the American Statistical Association, vol. 76, no. 373, pp. 131–139, 1981. View at: Publisher Site | Google Scholar
  13. J. P. Newhouse and S. D. Oman, An Evaluation of Ridge Estimators. A report prepared for United States Air Force project RAND, 1971.
  14. A. F. Lukman, K. Ayinde, and A. S. Ajiboye, “Monte Carlo study of some classification-based ridge parameter estimators,” Journal of Modern Applied Statistical Methods, vol. 16, no. 1, pp. 428–451, 2017. View at: Publisher Site | Google Scholar
  15. A. F. Lukman, K. Ayinde, B. Aladeitan, and R. Bamidele, “An unbiased estimator with prior information,” Arab Journal of Basic and Applied Sciences, vol. 27, no. 1, pp. 45–55, 2020. View at: Publisher Site | Google Scholar
  16. H. Woods, H. H. Steinour, and H. R. Starke, “Effect of composition of Portland cement on heat evolved during hardening,” Industrial and Engineering Chemistry, vol. 24, no. 11, pp. 1207–1214, 1932. View at: Publisher Site | Google Scholar
  17. S. Kaciranlar, S. Sakallioglu, F. Akdeniz, G. P. H. Styan, and H. J. Werner, “A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on Portland cement,” Sankhyā: The Indian Journal of Statistics, Series B, vol. 61, no. 3, pp. 443–459, 1999. View at: Google Scholar
  18. Y. Li and H. Yang, “A new Liu-type estimator in linear regression model,” Statistical Papers, vol. 53, no. 2, pp. 427–437, 2012. View at: Publisher Site | Google Scholar
  19. A. F. Lukman, Classification-Based Ridge, Lambert Academic Publishing, Mauritius, 2018.
  20. G. James, D. Witten, T. Hastie, and R. Tibshirani, An Introduction to Statistical Learning with Application to R, vol. 103 of Springer Texts in Statistics, 2013. View at: Publisher Site
  21. S. Chatterjee and A. S. Price, Regression Analysis by Example, Wiley, New York, 1977.
  22. E. Malinvard, Statistical Methods of Econometrics, North Holland, Amsterdam, 3rd edition, 1980.

Copyright © 2020 Adewale F. Lukman et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views412
Downloads460
Citations

Related articles

Article of the Year Award: Outstanding research contributions of 2020, as selected by our Chief Editors. Read the winning articles.