Abstract
This paper considers several estimators for estimating the stochastic restricted ridge regression estimators. A simulation study has been conducted to compare the performance of the estimators. The result from the simulation study shows that stochastic restricted ridge regression estimators outperform mixed estimator. A numerical example has been also given to illustrate the performance of the estimators.
1. Introduction
In regression analysis, researchers often encounter the problem of multicollinearity. Multicollinearity leads to high variance and instable parameter estimates when estimating linear regression models using ordinary least squares (OLS). So many researchers propose many ways to overcome this problem.
One method to overcome multicollinearity is to consider biased estimator, such as principal component regression estimator [1], ridge estimator [2], and Liu estimator (Liu, 1993). Ridge estimator is used by many researchers. When we use ridge estimator, how to choose the parameter is very important. A lot of ways of estimating the ridge parameter have been proposed. To mention a few, Hoerl and Kennard [2], Hoerl and Kennard [3], Hoerl et al. [4], McDonald and Galarneau [5], Kibria [6], Kibria et al. [7], and Najarian et al. [8].
Another method to overcome multicollinearity is to consider the restrictions. In the literature, the authors usually discuss two restrictions; one is linear restriction and the other is stochastic linear restriction. For the linear model with linear restriction, Sarkar [9], Kaciranlar et al. [10], and Xu and Yang [11–13] combined the restricted least squares estimator with the ridge estimator, the Liu estimator, almost unbiased ridge and Liu estimator, and and class estimator and proposed the restricted ridge estimator, restricted Liu estimator, restricted almost unbiased ridge and Liu estimator, and restricted and class estimator. For the linear model with stochastic linear restriction, Özkale [14] and Yang and Xu [15] proposed the stochastic restricted ridge estimator and the stochastic restricted Liu estimator.
The main objective of this paper is to propose some stochastic restricted ridge estimators. These estimators are introduced since they are expected to reduce the mean squared error (MSE) of the mixed method. In order to judge the performance of the estimators, the MSE is calculated. In this paper, we give a Monte simulation study to show the performance of the stochastic restricted ridge estimators. We will also analyze some important properties of the different estimators of the ridge parameter .
The paper is organized as follows. In Section 2, we describe the statistical methodology. The simulation results are discussed in Section 3 and a numerical example is given in Section 4. In Section 5 we give a brief summary and conclusions.
2. Methodology
In this section we introduce some stochastic restricted ridge estimators for estimating the ridge parameter based on the work of Hoerl and Kennard [2], Hoerl et al. [4], Schaeffer et al. [16], Kibria [6], and Kibria et al. [7].
2.1. Stochastic Restricted Ridge Estimator
Consider the following linear model: where is an vector of observation, is an design matrix of rank , is a vector denoting unknown coefficients, and is an random error vector with and .
Suppose that satisfies the following stochastic restriction; that is, where is a nonzero matrix with and is a known vector, and and . In this paper, we assumed that is independent of .
Using the mixed approach, Durbin [17], Theil and Goldberger [18], and Theil [19] introduced the mixed estimator (ME), which is defined as follows: The mixed estimator is an unbiased estimator. However, when multicollinearity exists, the mixed estimator is no longer a good estimator.
Özkale [14] proposed the following stochastic restricted ridge estimator (SRRE):
For any particular estimator of , the total MSE is defined as Thus we have where .
2.2. Estimating
The most classical ridge estimator for linear regression is the following: proposed by Hoerl and Kennard [2, 3], where denote the maximum element of , , , and is the estimator of . Hoerl et al. [4] introduce an alternative of the estimator of , which is defined as follows:
In Schaeffer et al. [16] a modified version of this estimator is proposed as follows:
In Kibria et al. [7], a new estimator is proposed as follows: Finally the following estimators are considered:
3. The Monte Carlo Simulation
The main purpose of this paper is to see the effect of multicollinearity on mixed estimator and SRRE.
3.1. Simulation Technique
To achieve different degrees of collinearity, following McDonald and Galarneau [5] and Liu [20], the explanatory variables were generated using the following method: where are independent standard normal pseudorandom numbers and is specified so that the correlation between any two explanatory variables is given by . These variables are then standardized so that and are in correlation forms. Four different sets of correlations corresponding to are discussed. Then the dependent variable are determined by where are independent normal pseudorandom numbers. In this paper, we consider , , and . Suppose that where
3.2. Results and Discussion
Tables 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, and 20 present the results of our Monte Carlo experiment concerning the MSEs of the different proposed stochastic restricted ridge estimators and the mixed estimator.
3.2.1. The Performance as a Function of
Increasing the number of observations has a negative effect on all of the estimators. With the increasing of observations, the MSE of SRRE and ME are decreasing, although SRRE is always better than the ME. However, the performance of the SSRE over the ME is not very obvious. Thus, both measurements of performances show that the advantage of SRRE decreases with the increase of the number of observations.
3.2.2. The Performance as a Function of
Increasing the independent variables has a positive effect on all of the estimators. With the increasing of observations, the MSE of SRRE and ME are increasing. When increasing the independent variables, the SRRE is better than the ME, and the performance of the SRRE over the ME is very obvious. Thus, both measurements of performances show that the advantage of SRRE increases with the increase of the number of observations.
3.2.3. The Performance as a Function of
Increasing the degree of correlation has a clear negative impact on the ME and SRRE. MSEs of these estimators actually increase as increases. Thus, the gain of applying SRRE increases as the degree of correlation increases.
3.2.4. The Performance as a Function of
Increasing has a clear negative impact on the ME and SRRE. MSEs of these estimators actually increase as increases. When is big, the performance of the SRRE over the ME is very obvious.
4. Numerical Example
In this example, we discuss the dataset on Portland cement originally due to Woods et al. [21], and the dataset has then been widely analyzed in the literature such as Kaciranlar et al. [10]. Firstly, we assemble the data as follows:
Consider the following matrix restriction: where The estimated regression coefficients along with the MSE are presented in Table 21.
From Table 21, we can see that the proposed SRRE are performing better than the ME in the sense of smaller MSE. We also noted that the estimator performed the best followed by .
5. Concluding Remarks
In this paper, we discussed the usefulness of the RR to obtain better parameter estimators in linear regressions model with stochastic restrictions when the independent variables are highly correlated. By a Monte Carlo experiment, we evaluate some recently proposed methods of estimating the ridge parameter that we apply in linear regressions model. The results show that the newly proposed estimators outperform the mixed estimator.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported by the Scientific Research Foundation of Chongqing University of Arts and Sciences (Grant no. R2013SC12) and the Program for Innovation Team Building at Institutions of Higher Education in Chongqing (Grant no. KJTD201321).