Research Article | Open Access
Mayuri Pandya, Krishnam Bhatt, Paresh Andharia, "Bayes Estimation of Two-Phase Linear Regression Model", Journal of Quality and Reliability Engineering, vol. 2011, Article ID 357814, 9 pages, 2011. https://doi.org/10.1155/2011/357814
Bayes Estimation of Two-Phase Linear Regression Model
Let the regression model be , where are i. i. d. N () random errors with variance but later it was found that there was a change in the system at some point of time and it is reflected in the sequence after by change in slope, regression parameter . The problem of study is when and where this change has started occurring. This is called change point inference problem. The estimators of , are derived under asymmetric loss functions, namely, Linex loss & General Entropy loss functions. The effects of correct and wrong prior information on the Bayes estimates are studied.
Regression analysis is an important statistical technique to analyze data in social, medical, and engineering sciences. Quite often, in practice, the regression coefficients are assumed constant. In many real-life problems, however, theoretical or empirical deliberations suggest models with occasionally changing one or more of its parameters. The main parameter of interest in such regression analyses is the shift point parameter, which indexes when or where the unknown change occurred.
A variety of problems, such as switching straight lines , shifts of level or slope in linear time series models , detection of ovulation time in women , and many others, have been studied during the last two decades. Holbert , while reviewing Bayesian developments in structural change from 1968 onward, gives a variety of interesting examples from economics and biology. The monograph by Broemeling and Tsurumi  provides a complete study of structural change in the linear model from the Bayesian viewpoint.
Bayesian inference of the shift point parameter assumes availability of the prior distribution of the changing model parameters. Bansal and Chakravarty  had proposed to study the effect of an ESD prior for the changed slope parameter of the two-phase linear regression (TPLR) model on the Bayes estimates of the shift point and also on the posterior odds ratio (POR) to detect a change in the simple regression model.
In this paper, we studied a TPLR model. In Section 2, we have given a change point model TPLR. In Sections 3.1 and 3.2, we obtained posterior densities of considering unknown and of and considering known, respectively. We derive Bayes estimators of , and under symmetric loss functions in Section 4 and asymmetric loss functions in Section 5. We have studied the sensitivity of the Bayes estimators of when prior specifications deviate from the true values in Section 6. In Section 7, we have presented a numerical study to illustrate the above technique on generated observations. In this study, we have generated observations from the proposed model and have computed the Bayes estimates of and of other parameters. Section 8 concludes the research paper.
2. Two-Phase Linear Regression Model
The TPLR model is one of the many models, which exhibits structural change. Holbert  used a Bayesian approach, based on TPLR model, to reexamine the McGee and Kotz  data for stock market sales volume and reached the same conclusion that the abolition of splitups did hurt the regional exchanges.
The TPLR model is defined as where ’s are i. i. d. N random errors with variance , is a nonstochastic explanatory variable, and the regression parameters (α1, β1) ≠ (α2, β2). The shift point is such that if there is no shift, but when exactly one shift has occurred.
3. Bayes Estimation
The ML method, as well as other classical approaches are based only on the empirical information provided by the data. However, when there is some technical knowledge on the parameters of the distribution available, a Bayes procedure seems to be an attractive inferential method. The Bayes procedure is based on a posterior density, say, , which is proportional to the product of the likelihood function , with a prior joint density, say, representing uncertainty on the parameters values where denotes .
The likelihood function of β1, β2, and , given the sample information , is where
3.1. Using Gamma Prior on and Conditional Informative Priors on 1, 2 with Unknown
We consider the TPLR model (1) with unknown . As in Broemeling and Tsurumi , we suppose that the shift point is a priori uniformly distributed over the set and is independent of β1 and β2. We also suppose that some information on β1 and β2 are available that can be expressed in terms of conditional prior probability densities on β1 and β2.
We have conditional prior density on β1 and β2 given , with as We also suppose that some information on is available and that technical knowledge can be given in terms of prior mean and coefficient of variation . We suppose marginal prior distribution of to be gamma distribution with mean where is gamma function same as explained in (8).
The integral representation of is as below , Gradshteyn and Ryzhik [8, page 934]
The gamma function (Euler’s integral of the second kind)
, (Euler) Gradshteyn and Ryzhik [8, page 933], is defined as
If the prior information is given in terms of prior mean and coefficient of variation , then the parameters and can be obtained by solving
Hence, joint prior pdf of and , say is where
Joint posterior density of , and say, (β1, β2, , ) is where , and are as given in (4).
is the gamma function as explained in (8).
Marginal posterior density of change point , say is is as given in (15).
3.2. Using Conditional Informative Priors on with Known
Hence, joint prior pdf of β1, β2, and , say β1, β2, ) is where Joint posterior density of β1, β2 and say, is where, , and are as given in (4).
Where is the marginal density of given by
And the integrals,
, and are as given in (4).
, are as given in (4).
4. Bayes Estimates under SymmetricLoss Function
The Bayes estimator of a generic parameter (or function there of) α based on a squared error loss (SEL) function: where is a decision rule to estimate α, is the posterior mean. As a consequence, the SEL function relative to an integer parameter,
Hence, the Bayesian estimate of an integer-valued parameter under the SEL function is no longer the posterior mean and can be obtained by numerically minimizing the corresponding posterior loss. Generally, such a Bayesian estimate is equal to the nearest integer value to the posterior mean. So, we consider the nearest value to the posterior mean as Bayes Estimate.
Other Bayes estimators of α based on the loss functions is the posterior median and posterior mode, respectively.
5. Asymmetric Loss Function
The Loss function provides a measure of the financial consequences arising from a wrong decision rule d to estimate an unknown quantity (a generic parameter or function thereof) α. The choice of the appropriate loss function depends on financial considerations only and is independent from the estimation procedure used. The use of symmetric loss function was found to be generally inappropriate, since for example, an overestimation of the reliability function is usually much more serious than an underestimation.
A useful asymmetric loss, known as the Linex loss function was introduced by Varian . Under the assumption that the minimal loss occurs at α, the Linex loss function can be expressed as
The sign of the shape parameter reflects the direction of the asymmetry, if overestimation is more serious than underestimation, and vice versa, and the magnitude of reflects the degree of asymmetry.
The posterior expectation of the Linex loss function is where denotes the expectation of with respect to the posterior density . The Bayes estimate is the value of that minimizes provided that exists and is finite.
5.1. Assuming Unknown
Minimizing the posterior expectation of the Linex loss function . Where denotes the expectation of with respect to the posterior density given in (17), we get the Bayes estimate of by means of the nearest integer value to (37), say , as under. We get the Bayes estimators of using Linex loss function, respectively, as
Another loss function, called general entropy (GE) loss function, proposed by Calabria and Pulcini , is given by The Bayes estimate is the value of that minimizes : provided that exists and is finite.
Combining the General Entropy Loss with the posterior density (17), we get the estimate by means of the nearest integer value to (40), say , as below. We get the Bayes estimates of using General Entropy loss function as where is as given in (15).
5.2. Assuming Known
Combining the Linex loss with the posterior distributions (27) and (28), respectively, we get the Bayes estimators of β1 and β2 using Linex loss function as where , , and are same as given in (21), (22), (23), and (24), respectively. are as given in (4).
Minimizing expectation and then taking expectation with respect to posterior density , we get the estimate by means of the nearest integer value to (44) say , as below. We get the Bayes estimates of using General Entropy loss function as where is same as given in (25).
Note 1. The confluent hypergeometric function of the first kind  is a degenerate form of the hypergeometric function which arises as a solution to the confluent hypergeometric differential equation. It is also known as Kummer's function of the first kind and denoted by , defined as follows:
With Pochhammer coefficients for and [12, page 755], also has an integral representation
The symbols Γ and denoting the usual functions gamma and beta, respectively.
When and are both integer, some special results are obtained. If , and either or , the series yields a polynomial with a finite number of terms. If integer , the function is undefined.
Note 2. ] is called a generalized hypergeometric series and defined as (Gradshteyn and Ryzhik [8, page 1045]).
] has series expansion In many special cases hypergeometric is automatically converted to other functions.
For , hypergeometric [ list, list, ] has a branch cut discontinuity in the complex plane running from 1 to .
Hypergeometric (Regularized) is finite for all finite values of its argument so long as .
Note 3. is the beta function Euler’s integral of the first kind defined as Gradshteyn and Ryzhik [8, pages 948, 950],
The gamma function is as explained in (8).
Minimizing expected loss function and using posterior distributions (27) and (28), we get the Bayes estimates of using General Entropy loss function, respectively, as where Hypergeometric and hypergeometric are hypergeometric functions same as explained in Notes 1 and 2, respectively. , and are as explained in (23) and (50), respectively. is as given in (21). and are gamma functions same as explained in (8). , and are as given in (4) where Hypergeometric and hypergeometric are hypergeometric functions same as explained in Notes 1 and 2, respectively. , and are as explained in (23) and (50), respectively. is as given in (21).
is as given in (21). and are gamma functions same as explained in (8). , and are as given in (4).
Remark 1. Putting in (40) and (44), we get the Bayes estimators of , posterior means under the squared error loss as given in (31) and (32). Note that for , the GE loss function reduces to the squared error loss function.
6. Numerical Study
Let us consider the two-phase regression model where 's are i.i.d. random errors. We take the first 15 values of and from Table 4.1 of Zellner  to generate 15 sample values . The generated sample values are given in Table 1. The , and themselves were random observations. and were from standard normal distribution and precision was from gamma distribution with =1 and coefficient of variation , respectively, in .
We have calculated posterior mean, posterior median and posterior mode of m. The results are shown in Table 2.
Table 3 shows that for small values of , 0.5, 0.2, 0.1 Linex loss function is almost symmetric and nearly quadratic and the values of the Bayes estimate under such a loss is not far from the posterior mean. Table 3 also shows that for , 1.2, Bayes estimates are less than actual value of .
It can be seen from Table 3 that positive sign of shape parameter of loss functions reflects overestimation is more serious than underestimation. Thus, problem of overestimation can be solved by taking the value of shape parameter of Linex and General Entropy loss functions positive and high.
For , −2, Bayes estimates are quite large than actual value . It can be seen from Table 3 that the negative sign of shape parameter of loss functions reflects underestimation is more serious than overestimation. Thus, problem of underestimation can be solved by taking the value of shape parameters of Linex and General Entropy loss functions negative.
7. Sensitivity of Bayes Estimates
In this section, we study the sensitivity of the Bayes estimator, obtained in Sections 4 and 5 with respect to change in the prior of parameters. The mean μ of gamma prior on has been used as prior information in computing the parameters of the prior. We have computed posterior mean using (31) and using (32) for the data given in Table 1 considering different sets of values of (μ). Following Calabria and Pulcini , we also assume the prior information to be correct if the true value of is closed to prior mean μ and is assumed to be wrong if is far from μ. We observed that the posterior mode appears to be robust with respect to the correct choice of the prior density of and also with a wrong choice of the prior density of . This can be seen from Table 5.
Table 5 shows that when prior mean μ = 1 = actual value of , it means correct choice of prior of , The values of Bayes estimator posterior mode is 4. It gives correct estimation of change point. Now, when μ = 0.5 and 1.5 (far from true value of = 1), it means wrong choice of prior of . The value of Bayes estimator of posterior mode remains 4. But, posterior mean and posterior median do not remain same for wrong choice of prior of . Thus, posterior mode is not sensitive with wrong choice of prior density of . While, posterior mean and posterior median are sensitive with wrong choice of prior density of .
8. Simulation Study
In Sections 4 and 5 we have obtained Bayes estimates of on the basis of the generated data given in Table 1 for given values of parameters. To justify the results, we have generated 10,000 different random samples with , 3.3, 3.4, , 3.6, 3.7 and obtained the frequency distributions of posterior mean, median of with the correct prior consideration. The result is shown in Tables 2 and 3. The value of shape parameter of the general entropy loss and Linex loss used in simulation study for shift point is taken as 0.1.
We have also simulated several standard normal samples. For each β1, β2, and and , 1000 pseudorandom samples from two-phase linear regression model discussed in Section 2 have been simulated and Bayes estimators of change point has been computed using and for different prior mean μ.
Table 6 leads to conclusion that performance of , posterior mode’s and posterior median’s has better performance than that of posterior mean of change point explained in Sections 4 and 5. 46% values of posterior mean are closed to actual value of change point with correct choice of prior. 62% values of posterior median are closed to actual value of change point with correct choice of prior. 70% values of posterior mode are close to correct values of change point with correct prior considerations. 65% values of are closed to actual values of . 66% values of are closed to actual values of .
In this study, we are discussing the Bayes estimator of shift point, the integer parameter, posterior mean is less appealing. Posterior median and posterior mode appear as better estimators as they would be always integer. Our numerical study showed that the Bayes estimators posterior mode of is robust with respect to the correct choice of the prior specifications on and wrong choice of the prior specifications on , posterior median and posterior mode are sensitive in case prior specifications on deviate simultaneously from the true values. Here, we discussed regression model with one change point, in practice it may have two or more change point models. One can apply these models to econometric data such as poverty and irrigation.
The authors would like to thank the editor and the referee for their valuable suggestions which improved the earlier version of the paper.
- P. E. Ferreira, “A bayesian analysis of switching linear model: a known number of regimes,” Journal of the American Statistical Association, vol. 70, pp. 370–374, 1975.
- A. F. M. Smith, “Change point problems: approaches and applications,” in Bayesian Statistics, J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, Eds., pp. 83–98, University Press, Valencia, España, 1980.
- R. L. Carter and B. J. N. Blight, “A bayesian change point problem with an application to the prediction and detection of ovulation in women,” in Practical Bayesian Statistics, G. K. Kanji, Ed., pp. 229–230, Longman, London, UK, 1983.
- D. Holbert, “A Bayesian analysis of a switching linear model: a known number of regimes,” Journal of the American Statistical Association, vol. 70, pp. 370–374, 1982.
- L. D. Broemeling and H. Tsurumi, Econometrics and Structural Change, Marcel Dekker, New York, NY, USA, 1987.
- A. K. Bansal and S. Chakravarty, “Bayes estimation and detection of a change in prior distribution of the regression parameter,” in Bayesian Analysis in Statistics and Econometrics, Donald A. Berry and M. Kathryn, Eds., pp. 257–266, Wiley-Interscience, New York, NY, USA, 1996.
- V. E. McGee and S. Kotz, “Piecewise regression,” Journal of the American Statistical Association, vol. 65, pp. 1109–1124, 1970.
- I. S. Gradshteyn and I. M. Ryzhik, Table of Integrals, Series and Products, Academic Press, New York, NY, USA, 1965.
- H. R. Varian, “A Bayesian approach to real estate assessment,” in Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage, S. E. Fienberg and A. Zellner, Eds., pp. 195–208, North-Holland, Amsterdam, The Netherlands, 1975.
- R. Calabria and G. Pulcini, “An engineering approach to Bayes estimation for the Wseibull distribution,” Microelectronics Reliability, vol. 34, no. 5, pp. 789–802, 1994.
- E. E. Kummer, “Über die hypergeometrische reihe F(a,b;x),” Journal für die reine und angewandte Mathematik, vol. 15, pp. 39–83, 1836.
- G. Arfken, “Confluent hypergeometric functions,” in Mathematical Methods for Physicists, chapter 13.6, pp. 753–758, Academic Press, Orlando, Fla, USA, 3rd edition, 1985.
- A. Zellner, An Introduction to Bayesian Inference in Econometrics, Wiley, New York, NY, USA, 1971.
Copyright © 2011 Mayuri Pandya et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.