- About this Journal ·
- Abstracting and Indexing ·
- Aims and Scope ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Volume 2012 (2012), Article ID 872956, 6 pages
Constrained Parameters in Applications: Review of Issues and Approaches
Office of Research and Development, US Environmental Protection Agency, 1200 Pennsylvania Avenue, Washington, DC 20460, USA
Received 6 January 2012; Accepted 23 February 2012
Academic Editors: J. Chow and J. Crezee
Copyright © 2012 Leonid Kopylev. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
This review article provides an introduction to statistical issues that arise when some statistical model parameters are constrained. This often happens in applications, in particular in testing for variance components (e.g., genomics) and construction of one-sided confidence intervals (e.g., environmental risk analysis). Heuristic explanations are provided, and a number of general and recent statistical results that appeared in statistical literature are summarized for use in applications. Simulation results are shown for illustration of consequences of ignoring parameters on the boundary. Special attention is paid to likelihood ratio tests, but other approaches to confidence interval construction, such as Wald, bootstrap, and Bayesian are also briefly discussed. This paper presents examples from the risk assessment field and genomics, but all conclusions apply to whenever one-sided testing is conducted. Recommendations are provided for dealing with parameters on the boundary for a range of situations.
Most standard results in statistical literature (e.g., ) are derived assuming that the true values of parameters are interior to the parametric space. It is well known that, under some very general conditions, the usual asymptotic theory of estimation based on the maximum likelihood estimates (MLEs) and the usual asymptotic theory of tests based on the likelihood ratio tests (LRTs) are valid and provide useful tools for meaningful statistical inference in large samples. Typically, we conclude the asymptotic normality of the MLE and the asymptotic chi-square distribution of −2 * ln (LRT) with suitable degrees of freedom (df) under a null hypothesis. This is what is generally covered in statistical textbooks.
However, the asymptotic distribution of the LRT in situations when regularity conditions do not hold, in particular when parameters of interest and/or nuisance parameters are on the boundary of a parameter space, is important in many applications, for example, in variance components testing in many natural science applications; however, statistical textbooks do not cover this subject. In the field of genetics and biology, there is indeed considerable amount of interest in statistical issues when some parameters may lie on the boundaries: in almost 1000 references to pioneering paper  on statistical results when parameters are on the boundary, about half come from publications in genetics and biology fields.
This review paper is organized as follows. Section 2 first introduces some examples from various applications that provide motivation. Next, it provides heuristic reasoning to demonstrate complicated statistical issues arising in boundary parameter situations using simple examples. Section 3 surveys literature on several approaches to parameters on the boundary and summarizes published results, paying most attention to the LRT. Finally, discussion section summarizes advice for scientists working in applications.
2. Examples and Heuristics
2.1. Statistical Issues due to Boundaries: Examples and Motivation
We start with basic examples of normal distribution that we will carry through the next section.
Example 1. Interior and boundary points of parametric spacefor parameters of interest and nuisance parameters.
Let ,(a) is known. versus results in the parametric space , 0 is an interior point, and is parameter of interest,(b) is known. versus results in the parametric space , 0 is a boundary point, and is parameter of interest,(c) is unknown. versus results in the parametric space and 1 being an interior point. is parameter of interest and is called a nuisance parameter.
We continue with an application from the risk assessment field.
Example 2 (Constraints in multistage model). Toxicological experience and principles indicate that the response will generally be bounded and nondecreasing in the dose level . Monotonicity may require constraints on the parameters of a dose response model, or constraints also may be needed to ensure that the probability lies within . Any of these constraints on a parameter leads to a parametric space that includes boundary. Quantal response models represent the probability of a quantal response, like presence or absence of a particular type of cancer, in relation to dose () in a model with parameter vector . The experimental data consist of counts of animals exposed to a chemical, the numbers exhibiting the response, and the dose levels (e.g., , , , ) from a bioassay conducted with mice or rats, typically in 2–5 dose groups and the control group, each having 10–50 animals. Commonly used model for modeling bioassay data is called multistage All parameters of the multistage model are nonnegative, resulting in parameter space [0, +∞), with true or estimated values of parameters may be 0 or be on the boundary, and is called the order of the multistage model, for example, model with is called cubic. Maximum likelihood estimation for the parameters employs the binomial likelihood where is the binomial coefficient. The benchmark dose method [4, 5] consists of estimating a lower one-sided confidence limit for the dose associated with a specified increase in adverse response (i.e., increased risk) above the background level.
In the context of dose-response modeling, the statistical problems arising when true values of parameters of a dose-response model may be on the boundary were acknowledged long ago  but have not been resolved clearly for the practitioners of dose-response modeling. Molenberghs and Verbeke  provide a nice discussion of these issues in the risk assessment context.
Our final example is variance component testing in genetics.
Example 3 (ACE model in twin studies). The ACE model used in twin studies consists of the three components: A (additive genetics), C (common environment), and E (unique environment). Given the ACE model, researchers can test which of the three variance components are present. The likelihood ratio statistic is generally used as the basis for these tests . The statistical test of AE against ACE alternative is the test of the hypothesis that there is no influence of environmental factors common among twins. This corresponds to the statistical test that variance component corresponding to C is 0. The test of E against ACE is the test of no familial correlation, that is, of the hypothesis that there is no influence of genetic or shared environmental factors. This corresponds to the statistical test of the null hypothesis that variance components corresponding to A and C are both 0. As variance is always nonnegative, the null hypothesis that variance component(s) is 0 is on the boundary of that parameter space.
What follows from the examples above is that most common causes of boundaries are one-sided testing and a dual problem of one-sided confidence interval construction. Suppose that we are testing the null hypothesis for a point . The two-sided test versus , and corresponding parameter space is . However, for one-sided test: , versus the corresponding parameter space is , and is necessarily a boundary point! Therefore, in one-sided testing, under null hypothesis, the parameter of interest is always on the boundary of the parametric space, and standard results, like one-sided t-test, are not applicable .
2.2.1. Maximum Likelihood Estimation
Example 1 (continued). Unrestricted and restricted maximum likelihood estimate (MLE) of mean of Normal , when true .(a)-unrestricted. In this case, is estimated by . is normal (Figure 1(a));(b). In this case, is estimated by which is a mixture of point mass at 0 and half-normal (Figure 1(b)).
Example 2 (continued). Figure 2 shows joint distribution of the parameters of cubic multistage model when no true parameters are on the boundary (top panel) and when a true quadratic term = 0 (bottom panel). When no parameters are on the boundary, the joint distribution of parameters is multivariate normal, as expected. When the quadratic parameter is on the boundary, it is half-normal, but the other parameters are affected and are further from normality.
In the simple univariate example above, the distribution of restricted MLE is easily described. However, in a general multivariate case, the joint distribution of MLE is generally not multivariate Normal when just a single parameter has true value on the boundary (Section 3).
2.2.2. Likelihood Ratio Tests
The following example is continuation of Example 1(b).
Example 4. Likelihood ratio test (LRT) for a single normal random variable (a)Two-sided test versus . The Likelihood Ratio (LR) is as supremum in the denominator is achieved when is equal to . The distribution of −2 * is a familiar with 1 degree of freedom.(b)One-sided test versus . The likelihood ratio is
In this situation, supremum in the denominator depends on whether is positive or negative, and distribution of −2 * is a mixture of point mass at 0 and with 1df.
Example 4 demonstrates that when the LRT is used for one-sided tests, the distribution is not with 1 degree of freedom anymore and can be fairly complicated. Various cases are described in the next section.
3.1. Likelihood Ratio Tests
A pioneering theoretical article , following mainly on earlier work [8–11], proposed a general approach to deriving distribution of the LRT when some parameters of interest and/or nuisance parameters may be on the boundary of a parameter space. They showed that, under fairly general regularity conditions, the distribution of the LRT depends only on number of parameters of interest and nuisance parameters on the boundary and off the boundary. They also explicitly derived the LRT for several particular situations but were not completely correct for some of these. Sinha et al.  and Kopylev and Sinha  solved some of the cases in  and derived asymptotic distribution of the LRT for some other important special cases. All these results are summarized below.(i)When one or several parameters of interest not on the boundary and some nuisance parameters are on the boundary, the asymptotic distribution of the LRT remains with corresponding degrees of freedom .(ii) When one parameter of interest is on the boundary and no nuisance parameters are on the boundary (e.g., Example 3 above with testing AE versus ACE), the asymptotic distribution of the LRT is a 50 : 50 mixture of point mass at 0 and with 1df .(iii) When one parameter of interest is on the boundary and one parameter of interest not on the boundary and no nuisance parameters are on the boundary, the asymptotic distribution of the LRT is 50 : 50 mixture of with 1df and with 2 df .(iv) When two parameters of interest are on the boundary and no nuisance parameters are on the boundary (e.g., Example 3 above), the asymptotic distribution of the LRT is a mixture of point mass at 0, with 1 df and with 2 df with mixing parameters , 1/2 and , where , /, and is an information matrix (e.g.,  for derivation of ). Unfortunately, this example in  has probabilities of the mixture components inverted, most likely a misprint. A lucky erroneous inversion of the sign in calculations  allowed them to make the correct recommendation.(v) When one parameter of interest is on the boundary and one nuisance parameter is on the boundary, the asymptotic distribution of the LRT is a mixture of point mass at 0, with 1df and with 2df with mixing parameters , 1/2 and , where , , and is an information matrix. When , asymptotic distribution is not a mixture of chi-squares but is explicitly derived .(vi) When one parameter of interest and two nuisance parameters are on the boundary or two parameters of interest and one nuisance parameter are on the boundary, Kopylev and Sinha  derive a straightforward way for simulation of the asymptotic distribution of the LRT in these cases.
Table 1 (adapted from ) demonstrates that, in case of one parameter of interest and two nuisance parameters on the boundary, distribution of the LRT can be very different from with 1 df and use of with 1 df can be anticonservative.
The correct joint asymptotic distribution of the MLE in case of some parameters on the boundary is described in , following earlier work . It turns out that the asymptotic distribution of the MLE is not multivariate normal distribution anymore (cf. Example 1). Sinha et al.  adapted general statistical theory  for the multistage model and showed that asymptotic joint distribution of the MLE can be easily simulated starting from the standard multivariate Normal distribution. They also demonstrated how to construct confidence intervals following  and investigated the coverage of Wald-like confidence intervals for the multistage model. For sample sizes common for bioassays, the two-sided coverage of Wald-like intervals was comparable with that of the profile likelihood method for a wide range of scenarios. However, one-sided coverage by Wald-like intervals was found to be different from the nominal coverage .
Bootstrap approaches have been suggested by many authors for use in the field of risk assessment. Bailer and Smith  investigated coverage of one-sided confidence intervals for the risk based upon parametric and nonparametric approaches to bootstrap, for a number of scenarios including true values of parameters on or near the boundary. They found that both parametric and nonparametric bootstraps may be anticonservative for some scenarios and very conservative for others.
There is only limited literature on Bayesian approaches to boundary parameter problems, especially when nuisance parameters on the boundary are involved. Gelfand et al.  recommended use of Bayesian methods when a parameter is constrained. However, this approach had been shown not to work well when distribution of the parameter has a nonzero mass concentrated on a boundary (e.g., ). For situations when there is a mass on the boundary, publications [16, 17] suggest a Bayesian approach.
In the field of science, there is indeed considerable amount of interest in statistical issues when some parameters may lie on the boundaries. However, a number of publications suggest, if one is willing to use a conservative test, to ignore the possibility of parameters on the boundary as these publications claim that using the usual chi-square distribution with appropriate degrees of freedom (as when no parameters are on the boundary) will result in a conservative test (e.g., [18, 19] in the field of genetics, and  in the field of psychology). Nevertheless, results in  demonstrate that when one or more nuisance parameters are on the boundary, the suggested strategy of ignoring parameters on the boundary may result in anticonservative tests for many different correlation structures between parameters that are on the boundary, as seen in Table 1 illustrating what happens with two nuisance parameters on the boundary.
Therefore, correct asymptotic distribution of the likelihood ratio test must be used whenever a parameter of interest on the boundary, for example, one-sided tests, is encountered, especially if nuisance parameters on the boundary are also present. Asymptotic results for many situations that are routine in applications are summarized in Section 3.1. However, in case a different situation is encountered in applications, the theory  and explanations and clarifications  can be used to derive correct asymptotic distribution of the LRT for that particular situation. In some cases, that asymptotic distribution may be explicit, but it is likely that in most cases the asymptotic distribution will have to be simulated.
From other approaches to confidence interval construction described in Section 3, the Bayesian approach (e.g., [16, 17]) is the most interesting, but more research is required especially when there are constrained nuisance parameters. Another approach useful in applications would be to try to use a different model without boundary parameters or, if possible, reparameterize existing model so that to avoid situations with parameters on the boundary.
The author wishes to thank Professor Bimal Sinha of the Department of Mathematics and Statistics at University of Maryland Baltimore County and John Fox of Office of Research and Development of US EPA for their advice and encouragement on this project. The views expressed in this paper are those of the author and do not necessarily reflect the views or policies of the US Environmental Protection Agency.
- D. R. Cox and D. V. Hinkley, Theoretical statistics, Chapman and Hall, London, UK, 1974.
- S. G. Self and K.-Y. Liang, “Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions,” Journal of American Statistical Association, vol. 82, no. 398, pp. 605–610, 1987.
- D. L. Eaton and C. D. Klaassen, “Principles of Toxicology,” in Toxicology: The Basic Science of Poisons, Doull and Casarett, Eds., McGraw-Hill, New York, NY, USA, 6th edition, 2001.
- K. S. Crump, “A new method for determining allowable daily intakes,” Fundamental and Applied Toxicology, vol. 4, no. 5, pp. 854–871, 1984.
- K. S. Crump, H. A. Guess, and K. L. Deal, “Confidence intervals and test of hypotheses concerning dose response relations inferred from animal carcinogenicity data,” Biometrics, vol. 33, no. 3, pp. 437–451, 1977.
- G. Molenberghs and G. Verbeke, “Likelihood ratio, score, and wald tests in a constrained parameter space,” American Statistician, vol. 61, no. 1, pp. 22–27, 2007.
- A. Dominicus, A. Skrondal, H. K. Gjessing, N. L. Pedersen, and J. Palmgren, “Likelihood ratio tests in behavioral genetics: problems and solutions,” Behavior Genetics, vol. 36, no. 2, pp. 331–340, 2006.
- H. Chernoff, “On the distribution of the likelihood ratio,” Annals of Mathematical Statistics, vol. 25, pp. 573–578, 1954.
- P. I. Feder, “On the distribution of the log likelihood ratio test statistic when the true parameter is “near” the boundaries of the hypothesis regions,” Annals of Mathematical Statistics, vol. 39, pp. 2044–2055, 1968.
- P. A. P. Moran, “Maximum-likelihood estimation in non-standard conditions,” Proceedings of the Cambridge Philosophical Society, vol. 70, pp. 441–450, 1971.
- D. Chant, “On asymptotic tests of composite hypotheses in nonstandard conditions,” Biometrika, vol. 61, no. 2, pp. 291–298, 1974.
- B. Sinha, L. Kopylev, and J. Fox, “Some new aspects of dose-response multistage models with applications,” UMBC technical report, 2009, http://www.math.umbc.edu/~kogan/technical_papers/2007/Sinha_Kopylev_Fox.pdf.
- L. Kopylev and B. Sinha, “On the asymptotic distribution of likelihood ratio test when parameters lie on the boundary,” Sankhya B, vol. 73, no. 1, pp. 20–41, 2011.
- A. J. Bailer and R. J. Smith, “Estimating upper confidence limits for extra risk in quantal multistage models,” Risk Analysis, vol. 14, no. 6, pp. 1001–1010, 1994.
- A. E. Gelfand, A. F. M. Smith, and T.-M. Lee, “Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling,” Journal of the American Statistical Association, vol. 87, no. 418, pp. 523–532, 1992.
- D. B. Dunson and B. Neelon, “Bayesian inference on order-constrained parameters in generalized linear models,” Biometrics, vol. 59, no. 2, pp. 286–295, 2003.
- C. Hans and D. B. Dunson, “Bayesian inferences on umbrella orderings,” Biometrics, vol. 61, no. 4, pp. 1018–1026, 2005.
- P. M. Visscher, “A note on the asymptotic distribution of likelihood ratio tests to test variance components,” Twin Research and Human Genetics, vol. 9, no. 4, pp. 490–495, 2006.
- K. Meyer, “Likelihood calculations to evaluate experimental designs to estimate genetic variances,” Heredity, vol. 101, no. 3, pp. 212–221, 2008.
- R. D. Stoel, F. G. Garre, C. Dolan, and G. Van Den Wittenboer, “On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints,” Psychological Methods, vol. 11, no. 4, pp. 439–455, 2006.