Table of Contents Author Guidelines Submit a Manuscript
Journal of Probability and Statistics
VolumeΒ 2010Β (2010), Article IDΒ 139856, 9 pages
http://dx.doi.org/10.1155/2010/139856
Research Article

A Note on Confidence Interval for the Power of the One Sample 𝑑 Test

Department of Mathematics and Statistics, York University, Toronto, Ontario, Canada M3J 1P3

Received 26 January 2010; Revised 22 June 2010; Accepted 12 August 2010

Academic Editor: Junbin B.Β Gao

Copyright Β© 2010 A. Wong. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

In introductory statistics texts, the power of the test of a one-sample mean when the variance is known is widely discussed. However, when the variance is unknown, the power of the Student's 𝑑-test is seldom mentioned. In this note, a general methodology for obtaining inference concerning a scalar parameter of interest of any exponential family model is proposed. The method is then applied to the one-sample mean problem with unknown variance to obtain a (1βˆ’π›Ύ)100% confidence interval for the power of the Student's 𝑑-test that detects the difference (πœ‡βˆ’πœ‡0). The calculations require only the density and the cumulative distribution functions of the standard normal distribution. In addition, the methodology presented can also be applied to determine the required sample size when the effect size and the power of a size 𝛼 test of mean are given.

1. Introduction

Let (π‘₯1,…,π‘₯𝑛) be a random sample from a normal distribution with mean πœ‡ and variance 𝜎2. As presented in any introductory statistics text, such as Mandenhall et al. [1, page 425], a (1βˆ’π›Ύ)100% confidence interval for 𝜎2 isξ€·πΏπœŽ2,π‘ˆπœŽ2ξ€Έ=(π‘›βˆ’1)𝑠2πœ’2π‘›βˆ’1,1βˆ’π›Ύ/2,(π‘›βˆ’1)𝑠2πœ’2π‘›βˆ’1,𝛾/2ξƒͺ,(1.1) where βˆ‘π‘₯π‘₯=𝑖/𝑛, 𝑠2=βˆ‘(π‘₯π‘–βˆ’π‘₯)2/(π‘›βˆ’1), and πœ’2𝜈,𝛿 is the 100𝛿th percentile of the πœ’2 distribution with 𝜈 degrees of freedom. Moreover, for testing𝐻0βˆΆπœ‡=πœ‡0versusπ»π‘ŽβˆΆπœ‡=πœ‡0+π‘˜πœŽπ‘˜>0,(1.2) the null hypothesis will be rejected at significance level 𝛼 ifπ‘₯βˆ’πœ‡0βˆšπ‘ /𝑛>π‘‘π‘›βˆ’1,1βˆ’π›Ό,(1.3) where π‘‘πœˆ,𝛿 is the 100𝛿th percentile of the Student's 𝑑 distribution with 𝜈 degrees of freedom. Although the power of this test is rarely discussed in introductory statistics texts, Lehmann [2] proved that the probability of committing Type II error of a size 𝛼 test with the hypotheses stated in (1.2) is𝛽=πΊβˆšπ‘›βˆ’1,π‘˜π‘›ξ€·π‘‘π‘›βˆ’1,1βˆ’π›Όξ€Έ,(1.4) where π‘˜=(πœ‡βˆ’πœ‡0)/𝜎 is the effect size and 𝐺𝜈,πœ†(β‹…) is the cumulative distribution function of the noncentral 𝑑 distribution with 𝜈 degrees of freedom and noncentrality πœ†. Note that the calculation of 𝛽 involves the unknown 𝜎. A naive point estimate of 𝛽 iŝ𝛽=πΊπ‘›βˆ’1,Μ‚π‘˜βˆšπ‘›ξ€·π‘‘π‘›βˆ’1,1βˆ’π›Όξ€Έ,(1.5) where Μ‚π‘˜=(π‘₯βˆ’πœ‡0)/𝑠. Thus, the corresponding point estimate of the power of the size 𝛼 test that detects the difference (πœ‡βˆ’πœ‡0) is ̂𝛽1βˆ’.

In Section 2, a general methodology is proposed for obtaining inference concerning a scalar parameter of interest of an exponential family model. Applying the general methodology to the one-sample mean problem with unknown variance, a (1βˆ’π›Ύ)100% confidence interval for 1βˆ’π›½ is derived. This interval estimate will depend only on the evaluation of the density and the cumulative distribution functions of the standard normal distribution. The methodology can also be used to determine the required sample size when the effect size and the power of a size 𝛼 test are fixed. Numerical examples are presented in Section 3 to illustrate the accuracy of the proposed method. Finally, some concluding remarks are given in Section 4.

2. Confidence Interval for the Power of the Test and Sample Size Calculation

From (1.1), for a given (πœ‡βˆ’πœ‡0) value, a (1βˆ’π›Ύ)100% confidence interval for (πœ‡βˆ’πœ‡0)βˆšπ‘›/𝜎 isξƒ©ξ€·πœ‡βˆ’πœ‡0ξ€Έβˆšπ‘›βˆšπ‘ˆπœŽ2,ξ€·πœ‡βˆ’πœ‡0ξ€Έβˆšπ‘›βˆšπΏπœŽ2ξƒͺ.(2.1) Hence, from (1.4), the corresponding confidence interval for 𝛽 is𝐿𝛽,π‘ˆπ›½ξ€Έ=ξ‚€πΊπ‘›βˆ’1,(πœ‡βˆ’πœ‡0)βˆšβˆšπ‘›/π‘ˆπœŽ2ξ€·π‘‘π‘›βˆ’1,1βˆ’π›Όξ€Έ,πΊπ‘›βˆ’1,(πœ‡βˆ’πœ‡0)βˆšβˆšπ‘›/𝐿𝜎2ξ€·π‘‘π‘›βˆ’1,1βˆ’π›Όξ€Έξ‚.(2.2) Finally, a (1βˆ’π›Ύ)100% confidence interval for the power of a size 𝛼 test that detects the difference (πœ‡βˆ’πœ‡0) isξ€·1βˆ’π‘ˆπ›½,1βˆ’πΏπ›½ξ€Έ.(2.3)

Evaluating (2.3) requires the cumulative distribution function of the noncentral 𝑑 distribution, which is generally not discussed in introductory statistics texts. In statistics literature, various approximations of 𝐺𝜈,πœ†(β‹…) have been proposed. For the rest of this section, a simple and accurate approximation of 𝐺𝜈,πœ†(β‹…) will be derived.

Let 𝑋1,…,𝑋𝑛 be identically independently normally distributed random variables with mean πœ‡ and variance 𝜎2. It is well known that βˆ‘π‘‹π‘‹=𝑖/𝑛 and (π‘›βˆ’1)𝑆2/𝜎2=βˆ‘(π‘‹π‘–βˆ’π‘‹)2/𝜎2 are independently distributed as normal with mean πœ‡ and variance 𝜎2/𝑛 and πœ’2 with (π‘›βˆ’1) degrees of freedom, respectively. Let π‘‡βˆ—=βˆšπ‘›ξ‚ƒξ‚€ξ‚ξ‚„βˆ’βˆšπ‘‹βˆ’πœ‡/πœŽπ‘›π‘§π‘=βˆšπ‘†/πœŽπ‘›π‘†ξ‚€π‘‹βˆ’πœ‡βˆ’π‘§π‘πœŽξ‚,(2.4) where 𝑧𝑝 denotes the 100𝑝th percentile of the standard normal distribution, then π‘‡βˆ— follows a noncentral 𝑑 distribution with (π‘›βˆ’1) degrees of freedom and noncentrality βˆ’βˆšπ‘›π‘§π‘.

Now, consider a sample (π‘₯1,…,π‘₯𝑛) from a normal distribution with mean πœ‡ and variance 𝜎2. Let the parameter of interest beβˆšπœ“=𝑛𝑠π‘₯βˆ’πœ‡βˆ’π‘§π‘πœŽξ€Έ,(2.5) where βˆ‘π‘₯π‘₯=𝑖/𝑛 and 𝑠2=βˆ‘(π‘₯π‘–βˆ’π‘₯)2/(π‘›βˆ’1), then the log-likelihood function can be written asξ€·β„“(πœƒ)=β„“πœ“,𝜎2𝑛=βˆ’2log𝜎2βˆ’12𝜎2𝑠2ξ€·π‘›βˆ’1+πœ“2ξ€Έπœ“+π›Ώπ‘ βˆšπœŽ2,(2.6) where βˆšπ›Ώ=βˆ’π‘›π‘§π‘. Denote that𝑝𝑇(πœ“)=π‘ƒβˆ—ξ€Έβ‰€πœ“=πΊπ‘‘π‘›βˆ’1,𝛿(πœ“).(2.7)

The overall maximum likelihood estimate (MLE) of πœƒ, Μ‚πœƒ=(ξπœ“,𝜎2)ξ…žβˆš=(𝛿(π‘›βˆ’1)/𝑛,((π‘›βˆ’1)/𝑛)𝑠2)ξ…ž is obtained by solving (πœ•β„“(πœƒ)/πœ•πœƒ)|πœƒ=Μ‚πœƒ=0, and the determinant of the observed information matrix evaluated at the overall mle is||π‘—πœƒπœƒβ€²ξ€·Μ‚πœƒξ€Έ||=||||πœ•2β„“(πœƒ)||||πœ•πœƒπœ•πœƒβ€²πœƒ=Μ‚πœƒ=𝑛42(π‘›βˆ’1)3𝑠4.(2.8) The constrained mle of πœƒ at a fixed πœ“, Μ‚πœƒπœ“=(πœ“,𝜎2πœ“)=(πœ“,𝑠2𝐴2/4𝑛2), where𝐴=𝛿2πœ“2ξ€·+4π‘›π‘›βˆ’1+πœ“2ξ€Έβˆ’π›Ώπœ“,(2.9) is obtained by solving (πœ•β„“(πœƒ)/πœ•πœŽ2)|πœƒ=Μ‚πœƒπœ“=0. Moreover, the determinant of the observed nuisance information matrix evaluated at the constrained mle is||π‘—πœŽ2𝜎2ξ€·Μ‚πœƒπœ“ξ€Έ||=||||πœ•2β„“(πœƒ)πœ•πœŽ2πœ•πœŽ2||||πœƒ=Μ‚πœƒπœ“=8𝑛5𝑠4𝐴5(𝐴+π›Ώπœ“).(2.10) Hence, the signed log-likelihood ratio statistic isξ€·Μ‚Μ‚πœƒπ‘Ÿ=π‘Ÿ(πœ“)=sgnξπœ“βˆ’πœ“ξ€Έξ€½2[β„“(πœƒ)βˆ’β„“(πœ“ξ€Ύ)]1/2ξ€·ξ€Έξ‚»ξ‚Έ=sgnξπœ“βˆ’πœ“βˆ’π‘›log4𝑛(π‘›βˆ’1)𝐴2ξ‚Ή+𝛿2βˆ’2π‘›π›Ώπœ“π΄ξ‚Ό1/2.(2.11)

It is well known that π‘Ÿ is asymptotically distributed as the standard normal distribution with rate of convergence 𝑂(π‘›βˆ’1/2). Hence, 𝑝(πœ“) can be approximated by Ξ¦(π‘Ÿ) where Ξ¦(β‹…) is the cumulative distribution function of the standard normal distribution. It is important to note that π‘Ÿ is reparameterization invariant.

In statistics literatures, various likelihood-based small sample asymptotic methods have been proposed. In particular, if the model is a canonical exponential family model and the canonical parameter is πœƒ=(πœ“,πœ†β€²)β€², Lugannani and Rice [3] deriveξ‚»1𝑝(πœ“)=1βˆ’Ξ¦(π‘Ÿ)βˆ’πœ™(π‘Ÿ)π‘Ÿβˆ’1π‘žξ‚Ό,(2.12) where πœ™(β‹…) is the density function of the standard normal distribution, π‘Ÿ is defined in (2.11), and π‘ž takes the formξ€·ξ€Έξƒ―||π‘—π‘ž=π‘ž(πœ“)=ξπœ“βˆ’πœ“πœƒπœƒβ€²ξ€·Μ‚πœƒξ€Έ||||π‘—πœ†πœ†β€²ξ€·Μ‚πœƒπœ“ξ€Έ||ξƒ°1/2.(2.13) This approximation has a rate of convergence 𝑂(π‘›βˆ’3/2). It is important to note that π‘Ÿ is reparameterization invariant whereas π‘ž is not.

For a general exponential family model with canonical parameter πœ‘=πœ‘(πœƒ) and a scalar parameter πœ“=πœ“(πœƒ), to obtain inference concerning πœ“ based on the Lugannani and Rice (1980) [3] method, π‘Ÿ remains unchanged as in (2.11) because it is reparameterization invariant, but π‘ž has to be re-expressed in the canonical parameter scale, πœ™ scale. To achieve this, let πœ‘πœƒ(πœƒ) and πœ‘πœ†(πœƒ) be the derivatives of πœ‘(πœƒ) with respect to πœƒ and πœ†, respectively. Denote πœ‘πœ“(πœƒ) to be the row of πœ‘πœƒβˆ’1(πœƒ) that corresponds to πœ“, and β€–πœ‘πœ“(πœƒ)β€–2 is the square length of the vector πœ‘πœ“(πœƒ). Let πœ’(πœƒ) be a rotated coordinate of πœ‘(πœƒ) that agrees with πœ“(πœƒ) at Μ‚πœƒπœ“. Thenπœ‘πœ’(πœƒ)=πœ“ξ€·Μ‚πœƒπœ“ξ€Έβ€–β€–πœ‘πœ“ξ€·Μ‚πœƒπœ“ξ€Έβ€–β€–πœ‘(πœƒ)(2.14) can be viewed operationally as the scalar parameter of interest in πœ‘(πœƒ) scale.

Since β„“(πœƒ)=β„“(πœ‘(πœƒ)), by the chain rule in differentiation, we have||π‘—πœ‘πœ‘β€²ξ€·Μ‚πœƒξ€Έ||=||π‘—πœƒπœƒβ€²ξ€·Μ‚πœƒξ€Έ||||πœ‘πœƒξ€·Μ‚πœƒξ€Έ||βˆ’2,||𝑗(πœ†πœ†β€²)ξ€·Μ‚πœƒπœ“ξ€Έ||=||π‘—πœ†πœ†β€²ξ€·Μ‚πœƒπœ“ξ€Έ||||πœ‘ξ…žπœ†ξ€·Μ‚πœƒπœ“ξ€Έπœ‘πœ†ξ€·Μ‚πœƒπœ“ξ€Έ||βˆ’1.(2.15) Hence, an estimated variance for Μ‚Μ‚πœƒ|πœ’(πœƒ)βˆ’πœ’(πœ“)| in πœ‘(πœƒ) scale is |𝑗(πœ†πœ†β€²)(Μ‚πœƒπœ“)|/|π‘—πœ‘πœ‘ξ…ž(Μ‚πœƒ)|. Thus, π‘ž=π‘ž(πœ“), as defined in (2.13) and expressed in πœ‘(πœƒ) scale, isξ€·ξ€Έ||πœ’ξ€·Μ‚πœƒξ€Έξ€·Μ‚πœƒπ‘ž=π‘ž(πœ“)=sgnξπœ“βˆ’πœ“βˆ’πœ’πœ“ξ€Έ||ξƒ―||π‘—πœ‘πœ‘β€²ξ€·Μ‚πœƒξ€Έ||||𝑗(πœ†πœ†β€²)ξ€·Μ‚πœƒπœ“ξ€Έ||ξƒ°1/2.(2.16) Therefore, 𝑝(πœ“) can be obtained from (2.12) with π‘Ÿ and π‘ž being defined in (2.11) and (2.17), respectively.

Note that the model being considered is an exponential family model with canonical parameter1πœ‘(πœƒ)=𝜎2,1𝜎2𝑠π‘₯βˆ’βˆšπ‘›πœ“βˆ’π‘§π‘βˆšπœŽ2ξƒͺξƒͺξ…ž.(2.17) From (2.17), we haveπœ‘πœƒβŽ›βŽœβŽœβŽœβŽ1(πœƒ)=0βˆ’πœŽ4βˆ’π‘ βˆšπ‘›πœŽ2βˆ’π‘₯𝜎4+π‘ πœ“βˆšπ‘›πœŽ4βˆ’π›Ώ2βˆšπ‘›ξ€·πœŽ2ξ€Έ3/2⎞⎟⎟⎟⎠,||πœ‘πœƒξ€·Μ‚πœƒξ€Έ||𝑠=βˆ’βˆšπ‘›ξπœŽ6,||πœ‘ξ…žπœŽ2ξ€·Μ‚πœƒπœ“ξ€Έπœ‘πœŽ2ξ€·Μ‚πœƒπœ“ξ€Έ||=1𝜎8πœ“ξ€·1+𝐡2ξ€Έ,(2.18) where𝐡=βˆ’π‘₯+π‘ πœ“βˆšπ‘›βˆ’π›Ώξ”ξπœŽ2πœ“2βˆšπ‘›.(2.19) Moreover, by obtaining the inverse of πœ‘πœƒ(πœƒ), we haveπœ‘πœ“βˆš(πœƒ)=π‘›πœŽ2𝑠(βˆ’π΅,βˆ’1).(2.20) Hence, from (2.14), we can obtain||πœ’ξ€·Μ‚πœƒξ€Έξ€·Μ‚πœƒβˆ’πœ’πœ“ξ€Έ||=1√1+𝐡2|||||βˆ’βˆšπ‘›πœ“+(π‘›βˆ’1)𝑠𝛿𝐴2+4𝛿𝑛(π‘›βˆ’1)4√|||||.𝑛(π‘›βˆ’1)𝑠𝐴(2.21) Thus, from (2.16), we haveξ€·ξ€Έ||π‘ž=π‘ž(πœ“)=sgnξπœ“βˆ’πœ“π›Ώπ΄2||βˆ’4π‘›πœ“π΄+4𝛿𝑛(π‘›βˆ’1)√√𝐴+π›Ώπœ“π‘›(π‘›βˆ’1)𝐴5/2.(2.22) Finally, 𝑝(πœ“)=πΊπ‘›βˆ’1,𝛿(πœ“) can be approximated from (2.12) with rate of convergence 𝑂(π‘›βˆ’3/2).

By reindexing all the necessary equations, we haveπΊβˆšπ‘›βˆ’1,π‘˜π‘›ξ€·π‘‘π‘›βˆ’1,1βˆ’π›Όξ€Έξ‚»1=1βˆ’Ξ¦(π‘Ÿ)βˆ’πœ™(π‘Ÿ)π‘Ÿβˆ’1π‘žξ‚Ό,(2.23) where πœ™(β‹…) and Ξ¦(β‹…) are the density and cumulative distribution functions of the standard normal distribution, andξ‚€π‘˜βˆšπ‘Ÿ=sgnπ‘›βˆ’1βˆ’π‘‘π‘›βˆ’1,1βˆ’π›Όξ‚ξƒ―βˆ’π‘›log4𝑛(π‘›βˆ’1)𝐴2+π‘›π‘˜2βˆ’2π‘˜π‘›3/2π‘‘π‘›βˆ’1,1βˆ’π›Όπ΄ξƒ°1/2,ξ‚€π‘˜βˆšπ‘ž=sgnπ‘›βˆ’1βˆ’π‘‘π‘›βˆ’1,1βˆ’π›Όξ‚π‘›(π‘›βˆ’1)𝐴5/2||π‘˜βˆšπ‘›π΄2βˆ’4π‘›π΄π‘‘π‘›βˆ’1,1βˆ’π›Ό+4π‘˜π‘›3/2||(π‘›βˆ’1)𝐴+π‘˜π‘›1/2π‘‘π‘›βˆ’1,1βˆ’π›Ό,√𝐴=βˆ’π‘˜π‘›π‘‘π‘›βˆ’1,1βˆ’π›Όξ”.+π‘˜2𝑛𝑑2π‘›βˆ’1,1βˆ’π›Όξ€·+4π‘›π‘›βˆ’1+π‘‘π‘›βˆ’1,1βˆ’π›Όξ€Έ.(2.24) Finally, with a predetermined effect size π‘˜ and power of a size 𝛼 test, the sample size can be obtained by iterations.

Note that DiCiccio and Martin [4] derived an asymptotic approximation of marginal tail probabilities for a real-valued function of a random vector where the function has continuous gradient that does not vanish at the mode of the joint density of the random vector. Applied to the noncentral 𝑑 distribution problem, the results are identical. Nevertheless, the approach of DiCiccio and Martin [4] is quite different from the proposed method. More specifically, DiCiccio and Martin [4] worked directly from the log density and treated the parameters as fixed whereas the proposed method works from the log-likelihood function where the data are observed.

3. Numerical Example

Figure 1 plots the power function of a one-sample 𝑑 test against the effect size π‘˜ for 𝑛=2,3 and 𝛼=0.05,0.01. The exact method is obtained from the built-in cumulative distribution function of the noncentral 𝑑 distribution in 𝑅. From the plot, it is clear that the signed log-likelihood ratio does not provide satisfactory results. The proposed method and the built-in function of 𝑅 are very close even when the sample size is 2. It is interesting to note that the built-in function of 𝑅 has a discontinuity point in the 𝑛=2,𝛼=0.01 case.

fig1
Figure 1

Now, consider the data set recorded in Mandenhall et al. [1, page 103] 0.46,0.61,0.52,0.48,0.57,0.54.(3.1)

For testing the hypothesis𝐻0βˆΆπœ‡=0.5versusπ»π‘ŽβˆΆπœ‡=πœ‡1>0.5,(3.2) the power function of a size 0.05 test and the corresponding 95% confidence bands are plotted in Figure 2. From Figure 2, the approximated power at πœ‡1=0.52 is 0.5764. Furthermore, the 95% confidence interval for the power of the above test when πœ‡1=0.52 is (0.1856,0.8992). At first, the confidence interval seems too wide. However, by examining (2.3), the result is not too surprising because (2.3) depends on (1.1). Since πœ’2 distribution is a skewed distribution, by defining the confidence interval of 𝜎2 to have equal tail coverage, (1.1) is a wide interval and hence (2.3) is a wide interval.

139856.fig.002
Figure 2: Power function and the 95% confidence band.

Finally, to illustrate the determination of the sample size, let the effect size be 0.8, and at 𝛼=0.025, let the power be at least 0.9, then the proposed method gives 𝑛=19 with power 0.909.

4. Summary and Conclusion

The (1βˆ’π›Ύ)100% confidence interval for the power of the size 𝛼 Student's 𝑑-test detecting the difference (πœ‡βˆ’πœ‡0) is presented. The major advantages of the presented confidence interval are that it depends only on the evaluations of the density and cumulative distribution functions of the standard normal distribution and that it is extremely accurate. The 𝑅 source code is available from the author upon request.

As a final note, the proposed method can be applied to any distribution that belongs to the exponential family model with known canonical parameters. Although the method depends on the correct specification of the underlying distribution, Fraser et al. [5] examined a special case when the error distribution of the regression model is misspecified and the likelihood-based method still gives results that are more accurate than the existing Central Limit Theorem-based approximations.

References

  1. W. Mandenhall, R. Beaver, B. Beaver, and S. Ahmed, Introduction to Probability and Statistics, Nelson, Torrent, Canada, 2009.
  2. E. L. Lehmann, Testing Statistical Hypotheses, John Wiley & Sons, New York, NY, USA, 1959.
  3. R. Lugannani and S. Rice, β€œSaddle point approximation for the distribution of the sum of independent random variables,” Advances in Applied Probability, vol. 12, no. 2, pp. 475–490, 1980. View at Publisher Β· View at Google Scholar
  4. T. J. DiCiccio and M. A. Martin, β€œApproximations of marginal tail probabilities for a class of smooth functions with applications to Bayesian and conditional inference,” Biometrika, vol. 78, no. 4, pp. 891–902, 1991. View at Publisher Β· View at Google Scholar Β· View at MathSciNet
  5. D. A. S. Fraser, A. Wong, and J. Wu, β€œRegression analysis, normal and nonnormal: accurate P-values from likelihood analysis,” Journal of the American Statistical Association, vol. 94, no. 448, pp. 1286–1295, 1999. View at Google Scholar