Journal of Probability and Statistics

Journal of Probability and Statistics / 2012 / Article
Special Issue

Advanced Designs and Statistical Methods for Genetic and Genomic Studies of Complex Diseases

View this Special Issue

Research Article | Open Access

Volume 2012 |Article ID 817948 | https://doi.org/10.1155/2012/817948

Shulian Shang, Qianhe Zhou, Mengling Liu, Yongzhao Shao, "Sample Size Calculation for Controlling False Discovery Proportion", Journal of Probability and Statistics, vol. 2012, Article ID 817948, 13 pages, 2012. https://doi.org/10.1155/2012/817948

Sample Size Calculation for Controlling False Discovery Proportion

Academic Editor: Xiaohua Douglas Zhang
Received31 Mar 2012
Accepted05 Jun 2012
Published16 Aug 2012

Abstract

The false discovery proportion (FDP), the proportion of incorrect rejections among all rejections, is a direct measure of abundance of false positive findings in multiple testing. Many methods have been proposed to control FDP, but they are too conservative to be useful for power analysis. Study designs for controlling the mean of FDP, which is false discovery rate, have been commonly used. However, there has been little attempt to design study with direct FDP control to achieve certain level of efficiency. We provide a sample size calculation method using the variance formula of the FDP under weak-dependence assumptions to achieve the desired overall power. The relationship between design parameters and sample size is explored. The adequacy of the procedure is assessed by simulation. We illustrate the method using estimated correlations from a prostate cancer dataset.

1. Introduction

Modern biomedical research frequently involves parallel measurements of a large number of quantities of interest, such as gene expression levels, single nucleotide polymorphism (SNP) and DNA copy number variations. The scientific question can often be formulated as a multiple testing problem. In order to address the multiplicity issue, many methods have been proposed to control the family-wise error rate (FWER), false discovery rate (FDR) or false discovery proportion (FDP). Controlling FDR has been widely used in high-dimensional data analysis [1ā€“3]. FDR is the expected value of the FDP, which is the proportion of incorrect rejections among all rejections. Controlling FDR ensures that the average of FDP from many independently repeated experiments is under control. However, the variability of FDP is ignored, and the actual FDP could be much greater than FDR with high probability, especially when test statistics are correlated. Consequently, researchers have proposed many procedures to control FDP directly [4ā€“11]: Pī€·FDPā‰¤š‘Ÿ1ī€øā‰„š‘1(1.1) for given š‘Ÿ1 and š‘1. This is a more stringent criterion than FDR because the proportion of false rejections is bounded above by š‘Ÿ1 with high probability. The FDP controlling procedures are generally too conservative to be useful for the purpose of study design or power analysis.

When we design studies involving multiple testing, it is important to determine sample size to ensure adequate statistical power. Methods for calculating sample size have been proposed to control various criteria, for example, FWER [12ā€“14], FDR [15ā€“20], the number of false discoveries [19, 21] and FDP [22]. For controlling FDP, Oura et al. [22] provided a method to calculate sample size using the beta-binomial model for the sum of rejection status of true alternative hypotheses. It is assumed that only test statistics of true alternative hypotheses are dependent, with a parametric correlation structure. This assumption is restrictive because null test statistics can also be correlated and the dependence structure can be more complicated than the assumed parametric correlation structure. Furthermore, the computation is intensive because computation of the beta-binomial distribution is required. However, to our knowledge this is the only paper that directly deals with this important design problem.

In this paper, we provide a more general method of sample size calculation for controlling FDP under weak-dependence assumptions. Under some assumptions on dependence among test statistics, explicit formulas for the mean and variance of FDP have been derived for each fixed effect size [23]. The formulas elucidate the effects of various design parameters on the variance of FDP. Moreover, the formulas provide a convenient tool to calculate sample size for controlling the FDP. As in [13, 18, 19, 24], we consider the probability of detecting at least a specified proportion of true alternative hypotheses as the power criterion. An iterative computation algorithm for calculating sample size is provided. Simulation experiments indicate that studies with the resultant sample sizes satisfy the power criterion at the given rejection threshold. We illustrate the sample size calculation procedure using a prostate cancer dataset.

2. Methods

2.1. Notation

Suppose that š‘š hypotheses are tested simultaneously. Let š‘€0 denote the index set of š‘š0 tests for which null hypotheses are true and š‘€1 the index set of š‘š1=š‘šāˆ’š‘š0 tests for which alternative hypotheses are true. Denote the proportion of true null hypotheses by šœ‹0=š‘š0/š‘š. We reject a hypothesis if the š‘ƒ value is less than some threshold š›¼, and denote the rejection status of the š‘–th test by š‘…š‘–(š›¼)=š¼(š‘š‘–<š›¼), where š‘š‘– denotes the š‘ƒ value of the š‘–th test and š¼(ā‹…) is an indicator function. The number of rejections is āˆ‘š‘…=š‘šš‘–=1š‘…š‘–(š›¼). Let the comparison-wise type II error of the š‘–th test be š›½š‘– and the average type II error be 1š›½=š‘š1ī“š‘–āˆˆš‘€1š›½š‘–.(2.1) Table 1 summarizes the outcomes of š‘š tests and their expected values.


ā€‰ Outcomes
ā€‰ Reject š» 0 Accept š» 0 Total

š» 0 is true š‘‰ š‘š 0 āˆ’ š‘‰ š‘š 0
š» 1 is true š‘ˆ š‘š 1 āˆ’ š‘ˆ š‘š 1
Total š‘… š‘š āˆ’ š‘… š‘š

ā€‰ Expected outcomes
ā€‰ Reject š» 0 Accept š» 0 Total

š» 0 is true š‘š 0 š›¼ š‘š 0 ( 1 āˆ’ š›¼ ) š‘š 0
š» 1 is true š‘š 1 ( 1 āˆ’ š›½ ) š‘š 1 š›½ š‘š 1
Total š‘š 0 š›¼ + š‘š 1 ( 1 āˆ’ š›½ ) š‘š 0 ( 1 āˆ’ š›¼ ) + š‘š 1 š›½ š‘š

Denote the Pearson correlation coefficient of two rejection indicators by šœƒš‘–š‘—ī€½š‘…=corrš‘–(š›¼),š‘…š‘—ī€¾.(š›¼)(2.2) Furthermore, for š‘–,š‘—āˆˆš‘€0, define šœƒš‘‰š‘–š‘—ī€½š‘…=corrš‘–(š›¼),š‘…š‘—ī€¾.(š›¼)(2.3) Let the average correlation be denoted as šœƒš‘‰=āˆ‘š‘–,š‘—āˆˆš‘€0,š‘–ā‰ š‘—šœƒš‘‰š‘–š‘—š‘š0ī€·š‘š0ī€ø.āˆ’1(2.4) Similarly, for š‘–,š‘—āˆˆš‘€1, we define šœƒš‘ˆš‘–š‘—ī€½š‘…=corrš‘–(š›¼),š‘…š‘—ī€¾.(š›¼)(2.5) The average correlation is šœƒš‘ˆ=āˆ‘š‘–,š‘—āˆˆš‘€1,š‘–ā‰ š‘—šœƒš‘ˆš‘–š‘—š‘š1ī€·š‘š1ī€ø.āˆ’1(2.6) In addition, for š‘–āˆˆš‘€1,š‘—āˆˆš‘€0, denote šœƒš‘–š‘—š‘ˆš‘‰ī€½š‘…=corrš‘–(š›¼),š‘…š‘—ī€¾.(š›¼)(2.7) Denote the average correlation by šœƒš‘ˆš‘‰=āˆ‘š‘–āˆˆš‘€1āˆ‘š‘—āˆˆš‘€0šœƒš‘–š‘—š‘ˆš‘‰š‘š0š‘š1.(2.8)

2.2. The Effect of Design Parameters on the Variance of FDP

It has been shown via numerical studies that the variability of FDP increases when test statistics are dependent [25, 26]. But the relationship between design parameters and the variance of FDP has not been examined through analytical formulas. Under the assumptions of common effect size and weak dependence among test statistics, explicit formulas for the mean (šœ‡š‘„) and variance (šœŽ2š‘„) of the FDP have been derived [23]: šœ‡š‘„ā‰ˆšœ‹0š›¼šœ‹0ī€·š›¼+1āˆ’šœ‹0ī€øī‚€1āˆ’š›½ī‚,šœŽ(2.9)2š‘„ā‰ˆšœ‹0ī€·1āˆ’šœ‹0ī€ø2ī‚€š›¼(1āˆ’š›¼)1āˆ’š›½ī‚ī‚†šœ‹0ī€·š›¼+1āˆ’šœ‹0ī€øī‚€1āˆ’š›½ī‚ī‚‡4Ī£,(2.10) where 1Ī£=š‘šī‚µ1āˆ’šœ‹š›½+01āˆ’šœ‹0šœ”š›½ī‚¶+ī‚€šœ‹0āˆ’1š‘šī‚ī‚€1āˆ’š›½ī‚šœƒš‘‰+šœ‹0šœ”š›½šœƒš‘ˆāˆ’2šœ‹0ī‚™šœ”š›½ī‚€1āˆ’š›½ī‚šœƒš‘ˆš‘‰(2.11) and šœ”=š›¼/(1āˆ’š›¼).

The variance formula (2.10) elucidates the effects of various design parameters on the variance of FDP. To explore the effects, in Figure 1 we calculated šœŽš‘„ using (2.10) and plotted it against š‘š for different correlations šœƒš‘‰. We set šœ‹0=0.7 and š‘š in the range of 1000 to 10000. The average correlations šœƒš‘ˆ and šœƒš‘ˆš‘‰ are fixed to be 0.001 and 0, respectively. The levels of š›¼ and š›½ are chosen such that FDR is 3% or 5%. At each value of šœƒš‘‰, šœŽš‘„ decreases as the number of tests š‘š increases. The solid line shows the standard deviation of the FDP when šœƒš‘‰ is 0. When šœƒš‘‰ is not 0, šœŽš‘„ increases evidently. If test statistics are highly correlated, FDP can be much greater than its mean FDR at a given rejection threshold due to its large variability.

In Figure 2, the relationship between šœŽš‘„ and šœ‹0 was investigated. When other parameters are fixed, šœŽš‘„ increases as šœ‹0 increases.

Figure 3 shows that šœŽš‘„ increases as š›½ increases. When other factors are fixed, the variability of FDP is smaller when the comparison-wise type II error is smaller.

2.3. Power and Sample Size Analysis

Under some general regularity conditions including weak dependence among test statistics, the FDP follows an asymptotic normal distribution š‘(šœ‡š‘„,šœŽ2š‘„) [23, 27]. As was pointed out by Shang et al. [23], š‘Œ=log(FDP) also has an asymptotic normal distribution by the delta method, and under weak dependence log(FDP) is closer to normal than the FDP itself. The approximate mean and variance of š‘Œ=log(FDP) are [23] šœ‡š‘Œī€·šœ‡ā‰ˆlogš‘„ī€øāŽ§āŽŖāŽØāŽŖāŽ©šœ‹ā‰ˆlog0š›¼šœ‹0ī€·š›¼+1āˆ’šœ‹0ī€øī‚€1āˆ’š›½ī‚āŽ«āŽŖāŽ¬āŽŖāŽ­,šœŽ(2.12)2š‘Œā‰ˆī€·1āˆ’šœ‹0ī€ø2ī‚€(1āˆ’š›¼)1āˆ’š›½ī‚šœ‹0š›¼ī‚†šœ‹0ī€·š›¼+1āˆ’šœ‹0ī€øī‚€1āˆ’š›½ī‚ī‚‡2Ī£,(2.13) where Ī£ is in (2.11).

To control FDP with desired power, criterion (1.1) has to be satisfied. Asymptotic normality of log(FDP) implies that Ī¦ī‚µlogš‘Ÿ1āˆ’šœ‡š‘ŒšœŽš‘Œī‚¶ā‰„š‘1,(2.14) where Ī¦(ā‹…) is the cumulative distribution function (CDF) of standard normal distribution, šœ‡š‘Œ is in (2.12), and šœŽ2š‘Œ is in (2.13).

There are two commonly used power criteria in multiple testing: the average power, defined as E(š‘ˆ/š‘š1), and the overall power, defined as P(š‘ˆ/š‘š1ā‰„š‘Ÿ2) for given š‘Ÿ2. When a study is designed using the average power criterion, the proportion of true alternative hypotheses rejected will be greater than a prespecified number on average. However, under dependence among test statistics the variability of š‘ˆ/š‘š1 increases [18], and the study can be underpowered with high probability. Consequently, the overall power has been used in [13, 18, 19, 24] and we also use this power criterion, Pī‚µš‘ˆš‘š1ā‰„š‘Ÿ2ī‚¶ā‰„š‘2(2.15) for given š‘Ÿ2 and š‘2.

Under the weak-dependence assumptions in [18, 23], š‘ˆ/š‘š1 has an asymptotic normal distribution: š‘ˆš‘š1āŽ›āŽœāŽœāŽāˆ¼š‘1āˆ’š›½,š›½ī‚€1āˆ’š›½ī‚š‘š1ī‚†1+šœƒš‘ˆī€·š‘š1ī€øī‚‡āŽžāŽŸāŽŸāŽ .āˆ’1(2.16) Setting the inequality in (2.15) to equality, the following equation for š›½ can be obtained as in [18]: š›½=1āˆ’š‘Ÿ2āˆ’1āˆ’2š‘Ÿ2+ī”4š‘šāˆ—1š‘Ÿ2ī€·1āˆ’š‘Ÿ2ī€ø+12š‘šāˆ—1,+2(2.17) where š‘šāˆ—1=š‘š1/{1+šœƒš‘ˆ(š‘š1āˆ’1)}š‘§21āˆ’š‘2 and Ī¦(š‘§1āˆ’š‘2)=š‘2.

For illustration, consider that a two-sample one-sided š‘”-test is performed. Let š›æ denote the effect size (mean difference divided by the common standard deviation), and š‘Ž1 and 1āˆ’š‘Ž1 denote the allocation proportion for two groups. We first find š›¼ and š›½ which fulfill criteria (1.1) and (2.15). The required sample size š‘› is the smallest integer satisfying the following inequality: 1āˆ’š›½ā‰¤1āˆ’Ī“š‘›āˆ’2ī‚µāˆ’š‘”š‘›āˆ’2,š›¼|||š›æī”š‘Ž1ī€·1āˆ’š‘Ž1ī€øš‘›ī‚¶,(2.18) where Ī“š‘›āˆ’2(ā‹…āˆ£š‘) is the CDF of a noncentral š‘”-distribution with š‘›āˆ’2 degrees of freedom and noncentrality parameter š‘ and š‘”š‘›āˆ’2,š›¼ is the upper š›¼ critical value of the central š‘”-distribution with š‘›āˆ’2 degrees of freedom.

Following the notation defined in Section 2.1, the correlations can be calculated as: šœƒš‘‰š‘–š‘—=ĪØš‘›āˆ’2ī€·āˆ’š‘”š‘›āˆ’2,š›¼,āˆ’š‘”š‘›āˆ’2,š›¼;šœŒš‘–š‘—ī€øāˆ’š›¼2,šœƒš›¼(1āˆ’š›¼)(2.19)š‘ˆš‘–š‘—=ĪØš‘›āˆ’2ī‚€š‘”š‘›āˆ’2,š›½,š‘”š‘›āˆ’2,š›½;šœŒš‘–š‘—ī‚āˆ’ī‚€1āˆ’š›½ī‚2š›½ī‚€1āˆ’š›½ī‚,šœƒ(2.20)š‘–š‘—š‘ˆš‘‰=ĪØš‘›āˆ’2ī‚€āˆ’š‘”š‘›āˆ’2,š›¼,š‘”š‘›āˆ’2,š›½;šœŒš‘–š‘—ī‚ī‚€āˆ’š›¼1āˆ’š›½ī‚ī‚™š›¼(1āˆ’š›¼)š›½ī‚€1āˆ’š›½ī‚,(2.21) where ĪØš‘›āˆ’2 is the CDF of a bivariate š‘”-distribution with š‘›āˆ’2 degrees of freedom and šœŒš‘–š‘— denotes the Pearson correlation between the š‘–th and š‘—th test statistics. As can be seen from these formulas, the correlations depend on š›¼ and š›½. No analytical solutions can be found for these two parameters. We use the following iterative computation algorithm to calculate sample size.

Algorithm.(1)Input design parameters š‘Ÿ1,š‘1,š‘Ÿ2,š‘2,š‘š,šœ‹0,š›æ,š‘Ž1,šœŒš‘ˆ,šœŒš‘‰andšœŒš‘ˆš‘‰. (2)Start from šœƒš‘ˆ = 0, šœƒš‘‰ = 0 and šœƒš‘ˆš‘‰ = 0. (3)Calculate š›½ from (2.17). (4)Using the current values of šœƒš‘ˆ, šœƒš‘‰, šœƒš‘ˆš‘‰ and š›½, solve for š›¼ from equation Ī¦((logš‘Ÿ1āˆ’šœ‡š‘Œ)/šœŽš‘Œ)=š‘1. (5)Using the current estimates of š›½ and š›¼, calculate šœƒš‘‰, šœƒš‘ˆ and šœƒš‘ˆš‘‰ from (2.19), (2.20) and (2.21), respectively. Obtain the average correlations šœƒš‘‰, šœƒš‘ˆ and šœƒš‘ˆš‘‰. (6)With updated estimates of šœƒš‘‰, šœƒš‘ˆ and šœƒš‘ˆš‘‰, repeat steps 3 to 5 until the estimates of š›½ and š›¼ converge.(7)Plug the estimated š›½ and š›¼ into (2.18) to solve the sample size.
The estimates of rejection threshold š›¼ and comparison-wise type II error š›½ can also be obtained.

3. Numerical Studies

3.1. Simulation

The proposed sample size calculation procedure was illustrated for one-sided š‘”-test comparing the mean of two groups. The effect size š›æ=1 and allocation proportion š‘Ž1=0.5. Two types of correlation structures were used: blockwise correlation and autoregressive correlation structure. In the blockwise correlation structure, a proportion of test statistics were correlated in units of blocks. The correlation coefficient within block was a constant, and test statistics were independent across blocks. True null test statistics and true alternative test statistics were independent. In the autoregressive correlation structure, the correlation matrix (šœŽš‘–š‘—) for dependent test statistics was parameterized by šœŽš‘–š‘—(šœŒ)=šœŒ|š‘–āˆ’š‘—|, where šœŽš‘–š‘—(šœŒ) is the Pearson correlation coefficient for the š‘–th and š‘—th test statistics and šœŒ is a correlation parameter.

Oura et al. [22] provided a sample size calculation method for controlling FDP using the beta-binomial model. Only test statistics of true alternative hypotheses are allowed to be dependent, with blockwise correlation structure. For comparison, this method and the sample size calculation procedure for controlling FDR with dependence adjustment in [18] were also assessed. Specifically, the criteria for controlling FDP are ī‚µš‘ˆP(FDPā‰¤0.05)ā‰„0.95,Pš‘š1ī‚¶ā‰„0.9ā‰„0.8.(3.1) The criteria for controlling FDR are ī‚µš‘ˆFDRā‰¤0.05,Pš‘š1ī‚¶ā‰„0.9ā‰„0.8.(3.2)

Table 2 presents the sample size estimates for the blockwise correlation structure. Several parameter configurations were used. The block size is 20 or 100, for š‘š = 2000 or 10000, respectively. We observe that the sample size increases as the correlation between test statistics gets stronger, represented by a greater correlation parameter or a larger proportion of correlated test statistics. When the correlation is fixed, as the number of tests š‘š increases, the required sample size decreases. With the other parameters fixed, when the number of true alternative hypotheses increases (šœ‹0 decreases), the required sample size decreases.


šœ‹ 0 šœŒ š‘¢ šœŒ š‘£ š‘ š‘¢ š‘ š‘£ š‘š = 2000 š‘š = 10000
FDP FDR FDP FDR

0.9 0 . 2 0 . 2 0 . 1 0 . 1 7 5 6 6 6 8 6 5
0 . 5 0 . 5 0 . 1 0 . 1 7 7 6 6 7 0 6 5
0 . 8 0 . 8 0 . 1 0 . 1 8 1 6 7 7 4 6 6
0 0 . 2 0 0 . 3 7 6 6 6 6 8 6 4
0 0 . 5 0 0 . 3 7 8 6 6 7 1 6 4
0 0 . 6 0 0 . 3 8 1 6 6 7 3 6 4
0 . 2 0 1 0 77 (74) 6 7 7 0 6 7
0 . 5 0 1 0 79 (75) 6 9 7 2 6 9
0 . 8 0 1 0 82 (77) 7 2 7 5 7 2

0.7 0 . 2 0 . 2 0 . 1 0 . 1 5 3 4 9 5 0 4 8
0 . 5 0 . 5 0 . 1 0 . 1 5 4 4 9 5 1 4 8
0 . 8 0 . 8 0 . 1 0 . 1 5 5 4 9 5 3 4 9
0 0 . 2 0 0 . 3 5 3 4 8 5 0 4 8
0 0 . 5 0 0 . 3 5 5 4 8 5 2 4 8
0 0 . 8 0 0 . 3 5 8 4 8 5 6 4 8
0 . 2 0 1 0 54 (53) 4 9 5 0 4 9
0 . 5 0 1 0 55 (54) 5 0 5 0 5 0
0 . 8 0 1 0 56 (55) 5 2 5 0 5 2

šœŒ š‘¢ : correlation between test statistics for which the alternative hypotheses are true; šœŒ š‘£ : correlation between test statistics for which the null hypotheses are true; š‘ š‘¢ : proportion of correlated test statistics for which the alternative hypotheses are true; š‘ š‘£ : proportion of correlated test statistics for which the null hypotheses are true.

The sample sizes for controlling FDP are greater than those for controlling FDR because controlling FDP is in general more stringent. In the case that šœ‹0 = 0.9, š‘š‘£ = 0.3, šœŒš‘£ = 0.6 and š‘š = 2000 (see Table 2), the sample size for controlling FDP is 81, which is 23% greater than the sample size for controlling FDR. The sample sizes using the method in [22] are in parentheses and are slightly smaller than ours. In terms of computational efficiency, our algorithm converges very fast and generally within 10 steps. The computation is not heavy, and in fact, very similar and comparable to that in [18] for controlling FDR with dependence adjustment. The method of Oura et al. [22] is more computationally intensive. It becomes not feasible when the number of tests or the number of blocks of dependent test statistics is large. Simulation studies show that FDP is controlled and the power is achievable with the sample size given by our procedure at the calculated rejection threshold š›¼ (results not shown).

Table 3 presents the sample sizes for the autoregressive correlation structure. Similar trends for sample size are observed as the design parameters vary. The method in [22] is not applicable to this dependence structure.


šœ‹ 0 šœŒ š‘¢ šœŒ š‘£ š‘ š‘¢ š‘ š‘£ š‘š = 2 0 0 0 š‘š = 1 0 0 0 0
FDP FDR FDP FDR

0.9 0.2 0.2 0.1 0.1 75 66 67 64
0.5 0.5 0.1 0.1 75 66 67 64
0.8 0.8 0.1 0.1 77 66 68 64
0.2 0.2 0.4 0.4 75 66 67 64
0.5 0.5 0.4 0.4 76 66 68 64
0.8 0.8 0.4 0.4 81 66 69 64

0.7 0.2 0.2 0.1 0.1 53 48 49 48
0.5 0.5 0.1 0.1 53 48 49 48
0.8 0.8 0.1 0.1 54 48 50 48
0.2 0.2 0.4 0.4 53 49 49 48
0.5 0.5 0.4 0.4 54 49 50 48
0.8 0.8 0.4 0.4 56 49 51 48

šœŒ š‘¢ : correlation parameter for test statistics for which the alternative hypotheses are true; šœŒ š‘£ : correlation parameter for test statistics for which the null hypotheses are true; š‘ š‘¢ : proportion of correlated test statistics for which the alternative hypotheses are true; š‘ š‘£ : proportion of correlated test statistics for which the null hypotheses are true.
3.2. Sample Size Calculation Based on a Prostate Cancer Dataset

We use a prostate cancer dataset as source of correlation structure to illustrate the proposed sample size calculation method while ensuring overall power. The study by Wang et al. [28] investigated the association between mRNA gene expression levels and the aggressive phenotype of prostate cancer. The dataset contains 13935 mRNA measured from 62 patients with aggressive prostate cancer and 63 patients with nonaggressive disease. The method in [23] was used to estimate the correlation between gene expression levels. The estimated average correlation of expression levels of null genes, alternative genes and between null genes and alternative genes are 0.0040, 0.0043 and āˆ’0.0005, respectively.

Sample size was calculated for one-sided š‘”-test. The total number of genes is š‘š=10000, and the following criteria are to be satisfied: P(FDPā‰¤0.10)ā‰„0.7 and P(š‘ˆ/š‘š1ā‰„0.9)ā‰„0.8. Table 4 presents the sample sizes for various values of š‘š1. We performed simulation studies to confirm that these sample sizes provided adequate power at the rejection threshold given by our algorithm. Simulation data were generated with blockwise dependence structure such that the average correlation was close to the estimated correlation from the real dataset.


šœ‹ 0 š‘š 1 š‘› Ģ‚ P ( F D P ā‰¤ 0 . 1 0 ) Ģ‚ P ( ( š‘ˆ / š‘š 1 ) ā‰„ 0 . 9 ) Ģ‚ š›¼

0.95 500 74 0.830 0.915 0.003
0.9 1000 63 0.855 0.970 0.007
0.85 1500 57 0.885 0.955 0.012
0.8 2000 52 0.875 0.945 0.018
0.75 2500 48 0.870 0.955 0.026
0.7 3000 44 0.915 0.925 0.034

Ģ‚ P ( F D P ā‰¤ 0 . 1 0 ) and Ģ‚ P ( ( š‘ˆ / š‘š 1 ) ā‰„ 0 . 9 ) : empirical probability from 200 simulation runs.

4. Discussion

In practice, when planning a study one typically needs to make some assumptions. For designing multiple testing studies, a common assumption is that the dependence between test statistics is weak. In this paper, we provide a computationally effective method of sample size calculation for controlling FDP under weak dependence while achieving the desired overall power. This approach uses semiparametric assumptions on dependence structure. We only need to estimate the Pearson correlation between test statistics, and thus this method is applicable to many realistic settings where weak dependence can be assumed. The variance formula of FDP provides a convenient tool to uncover the relationship between the design parameters and the variability of FDP under the assumption of weak dependence. Simulation studies indicate that the algorithm is computationally efficient and stable.

We have used one-sided š‘”-test to illustrate the method, and the procedure can be easily extended to two-sided š‘”-test and other tests. Common effect size is assumed in the sample size calculation algorithm. In practice, one can try different effect sizes, see the range of sample sizes and the variability and then make a decision. Effects of variations on other parameters such as šœ‹0 can be examined similarly.

Acknowledgments

We would like to thank the reviewers and the Editor for their careful reading of our paper and the constructive suggestions. This research was partially supported by a Stony Wold-Herbert Foundation grant and NIH grants 2P30 CA16087, 5P30 ES00260, UL1 TR000038 and R03 CA153083.

References

  1. Y. Benjamini and Y. Hochberg, ā€œControlling the false discovery rate: a practical and powerful approach to multiple testing,ā€ Journal of the Royal Statistical Society B, vol. 57, no. 1, pp. 289ā€“300, 1995. View at: Google Scholar
  2. J. D. Storey, ā€œA direct approach to false discovery rates,ā€ Journal of the Royal Statistical Society B, vol. 64, no. 3, pp. 479ā€“498, 2002. View at: Publisher Site | Google Scholar
  3. J. D. Storey, J. E. Taylor, and D. Siegmund, ā€œStrong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach,ā€ Journal of the Royal Statistical Society B, vol. 66, no. 1, pp. 187ā€“205, 2004. View at: Publisher Site | Google Scholar
  4. A. Farcomeni, ā€œGeneralized augmentation to control the false discovery exceedance in multiple testing,ā€ Scandinavian Journal of Statistics, vol. 36, no. 3, pp. 501ā€“517, 2009. View at: Publisher Site | Google Scholar
  5. Y. Ge, S. C. Sealfon, and T. P. Speed, ā€œMultiple testing and its applications to microarrays,ā€ Statistical Methods in Medical Research, vol. 18, no. 6, pp. 543ā€“563, 2009. View at: Publisher Site | Google Scholar
  6. C. R. Genovese and L. Wasserman, ā€œExceedance control of the false discovery proportion,ā€ Journal of the American Statistical Association, vol. 101, no. 476, pp. 1408ā€“1417, 2006. View at: Publisher Site | Google Scholar
  7. C. Genovese and L. Wasserman, ā€œA stochastic process approach to false discovery control,ā€ The Annals of Statistics, vol. 32, no. 3, pp. 1035ā€“1061, 2004. View at: Publisher Site | Google Scholar
  8. E. L. Korn, M.-C. Li, L. M. McShane, and R. Simon, ā€œAn investigation of two multivariate permutation methods for controlling the false discovery proportion,ā€ Statistics in Medicine, vol. 26, no. 24, pp. 4428ā€“4440, 2007. View at: Publisher Site | Google Scholar
  9. E. L. Korn, J. F. Troendle, L. M. McShane, and R. Simon, ā€œControlling the number of false discoveries: application to high-dimensional genomic data,ā€ Journal of Statistical Planning and Inference, vol. 124, no. 2, pp. 379ā€“398, 2004. View at: Publisher Site | Google Scholar
  10. N. Meinshausen, ā€œFalse discovery control for multiple tests of association under general dependence,ā€ Scandinavian Journal of Statistics, vol. 33, no. 2, pp. 227ā€“237, 2006. View at: Publisher Site | Google Scholar
  11. M. J. van der Laan, S. Dudoit, and K. S. Pollard, ā€œAugmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives,ā€ Statistical Applications in Genetics and Molecular Biology, vol. 3, article 15, 2004. View at: Publisher Site | Google Scholar
  12. S. H. Jung, H. Bang, and S. Young, ā€œSample size calculation for multiple testing in microarray data analysis,ā€ Biostatistics, vol. 6, no. 1, pp. 157ā€“169, 2005. View at: Publisher Site | Google Scholar
  13. S. J. Wang and J. J. Chen, ā€œSample size for identifying differentially expressed genes in microarray experiments,ā€ Journal of Computational Biology, vol. 11, no. 4, pp. 714ā€“726, 2004. View at: Publisher Site | Google Scholar
  14. M. C. K. Yang, J. J. Yang, R. A. McIndoe, and J. X. She, ā€œMicroarray experimental design: power and sample size considerations,ā€ Physiological Genomics, vol. 16, pp. 24ā€“28, 2003. View at: Publisher Site | Google Scholar
  15. S. H. Jung, ā€œSample size for fdr-control in microarray data analysis,ā€ Bioinformatics, vol. 21, no. 14, pp. 3097ā€“3104, 2005. View at: Publisher Site | Google Scholar
  16. S. S. Li, J. Bigler, J. W. Lampe, J. D. Potter, and Z. Feng, ā€œFDR-controlling testing procedures and sample size determination for microarrays,ā€ Statistics in Medicine, vol. 24, no. 15, pp. 2267ā€“2280, 2005. View at: Publisher Site | Google Scholar
  17. S. Pounds and C. Cheng, ā€œSample size determination for the false discovery rate,ā€ Bioinformatics, vol. 21, no. 23, pp. 4263ā€“4271, 2005. View at: Publisher Site | Google Scholar
  18. Y. Shao and C.-H. Tseng, ā€œSample size calculation with dependence adjustment for FDR-control in microarray studies,ā€ Statistics in Medicine, vol. 26, no. 23, pp. 4219ā€“4237, 2007. View at: Publisher Site | Google Scholar
  19. C. A. Tsai, S. J. Wang, D. T. Chen, and J. J. Chen, ā€œSample size for gene expression microarray experiments,ā€ Bioinformatics, vol. 21, no. 8, pp. 1502ā€“1508, 2005. View at: Publisher Site | Google Scholar
  20. X. D. Zhang, Optimal High-Throughput Screening: Practical Experimental Design and Data Analysis for Genome-Scale Rnai Research, Cambridge University Press, 2011. View at: Publisher Site
  21. M. L. T. Lee and G. A. Whitmore, ā€œPower and sample size for dna microarray studies,ā€ Statistics in Medicine, vol. 21, no. 23, pp. 3543ā€“3570, 2002. View at: Publisher Site | Google Scholar
  22. T. Oura, S. Matsui, and K. Kawakami, ā€œSample size calculations for controlling the distribution of false discovery proportion in microarray experiments,ā€ Biostatistics, vol. 10, no. 4, pp. 694ā€“705, 2009. View at: Publisher Site | Google Scholar
  23. S. Shang, M. Liu, and Y. Shao, ā€œA tight prediction interval for false discovery proportion under dependence,ā€ Open Journal of Statistics, vol. 2, no. 2, pp. 163ā€“171, 2012. View at: Publisher Site | Google Scholar
  24. W.-J. Lin, H.-M. Hsueh, and J. J. Chen, ā€œPower and sample size estimation in microarray studies,ā€ Bmc Bioinformatics, vol. 11, article 48, 2010. View at: Publisher Site | Google Scholar
  25. R. Heller, ā€œCorrelated z-values and the accuracy of large-scale statistical estimates,ā€ Journal of the American Statistical Association, vol. 105, no. 491, pp. 1057ā€“1059, 2010. View at: Publisher Site | Google Scholar
  26. Y. Pawitan, S. Calza, and A. Ploner, ā€œEstimation of false discovery proportion under general dependence,ā€ Bioinformatics, vol. 22, no. 24, pp. 3025ā€“3031, 2006. View at: Publisher Site | Google Scholar
  27. A. Farcomeni, ā€œSome results on the control of the false discovery rate under dependence,ā€ Scandinavian Journal of Statistics, vol. 34, no. 2, pp. 275ā€“297, 2007. View at: Publisher Site | Google Scholar
  28. L. Wang, H. Tang, V. Thayanithy et al., ā€œGene networks and microRNAs implicated in aggressive prostate cancer,ā€ Cancer Research, vol. 69, no. 24, pp. 9490ā€“9497, 2009. View at: Publisher Site | Google Scholar

Copyright © 2012 Shulian Shang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views2313
Downloads610
Citations

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.