Research Article

Invalid Permutation Tests

Table 1

Simulated means of effect estimates.

šœƒ NullConventionalAdjustedOrthogonal

0 āˆ’ 0 . 0 0 0 2 7 āˆ’ 0 . 0 0 0 2 7 āˆ’ 0 . 0 0 0 2 1 āˆ’ 0 . 0 0 0 0 3
0.1 āˆ’ 0 . 0 0 0 0 7 āˆ’ 0 . 0 0 0 0 6 0 . 0 0 0 0 2 0 . 0 0 0 1 0
0.2 āˆ’ 0 . 0 0 0 0 5 āˆ’ 0 . 0 0 0 1 7 0 . 0 0 0 0 5 āˆ’ 0 . 0 0 0 4 9
0.3 0 . 0 0 0 2 2 0 . 0 0 0 2 2 0 . 0 0 0 2 6 āˆ’ 0 . 0 0 0 2 4
0.4 0 . 0 0 0 2 1 0 . 0 0 0 2 7 0 . 0 0 0 2 1 0 . 0 0 0 3 3
0.5 0 . 0 0 0 5 1 0 . 0 0 0 4 7 0 . 0 0 0 4 8 āˆ’ 0 . 0 0 0 0 9
0.6 āˆ’ 0 . 0 0 0 1 5 āˆ’ 0 . 0 0 0 0 5 āˆ’ 0 . 0 0 0 0 5 āˆ’ 0 . 0 0 0 3 8
0.7 0 . 0 0 0 5 0 0 . 0 0 0 5 3 0 . 0 0 0 4 6 0 . 0 0 0 1 8
0.8 āˆ’ 0 . 0 0 0 2 5 āˆ’ 0 . 0 0 0 3 5 āˆ’ 0 . 0 0 0 3 3 0 . 0 0 0 0 7
0.9 āˆ’ 0 . 0 0 0 6 6 āˆ’ 0 . 0 0 0 7 0 āˆ’ 0 . 0 0 0 6 0 āˆ’ 0 . 0 0 0 3 8
1 āˆ’ 0 . 0 0 0 0 7 āˆ’ 0 . 0 0 0 3 0 āˆ’ 0 . 0 0 0 0 5 0 . 0 0 0 0 4
1.1 āˆ’ 0 . 0 0 0 1 5 0 . 0 0 0 3 2 āˆ’ 0 . 0 0 0 0 7 0 . 0 0 0 0 6
1.2 0 . 0 0 0 1 3 āˆ’ 0 . 0 0 0 1 2 0 . 0 0 0 1 2 āˆ’ 0 . 0 0 0 0 9
1.3 0 . 0 0 0 1 4 āˆ’ 0 . 0 0 0 3 5 0 . 0 0 0 1 1 āˆ’ 0 . 0 0 0 1 9

Means of the sampling distributions of estimates of the null mean, using four procedures: ( 1 ) null, the usual method when the null is true, ( 2 ) conventional, the usual method when the null is false, ( 3 ) adjusted, subtracting off the estimate, ( 4 ) orthogonal, using only orthogonal permutations.