Misuse of Statistical Methods in 10 Leading Chinese Medical Journals in 1998 and 2008
Table 1
Errors/defects in statistical methods.
Types of errors
Incorrect use in 1998 (%)
Incorrect use in 2008 (%)
value
OR* (1998/2008)
95% CI**
-test
305 (62.0%)
253 (44.4%)
32.83
<0.001
2.04
(1.60,2.61)
(1) Using multiple -test for multiple group comparison
153 (31.1%)
129 (22.6％)
9.71
0.002
1.54
(1.17,2.03)
(2) Using paired -test for unpaired data or vice versa
40 (8.1%)
34 (6.0%)
1.91
0.167
1.40
(0.87,2.24)
(3) Using -test under nonparametric setting
89 (18.1%)
60 (10.5%)
12.52
<0.001
1.88
(1.32,2.67)
(4) Using -test without considering the baseline
52 (10.6%)
33 (5.8%)
8.19
0.004
1.92
(1.22,3.03)
(5) Using -test to conduct repeated-measure data
73 (14.8%)
60 (10.5%)
4.48
0.034
1.48
(1.03,2.13)
Others
28 (5.7%)
8 (1.4%)
14.82
<0.001
4.24
(1.91,9.39)
Contingency tables
154 (48.3%)
169 (32.3%)
21.35
<0.001
1.96
(1.47,2.60)
(1) No continuity correction or Fisher exact test if needed
52 (16.3%)
53 (10.1%)
6.90
0.009
1.73
(1.15,2.61)
(2) No significant level adjustment for multiple comparison
82 (25.7%)
74 (14.2%)
17.53
<0.001
2.10
(1.48,2.98)
(3) Misusing Chi-square test for paired fourfold table
10 (3.1%)
12 (2.3%)
0.55
0.458
1.38
(0.59,3.23)
(4) Using Chi-square test for ranked data
29 (9.1%)
31 (5.9%)
3.00
0.083
1.59
(0.94,2.69)
(5) Ignorance of stratification factors
12 (3.8%)
12 (2.3%)
1.54
0.215
1.66
(0.74,3.75)
(6) Using value of Chi-square test instead of contingency coefficient to describe the correlation of two variables
8 (2.5%)
4 (0.8%)
3.13
0.077
3.34
(0.98,11.18)
Others
21 (6.6%)
19 (3.6%)
3.81
0.051
1.87
(0.99,3.53)
ANOVA***
128 (63.4%)
263 (59.0%)
1.12
0.289
1.20
(0.85,1.70)
(1) Using one-factorial ANOVA to analyse data from multifactorial designs
10 (5.0%)
31 (7.0%)
0.94
0.333
0.70
(0.34,1.45)
(2) Ignoring the setting of ANOVA for completely random design data
25 (12.4%)
53 (11.9%)
0.03
0.858
1.05
(0.63,1.74)
(3) No multiple pair-wise comparison of ANOVA when needed
25 (12.4%)
28 (6.3%)
6.89
0.009
2.11
(1.20,3.72)
(4) Misusing the method of multiple pair-wise comparison of ANOVA
51 (25.3%)
132 (29.6%)
1.30
0.255
0.80
(0.55,1.17)
(5) Using ANOVA to analyse repeated-measures data
45 (22.3%)
63 (14.1%)
6.65
0.010
1.74
(1.14,2.67)
Others
16 (7.9%)
10 (2.2%)
11.64
0.001
3.75
(1.67,8.42)
Rank transformation nonparametric test
29 (43.3%)
33 (17.7%)
17.57
<0.001
3.56
(1.93,6.57)
(1) Using multiple pair-wise comparison for multiple group comparison
14 (20.9%)
20 (10.7%)
4.43
0.035
2.21
(1.04,4.47)
(2) Using wrong type of rank sum test for different study types
4 (6.0%)
3 (1.6%)
2.07
0.150
3.89
(0.85,17.88)
Others
20 (29.9%)
6 (3.2%)
38.11
<0.001
12.84
(4.88,33.77)
*OR: odds ratio; **CI: confidence interval; ***ANOVA: analysis of variance. Incorrect use of (%): for each statistical method, is the number of articles using this statistical methods incorrectly and the percentage = n/the number of papers using this statistical methods × 100%; for each error under certain statistical methods, is the number of articles with this mistake and the percentage = n/the number of papers using these statistical methods × 100%.
