Noninferiority and Equivalence Evaluation of Clinical Performance among Computed Radiography, Film, and Digitized Film for Telemammography Services
Table 6
Equivalence tests for accuracy, PPV, sensitivity, specificity, and AUC by paired devices ().
Variable
Compared devices
Mean difference ()
SE
(1-2α)% confidence interval for equivalence testing
() device
() device
Lower
Upper
Accuracy
LUMIX
ICR
0.010
0.0142
−0.013
0.034
−6.31
<0.001
Ha
LUMIX
CR
−0.014
0.0133
−0.036
0.008
−6.46
<0.001
Ha
LUMIX
FILM
−0.047
0.0152
−0.072
−0.022
−3.49
<0.001
Ha
ICR
CR
−0.024
0.0138
−0.047
−0.002
−5.46
<0.001
Ha
ICR
FILM
−0.057
0.0162
−0.084
−0.031
−2.65
0.004
Ha
FILM
CR
0.033
0.0164
0.006
0.060
−4.11
<0.001
Ha
PPV
LUMIX
ICR
−0.003
0.0052
−0.011
0.006
−18.69
<0.001
Ha
LUMIX
CR
−0.007
0.0049
−0.015
0.001
−18.79
<0.001
Ha
LUMIX
FILM
−0.009
0.0050
−0.017
−0.001
−18.28
<0.001
Ha
ICR
CR
−0.005
0.0047
−0.012
0.003
−20.32
<0.001
Ha
ICR
FILM
−0.007
0.0040
−0.013
0.000
−23.08
<0.001
Ha
FILM
CR
0.002
0.0027
−0.002
0.007
−35.76
<0.001
Ha
Sensitivity
LUMIX
ICR
0.035
0.0203
0.001
0.068
−3.21
<0.001
Ha
LUMIX
CR
−0.017
0.0219
−0.053
0.019
−3.78
<0.001
Ha
LUMIX
FILM
−0.065
0.0246
−0.105
−0.025
−1.43
0.08
Ho
ICR
CR
−0.052
0.0210
−0.087
−0.017
−2.28
0.01
Ha
ICR
FILM
−0.100
0.0256
−0.142
−0.057
−0.02
0.49
Ho
FILM
CR
0.048
0.0316
−0.004
0.100
−1.66
0.05
Ha
Specificity
LUMIX
ICR
−0.012
0.0192
−0.043
0.020
−4.60
<0.001
Ha
LUMIX
CR
−0.012
0.0158
−0.038
0.014
−5.59
<0.001
Ha
LUMIX
FILM
−0.031
0.0182
−0.061
−0.001
−3.79
<0.001
Ha
ICR
CR
0.000
0.0173
−0.028
0.028
−5.79
<0.001
Ha
ICR
FILM
−0.019
0.0182
−0.049
0.011
−4.42
<0.001
Ha
FILM
CR
0.019
0.0124
−0.001
0.040
−6.50
<0.001
Ha
AUC
LUMIX
ICR
0.009
0.0646
−0.098
0.115
−1.412
0.08
Ho
LUMIX
CR
−0.016
0.0350
−0.073
0.042
−2.408
0.008
Ha
LUMIX
FILM
−0.026
0.0412
−0.093
0.042
−1.807
0.04
Ha
ICR
CR
−0.024
0.0660
−0.133
0.084
−1.145
0.13
Ho
ICR
FILM
−0.034
0.0508
−0.118
0.049
−1.292
0.10
Ho
FILM
CR
0.010
0.0463
−0.066
0.086
−1.945
0.03
Ha
Each comparison was calculated from 980 observations (70 cases, seven radiologists, and two devices). PPV: positive predictive value, ROC: receiver operating characteristic, AUC: area under ROC curve, SE: standard error of the mean, Ho: null hypothesis, Ha: alternative hypothesis for testing equivalence, α: significance of the test (0.05), δ: difference of the means allowed to achieve equivalence, : test for difference of compared devices, that is, = (Difference () − δ)/SE, : retained hypothesis equivalence at δ level (“Ha” indicates equivalence achieved and “Ho” indicates failing to reject the null hypothes). Ho: difference () − δ = 0. Ha: difference () − δ < 0.