Noninferiority and Equivalence Evaluation of Clinical Performance among Computed Radiography, Film, and Digitized Film for Telemammography Services

Table 6

Equivalence tests for accuracy, PPV, sensitivity, specificity, and AUC by paired devices ().

Variable	Compared devices		Mean difference ()	SE	⁢(1-2α)% confidence interval for equivalence testing
	() device	() device			Lower	Upper
Accuracy	LUMIX	ICR	0.010	0.0142	−0.013	0.034	−6.31	<0.001	Ha
	LUMIX	CR	−0.014	0.0133	−0.036	0.008	−6.46	<0.001	Ha
	LUMIX	FILM	−0.047	0.0152	−0.072	−0.022	−3.49	<0.001	Ha
	ICR	CR	−0.024	0.0138	−0.047	−0.002	−5.46	<0.001	Ha
	ICR	FILM	−0.057	0.0162	−0.084	−0.031	−2.65	0.004	Ha
	FILM	CR	0.033	0.0164	0.006	0.060	−4.11	<0.001	Ha
PPV	LUMIX	ICR	−0.003	0.0052	−0.011	0.006	−18.69	<0.001	Ha
	LUMIX	CR	−0.007	0.0049	−0.015	0.001	−18.79	<0.001	Ha
	LUMIX	FILM	−0.009	0.0050	−0.017	−0.001	−18.28	<0.001	Ha
	ICR	CR	−0.005	0.0047	−0.012	0.003	−20.32	<0.001	Ha
	ICR	FILM	−0.007	0.0040	−0.013	0.000	−23.08	<0.001	Ha
	FILM	CR	0.002	0.0027	−0.002	0.007	−35.76	<0.001	Ha
Sensitivity	LUMIX	ICR	0.035	0.0203	0.001	0.068	−3.21	<0.001	Ha
	LUMIX	CR	−0.017	0.0219	−0.053	0.019	−3.78	<0.001	Ha
	LUMIX	FILM	−0.065	0.0246	−0.105	−0.025	−1.43	0.08	Ho
	ICR	CR	−0.052	0.0210	−0.087	−0.017	−2.28	0.01	Ha
	ICR	FILM	−0.100	0.0256	−0.142	−0.057	−0.02	0.49	Ho
	FILM	CR	0.048	0.0316	−0.004	0.100	−1.66	0.05	Ha
Specificity	LUMIX	ICR	−0.012	0.0192	−0.043	0.020	−4.60	<0.001	Ha
	LUMIX	CR	−0.012	0.0158	−0.038	0.014	−5.59	<0.001	Ha
	LUMIX	FILM	−0.031	0.0182	−0.061	−0.001	−3.79	<0.001	Ha
	ICR	CR	0.000	0.0173	−0.028	0.028	−5.79	<0.001	Ha
	ICR	FILM	−0.019	0.0182	−0.049	0.011	−4.42	<0.001	Ha
	FILM	CR	0.019	0.0124	−0.001	0.040	−6.50	<0.001	Ha
AUC	LUMIX	ICR	0.009	0.0646	−0.098	0.115	−1.412	0.08	Ho
	LUMIX	CR	−0.016	0.0350	−0.073	0.042	−2.408	0.008	Ha
	LUMIX	FILM	−0.026	0.0412	−0.093	0.042	−1.807	0.04	Ha
	ICR	CR	−0.024	0.0660	−0.133	0.084	−1.145	0.13	Ho
	ICR	FILM	−0.034	0.0508	−0.118	0.049	−1.292	0.10	Ho
	FILM	CR	0.010	0.0463	−0.066	0.086	−1.945	0.03	Ha

Each comparison was calculated from 980 observations (70 cases, seven radiologists, and two devices). PPV: positive predictive value, ROC: receiver operating characteristic, AUC: area under ROC curve, SE: standard error of the mean, Ho: null hypothesis, Ha: alternative hypothesis for testing equivalence, α: significance of the test (0.05), δ: difference of the means allowed to achieve equivalence, : test for difference of compared devices, that is, = (Difference () − δ)/SE, : retained hypothesis equivalence at δ level (“Ha” indicates equivalence achieved and “Ho” indicates failing to reject the null hypothes).
Ho: difference () − δ = 0.
Ha: difference () − δ < 0.