Essentials of a Robust Deep Learning System for Diabetic Retinopathy Screening: A Systematic Literature Review

Table 3

Sensitivity, specificity, and AUC validation results of the 15 included studies. Differences in target condition and grading scales made direct comparisons of validation results difficult. AUC = area under the receiver operating characteristic curve; CI = confidence interval; DMO = diabetic macular oedema; DR = diabetic retinopathy; ETDRS = Early Treatment Diabetic Retinopathy Study; ICDR = International Clinical Diabetic Retinopathy Disease Severity Scale; NHS = English National Health Service; NZ MoH = New Zealand Ministry of Health.

	% (95% CI)
	Sensitivity	Specificity	AUC (95% CI)	Target condition	DR grading scale
Gulshan et al. [24]	97.5 (95.8–98.7)	93.4 (92.8–94.0)	—	Moderate or worse DR, referable DMO, or both	ICDR
Gulshan et al. [32]	92.1 (90.1–93.8)	95.2 (94.2–96.1)	0.980	Moderate or worse DR	ICDR
Krause et al. [33]	97.1	92.3	0.986	Moderate or worse DR	ICDR
Ruamviboonsuk et al. [34]	96.8 (93.9–100)	95.6 (98.3–96.8)	0.987 (0.977–0.995)	Moderate or worse DR	ICDR
Voets et al. [35]	90.6	84.7	0.951 (0.947–0.956)	Moderate or worse DR	ICDR
Ramachandran et al. [36]	96.0	90.0	0.980 (0.973–0.986)	Moderate or worse DR, or exudates in 1 disc diameter of the fovea	ICDR and NZ MoH guidelines
Li et al. [37]	97.0	91.4	0.989	Preproliferative or worse DR, DMO, or both	NHS diabetic eye screening guidelines
Gargeya et al. [38]	93.0	87.0	0.940	No DR or any DR	—
Abràmoff et al. [39]	96.8 (93.3–98.8)	87.0 (84.2–89.4)	0.980 (0.968–0.992)	Moderate or worse DR, DMO, or both	ICDR
Verbraak et al. [40]	79.4 (66.5–87.9)	93.8 (92.1–94.9)	—	More than mild DR	ICDR
Abràmoff et al. [41]	87.2 (81.8–91.2)	90.7 (88.3–92.7)	—	More than mild, or DMO, or both	ETDRS
Ting et al. [42]	98.9 (97.5–99.6)	92.2 (89.5–94.3)	0.983 (0.972–0.991)	Moderate or worse DR, DMO, and/or ungradable image	ICDR
Bellemo et al. [43]	92.3 (90.1–94.1)	89.0 (87.9–90.3)	0.973 (0.969–0.978)	Moderate or worse DR, DMO, and/or ungradable image	ICDR
Rogers et al. [44]	93.4% (95% CI: 90.8–95.8)	94.2% (95% CI: 91.0–97.2)	98.5% (95% CI: 97.8–99.2)	Referable DR (RDR) and proliferative DR (PDR)	Scottish DR grading scheme
Bhaskaranand et al. [45]	91.3% (95% CI: 90.9%–91.7%)	91.1% (95% CI: 90.9%–91.3%)	0.965 (95% CI: 0.963–0.966)	Severe NPDR, proliferative DR	American academy of ophthalmology