Constructing Benchmark Databases and Protocols for Medical Image Analysis: Diabetic Retinopathy

Table 1

Summary of the current state of the reference image databases in terms of the key questions addressed in Section 2.1.

Key questions	STARE; (vessel)	STARE; (disc)	DRIVE	MESSIDOR	CMIF	ROC	REVIEW
C2: “Is there a data set for which the correct answers are known?”	x		x	x		x	x
C3: “Are there data sets in common use?”	x	x	x	x	x	x	x
C4: “Are there experiments which show algorithms are stable and work as expected?”	x		x			x
C5: “Are there any strawman algorithms?”	x	x	x
C6.1: “What code is available?”						x
C6.2: “What data is available?”	x	x	x	x	x	x	x
C7: “Is there a quantitative methodology for the design of algorithms?”
C8.1: “What should we be measuring to quantify performance?”	x	x	x			x	x
C8.2: “What metrics are used?”		x	x			x	x
∑	6	5	7	3	2	7	5