Formal Implementation of a Performance Evaluation Model for the Face Recognition System

Table 5

Comparison between evaluation methods (FERET, FRVT, and the proposed performance evaluation tool).

	FERET	FRVT	Proposed evaluation tool
		Face recognition system
Evaluation target	Face recognition system	Scenario	Face recognition system
		Operation
Face DB distribution method	DB distribution onsite (possibility of face DB disclosure)	DB distribution on-site (possibility of face DB disclosure)	Vendors provide the face recognition module. (No risk of disclosing face DB)
		FRVT 2006: Vendors provide the face recognition module. (No risk of disclosing face DB)
Performance evaluator	Vendor staff	Vendor staff	Evaluator
		FRVT 2006: Evaluator
Result analysis tool	Performance result analysis tool is required	Performance result analysis tool is required	No analysis tool is required
Result report	Manual work	Manual work	Automatically generated by the
generation			tool
Individual evaluation	Impossible	Impossible	Individual valuation by vendor
Evaluation costs	All related vendor staff should congregate in one place (expensive)	All related vendor staff should congregate in one place (expensive)	No meeting is required (low cost)
		FRVT 2006: No meeting is required (low cost)
Responsibility of the	I/O file format should be complied	I/O file format should be complied	Complies with recognition module
vendor	with	with	interface