Bag-of-Words Representation in Image Annotation: A Review

Table 2

Comparisons of datasets used and annotation performance.

Work	Categories		Dataset	No. of categories	No. of images	Baseline
	Scene	Object
2012
de Campos et al. [70]		v	PASCAL′07/′08^16	20	9292
Elfiky et al. [97]	v	v	Sport event/15 scene/butterflies^17/; PASCAL′07/′09	15/20	6000/21000/2000/160k/4194k	Spatial pyramid
Fernando et al. [68]		v	PASCAL′06/Caltech 10^18	10/10/11	5304/3044	BoW
Gavves et al. [77]	v		Oxford 5k^19	11	5062
Kesorn and Poslad [80]	v		Olympic organization website + Google images	8	16000	pLSA
Lee and Grauman [103]	v	v	MSRC-v0^20/-v2/PASCAL′08/Corel/Gould′09	21/20/7/14	3457/591/1023/100/715	LDA
Qin and Yung [64]	v		SCENE-8/-15	8/15	2688/4485	BoW
Romberg et al. [102]	v	v	Flickr-10M	>300	10080251	pLSA
Shang and Xiao [99]		v	Caltech 256/MSRC	20/20		BoW
Stottinger et al. [104]			PASCAL′07	20	9963
Tong et al. [100]	v	v	Tattoo dataset/Oxford/Flickr		101745/5062/1002805	RS^21/HKM/AKM
2011
Hare et al. [73]	v	v	UK Bench/MIR Flickr-25000^22			BoW
López-Sastre et al. [78]		v	Caltech 101	10	890	Mikolajczyk et al. [25]; Stark and Schiele [119]
Luo et al. [18]		v	Caltech 4/Graz-02^23	5/2	400/200	Li and Perona [31]; Moosmann et al. [72]
Van Gemert [65]	v	v	Corel/PASCAL′09	20	2000/7054	BoW/spatial pyramid
Yang et al. [37]		v	PASCAL′08	20	8445	Divvala et al. [120]; Zhong et al. [109]
Zhang et al. [76]	v	v	Google images/Caltech 101and256	15	376500	BoW
Zhang et al. [38]	v	v	ImageNet^24	15 queries	1.5 million	Nister and Stewenius [121]; Zhong et al. [61]
2010
Bae and Juang [79]	v		Corel	15	20000	LSA
Chen et al. [62]		v	Oxford buildings/Flickr 1k	11 (55 queries)/7; (56 queries)	5062/11282	Sivic and Zisserman [14]; Philbin et al. [45]; Lazebnik et al. [48]
Cheng and Wang [82]	v		6-scene dataset	6	700	Vogel and Schiele [122]; Bosch et al. [98]; Quelhas et al. [13]; Boutell et al. [123]
Ding et al. [105]	v		TRECVID′06^25	20	61901	Binary/TF/TF-IDF weighting
Jégou et al. [22]	v	v	Holidays^26/Oxford 5k/U. of Kentucky object recognition^27	500/11 (55 queries)	1491/5062/6376	BoW by HE^28/
Jiang et al. [17]		v	TRECVID′06	20	79484
Li and Godli [87]	v	v	Corel	50	5000	Duygulu et al. [118]; Jeon et al. [124]; Lavrenko et al. [125]; Monay and Gatica-Perez, 2007 [126]
Qin and Yung [106]	v	v		8/13/15	2688/3759/4485	Siagian and Itti [127, 128]; Bosch et al. [29]; Li and Perona [31]; Quelhas et al. [60]; Lazebnik et al. [48]
Tirilly et al. [107]	v	v	U. of Kentucky object recognition/Oxford 5k/Caltech 6 & 101	300/55/200 queries	10200/5062/8197	TF-IDF weighting
Uijlings et al. [33]		v	PASCAL′07/TRECVID′05/Caltech 101	20/101/15	9963/12914/4485	BoW
Wu et al. [69]		v	LabelMe^29/PASCAL′06	495/10		BoW; Bar-Hillel et al. [129]; Davis et al. [130]; Goldberger et al. [131]; Perronnin et al. [113]; Weinberger et al. [132]
2009
Chen et al. [39]	v		LabelMe	8 (448 queries)	2689	Yang et al. [133]
Lu and Ip (a) [41]	v		LabelMe + Web images	3	1239	k-NN; LDA
Lu and Ip (b) [42]	v	v	Corel/histological images	10/5	1000	pLSA/SVM
S. Kim and D. Kim [40]	v	v	Corel/histological images	10/5	1000	LLP/GLP/SVM/pLSA
Uijlings et al. [43]		v	PASCAL′07	20	9963	BoW
Xiang et al. [108]		v	Corel/TRECVID′05	50/39	5000	Feng et al. [134]
Zhang et al. [94]	v	v	Google images/Corel/Caltech 101 and 256	1506 queries/50/15	376500/500/2250	BoW
2008
Bosch et al. [98]	v		6-/8-/13-/15-scene	6/8/13/15	2688/702	BoW
Liu et al. [96]	v		PASCAL′06/Caltech 4/MSRC-v2	20/5/15		Savarese et al. [135]
Marszalek and Schmid [109]	v	v	Caltech 256	256		Lazebnik et al.[48]; Zhang et al. [35]
Rasiwasia and Vasconcelos [66]	v		15-natural scene/Corel	15/50		Bosch et al. [29]; Lazebnik et al. [48]; Li and Perona [31]; Liu and Shah [136]
Tirilly et al. [81]		v	Caltech 6 and 101	6/101	5435/8697	SVM
Van de Sande et al. [110]	v	v	PASCAL′07/TRECVID′05	20
Zheng et al. [71]		v	Caltech 101/PASCAL′05	12/4		BoW
2007
Bosch et al. [24]	v		Corel	6	700	Global and block-based features + k-NN; Vogel and Schiele [122]
Chum et al. [52]	v	v	Oxford + Flickr		104844	BoW
Gökalp and Aksoy [28]	v		LabelMe	7	1050	Bag of individual regions/bag of region pairs
Hörster and Lienhart [21]	v		Flickr	12 (60 queries)	246348	BoW/color based BoW
Jegou et al. [74]	v	v	Object recognition benchmark^30		10200	Object recognition benchmark
Li and Fei-Fei [111]	v		8 events	8	240	LDA
Lienhart and Slaney [93]	v		Flickr	12 (60 queries)	253460	LSA
Philbin et al. [45]	v	v	Oxford 5 k/Flickr 1 and 2	11/145 and 450 tags	5062/99782/1040801	BoW
Quelhas et al. [13]	v		Corel + Web images	5	6680/3805/9457/6364	BoW; Vailaya et al. [137]
Wu et al. [46]	v	v	Caltech 7/Corel	8/6	600	LDA/pLSA
Yuan et al. [112]		v	Caltech 101	2	558	BoW
2006
Agarwal and Triggs [47]		v	Caltech 7 + Graz/KTH-TIPS^31/Cal-IPNP^32	4/10/2	1337/810/360	LDA
Bosch et al. [29]	v		6-/8-/13-scene	6/8/13	2688/702/1071	BoW
Lazebnik et al. [48]	v	v	15-scene/Caltech 101/Graz	15/101/2		Zhang et al. [138]; Opelt et al. [139]
Marszalek and Schmid [49]		v	PASCAL′05			Wang et al. [20]
Monay et al. [50]	v		Corel	4	6600
Moosmann et al. [72]		v	Graz-02/PASCAL′05	3/4		BoW
Perronnin et al. [113]	v	v	Corel	10	1000	BoW; Farquhar et al. [140]; Deselaers et al. [141]

¹⁶http://pascallin.ecs.soton.ac.uk/challenges/VOC/.
¹⁷http://www.comp.leeds.ac.uk/scs6jwks/dataset/leedsbutterfly/.
¹⁸http://www.vision.caltech.edu/Image_Datasets/Caltech101/.
¹⁹http://www.robots.ox.ac.uk/~vgg/data/oxbuildings/.
²⁰http://www.cs.utexas.edu/~grauman/research/datasets.html.
²¹Random seed [142].
²²http://press.liacs.nl/mirflickr/.
²³http://lear.inrialpes.fr/people/marszalek/data/ig02/.
²⁴http://www.image-net.org/.
²⁵http://www-nlpir.nist.gov/projects/tv2006/tv2006.html.
²⁶http://lear.inrialpes.fr/~jegou/data.php.
²⁷http://vis.uky.edu/.
²⁸Hamming embedding.
²⁹http://labelme.csail.mit.edu/.
³⁰http://vis.uky.edu/%7Estewe/ukbench/.
³¹http://www.nada.kth.se/cvap/databases/kth-tips/.
³²http://crl.ucsd.edu/.