Bag-of-Words Representation in Image Annotation: A Review

Table 1

Comparisons of interest point detection, visual words generation, and learning models.

Work	Region/point detection	Local descriptor	Clustering algorithm	No. of visual words	Weighting scheme	Learning model
2012
de Campos et al. [70]	DoG	SIFT				Logistic regression
Elfiky et al. [97]	Harris-Laplace	SIFT/HSV; color + SIFT	k-means			SVM
Fernando et al. [68]	Harris-Laplace	PCA-SIFT/SIFT/SURF^1	k-means	2000		SVM
Gavves et al. [77]		SIFT/SURF		200000
Kesorn and Poslad [80]	DoG	SIFT	SLAC^2		Binary/TF/ TF-IDF	Naïve bayes/ SVM-linear/ SVM-RBF
Lee and Grauman [103]	NCuts^3	Texton; histogram	k-means	400		SVM
Qin and Yung [64]		Color SIFT	k-means			SVM-linear/ SVM-poly/ SVM-RBF
Romberg et al. [102]		SIFT	k-means			mm-pLSA^4
Shang and Xiao [99]		SIFT	k-means	1000		SVM
Stottinger et al. [104]	Harris-Laplace	RGB Harris with Laplacian scale selection	k-means	4000		SVM
Tong et al. [100]	Harris-Laplace	SIFT	AKM^5
2011
Hare et al. [73]	DoG/MSER	SIFT	AKM	1000–100000	IDF
López-Sastre et al. [78]	Hessian-Laplace	SIFT	CPM and Adaptive Refinement	3818		SVM
Luo et al. [18]	DoG	SIFT	k-means	500	TF	SVM
Van Gemert [65]	Harris and Hessian-Laplace	SIFT	k-means	2000
Yang et al. [37]		SIFT	k-means	1000		SVM
Zhang et al. [76]	DoG	SIFT	HKM^6	32357	TF-IDF
Zhang et al. [38]	DoG	SIFT	HKM	32400	TF-IDF
2010
Bae and Juang [79]	Dense sampling			171329
Chen et al. [62]	Hessian-Laplace	SIFT	GMM-BIC^7	3500	TF
Cheng and Wang [82]	Mean-shift^8	HSV color histogram and co-occurrence matrix				SVM
Ding et al. [105]	DoG	PCA-SIFT	k-means	2000		SVM
Jégou et al. [22]	Hessian-Laplace	SIFT	k-means	200000	TF-IDF
Jiang et al. [17]	DoG	SIFT	k-means	500–10000	Binary/TF/ TF-IDF/soft-weighting	SVM
Li and Godil [87]	DoG	SIFT	k-means	500/700/800	TF	pLSA
Qin and Yung [106]		PCA-SIFT	Accelerated k-means	32/128/2048/ 4096		SVM
Tirilly et al. [107]	Hessian-Laplace	SIFT	HKM	6556 to 117151
Uijlings et al. [33]		PCA-SIFT	k-means/ random forest	4096		SVM
Wu et al. [69]		SIFT	k-means	2500–4500		Naïve Bayes/ SVM
2009
Chen et al. [39]	DoG	SIFT	k-means	1000	Spatial weighting
Lu and Ip [41]	Dense sampling	HSV color + Gabor txture	k-means	100/200		SVM
Lu and Ip [42]	Dense sampling	HSV color + Gabor txture	k-means	100/200		LLP^9/GLP^10/ SVM
S. Kim and D. Kim [40]	Dense sampling	SIFT/SURF	k-means	500/1500/3000	TF	pLSA/SVM
Uijlings et al. [43]	Dense sampling	SIFT	k-means	4096		SVM
Xiang et al. [108]	NCuts	36 region features^11				MRFA^12
Zhang et al. [94]		SIFT	HKM	32357	TF-IDF
2008
Bosch et al. [98]	Harris-Laplace	Color SIFT	k-means	1500		k-NN/SVM
Liu et al. [96]	Harris-Laplace	SIFT	k-means	1000		SVM-linear
Marszałek and Schmid [109]	Harris-Laplace	SIFT	k-means	8000		SVM
Rasiwasia and Vasconcelos [66]		DCT^13 coefficients				Hierarchical Dirichlet models/SVM
Tirilly et al. [81]		SIFT	HKM	6556/61687	TF-IDF	SVM
Van de Sande et al. [110]	Harris-Laplace	Color SIFT	k-means	4000		SVM
Zheng et al. [71]	DoG + Hessian-Laplace	SIFT + Spin^14	k-means	1010		SVM
2007
Bosch et al. [24]	Dense sampling	HSV color + co-occurrence + edge	k-means	700		pLSA
Chum et al. [52]	Hessian-Laplace	SIFT	k-means		TF-IDF
Gökalp and Aksoy [28]	Dense sampling	HSV color	k-means			Bayesian classifier
Hörster and Lienhart [21]	DoG/dense sampling	Color SIFT	k-means			LDA
Jegou et al. [74]		SIFT	k-means	30000
Li and Fei-Fei [111]	Dense sampling	SIFT	k-means	300	TF	LDA
Lienhart and Slaney [93]		SIFT	k-means		TF	pLSA
Philbin et al. [45]	Hessian-Laplace	SIFT	AKM	1 M
Quelhas et al. [13]	DoG	SIFT	k-means	1000		SVM/pLSA
Wu et al. [46]	Dense sampling	Texture histogram				Unigram/ bigram/trigram models
Junsong et al. [112]	DoG	PCA-SIFT	k-means	160/500
2006
Agarwal and Triggs [47]	Dense sampling	SIFT	EM^15			LDA/SVM
Bosch et al. [29]	Dense sampling	Color SIFT	k-means	1500		k-NN/pLSA
Lazebnik et al. [48]	Dense sampling	SIFT	k-means	200/400		SVM
Marszałek and Schmid [49]	Harris-Laplace	SIFT	k-means	1000	TF	SVM
Monay et al. [50]	DoG	SIFT	k-means	1000	TF	pLSA
Moosmann et al. [72]	Dense sampling/DoG	HSV color + wavelet/ SIFT	Extremely randomized trees			SVM
Perronnin et al. [113]	DoG	PCA-SIFT		1024		SVM-linear

¹Speeded up robust features [114].
²Search ant and labor ant clustering algorithm [115].
³Normalized cuts [116].
⁴Multilayer modality pLSA.
⁵Approximate k-means.
⁶Hierarchical k-means.
⁷Gaussian mixture model with Bayesian information criterion.
⁸Mean shift region segmentation algorithm [117].
⁹Local label propagation on the k-NN graph.
¹⁰Global label propagation on the complete graph.
¹¹Region color and standard deviation, region average orientation energy (12 filters), region size, location, convexity, first moment, and ratio of region area to boundary length squared [118].
¹²Multiple Markov random fields.
¹³Discrete cosine transform.
¹⁴A rotation-invariant two-dimensional histogram of intensities within an image region [71].
¹⁵Expectation maximization.