Table of Contents
ISRN Machine Vision
Volume 2013 (2013), Article ID 428746, 16 pages
Research Article

Multimodal Markov Random Field for Image Reranking Based on Relevance Feedback

Department of Computer Sciences, Instituto Nacional de Astrofśsica, Óptica y Electrónica, Luis Enrique Erro No. 1, 72840 Tonantzintla, PUE, Mexico

Received 4 December 2012; Accepted 30 December 2012

Academic Editors: H. Erdogan, N. Grammalidis, N. D. A. Mascarenhas, and W. L. Woo

Copyright © 2013 Ricardo Omar Chávez et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


This paper introduces a multimodal approach for reranking of image retrieval results based on relevance feedback. We consider the problem of reordering the ranked list of images returned by an image retrieval system, in such a way that relevant images to a query are moved to the first positions of the list. We propose a Markov random field (MRF) model that aims at classifying the images in the initial retrieval-result list as relevant or irrelevant; the output of the MRF is used to generate a new list of ranked images. The MRF takes into account (1) the rank information provided by the initial retrieval system, (2) similarities among images in the list, and (3) relevance feedback information. Hence, the problem of image reranking is reduced to that of minimizing an energy function that represents a trade-off between image relevance and interimage similarity. The proposed MRF is a multimodal as it can take advantage of both visual and textual information by which images are described with. We report experimental results in the IAPR TC12 collection using visual and textual features to represent images. Experimental results show that our method is able to improve the ranking provided by the base retrieval system. Also, the multimodal MRF outperforms unimodal (i.e., either text-based or image-based) MRFs that we have developed in previous work. Furthermore, the proposed MRF outperforms baseline multimodal methods that combine information from unimodal MRFs.