Qingyong Li
ABSTRACT
Relevance feedback is one of approach to improve the performance of content-based image retrieval system, and implicit feedback approaches, which gather users' feedback by biometric devices (e.g. eye tracker), are extensively investigated in recent years. This paper proposes a novel image retrieval system with eye tracking (IRSET). IRSET is composed of three modules: image retrieval module based on standard bag-of-words, eye tracking module to obtain a user's fixation data and to infer feedback information, and query expansion module that fuses the user's feedback and the input query to form a richer latent query. The implicit feedback of IRSET is implemented online and real-time, which makes IRSET remarkably distinguish from other systems with implicit feedback. We conduct experiments on the dataset of Oxford building for ten participants. The experimental results demonstrate that IRSET is an attractive interface to image retrieval and improves the retrieval performance.
- Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys (CSUR), 40(2):5, 2008. Google Scholar
Digital Library
- David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91--110, 2004. Google ScholarDigital Library
- Josef Sivic and Andrew Zisserman. Video google: A text retrieval approach to object matching in videos. In IEEE International Conference on Computer Vision, pages 1470--1477. IEEE, 2003. Google ScholarDigital Library
- Yong Rui, Thomas S Huang, Michael Ortega, and Sharad Mehrotra. Relevance feedback: a power tool for interactive content-based image retrieval. Circuits and Systems for Video Technology, IEEE Transactions on, 8(5):644--655, 1998. Google ScholarDigital Library
- Xiang Sean Zhou and Thomas S Huang. Relevance feedback in image retrieval: A comprehensive review. Multimedia systems, 8(6):536--544, 2003.Google ScholarCross Ref
- G Papadopoulos, K Apostolakis, and Petros Daras. Gaze-based relevance feedback for realizing region-based image retrieval. IEEE Transactions on Multimedia, 16(2):440--454, 2014.Google ScholarDigital Library
- S Navid Hajimirza, Michael J Proulx, and Ebroul Izquierdo. Reading users' minds from their eyes: A method for implicit image annotation. IEEE Transactions on Multimedia, 14(3):805--815, 2012.Google ScholarDigital Library
- Yun Zhang, Hong Fu, Zhen Liang, Zheru Chi, and Dagan Feng. Eye movement as an interaction mechanism for relevance feedback in a content-based image retrieval system. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, pages 37--40. ACM, 2010. Google ScholarDigital Library
- Ming Qian, Mario Aguilar, Karen N Zachery, Claudio Privitera, Stanley Klein, Thom Carney, and Loren W Nolte. Decision-level fusion of eeg and pupil features for single-trial visual detection analysis. IEEE Transactions on Biomedical Engineering, 56(7):1929--1937, 2009.Google ScholarCross Ref
- Arto Klami, Craig Saunders, Teófilo E de Campos, and Samuel Kaski. Can relevance of images be inferred from eye movements? In ACM international conference on Multimedia information retrieval, pages 134--140. ACM, 2008. Google ScholarDigital Library
- Stefanos Vrochidis, Ioannis Patras, and Ioannis Kompatsiaris. An eye-tracking-based approach to facilitate interactive video search. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval, page 43. ACM, 2011. Google ScholarDigital Library
- Keith Rayner. Eye movements in reading and information processing. Psychological bulletin, 85(3):618, 1978.Google ScholarCross Ref
- Laura A Granka, Thorsten Joachims, and Geri Gay. Eye-tracking analysis of user behavior in www search. In 27th annual international ACM SIGIR, pages 478--479. ACM, 2004. Google ScholarDigital Library
- Kai Puolamäki, Jarkko Salojärvi, Eerika Savia, Jaana Simola, and Samuel Kaski. Combining eye movements and collaborative filtering for proactive information retrieval. In The 28th annual international ACM SIGIR, pages 146--153. ACM, 2005. Google ScholarDigital Library
- David R Hardoon, John Shawe-Taylor, Antti Ajanki, Kai Puolamäki, and Samuel Kaski. Information retrieval by inferring implicit queries from eye movements. In International Conference on Artificial Intelligence and Statistics, pages 179--186, 2007.Google Scholar
- Georg Buscher, Andreas Dengel, and Ludger van Elst. Query expansion using gaze-based feedback on the subdocument level. In The 31st annual international ACM SIGIR, pages 387--394. ACM, 2008. Google ScholarDigital Library
- Michael J Cole, Jacek Gwizdka, Chang Liu, Nicholas J Belkin, and Xiangmin Zhang. Inferring user knowledge level from eye movement patterns. Information Processing & Management, 49(5):1075--1091, 2013. Google ScholarDigital Library
- Anthony Hughes, Todd Wilkens, Barbara M Wildemuth, and Gary Marchionini. Text or pictures? an eyetracking study of how people view digital video surrogates. In International Conference on Image and Video Retrieval, pages 271--280. Springer, 2003. Google ScholarDigital Library
- Ok Oyekoya and Fred Stentiford. Eye tracking as a new interface for image retrieval. BT Technology Journal, 22(3):161--169, 2004. Google ScholarDigital Library
- Oyewole Oyekoya and Fred Stentiford. Exploring human eye behaviour using a model of visual attention. In 17th International Conference on Pattern Recognition, volume 4, pages 945--948. IEEE, 2004. Google ScholarDigital Library
- László Kozma, Arto Klami, and Samuel Kaski. Gazir: gaze-based zooming interface for image retrieval. In Proceedings of the 2009 international conference on Multimodal interfaces, pages 305--312. ACM, 2009. Google ScholarDigital Library
- David R Hardoon and Kitsuchart Pasupa. Image ranking with implicit feedback from eye movements. In 2010 Symposium on Eye-Tracking Research & Applications, pages 291--298. ACM, 2010. Google ScholarDigital Library
- Zhen Liang, Hong Fu, Yun Zhang, Zheru Chi, and Dagan Feng. Content-based image retrieval using a combination of visual features and eye tracking data. In Symposium on Eye-Tracking Research & Applications, pages 41--44. ACM, 2010. Google ScholarDigital Library
- Alberto Faro, Daniela Giordano, Carmelo Pino, and Concetto Spampinato. Visual attention for implicit relevance feedback in a content based image retrieval. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, pages 73--76. ACM, 2010. Google ScholarDigital Library
- James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. Object retrieval with large vocabularies and fast spatial matching. In IEEE Conference on Computer Vision and Pattern Recognition., pages 1--8. IEEE, 2007.Google ScholarCross Ref
Index Terms
- A Novel Image Retrieval System with Real-Time Eye Tracking
Recommendations
Visual attention for implicit relevance feedback in a content based image retrieval
ETRA '10: Proceedings of the 2010 Symposium on Eye-Tracking Research & ApplicationsIn this paper we propose an implicit relevance feedback method with the aim to improve the performance of known Content Based Image Retrieval (CBIR) systems by re-ranking the retrieved images according to users' eye gaze data. This represents a new ...
An implicit relevance feedback method for CBIR with real-time eye tracking
Relevance feedback is an efficient approach to improve the performance of content-based image retrieval systems, and implicit relevance feedback approaches, which gather users' feedback by biometric devices (e.g. eye tracker), have extensively ...
A graph-based relevance feedback mechanism in content-based image retrieval
Content-Based Image Retrieval (CBIR) is an important problem in the domain of digital data management. There is indeed a growing availability of images, but unfortunately the traditional metadata-based search systems are unable to properly exploit their ...
Comments