Abstract
Eye tracking information can be used to assign given tags to image regions in order to describe the depicted scene in more details. We introduce and compare two novel eye-tracking-based measures for conducting such assignments: The segmentation measure uses automatically computed image segments and selects the one segment the user fixates for the longest time. The heat map measure is based on traditional gaze heat maps and sums up the users’ fixation durations per pixel. Both measures are applied on gaze data obtained for a set of social media images, which have manually labeled objects as ground truth. We have determined a maximum average precision of 65% at which the segmentation measure points to the correct region in the image. The best coverage of the segments is obtained for the segmentation measure with a F-measure of 35%. Overall, both newly introduced gaze-based measures deliver better results than baseline measures that selects a segment based on the golden ratio of photography or the center position in the image. The eye-tracking-based segmentation measure significantly outperforms the baselines for precision and F-measure.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Tobii studio 2.x - user manual (2010), http://www.tobii.com
Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE TPAMI 33(5), 898–916 (2011)
Bartelma, J.M.: Flycatcher: Fusion of gaze with hierarchical image segmentation for robust object detection. PhD thesis, Massachusetts Institute of Technology (2004)
Belle, W.V., Laeng, B., Brennen, T., et al.: Anchoring gaze when categorizing faces sex: Evidence from eye-tracking data. Vision Research 49(23), 2870–2880 (2009)
Bojko, A.: Informative or misleading? heatmaps deconstructed. In: Human-Computer Interaction. New Trends, pp. 30–39 (2009)
Essig, K.: Vision-Based Image Retrieval (VBIR)-A New Approach for Natural and Intuitive Image Retrieval. PhD thesis (2008)
Freeman, M.: The Photographer’s Eye: Composition and Design for Better Digital Photos. Focal Press (2007)
Kim, D.H., Yu, S.H.: A new region filtering and region weighting approach to relevance feedback in content-based image retrieval. Journal of Systems and Software 81(9), 1525–1538 (2008)
Klami, A.: Inferring task-relevant image regions from gaze data. In: Workshop on Machine Learning for Signal Processing. IEEE (2010)
Ramanathan, S., Katti, H., Huang, R., Chua, T., Kankanhalli, M.: Automated localization of affective objects and actions in images via caption text-cum-eye gaze analysis. In: Multimedia. ACM, New York (2009)
Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. J. of Comp. Vision 77(1), 157–173 (2008)
San Agustin, J., Skovsgaard, H., Hansen, J.P., Hansen, D.W.: Low-cost gaze interaction: ready to deliver the promises. In: CHI, pp. 4453–4458. ACM (2009)
Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., Cohen, M.: Gaze-based interaction for semi-automatic photo cropping. In: CHI, p. 780. ACM (2006)
Walber, T., Scherp, A., Staab, S.: Identifying Objects in Images from Analyzing the Users’ Gaze Movements for Provided Tags. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 138–148. Springer, Heidelberg (2012)
Yarbus, A.L.: Eye movements and vision. Plenum (1967)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Walber, T., Scherp, A., Staab, S. (2013). Can You See It? Two Novel Eye-Tracking-Based Measures for Assigning Tags to Image Regions. In: Li, S., et al. Advances in Multimedia Modeling. MMM 2013. Lecture Notes in Computer Science, vol 7732. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35725-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-35725-1_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35724-4
Online ISBN: 978-3-642-35725-1
eBook Packages: Computer ScienceComputer Science (R0)