skip to main content
10.1145/3077136.3080799acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Investigating Examination Behavior of Image Search Users

Published: 07 August 2017 Publication History

Abstract

Image search engines show results differently from general Web search engines in three key ways: (1) most Web-based image search engines adopt the two-dimensional result placement instead of the linear result list; (2) image searches show snapshots instead of snippets (query-dependent abstracts of landing pages) on search engine result pages (SERPs); and (3) pagination is usually not (explicitly) supported on image search SERPs, and users can view results without having to click on the "next page'' button. Compared with the extensive study of user behavior in general Web search scenarios, there exists no thorough investigation how the different interaction mechanism of image search engines affects users' examination behavior. To shed light on this research question, we conducted an eye-tracking study to investigate users' examination behavior in image searches. We focus on the impacts of factors in examination including position, visual saliency, edge density, the existence of textual information, and human faces in result images. Three interesting findings indicate users' behavior biases: (1) instead of the traditional "Golden Triangle'' phenomena in the user examination patterns of general Web search, we observe a middle-position bias, (2) besides the position factor, the content of image results (e.g., visual saliency) affects examination behavior, and (3) some popular behavior assumptions in general Web search (e.g., examination hypothesis) do not hold in image search scenarios. We predict users' examination behavior with different impact factors. Results show that combining position and visual content features can improve prediction in image searches.

References

[1]
Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving web search ranking by incorporating user behavior information Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 19--26.
[2]
Paul André, Edward Cutrell, Desney S. Tan, and Greg Smith 2009. Designing novel image search interfaces by understanding unique characteristics and usage IFIP Conference on Human-Computer Interaction. Springer, 340--353.
[3]
Ioannis Arapakis and Luis A. Leiva 2016. Predicting user engagement with direct displays using mouse cursor information Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 599--608.
[4]
Roland J. Baddeley and Benjamin W. Tatler 2006. High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis. Vision research, Vol. 46, 18 (2006), 2824--2833.
[5]
Georg Buscher, Edward Cutrell, and Meredith Ringel Morris. 2009. What do you see when you're surfing? Using eye tracking to predict salient regions of web pages. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 21--30.
[6]
John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679--698.
[7]
Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, and Hang Li 2008. Context-aware query suggestion by mining click-through and session data Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 875--883.
[8]
Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey 2008. An experimental comparison of click position-bias models Proceedings of the 2008 International Conference on Web Search and Data Mining. ACM, 87--94.
[9]
Edward Cutrell and Zhiwei Guan 2007. What are you looking for? an eye-tracking study of information usage in web search Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 407--416.
[10]
Georges E. Dupret and Benjamin Piwowarski 2008. A user browsing model to predict search engine click data from past observations. Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 331--338.
[11]
Abby Goodrum and Amanda Spink 1999. Visual Information Seeking: A Study of Image Queries on the World Wide Web. Proceedings of the ASIS Annual Meeting, Vol. Vol. 36. ERIC, 665--74.
[12]
Jonathan Harel, Christof Koch, and Pietro Perona. 2006. Graph-based visual saliency. In Advances in neural information processing systems. 545--552.
[13]
Gord Hotchkiss, Steve Alston, and Greg Edwards. 2005. Eye tracking study. Research white paper, Enquiro Search Solutions Inc (2005).
[14]
Xian-Sheng Hua, Linjun Yang, Jingdong Wang, Jing Wang, Ming Ye, Kuansan Wang, Yong Rui, and Jin Li. 2013. Clickage: towards bridging semantic and intent gaps via mining click logs of search engines Proceedings of the 21st ACM international conference on Multimedia. ACM, 243--252.
[15]
Vidit Jain and Manik Varma 2011. Learning to re-rank: query-dependent image re-ranking using click data Proceedings of the 20th international conference on World wide web. ACM, 277--286.
[16]
Jyun-Yu Jiang, Yen-Yu Ke, Pao-Yu Chien, and Pu-Jen Cheng. 2014. Learning user reformulation behavior for query auto-completion Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, 445--454.
[17]
Yanen Li, Anlei Dong, Hongning Wang, Hongbo Deng, Yi Chang, and ChengXiang Zhai. 2014. A two-dimensional click model for query auto-completion. Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, 455--464.
[18]
Yiqun Liu, Zeyang Liu, Ke Zhou, Meng Wang, Huanbo Luan, Chao Wang, Min Zhang, and Shaoping Ma. 2016. Predicting Search User Examination with Visual Saliency Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 619--628.
[19]
Yiqun Liu, Chao Wang, Ke Zhou, Jianyun Nie, Min Zhang, and Shaoping Ma. 2014. From skimming to reading: A two-stage examination model for web search Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. ACM, 849--858.
[20]
Neil O'Hare, Paloma de Juan, Rossano Schifanella, Yunlong He, Dawei Yin, and Yi Chang. 2016. Leveraging user interaction signals for web image search Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 559--568.
[21]
Bing Pan, Helene A. Hembrooke, Geri K. Gay, Laura A. Granka, Matthew K. Feusner, and Jill K. Newman. 2004. The determinants of web page viewing behavior: an eye-tracking study Proceedings of the 2004 symposium on Eye tracking research & applications. ACM, 147--154.
[22]
Yingwei Pan, Ting Yao, Tao Mei, Houqiang Li, Chong-Wah Ngo, and Yong Rui. 2014. Click-through-based cross-view learning for image search Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, 717--726.
[23]
Jaimie Y. Park, Neil O'Hare, Rossano Schifanella, Alejandro Jaimes, and Chin-Wan Chung. 2015. A large-scale study of user image search behavior on the web Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 985--994.
[24]
Hsiao-Tieh Pu. 2005. A comparative analysis of web image and textual queries. Online Information Review Vol. 29, 5 (2005), 457--467.
[25]
Stephen W. Raudenbush and Anthony S. Bryk 2002. Hierarchical linear models: Applications and data analysis methods. Vol. Vol. 1.
[26]
Sara Shatford. 1986. Analyzing the subject of a picture: a theoretical approach. Cataloging & classification quarterly Vol. 6, 3 (1986), 39--62.
[27]
Arnold W. M. Smeulders, Marcel Worring, Simone Santini, Amarnath Gupta, and Ramesh Jain. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on pattern analysis and machine intelligence, Vol. 22, 12 (2000), 1349--1380.
[28]
Gavin Smith and Helen Ashman 2009. Evaluating implicit judgements from image search interactions. (2009).
[29]
Benjamin W. Tatler and Benjamin T. Vincent 2009. The prominence of behavioural biases in eye guidance. Visual Cognition, Vol. 17, 6-7 (2009), 1029--1054.
[30]
Dian Tjondronegoro, Amanda Spink, and Bernard J. Jansen. 2009. A study and comparison of multimedia Web searching: 1997--2006. Journal of the American Society for Information Science and Technology, Vol. 60, 9 (2009), 1756--1768.
[31]
Geoffrey Underwood and Tom Foulsham 2006. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. The Quarterly journal of experimental psychology, Vol. 59, 11 (2006), 1931--1949.
[32]
Chao Wang, Yiqun Liu, Meng Wang, Ke Zhou, Jian-yun Nie, and Shaoping Ma. 2015. Incorporating non-sequential behavior into click models Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 283--292.
[33]
Kuansan Wang, Nikolas Gloy, and Xiaolong Li 2010. Inferring search behaviors using partially observable Markov (POM) model Proceedings of the third ACM international conference on Web search and data mining. ACM, 211--220.
[34]
Yue Wang, Dawei Yin, Luo Jie, Pengyuan Wang, Makoto Yamada, Yi Chang, and Qiaozhu Mei 2016. Beyond ranking: Optimizing whole-page presentation Proceedings of the Ninth ACM International Conference on Web Search and Data Mining. ACM, 103--112.
[35]
Wei Wu, Hang Li, and Jun Xu 2013. Learning query and document similarities from click-through bipartite graph with metadata Proceedings of the sixth ACM international conference on Web search and data mining. ACM, 687--696.
[36]
Danqing Xu, Yiqun Liu, Min Zhang, Shaoping Ma, and Liyun Ru 2012. Incorporating revisiting behaviors into click models Proceedings of the fifth ACM international conference on Web search and data mining. ACM, 303--312.
[37]
Yongdong Zhang, Xiaopeng Yang, and Tao Mei 2014. Image search reranking with query-dependent click-based relevance feedback. IEEE transactions on image processing Vol. 23, 10 (2014), 4448--4459.

Cited By

View all
  • (2024)Known-Item Search in Video: An Eye Tracking-Based StudyProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658119(311-319)Online publication date: 30-May-2024
  • (2024)Perceptions in Pixels: Analyzing Perceived Gender and Skin Tone in Real-world Image Search ResultsProceedings of the ACM Web Conference 202410.1145/3589334.3645666(1249-1259)Online publication date: 13-May-2024
  • (2024)Towards Optimizing Ranking in Grid-Layout for Provider-Side FairnessAdvances in Information Retrieval10.1007/978-3-031-56069-9_7(90-105)Online publication date: 23-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval
August 2017
1476 pages
ISBN:9781450350228
DOI:10.1145/3077136
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. examination behavior
  2. eye-tracking
  3. image search

Qualifiers

  • Research-article

Funding Sources

Conference

SIGIR '17
Sponsor:

Acceptance Rates

SIGIR '17 Paper Acceptance Rate 78 of 362 submissions, 22%;
Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)21
  • Downloads (Last 6 weeks)4
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Known-Item Search in Video: An Eye Tracking-Based StudyProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658119(311-319)Online publication date: 30-May-2024
  • (2024)Perceptions in Pixels: Analyzing Perceived Gender and Skin Tone in Real-world Image Search ResultsProceedings of the ACM Web Conference 202410.1145/3589334.3645666(1249-1259)Online publication date: 13-May-2024
  • (2024)Towards Optimizing Ranking in Grid-Layout for Provider-Side FairnessAdvances in Information Retrieval10.1007/978-3-031-56069-9_7(90-105)Online publication date: 23-Mar-2024
  • (2023)Reinforcement Re-ranking with 2D Grid-based Recommendation PanelsProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3624918.3625311(282-287)Online publication date: 26-Nov-2023
  • (2023)Asking Clarifying Questions: To benefit or to disturb users in Web search?Information Processing & Management10.1016/j.ipm.2022.10317660:2(103176)Online publication date: Mar-2023
  • (2023)Contrasting Neural Click Models and Pointwise IPS RankersAdvances in Information Retrieval10.1007/978-3-031-28244-7_26(409-425)Online publication date: 17-Mar-2023
  • (2022)Representativeness and face-ism: Gender bias in image searchNew Media & Society10.1177/1461444822110069926:6(3541-3567)Online publication date: 19-Jun-2022
  • (2022)Reflection on future directions: a systematic review of reported limitations and solutions in interactive information retrieval user studiesAslib Journal of Information Management10.1108/AJIM-05-2022-0253Online publication date: 19-Dec-2022
  • (2022)From linear to non-linear: investigating the effects of right-rail results on complex SERPsAdvances in Computational Intelligence10.1007/s43674-021-00028-22:1Online publication date: 10-Jan-2022
  • (2021)Cross-Positional Attention for Debiasing ClicksProceedings of the Web Conference 202110.1145/3442381.3450098(788-797)Online publication date: 19-Apr-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media