research-article

The role of attractiveness in web image search

Authors:

Xian-Sheng Hua,

Shipeng LiAuthors Info & Claims

MM '11: Proceedings of the 19th ACM international conference on Multimedia

Pages 63 - 72

https://doi.org/10.1145/2072298.2072308

Published: 28 November 2011 Publication History

Abstract

Existing web image search engines are mainly designed to optimize topical relevance. However, according to our user study, attractiveness is becoming a more and more important factor for web image search engines to satisfy users' search intentions. Important as it can be, web image attractiveness from the search users' perspective has not been sufficiently recognized in both the industry and the academia. In this paper, we present a definition of web image attractiveness with three levels according to the end users' feedback, including perceptual quality, aesthetic sensitivity and affective tune. Corresponding to each level of the definition, various visual features are investigated on their applicability to attractiveness estimation of web images. To further deal with the unreliability of visual features induced by the large variations of web images, we propose a contextual approach to integrate the visual features with contextual cues mined from image EXIF information and the associated web pages. We explore the role of attractiveness by applying it to various stages of a web image search engine, including the online ranking and the interactive reranking, as well as the offline index selection. Experimental results on three large-scale web image search datasets demonstrate that the incorporation of attractiveness can bring more satisfaction to 80% of the users for ranking/reranking search results and 30.5% index coverage improvement for index selection, compared to the conventional relevance based approaches.

References

[1]

S. Bhattacahrya, R. Sukthankar, and M. Shah. A framework for photo-quality assessment and enhancement based on visual aesthetics. In SIGMM, 2010.

Digital Library

[2]

R. Datta, D. Joshi, J. Li, and J. Z. Wang. Studying aesthetics in photographic images using a computational approach. In ECCV, 2006.

Digital Library

[3]

B. Geng, L. Yang, C. Xu, and X.-S. Hua. Content-aware ranking for visual search. In CVPR, 2010.

[4]

K.-Q. Huang, Q. Wang, and Z.-Y. Wu. Natural color image enhancement and evaluation algorithm based on human visual system. CVIU, 2006.

Digital Library

[5]

T. Joachims. Optimizing search engines using clickthrough data. In SIGKDD, 2002.

Digital Library

[6]

Y. Ke, X. Tang, and F. Jing. The design of high-level features for photo quality assessment. In CVPR, 2006.

Digital Library

[7]

Y. Luo and X. Tang. Photo and video quality evaluation: Focusing on the subject. In ECCV, 2008.

Digital Library

[8]

Y.-F. Ma, X.-S. Hua, L. Lu, and H.-J. Zhang. A generic framework of user attention model and its application in video summarization. IEEE TMM, 2005.

Digital Library

[9]

J. Machajdik and A. Hanbury. Affective image classification using features inspired by psychology and art theory. In SIGMM, 2010.

Digital Library

[10]

M. Richardson, A. Prakash, and E. Brill. Beyond pagerank: machine learning for static ranking. In WWW, 2006.

Digital Library

[11]

S. Robertson and D. A. Hull. The trec-9 filtering track final report. In TREC, 2000.

[12]

J. San Pedro and S. Siersdorfer. Ranking and classifying attractiveness of photos in folksonomies. In WWW, 2009.

Digital Library

[13]

T. Saracevic. Relevance: A review of the literature and a framework for thinking on the notion in information science. In JASIS, 1976.

[14]

H. Sheikh, A. Bovik, and L. Cormack. No-reference quality assessment using natural scene statistics: Jpeg2000. IEEE TIP, 2005.

Digital Library

[15]

C. G. M. Snoek and A. W. M. Smeulders. Visual-concept search solved? IEEE Computer, 2010.

Digital Library

[16]

X. Tian, L. Yang, J. Wang, Y. Yang, X. Wu, and X.-S. Hua. Bayesian video search reranking. In SIGMM, 2008.

Digital Library

[17]

G. Wang and F. H. Lochovsky. Feature selection with conditional mutual information maximin in text categorization. In CIKM, 2004.

Digital Library

[18]

W. Wei-ning, Y. Ying-lin, and J. Sheng-ming. Image retrieval by emotional semantics: A study of emotional space and feature extraction. In ICSMC, 2006.

[19]

L. Yang and A. Hanjalic. Supervised reranking for web image search. In SIGMM, 2010.

Digital Library

[20]

Y. Yue, R. Patel, and H. Roehrig. Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data. In WWW, 2010.

Digital Library

[21]

Z.-J. Zha, L. Yang, T. Mei, M. Wang, and Z. Wang. Visual query suggestion. In SIGMM, 2009.

Digital Library

[22]

L. Zhang, L. Chen, F. Jing, K. Deng, and W.-Y. Ma. Enjoyphoto: a vertical image search engine for enjoying high-quality photos. In SIGMM, 2006.

Digital Library

Cited By

Tavakoli LTrippas JZamani HScholer FSanderson M(2024)Online and Offline Evaluation in Search ClarificationACM Transactions on Information Systems10.1145/368178643:1(1-30)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681786
Zhu SXie XYe ZAi QLiu Y(2024)Comparing point‐wise and pair‐wise relevance judgment with brain signalsJournal of the Association for Information Science and Technology10.1002/asi.24936Online publication date: 18-Jun-2024
https://doi.org/10.1002/asi.24936
Yu YShi SWang YLian XLiu JLei F(2021)Learning to Predict Page View on College Official Accounts With Quality-Aware FeaturesFrontiers in Neuroscience10.3389/fnins.2021.76639615Online publication date: 28-Oct-2021
https://doi.org/10.3389/fnins.2021.766396
Show More Cited By

Index Terms

The role of attractiveness in web image search
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Leveraging User Interaction Signals for Web Image Search
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

User interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, ...
Constructing an Interaction Behavior Model for Web Image Search
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

User interaction behavior is a valuable source of implicit relevance feedback. In Web image search a different type of search result presentation is used than in general Web search, which leads to different interaction mechanisms and user behavior. For ...
How Well do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

Comparing to general Web search engines, image search engines present search results differently, with two-dimensional visual image panel for users to scroll and browse quickly. These differences in result presentation can significantly impact the way ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '11: Proceedings of the 19th ACM international conference on Multimedia

November 2011

944 pages

ISBN:9781450306164

DOI:10.1145/2072298

General Chairs:
K. Selçuk Candan
Arizona State University, USA
,
Sethuraman Panchanathan
Arizona State University, USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA
,
Program Chairs:
Hari Sundaram
Arizona State University, USA
,
Wu-Chi Feng
Portland State University, USA
,
Nicu Sebe
University of Trento, Italy

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '11

Sponsor:

SIGMM

MM '11: ACM Multimedia Conference

November 28 - December 1, 2011

Arizona, Scottsdale, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
1,071
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tavakoli LTrippas JZamani HScholer FSanderson M(2024)Online and Offline Evaluation in Search ClarificationACM Transactions on Information Systems10.1145/368178643:1(1-30)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3681786
Zhu SXie XYe ZAi QLiu Y(2024)Comparing point‐wise and pair‐wise relevance judgment with brain signalsJournal of the Association for Information Science and Technology10.1002/asi.24936Online publication date: 18-Jun-2024
https://doi.org/10.1002/asi.24936
Yu YShi SWang YLian XLiu JLei F(2021)Learning to Predict Page View on College Official Accounts With Quality-Aware FeaturesFrontiers in Neuroscience10.3389/fnins.2021.76639615Online publication date: 28-Oct-2021
https://doi.org/10.3389/fnins.2021.766396
Ma JCzerniawski TLeite F(2021)An application of metadata-based image retrieval system for facility managementAdvanced Engineering Informatics10.1016/j.aei.2021.10141750:COnline publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1016/j.aei.2021.101417
Cui CLin PNie XJian MYin Y(2020)Social-sensed Image Aesthetics AssessmentACM Transactions on Multimedia Computing, Communications, and Applications10.1145/341484316:3s(1-19)Online publication date: 31-Dec-2020
https://dl.acm.org/doi/10.1145/3414843
Xie XMao JLiu Yde Rijke MChen HZhang MMa SHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Preference-based Evaluation Metrics for Web Image SearchProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401146(369-378)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401146
Zeng HCao ZZhang LBovik A(2020)A Unified Probabilistic Formulation of Image Aesthetic AssessmentIEEE Transactions on Image Processing10.1109/TIP.2019.294177829(1548-1561)Online publication date: 2020
https://doi.org/10.1109/TIP.2019.2941778
Shao YLiu YZhang FZhang MMa S(2019)On Annotation Methodologies for Image Search EvaluationACM Transactions on Information Systems10.1145/330999437:3(1-32)Online publication date: 27-Mar-2019
https://dl.acm.org/doi/10.1145/3309994
Xie XMao JLiu Yde Rijke MShao YYe ZZhang MMa S(2019)Grid-based Evaluation Metrics for Web Image SearchThe World Wide Web Conference10.1145/3308558.3313514(2103-2114)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313514
Xie XCulpepper JMoffat ABennett PLerman K(2019)User Behavior Modeling for Web Image SearchProceedings of the Twelfth ACM International Conference on Web Search and Data Mining10.1145/3289600.3291597(826-827)Online publication date: 30-Jan-2019
https://dl.acm.org/doi/10.1145/3289600.3291597
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten