research-article

An Interactive Image Retrieval Approach to Searching for Images on Social Media

Authors:

Manali Gaikwad,

Orland HoeberAuthors Info & Claims

CHIIR '19: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval

Pages 173 - 181

https://doi.org/10.1145/3295750.3298930

Published: 08 March 2019 Publication History

Abstract

Searching for images posted within social media services such as Twitter relies on matching textual queries to the contents of the posts that include the images. Unfortunately, social media posts may not always provide accurate or meaningful descriptions of the contents of the embedded images, making searching for images a challenging task. In this research, we augment the textual contents of the posts with new information extracted from the images using image processing and deep learning methods, and provide a visual interface to enable interactive image retrieval. A user study was conducted with 28 participants to collect evidence on how our approach was used in relation to Vakkari's three-stage model of information seeking. We also analyzed participants' perceptions of usefulness, ease of use, and satisfaction in comparison to a common grid-based image search interface. The results from this study highlight the value of providing visual and interactive features to enable searchers to discover images from social media sources.

References

[1]

Paul André, Edward Cutrell, Desney S. Tan, and Greg Smith. 2009. Designing Novel Image Search Interfaces by Understanding Unique Characteristics and Usage. In Proceedings of the IFIP TC 13 International Conference on Human-Computer Interaction (INTERACT). Springer, Berlin, Heidelberg, 340--353.

Digital Library

[2]

Nicholas J. Belkin. 2016. People, interacting with information. ACM SIGIR Forum, Vol. 49, 2 (2016), 13--27.

Digital Library

[3]

Matthew Brehmer, Bongshin Lee, Benjamin Bach, Nathalie Henry Riche, and Tamara Munzner. 2017. Timelines revisited: A design space and considerations for expressive storytelling. IEEE Transactions on Visualization and Computer Graphics, Vol. 29, 9 (2017), 2151--2164.

[4]

Nilgün Camgöz, Cengiz Yener, and Dilek Güvencc. 2004. Effects of hue, saturation, and brightness: Part 2: Attention. Color Research & Application, Vol. 29, 1 (2004), 20--28.

[5]

Ed H. Chi, Peter Pirolli, Kim Chen, and James Pitkow. 2001. Using information scent to model user information needs and actions and the web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, USA, 490--497.

Digital Library

[6]

M. Cole, J. Liu, N. Belkin, R. Bierig, J. Gwizdka, C. Liu, J. Zhang, and X. Zhang. 2009. Usefulness as the Criterion for Evaluation of Interactive Information Retrieval. In Proceedings of the Workshop on Human-Computer Interaction and Information Retrieval. 1--4.

[7]

Orland Hoeber, Larena Hoeber, Maha El Meseery, Kenneth Odoh, and Radhika Gopi. 2016. Visual Twitter analytics (Vista): Temporally changing sentiment and the discovery of emergent themes within sport event tweets. Online Information Review, Vol. 40, 1 (2016), 25--41.

[8]

Enamul Hoque, Orland Hoeber, and Minglun Gong. 2013. CIDER: Concept-based image diversification, exploration, and retrieval. Information Processing & Management, Vol. 49, 5 (2013), 1122--1138.

Digital Library

[9]

Leo M. Hurvich and Dorothea Jameson. 1957. An opponent-process theory of color vision. Psychological Review, Vol. 64, 6, Pt.1 (1957), 384--404.

[10]

Yushi Jing, Henry Rowley, Jingbin Wang, David Tsai, Chuck Rosenberg, and Michele Covell. 2012. Google image swirl: A large-scale content-based image visualization system. In Proceedings of the International Conference on World Wide Web. 539--540.

Digital Library

[11]

Christoph Kofler, Martha Larson, and Alan Hanjalic. 2016. User Intent in Multimedia Search: A Survey of the State of the Art and Future Challenges. Comput. Surveys, Vol. 49, 2 (2016), 36:1--36:37.

Digital Library

[12]

Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon. 2010. What is Twitter, a social network or a news media?. In Proceedings of the International Conference on World Wide Web. ACM, New York, USA, 519--600.

Digital Library

[13]

Bjorn Lantz. 2013. Equidistance of Likert-type scales and validation of inferential methods using experiments and simulations. The Electronic Journal of Business Research Methods, Vol. 11, 1 (2013), 16--28.

[14]

Mathias Lux, Michael Riegler, Pål Halvorsen, Konstantin Pogorelov, and Nektarios Anagnostopoulos. 2016. LIRE: Open source visual information retrieval. In Proceedings of the International Conference on Multimedia Systems. ACM, New York, USA, 30:1--30:4.

Digital Library

[15]

Lokman I. Meho and Helen R. Tibbo. 2003. Modeling the Information-Seeking Behavior of Social Scientists: Ellis's Study Revisited. Journal of the American Society for Information Science and Technology, Vol. 54, 6 (2003), 570--587.

Digital Library

[16]

Vicki L. O'Day and Robin Jeffries. 1993. Orienteering in an information landscape: How information seekers get from here to there. In Proceedings of the INTERCHI Conference on Human Factors in Computing Systems. ACM, New York, USA, 438--445.

Digital Library

[17]

Peter Pirolli and Stuart Card. 1999. Information foraging. Psychological Review, Vol. 106, 4 (1999), 643--675.

[18]

Peter Pirolli and Daniel M. Russell. 2011. Introduction to this special issue on sensemaking. Human-Computer Interaction, Vol. 26, 1--2 (2011), 1--8.

[19]

Grant Strong, Orland Hoeber, and Minglun Gong. 2010. Visual image browsing and exploration (Vibe): User evaluations of image search tasks. In International Conference on Active Media Technology. Springer, Berlin, Heidelberg, 424--435.

Digital Library

[20]

Pertti Vakkari. 2003. Task-based information searching. Annual Review of Information Science and Technology, Vol. 37, 1 (2003), 413--464.

[21]

Viswanath Venkatesh and Fred D. Davis. 2000. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, Vol. 46, 2 (2000), 186--204.

Digital Library

[22]

Matthew O. Ward, Georges Grinstein, and Daniel Keim. 2015. Interactive Data Visualization: Foundations, Techniques, and Applications 2nd ed.). A. K. Peters, Ltd., Natick, MA, USA.

Digital Library

[23]

Ryen W. White and Resa A. Roth. 2009. Exploratory Search: Beyond the Query-Response Paradigm.Morgan & Claypool Publisher, San Rafael, CA.

Cited By

Sharma DSingh BAgarwal SGarg LKim CJung K(2023)A Survey of Detection and Mitigation for Fake Images on Social Media PlatformsApplied Sciences10.3390/app13191098013:19(10980)Online publication date: 5-Oct-2023
https://doi.org/10.3390/app131910980
de Jesus Oliveira VRottermanner GGrößbacher SBoucher MJudmaier P(2022)Requirements and Concepts for Interactive Media Retrieval User InterfacesNordic Human-Computer Interaction Conference10.1145/3546155.3546701(1-10)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546701

Index Terms

An Interactive Image Retrieval Approach to Searching for Images on Social Media
1. Human-centered computing
  1. Visualization
    1. Visualization techniques
2. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Image search
    2. Users and interactive retrieval
      1. Search interfaces

Recommendations

On the learning of image social relevance from heterogeneous social network

Social media has been a convenient platform for users to maintain and share messages in different media types (e.g., image and video). Besides the sheer volume of data, various user social behaviors make the data socially related rather than existing ...
Reachability based Ranking in Interactive Image Retrieval
SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

In some interactive image retrieval systems, users can select images from image search results and click to view their similar or related images until they reach the targets. Existing image ranking options are based on relevance, update time, ...
Lire: lucene image retrieval: an extensible java CBIR library
MM '08: Proceedings of the 16th ACM international conference on Multimedia

LIRe (Lucene Image Retrieval) is a light weight open source Java library for content based image retrieval. It provides common and state of the art global image features and offers means for indexing and retrieval. Due to the fact that it is based on a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIIR '19: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval

March 2019

463 pages

ISBN:9781450360258

DOI:10.1145/3295750

General Chairs:
Leif Azzopardi
University of Strathclyde, UK
,
Martin Halvey
University of Strathclyde, UK
,
Ian Ruthven
University of Strathclyde, UK
,
Program Chairs:
Hideo Joho
University of Tsukuba, Japan
,
Vanessa Murdock
Amazon, USA
,
Pernilla Qvarfordt
pxq Labs, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Student Paper

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Sciences and Engineering Research Council of Canada (NSERC)

Conference

CHIIR '19

Sponsor:

CHIIR '19: Conference on Human Information Interaction and Retrieval

March 10 - 14, 2019

Glasgow, Scotland UK

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
453
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sharma DSingh BAgarwal SGarg LKim CJung K(2023)A Survey of Detection and Mitigation for Fake Images on Social Media PlatformsApplied Sciences10.3390/app13191098013:19(10980)Online publication date: 5-Oct-2023
https://doi.org/10.3390/app131910980
de Jesus Oliveira VRottermanner GGrößbacher SBoucher MJudmaier P(2022)Requirements and Concepts for Interactive Media Retrieval User InterfacesNordic Human-Computer Interaction Conference10.1145/3546155.3546701(1-10)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546701

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten