James Fogarty
Desney Tan
ABSTRACT
Web image search is difficult in part because a handful of keywords are generally insufficient for characterizing the visual properties of an image. Popular engines have begun to provide tags based on simple characteristics of images (such as tags for black and white images or images that contain a face), but such approaches are limited by the fact that it is unclear what tags end users want to be able to use in examining Web image search results. This paper presents CueFlik, a Web image search application that allows end users to quickly create their own rules for re ranking images based on their visual characteristics. End users can then re rank any future Web image search results according to their rule. In an experiment we present in this paper, end users quickly create effective rules for such concepts as "product photos", "portraits of people", and "clipart". When asked to conceive of and create their own rules, participants create such rules as "sports action shot" with images from queries for "basketball" and "football". CueFlik represents both a promising new approach to Web image search and an important study in end user interactive machine learning.
- Bederson, B.B. (2001). PhotoMesa: A Zoomable Image Browser Using Quantum Treemaps and Bubblemaps. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2001), 71--80. Google Scholar
Digital Library
- Cai, D., He, X., Li, Z., Ma, W.-Y. and Wen, J.-R. (2004). Hierarchical Clustering of WWW Image Search Results Using Visual, Textual, and Link Information. Proceedings of the ACM Conference on Multimedia (Multimedia 2004), 952--959. Google ScholarDigital Library
- Fails, J.A. and Olsen, D.R. (2003). Interactive Machine Learning. Proceedings of the International Conference on Intelligent User Interfaces (IUI 2003), 39--45. Google ScholarDigital Library
- Fass, A.M., Bier, E.A. and Adar, E. (2000). PicturePiper: Using a Re-Configurable Pipeline to Find Images on the Web. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2000), 51--62. Google ScholarDigital Library
- Gajos, K. and Weld, D.S. (2004). SUPPLE: Automatically Generating User Interfaces. Proceedings of the International Conference on Intelligent User Interfaces (IUI 2004), 93--100. Google ScholarDigital Library
- Gajos, K. and Weld, D.S. (2005). Preference Elicitation for Interface Optimization. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2005), 173--182. Google ScholarDigital Library
- Globerson, A. and Roweis, S. (2005). Metric Learning by Collapsing Classes. Proceedings of the Conference on Neural Information Processing Systems (NIPS 2005), 451--458.Google Scholar
- Gorkani, M.M. and Picard, R.W. (1994). Texture Orientation for Sorting Photos 'At a Glance'. Proceedings of the International Conference on Pattern Recognition (ICPR 1994), 459--464.Google ScholarCross Ref
- Hartmann, B., Abdulla, L., Mittal, M. and Klemmer, S.R. (2007). Authoring Sensor-Based Interactions by Demonstration with Direct Manipulation and Pattern Recognition. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2007), 145--154. Google ScholarDigital Library
- Hastie, T., Tibshirani, R. and Friedman, J. (2001). The Elements of Statistical Learning. Springer.Google ScholarCross Ref
- Kristjannson, T., Culotta, A., Viola, P. and McCallum, A. (2004). Interactive Information Extraction with Constrained Conditional Random Fields. Proceedings of the National Conference on Artificial Intelligence (AAAI 2004), 412--418. Google ScholarDigital Library
- Li, F.-F. and Perona, P. (2005). A Bayesian Hierarchical Model for Learning Natural Scene Categories. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005), 524--531. Google ScholarDigital Library
- Niblack, C.W., Barber, R., Equitz, W., Flickner, M.D., Glasman, E.H., Petkovic, D., Yanker, P., Faloutsos, C. and Taubin, G. (1993). QBIC Project: Querying Images by Content, Using Color, Texture, and Shape. Proceedings of the Conference on Storage and Retrieval for Image and Video Databases 173--187.Google ScholarCross Ref
- Nistér, D. and Stewénius, H. (2006). Scalable Recognition with a Vocabulary Tree. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2006), 2161--2168. Google ScholarDigital Library
- Nocedal, J. and Wright, S.J. (2006). Numerical Optimization. Science Press.Google Scholar
- Platt, J.C., Czerwinski, M. and Field, B.A. (2003). PhotoTOC: Automatic Clustering for Browsing Personal Photographs. Proceedings of the IEEE Pacific Rim Conference on Multimedia 6--10.Google ScholarCross Ref
- Schettini, R., Ciocca, G., Valsasna, A., Brambilla, C. and De Ponti, M. (2002). A Hierarchical Classification Strategy for Digital Documents. Pattern Recognition 35(8). 1759--1769.Google ScholarCross Ref
- Shilman, M., Tan, D.S. and Simard, P. (2006). CueTIP: A Mixed-Initiative Interface for Correcting Handwriting Errors. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2006), 323--332. Google ScholarDigital Library
- Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A. and Jain, R. (2000). Content-Based Image Retrieval at the End of the Early Years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12). 1349--1380. Google ScholarDigital Library
- Smith, J.R. and Chang, S.-F. (1997). VisualSEEK: A Fully Automated Content-Based Image Query System. Proceedings of the ACM Conference on Multimedia (Multimedia 1997), 87--98. Google ScholarDigital Library
- Vailaya, A., Figueiredo, M., Jain, A. and Zhang, H.J. (1999). Content-Based Hierarchical Classification of Vacation Images. Porceedings of the IEEE International Conference on Multimedia Computing and Systems (ICMCS 1999), 518--523. Google ScholarDigital Library
- Wang, S., Jing, F., He, J., Du, Q. and Zhang, L. (2007). IGroup: Presenting Web Image Search Results in Semantic Clusters. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2007), 587--596. Google ScholarDigital Library
- Winder, S.A. and Brown, M. (2007). Learning Local Image Descriptors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2007), 1--8.Google ScholarCross Ref
- Yee, K.-P., Swearingen, K., Li, K. and Hearst, M. (2003). Faceted Metadata for Image Search and Browsing. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2003), 401--408. Google ScholarDigital Library
Index Terms
- CueFlik: interactive concept learning in image search
Recommendations
How Well do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalComparing to general Web search engines, image search engines present search results differently, with two-dimensional visual image panel for users to scroll and browse quickly. These differences in result presentation can significantly impact the way ...
Constructing an Interaction Behavior Model for Web Image Search
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalUser interaction behavior is a valuable source of implicit relevance feedback. In Web image search a different type of search result presentation is used than in general Web search, which leads to different interaction mechanisms and user behavior. For ...
Leveraging User Interaction Signals for Web Image Search
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information RetrievalUser interfaces for web image search engine results differ significantly from interfaces for traditional (text) web search results, supporting a richer interaction. In particular, users can see an enlarged image preview by hovering over a result image, ...
Comments