Abstract
Searching online information resources using mobile devices is affected by small screens which can display only a fraction of ranked search results. In this paper we investigate whether the search effort can be reduced by means of a simple user feedback: for a screenful of search results the user is encouraged to indicate a single most relevant document. In our approach we exploit the fact that, for small display sizes and limited user actions, we can construct a user decision tree representing all possible outcomes of the user interaction with the system. Examining the trees we can compute an upper limit on relevance feedback performance. In this study we consider three standard feedback algorithms: Rocchio, Robertson/Sparck-Jones (RSJ) and a Bayesian algorithm. We evaluate them in conjunction with two strategies for presenting search results: a document ranking that attempts to maximize information gain from the user’s choices and the top-D ranked documents. Experimental results indicate that for RSJ feedback which involves an explicit feature selection policy, the greedy top-D display is more appropriate. For the other two algorithms, the exploratory display that maximizes information gain produces better results. We conducted a user study to compare the performance of the relevance feedback methods with real users and compare the results with the findings from the tree analysis. This comparison between the simulations and real user behaviour indicates that the Bayesian algorithm, coupled with the sampled display, is the most effective.
Article PDF
Similar content being viewed by others
References
Aalbersberg, I. J. (1992). Incremental relevance feedback. In: Proceedings of the Fifteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 11–22). Copenhagen.
Buyukkokten, O., Garcia-Molina, H., Paepcke, A., &Winograd, T. (2000). Power browser: Efficient web browsing for PDAs. In: Proceedings of the ACM Conference on Computers and Human Interaction, (CHI’00).
Buyukkokten, O., Garcia-Molina, H., &Paepcke, A. (2001). Seeing the whole in parts: Text Summarization for Web Browsing on Handheld Devices. In: Proceedings of the Tenth International World Wide Web Conference (WWW).
Campbell, I., &van Rijsbergen, C. J. (1996). The ostensive model of developing information needs. In: P., Ingwersen, &N.O., Pors (Eds.), Information science: Integration in perspective, Proceedings of CoLIS 2. (pp. 251–268).
Chen, Y., Ma, W.-Y., &Zhang, H.-J. (2003). Detecting web page structure for adaptive viewing on small form factor devices. In: Proceedings of the Twelfth World Wide Web Conference. Budapest.
Cox, I. J., Miller, M. L., Minka, T. P., Papathomas, T. V., &Yianilos, P. N. (2000). The Bayesian image retrieval system, PicHunter: Theory, implementation and psychophysical experiments. IEEE Transactions on Image Processing, 9(1):20–37.
Dunlop, M. D. (1997). The effect of accessing non-matching documents on relevance feedback. ACM Transactions on Information Systems, 15(2):137–153.
Harman, D. (1992). Relevance feedback revisited. In: Proceedings of 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Copenhagen, 1.10.
Jones, M., Marsden, G., Mohd-Nasir, N., Boone, K., &Buchanan, G. (1999). Improving web interaction on small displays. In: Proceedings of the 8th World Wide Web Conference. Toronto, Canada.
Jones, M., &Marsden, G. (1997). From the large screen to the small screen-. Retaining the designer’s design for effective user interaction. In: IEEE Colloquium on Issues for Networked Interpersonal Communicators, 239(3):1–4.
Levene, M., &Loizou, G. (1999). A probabilistic approach to navigation in Hypertext. Information Sciences, 114:165–186.
Magennis, M., &van Rijsbergen, C. J. (1997). The potential and actual effectiveness of interactive query expansion. In: Proceedings of 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Philadelphia.
Harman, (Ed.) (1997). Overview of the fifth text retrieval conference (TREC-5). Gaithersburg, MD: NIST.
Rocchio, J. (1971). Relevance feedback information retrieval. In: Gerard Salton (Ed.), The smart retrieval system—experiments in automatic document processing (pp. 313–323). Prentice-Hall, Englewood Cliffs, N.J.
Robertson, S. E., &Sparck Jones, K. (1976). Relevance weighting of search terms. Journal of the American Society for Information Science, 27, 129–146.
Rodden, K., Milic-Frayling, N., Sommerer, R., &Blackwell, A. (2003). Effective Web Searching on Mobile Devices. In: Proceedings of the HCI Conference, Bath.
Ruvini, J.-D. (2003). Adapting to the user’s internet search strategy. IUI’03, Miami, Florida.
Sellen, A. J., Murphy, R., &Shaw, K. L. (2002). How knowledge workers use the web. In: D., Wixon (Ed.), Proceedings of CHI 2002, ACM (pp. 227–234).
White, R. W., Jose, J. M., van Rijsbergen Cornelis, J., &Ruthven, I. (2004). A simulated study of implicit feedback models. In: Proceedings of ECIR.
Author information
Authors and Affiliations
Corresponding author
Additional information
Extended version of “Evaluating Relevance Feedback Algorithms for Searching on Small Displays, ” Vishwa Vinay, Ingemar J. Cox, Natasa Milic-Frayling, Ken Wood published in the proceedings of ECIR 2005, David E. Losada, Juan M. Fernández-Luna (Eds.), Springer 2005, ISBN 3-540-25295-9
Rights and permissions
About this article
Cite this article
Vinay, V., Cox, I.J., Milic-Frayling, N. et al. Can constrained relevance feedback and display strategies help users retrieve items on mobile devices?. Inf Retrieval 9, 435–453 (2006). https://doi.org/10.1007/s10791-006-6391-7
Received:
Revised:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10791-006-6391-7