Ashlee Edwards
ABSTRACT
One goal of using simulated work tasks in interactive information retrieval (IIR) experiments is to create a more relevant and interesting search experience for study participants. However, there is not much guidance about how to identify interesting tasks or how interest in a task impacts search behaviors and experiences, which is the purpose of this study. In this study, we created eight work tasks and asked forty participants to rank these tasks from most interesting to least interesting before they came into the lab for an IIR experiment. During the experiment, we asked participants to conduct searches for the two tasks they ranked as most interesting and the two they ranked as least interesting. Participants completed pre- and post-search questionnaires to characterize their interests in the tasks and their search experiences, including engagement. Participants rated their interest, prior knowledge and search experience, and the relevancy of interesting tasks significantly higher than uninteresting tasks. They also predicted these tasks would be significantly less difficult to complete. Participants reported significantly greater engagement with interesting tasks and they spent longer completing these tasks. However, there were no significant differences in participants' search behaviors including number of queries issued, number of SERPs, or number of documents bookmarked. These results provide evidence that our method of assigning tasks to participants that would interest and engage them, at least cognitively, if not behaviorally, was somewhat successful. This method can be used by others conducting laboratory IIR studies.
- Borlund, P. (2003). The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information Research, 8.Google Scholar
- Borlund, P., Dreier, S., & Byström, K. (2012). What does time spent on searching indicate? Proc. of IIiX, 184--193. Google ScholarDigital Library
- Borlund, P., & Schneider, J. W. (2010). Reconsideration of the simulated work task situation: A context instrument for evaluation of information retrieval interaction. In Proceedings of the Third Symposium on Information Interaction in Context, 155--164. Google ScholarDigital Library
- Byström, K. & Hansen, P. (2005). Conceptual framework for tasks in information studies. JASIST, 56, 1050--1061. Google ScholarDigital Library
- Kelly, D. (2009). Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval, 3(1--2). Google ScholarDigital Library
- Kelly, D., Arguello, J., Edwards, A., & Wu, W. C. (2015). Development and evaluation of search tasks for IIR experiments using a cognitive complexity framework. Proc. of ICTIR, 101--110. Google ScholarDigital Library
- O'Brien, H.L., & Toms, E.G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. JASIST, 59, 938--955. Google ScholarDigital Library
- O'Brien, H.L., & Toms, E.G. (2013). Examining the generalizability of the User Engagement Scale (UES) in exploratory search. IP&M, 49, 1092--1107. Google ScholarDigital Library
- Peters, L. H., O'Connor, E. J., & Rudolf, C. J. (1980). The behavioral and affective consequences of performance- relevant situational variables. Organizational Behavior and Human Performance, 25, 79--96.Google ScholarCross Ref
- Poddar, A., and Ruthven, I. (2010). The emotional impact of search tasks. Proc. of IIiX, 35--44. Google ScholarDigital Library
- Vakkari, P. (2003). Task-based information searching. ARIST, 37, 413--464.Google Scholar
- White, R. W., Ruthven, I., & Jose, J. M. (2005). A study of factors affecting the utility of implicit relevance feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 35--42. Google ScholarDigital Library
- Wildemuth, B. W., Freund, L. & Toms, E. G. (2014). Untangling search task complexity and difficulty in the context of interactive information retrieval studies. Journal of Documentation, 70, 1118--1140.Google ScholarCross Ref
Index Terms
- How does Interest in a Work Task Impact Search Behavior and Engagement?
Recommendations
Engaged or Frustrated?: Disambiguating Emotional State in Search
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information RetrievalOne of the primary ways researchers have characterized engagement during information search is by increases in search behaviors, such as queries and clicks. However, studies have shown that frustration is also characterized by increases in these same ...
Evaluation of interest-bridge model: older adults meditated learning of mobile technology
OzCHI '16: Proceedings of the 28th Australian Conference on Computer-Human InteractionThis research investigates older adults' adoption of mobile touch screen technologies based on the Interest-Bridge Model. According to the model, interest is a main driver in adoption. 60 independently living participants aged 60 years and older took ...
Where is the "I" in iPad?: The Role of Interest in Older Adults' Learning of Mobile Touch Screen Technologies
OzCHI '15: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human InteractionThis research explores the role of individual interest in the adoption of mobile touch screen technologies by older adults (above 60 years of age), based on the Four-Phase Model of Interest Development by Hidi and Renninger (2006) and Self-Determination ...
Comments