Kun Huang
ABSTRACT
This study uses comparative research to compare the effects of the subjective difficulty and objective difficulty of search tasks on users' search behaviors. Data regarding users' opinions about task difficulty was obtained via a post-search questionnaire using a 5-Likert scale. When measuring subjective difficulty, tasks with ratings above 3 were considered difficult. When measuring objective difficulty, tasks with ratings higher than the average difficulty score were considered difficult. The study's findings indicate that it is better to develop task difficulty prediction models that are based on subjective difficulty because these models are more stable. Models based on objective difficulty could not match the performance of models based upon subjective difficulty. The findings shed light on issues related to experiment design that will be valuable for future research.
Supplemental Material
- Yuelin Li and Nicholas J. Belkin. 2008. A faceted approach to conceptualizing tasks in information seeking. Information Processing & Management 44, 6 (2008), 1822--1837. DOI: http://dx.doi.org/10.1016/j.ipm. 2008.07.005.Google Scholar
Digital Library
- Chang Liu, Jingjing Liu, and Nicholas J. Belkin. 2014. Predicting Search Task Difficulty at Different Search Stages. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM '14). Association for Computing Machinery, New York, NY, USA, 569--578. DOI: https://doi.org/10.1145/2661829.2661939.Google Scholar
- Jingjing Liu, Yuan Li, and Samantha K. Hastings. 2019. Simplified Scheme of Search Task Difficulty Reasons. Journal of the Association for Information Science and Technology 70, 5 (2019), 526--529. DOI: http://dx.doi.org/10.1002/asi.24125.Google ScholarDigital Library
- Edwin A. Locke. 1968. Toward a theory of task motivation and incentives. Organizational Behavior and Human Performance 3, 2 (1968), 157--189. DOI: http://dx.doi.org/10.1016/0030--5073(68)90004--4.Google ScholarCross Ref
- Jingjing Liu, Jacek Gwizdka, Chang Liu, and Nicholas J. Belkin. 2010. Predicting task difficulty for different task types. Proceedings of the American Society for Information Science and Technology 47, 1 (2010), 1--10. DOI: http://dx.doi.org/10.1002/meet.14504701173.Google ScholarDigital Library
- Jingjing Liu, Chang Liu, Michael Cole, Nicholas J. Belkin, and Xiangmin Zhang. 2012. Exploring and predicting search task difficulty. In Proceedings of the 21st ACM international conference on Information and knowledge management (CIKM '12). Association for Computing Machinery, New York, NY, USA, 1313--1322. DOI: https://doi.org/10.1145/2396761.2398434.Google ScholarDigital Library
- Xiao Hu and Noriko Kando. 2017. Task complexity and difficulty in music information retrieval. Journal of the Association for Information Science and Technology 68, 7 (2017), 1711--1723. DOI: http://dx.doi.org/10.1002/asi.23803.Google ScholarDigital Library
- Daniel Hienert, Matthew Mitsui, Philipp Mayr, Chirag Shah, and Nicholas J. Belkin. 2018. The Role of the Task Topic in Web Search of Different Task Types. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR '18). Association for Computing Machinery, New York, NY, USA, 72--81. DOI: https://doi.org/10.1145/3176349.3176382.Google Scholar
- Saraschandra Karanam and Herre Van Oostendorp. 2017. Age-Related Effects of Task Difficulty on the Semantic Relevance of Query Reformulations. In Proceedings of the 16th IFIP TC 13 International Conference on Human-Computer Interaction-INTERACT (2017), 77--96. DOI: http://dx.doi.org/10. 1007/978--3--319--67744--6_6.Google ScholarCross Ref
- Peter Pirolli, Patricia Schank, Marti Hearst, and Christine Diehl. 1996. Scatter/gather browsing communicates the topic structure of a very large text collection. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '96). Association for Computing Machinery, New York, NY, USA, 213--220. DOI: https://doi.org/10.1145/238386.238489.Google ScholarDigital Library
- Jaime Arguello. 2014. Predicting Search Task Difficulty. In Proceedings of European Conference on Information Retrieval (2014), 88--99. DOI:http://dx.doi. org/10.1007/978--3--319-06028--6_8.Google ScholarDigital Library
- Anne Aula, Rehan M. Khan, and Zhiwei Guan. 2010. How does search behavior change as search becomes more difficult? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). Association for Computing Machinery, New York, NY, USA, 35--44. DOI: https://doi.org/10.1145/1753326.1753333.Google ScholarDigital Library
Index Terms
- A Comparative Study of the Relationship between the Subjective Difficulty, Objective Difficulty of Search Tasks and Search Behaviors
Recommendations
Why Do Users Perceive Search Tasks As Difficult? Exploring Difficulty in Different Task Types
HCIR '13: Proceedings of the Symposium on Human-Computer Interaction and Information RetrievalIn this paper, we examined why information searchers perceive search tasks as difficult, and what factors/reasons make them perceive tasks as difficult. We also examined if task difficulty reasons vary across different tasks (task types). Data was ...
Exploring and predicting search task difficulty
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge managementWe report on an investigation of behavioral differences between users in difficult and easy search tasks. Behavioral factors that can be used in real-time to predict task difficulty are identified. User data was collected in a controlled lab experiment (...
Reducing Objective Difficulty Without Influencing Subjective Difficulty in a Video Game
MMAsia '23: Proceedings of the 5th ACM International Conference on Multimedia in AsiaWhile dynamic difficulty adjustment techniques can enhance the player engagement, overly apparent difficulty adjustments may have a detrimental effect on the overall gaming experience. Hence, it is desirable to implement game difficulty controls as ...
Comments