Tung Vuong
Giulio Jacucci
Skip Abstract Section
Abstract
We investigate to what extent it is possible to infer a user’s work tasks by digital activity monitoring and use the task models for proactive information retrieval. Ten participants volunteered for the study, in which their computer screen was monitored and related logs were recorded for 14 days. Corresponding diary entries were collected to provide ground truth to the task detection method. We report two experiments using this data. The unsupervised task detection experiment was conducted to detect tasks using unsupervised topic modeling. The results show an average task detection accuracy of more than 70% by using rich screen monitoring data. The single-trial task detection and retrieval experiment utilized unseen user inputs in order to detect related work tasks and retrieve task-relevant information on-line. We report an average task detection accuracy of 95%, and the corresponding model-based document retrieval with Normalized Discounted Cumulative Gain of 98%. We discuss and provide insights regarding the types of digital tasks occurring in the data, the accuracy of task detection on different task types, and the role of using different data input such as application names, extracted keywords, and bag-of-words representations in the task detection process. We also discuss the implications of our results for ubiquitous user modeling and privacy.
- Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel. 2009. Improvements That Don’T Add Up: Ad-hoc Retrieval Results Since 1998. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM ’09). ACM, New York, NY, USA, 601--610. Google Scholar
Digital Library
- N.J. Belkin, R. N. Oddy, and H. M. Brooks. 1982. Ask for Information Retrieval: Part I.: Background and Theory. Journal of Documentation 38, 2 (1982), 61--71.Google ScholarCross Ref
- Daniel Billsus and Michael J. Pazzani. 2000. User Modeling for Adaptive News Access. User Modeling and User-Adapted Interaction 10, 2 (2000), 147--180. Google ScholarDigital Library
- David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet Allocation. J. Mach. Learn. Res. 3 (March 2003), 993--1022. http://dl.acm.org/citation.cfm?id=944919.944937 Google ScholarDigital Library
- Pam Briggs, Elizabeth Churchill, Mark Levine, James Nicholson, Gary W. Pritchard, and Patrick Olivier. 2016. Everyday Surveillance. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’16). ACM, New York, NY, USA, 3566--3573. Google ScholarDigital Library
- Katriina Byström and Preben Hansen. 2005. Conceptual Framework for Tasks in Information Studies. J. Am. Soc. Inf. Sci. Technol. 56, 10 (Aug. 2005), 1050--1061. Google ScholarDigital Library
- Sergey Chernov. 2008. Task Detection for Activity-based Desktop Search. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’08). ACM, New York, NY, USA, 894--894. Google ScholarDigital Library
- Eun Kyoung Choe, Nicole B. Lee, Bongshin Lee, Wanda Pratt, and Julie A. Kientz. 2014. Understanding Quantified-selfers’ Practices in Collecting and Exploring Personal Data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 1143--1152. Google ScholarDigital Library
- Elizabeth F. Churchill. 2014. Scrupulous, Scrutable, and Sumptuous: Personal Data Futures. interactions 21, 5 (Sept. 2014), 20--21. Google ScholarDigital Library
- Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science 41, 6 (1990), 391--407.Google ScholarCross Ref
- Carsten Eickhoff, Jaime Teevan, Ryen White, and Susan Dumais. 2014. Lessons from the Journey: A Query Log Analysis of Within-session Learning. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM ’14). ACM, New York, NY, USA, 223--232. Google ScholarDigital Library
- Gerhard Fischer. 2001. User Modeling in Human--Computer Interaction. User Modeling and User-Adapted Interaction 11, 1-2 (March 2001), 65--86. Google ScholarDigital Library
- Ramanathan Guha, Vineet Gupta, Vivek Raghunathan, and Ramakrishnan Srikant. 2015. User Modeling for a Personal Assistant. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (WSDM ’15). ACM, New York, NY, USA, 275--284. Google ScholarDigital Library
- Kirstie Hawkey and Kori M. Inkpen. 2006. Keeping Up Appearances: Understanding the Dimensions of Incidental Information Privacy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’06). ACM, New York, NY, USA, 821--830. Google ScholarDigital Library
- Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. 1998. The LumièRe Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI’98). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 256--265. http://dl.acm.org/citation.cfm?id=2074094.2074124 Google ScholarDigital Library
- Wen Hua, Yangqiu Song, Haixun Wang, and Xiaofang Zhou. 2013. Identifying Users’ Topical Tasks in Web Search. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining (WSDM ’13). ACM, New York, NY, USA, 93--102. Google ScholarDigital Library
- Peter Ingwersen and Kalervo Järvelin. 2005. The Turn: Integration of Information Seeking and Retrieval in Context (The Information Retrieval Series). Springer-Verlag New York, Inc., Secaucus, NJ, USA. Google ScholarDigital Library
- Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated Gain-based Evaluation of IR Techniques. ACM Trans. Inf. Syst. 20, 4 (Oct. 2002), 422--446. Google ScholarDigital Library
- Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately Interpreting Clickthrough Data As Implicit Feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’05). ACM, New York, NY, USA, 154--161. Google ScholarDigital Library
- Eunju Kim, Sumi Helal, and Diane Cook. 2010. Human Activity Recognition and Pattern Discovery. IEEE Pervasive Computing 9, 1 (Jan. 2010), 48--53. Google ScholarDigital Library
- Alfred Kobsa. 2007. Generic User Modeling Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 136--154. Google ScholarDigital Library
- Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, and Gabriele Tolomei. 2011. Identifying Task-based Sessions in Search Engine Query Logs. In Proceedings of the Fourth ACM International Conference on Web Search and Data Mining (WSDM ’11). ACM, New York, NY, USA, 277--286. Google ScholarDigital Library
- Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, and Gabriele Tolomei. 2013. Discovering Tasks from Search Engine Query Logs. ACM Trans. Inf. Syst. 31, 3, Article 14 (Aug. 2013), 43 pages. Google ScholarDigital Library
- J. MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. University of California Press, Berkeley, Calif., 281--297. http://projecteuclid.org/euclid.bsmsp/1200512992Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. CoRR abs/1301.3781 (2013). http://arxiv.org/abs/1301.3781Google Scholar
- Feng Qiu and Junghoo Cho. 2006. Automatic Identification of User Interest for Personalized Search. In Proceedings of the 15th International Conference on World Wide Web (WWW ’06). ACM, New York, NY, USA, 727--736. Google ScholarDigital Library
- Andreas S. Rath, Didier Devaurs, and Stefanie N. Lindstaedt. 2009. UICO: An Ontology-based User Interaction Context Model for Automatic Task Detection on the Computer Desktop. In Proceedings of the 1st Workshop on Context, Information and Ontologies (CIAO ‘09). ACM, New York, NY, USA, Article 8, 10 pages. Google ScholarDigital Library
- Xuehua Shen, Bin Tan, and ChengXiang Zhai. 2005. Context-sensitive Information Retrieval Using Implicit Feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’05). ACM, New York, NY, USA, 43--50. Google ScholarDigital Library
- Fabrizio Silvestri. 2010. Mining Query Logs: Turning Search Usage Data into Knowledge. Found. Trends Inf. Retr. 4, 1--2 (Jan. 2010), 1--174. Google ScholarDigital Library
- Karen Sparck Jones. 1988. Document Retrieval Systems. Taylor Graham Publishing, London, UK, UK, Chapter A Statistical Interpretation of Term Specificity and Its Application in Retrieval, 132--142. http://dl.acm.org/citation.cfm?id=106765.106782 Google ScholarDigital Library
- Bin Tan, Xuehua Shen, and ChengXiang Zhai. 2006. Mining Long-term Search History to Improve Search Accuracy. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’06). ACM, New York, NY, USA, 718--723. Google ScholarDigital Library
- Jaime Teevan, Susan T. Dumais, and Eric Horvitz. 2005. Personalizing Search via Automated Analysis of Interests and Activities. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’05). ACM, New York, NY, USA, 449--456. Google ScholarDigital Library
- Jaime Teevan, Susan T. Dumais, and Eric Horvitz. 2010. Potential for Personalization. ACM Trans. Comput.-Hum. Interact. 17, 1, Article 4 (April 2010), 31 pages. Google ScholarDigital Library
- Eran Toch, Yang Wang, and Lorrie Faith Cranor. 2012. Personalization and privacy: a survey of privacy risks and remedies in personalization-based systems. User Modeling and User-Adapted Interaction 22, 1 (2012), 203--220. Google ScholarDigital Library
- Manisha Verma and Emine Yilmaz. 2014. Entity Oriented Task Extraction from Query Logs. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM ’14). ACM, New York, NY, USA, 1975--1978. Google ScholarDigital Library
- Chirayu Wongchokprasitti, Jaakko Peltonen, Tuukka Ruotsalo, Payel Bandyopadhyay, Giulio Jacucci, and Peter Brusilovsky. 2015. User Model in a Box: Cross-System User Model Transfer for Resolving Cold Start Problems. Springer International Publishing, Cham, 289--301.Google Scholar
- Dingqi Yang, Daqing Zhang, Longbiao Chen, and Bingqing Qu. 2015. NationTelescope: Monitoring and visualizing large-scale collective behavior in {LBSNs}. Journal of Network and Computer Applications 55 (2015), 170 -- 180.Google ScholarCross Ref
- Zack Zhu, Ulf Blanke, Alberto Calatroni, and Gerhard Tröster. 2013. Human Activity Recognition Using Social Media Data. In Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia (MUM ’13). ACM, New York, NY, USA, Article 21, 10 pages. Google ScholarDigital Library
Index Terms
- Watching inside the Screen: Digital Activity Monitoring for Task Recognition and Proactive Information Retrieval
Recommendations
Proactive Information Retrieval via Screen Surveillance
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information RetrievalWe demonstrate proactive information retrieval via screen surveillance. A user's digital activities are continuously monitored by capturing all content on a user's screen using optical character recognition. This includes all applications and services ...
Bootstrapping activity modeling for ambient assisted living
ICSH'13: Proceedings of the 2013 international conference on Smart HealthIn many societies, the age profile of the population is increasing, posing many challenges for societies, health services and carers. One response to this unfolding situation has been to direct research effort towards Ambient Assisted Living (AAL), ...
A multi-scale fuzzy entropy measure for anomaly detection in activities of daily living
PETRA '20: Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive EnvironmentsAnomaly detection in the Activities of Daily Living (ADL) of older adults is essential for healthcare management, to act to avoid prospective problems early and improve this group's quality of life. Once ADLs are recognised, the gathered information ...
Comments