research-article

Mining problem-solving strategies from HCI data

Authors:

Chaitanya Komireddy,

Valentina Grigoreanu,

Margaret BurnettAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 17, Issue 1

Article No.: 3, Pages 1 - 22

https://doi.org/10.1145/1721831.1721834

Published: 06 April 2010 Publication History

Abstract

Can we learn about users' problem-solving strategies by observing their actions? This article introduces a data mining system that extracts complex behavioral patterns from logged user actions to discover users' high-level strategies. Our application domain is an HCI study aimed at revealing users' strategies in an end-user debugging task and understanding how the strategies relate to gender and to success. We cast this problem as a sequential pattern discovery problem, where user strategies are manifested as sequential behavior patterns. Problematically, we found that the patterns discovered by standard data mining algorithms were difficult to interpret and provided limited information about high-level strategies. To help interpret the patterns as strategies, we examined multiple ways of clustering the patterns into meaningful groups. This collectively led to interesting findings about users' behavior in terms of both gender differences and debugging success. These common behavioral patterns were novel HCI findings about differences in males' and females' behavior with software, and were verified by a parallel study with an independent data set on strategies. As a research endeavor into the interpretability issues faced by data mining techniques, our work also highlights important research directions for making data mining more accessible to non-data-mining experts.

References

[1]

Agrawal, R. and Srikant, R. 1995. Mining sequential patterns. In Proceedings of the 11th International Conference on Data Engineering. 3--14.

Digital Library

[2]

Ayres, J., Flannick, J., Gehrke, J., and Yiu, T. 2002. Sequential pattern mining using a bitmap representation. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 429--435.

Digital Library

[3]

Beckwith, L. and Burnett, M. 2004. Gender: An important factor in end-user programming environments&quest; In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing. 107--114.

Digital Library

[4]

Beckwith, L., Burnett, M., Grigoreanu, V., and Wiedenbeck, S. 2006a. Gender HCI: What about the software&quest; Computer, 83--87.

Digital Library

[5]

Beckwith, L., Burnett, M., Wiedenbeck, S., Cook, C., Sorte, S., and Hastings, M. 2005. Effectiveness of end-user debugging software features: Are there gender issues&quest; In Proceedings of the ACM Conference on Human-Computer Interaction. 869--878.

Digital Library

[6]

Beckwith, L., Inman, D., Rector, K., and Burnett, M. 2007. On to the real world: Gender and self-efficacy in excel. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing. 119--126.

Digital Library

[7]

Beckwith, L., Kissinger, C., Burnett, M., Wiedenbeck, S., Lawrance, J., Blackwell, A., and Cook, C. 2006b. Tinkering and gender in end-user programmers' debugging. In Proceedings of the ACM Conference on Human Factors in Computing Systems. 231--240.

Digital Library

[8]

Brewer, J. and Bassoli, A. 2006. Reflections of gender, reflections on gender: Designing ubiquitous computing technologies. In Proceedings of Gender & Interaction: Real and Virtual Women in a Male World, (Workshop at AVI). 9--12.

[9]

Burnett, M., Atwood, J., Djang, R., Gottfried, H., Reichwein, J., and Yang, S. 2001. Forms/3: A first-order visual language to explore the boundaries of the spreadsheet paradigm. J. Funct. Program. 11, 155--206.

Digital Library

[10]

Burnett, M., Cook, C., and Rothermel, G. 2004. End-user software engineering. Comm. ACM, 53--58.

Digital Library

[11]

Casella, G. and Berger, R. L. 1990. Statistical Inference. Duxbury Press.

[12]

Cervone, G. and Michalski, R. 2002. Modeling user behavior by integrating aq learning with a database: Initial results. Intell. Inform. Syst., 43--56.

Digital Library

[13]

Czerwinski, M., Tan, D. S., and Robertson, G. G. 2002. Women take a wider view. In Proceedings of ACM Conference on Factors in Computing Systems. ACM Press, 195--202.

Digital Library

[14]

Dettling, M. and Buehlmann, P. 2002. Supervised clustering of genes. Genome Biol. 3.

[15]

Dhillon, I. S., Mallela, S., and Kumar, R. 2003. A divisive information-theoretic feature clustering algorithm for text classication. J. Mach. Learn. Resear. 3, 1265--1287.

Digital Library

[16]

El-Ramly, M., Stroulia, E., and Sorenson, P. 2002. Interaction-pattern mining: Extracting usage scenarios from run-time behavior traces. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'02).

Digital Library

[17]

Garofalakis, M. N., Rastogi, R., and Shim, K. 1999. SPIRIT: Sequential pattern mining with regular expression constraints. In VLDB J. 223--234.

Digital Library

[18]

Gouda, K. and Zaki, M. 2001. Efficiently mining maximal frequent itemsets. In Proceedings of the International Conference on Data Mining.

Digital Library

[19]

Hastie, T., Tibshirani, B., and Friedman, J. 2001. Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer-Verlag.

[20]

Hatonen, K., Klemettinen, M., Ronkainen, P., and Toivonen, H. 1996. Knowledge discovery from telecommunication network alarm data bases. In Proceedings of the 12th International Conference on Data Engineering. 115--122.

Digital Library

[21]

Hilbert, D. and Redmiles, D. 2000. Extracting usability information from user interface events. ACM Comput. Surv. 32, 4, 384--421.

Digital Library

[22]

Jain, A. K., Murty, M. N., and Flynn, P. 1999. Data clustering: a review. ACM Comput. Surv. 31, 3.

Digital Library

[23]

Kelleher, C., Pausch, R., and Kiesler, S. 2007. Storytelling alice motivates middle school girls to learn computer programming. In Proceedings of the ACM Conference on Factors in Computing Systems. 1455--1464.

Digital Library

[24]

Khardon, R. 1999. Learning action strategies for planning domains. Artif. Intell. 113, 125--148.

Digital Library

[25]

Lorigo, L., Pan, B., Hembrooke, H., Joachims, T., Granka, L., and Gay, G. 2006. The influence of task and gender on search and evaluation behavior using google. In Information Processing and Management, 1123--1131.

Digital Library

[26]

Mannila, H., Toivonen, H., and Verkamo, A. 1997. Discovery of frequent episodes in event sequences. In Proceedings of the 1st International Conference on Data Mining and Knowledge Discovery, 259--289.

[27]

Mei, Q., Xin, D., Cheng, H., Han, J., and Zhai, C. 2006. Generating semantic annotations for frequent patterns with context analysis. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'06).

Digital Library

[28]

Mobasher, B., Cooley, R., and Srivastava, J. 2000. Automatic personalization based on Web usage mining. Comm. ACM 8, 142--151.

Digital Library

[29]

Pasquier, N., Bastide, Y., Taouil, R., and Lakhal, L. 1999. Discovering frequent closed itemsets for association rules. In Proceedings of the 7th International Conference on Database Theory.

Digital Library

[30]

Pei, J., Han, J., Mortazavi-Asl, B., and Pinto, H. 2001. Prefixspan: Mining sequential patterns efficiently by prefix-projected pattern growth. In Proceedings of International Conference on Data Engineering.

Digital Library

[31]

Perkowitz, M. and Etzioni, O. 1998. Adaptive Web sites: Automatically synthesizing Web pages. In Proceedings of the 15th National Conference on Artificial Intelligence.

Digital Library

[32]

Rode, J. 2008. An ethnographic examination of the relationship of gender & end-user programming. Ph.D. thesis, University of California Irvine.

Digital Library

[33]

Rode, J. A., Toye, E. F., and Blackwell, A. F. 2004. The fuzzy felt ethnography—understanding the programming patterns of domestic appliances. Person. Ubiq. Comput. 8, 161--176.

Digital Library

[34]

Rosson, M., Sinha, H., Bhattacharya, M., and Zhao, D. 2007. Design planning in end-user web development. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing. 189--196.

Digital Library

[35]

Seno, M. and Karypis, G. 2002. Slpminer: An algorithm for finding frequent sequential patterns using length decreasing support constraint. In Proceedings of the 2nd IEEE International Conference on Data Mining. 418--425.

Digital Library

[36]

Slonim, N. and Tishby, N. 2001. The power of word clusters for text classification. In Proceedings of the 23rd European Colloquium on Information Retrieval Research.

[37]

Srivastava, J., Cooley, R., Deshpande, M., and Tan, P.-N. 2000. Web usage mining: discovery and applications of usage patterns from web data. SIGKDD Explor. 1, 2, 12--23.

Digital Library

[38]

Subrahmaniyan, N., Beckwith, L., Grigoreanu, V., Burnett, M., Wiedenbeck, S., Narayanan, V., Bucht, K., Drummond, R., and Fern, X. 2008. Testing vs. code inspection vs. … what else&quest; Male and female end users' debugging strategies. In Proceedings of the ACM Conference on Factors in Computing Systems. 617--626.

Digital Library

[39]

Tan, D. S., Czerwinski, M., and Robertson, G. G. 2003. Women go with the (optical) flow. In Proceedings of the ACM Conference on Factors in Computing Systems. ACM Press, 209--215.

Digital Library

[40]

Wang, C. and Parthasarathy, S. 2006. Summarizing itemset patterns using probabilistic models. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 730--735.

Digital Library

[41]

Xin, D., Cheng, H., Yan, X., and Han, J. 2006. Extracting redundancy-aware top-k patterns. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'06).

Digital Library

[42]

Xin, D., Han, J., Yan, X., and Cheng, H. 2005. Mining compressed frequent-pattern sets. In Proceedings of the International Conference on Very Large Data Bases.

Digital Library

[43]

Yan, X., Cheng, H., Han, J., and Xin, D. 2005. Summarizing itemset patterns: A profile-based approach. In Proceedings of 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.

Digital Library

[44]

Yan, X., Han, J., and Afshar, R. 2003. Clospan: Mining closed sequential patterns in large datasets. In Proceedings of the 3rd SIAM International Conference on Data Mining.

Cited By

Abb LRehse J(2024)Process-related user interaction logs: State of the art, reference model, and object-centric implementationInformation Systems10.1016/j.is.2024.102386(102386)Online publication date: Apr-2024
https://doi.org/10.1016/j.is.2024.102386
Rehse JAbb LBerg GBormann CKampik TWarmuth C(2024)User Behavior MiningBusiness & Information Systems Engineering10.1007/s12599-023-00848-166:6(799-816)Online publication date: 5-Jan-2024
https://doi.org/10.1007/s12599-023-00848-1
Barros De Amorim ISantos Dos Santos SMoreira Miranda Da Silva IRios Da Hora Rodrigues KPereira Mota M(2023)Gender Nuances in Human-Computer Interaction ResearchProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638077(1-12)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638077
Show More Cited By

Index Terms

Mining problem-solving strategies from HCI data

Recommendations

A data mining approach to discovering reliable sequential patterns

Sequential pattern mining is a data mining method for obtaining frequent sequential patterns in a sequential database. Conventional sequence data mining methods could be divided into two categories: Apriori-like methods and pattern growth methods. In a ...
Effective database transformation and efficient support computation for mining sequential patterns
Abstract
In this paper, we propose a novel algorithm for mining frequent sequences from transaction databases. The transactions of the same customers form a set of customer sequences. A sequence (an ordered list of itemsets) is frequent if the number of ...
Mining Sequential Patterns across Time Sequences
Abstract
In this paper, we deal with mining sequential patterns in multiple time sequences. Building on a state-of-the-art sequential pattern mining algorithm PrefixSpan for mining transaction databases, we propose MILE (MIning in muLtiple sEquences), an ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 17, Issue 1

March 2010

130 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/1721831

Issue’s Table of Contents

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 April 2010

Accepted: 01 November 2009

Revised: 01 August 2008

Received: 01 December 2007

Published in TOCHI Volume 17, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
1,430
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Abb LRehse J(2024)Process-related user interaction logs: State of the art, reference model, and object-centric implementationInformation Systems10.1016/j.is.2024.102386(102386)Online publication date: Apr-2024
https://doi.org/10.1016/j.is.2024.102386
Rehse JAbb LBerg GBormann CKampik TWarmuth C(2024)User Behavior MiningBusiness & Information Systems Engineering10.1007/s12599-023-00848-166:6(799-816)Online publication date: 5-Jan-2024
https://doi.org/10.1007/s12599-023-00848-1
Barros De Amorim ISantos Dos Santos SMoreira Miranda Da Silva IRios Da Hora Rodrigues KPereira Mota M(2023)Gender Nuances in Human-Computer Interaction ResearchProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638077(1-12)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638077
Liu TIsrael M(2022)Uncovering students’ problem-solving processes in game-based learning environmentsComputers & Education10.1016/j.compedu.2022.104462182:COnline publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.compedu.2022.104462
Abb LRehse J(2022)A Reference Data Model for Process-Related User Interaction LogsBusiness Process Management10.1007/978-3-031-16103-2_7(57-74)Online publication date: 11-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-16103-2_7
Saarinen SHeimonen TTurunen MMikkilä-Erdmann MRaisamo RErdmann NYrjänäinen SKeskinen T(2015)Identifying User Interaction Patterns in E‐TextbooksThe Scientific World Journal10.1155/2015/9815202015:1Online publication date: 29-Oct-2015
https://doi.org/10.1155/2015/981520
Blikstein PWorsley MPiech CSahami MCooper SKoller D(2014)Programming Pluralism: Using Learning Analytics to Detect Patterns in the Learning of Computer ProgrammingJournal of the Learning Sciences10.1080/10508406.2014.95475023:4(561-599)Online publication date: 24-Oct-2014
https://doi.org/10.1080/10508406.2014.954750
Srimani PKoti M(2014)Rough Set Approach for Novel Decision Making in Medical Data for Rule Generation and Cost SensitivenessICT and Critical Infrastructure: Proceedings of the 48th Annual Convention of Computer Society of India- Vol II10.1007/978-3-319-03095-1_33(303-311)Online publication date: 2014
https://doi.org/10.1007/978-3-319-03095-1_33
Heidl WThumfart SLughofer EEitzinger CKlement E(2013)Machine learning based analysis of gender differences in visual inspection decision makingInformation Sciences: an International Journal10.1016/j.ins.2012.09.054224(62-76)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.1016/j.ins.2012.09.054
Chan HGoswami SKim H(2012)An Alternative Fit through Problem Representation in Cognitive Fit TheoryJournal of Database Management10.4018/jdm.201204010223:2(22-43)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.4018/jdm.2012040102
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents