Robert Bixler
ABSTRACT
It is hypothesized that the ability for a system to automatically detect and respond to users' affective states can greatly enhance the human-computer interaction experience. Although there are currently many options for affect detection, keystroke analysis offers several attractive advantages to traditional methods. In this paper, we consider the possibility of automatically discriminating between natural occurrences of boredom, engagement, and neutral by analyzing keystrokes, task appraisals, and stable traits of 44 individuals engaged in a writing task. The analyses explored several different arrangements of the data: using downsampled and/or standardized data; distinguishing between three different affect states or groups of two; and using keystroke/timing features in isolation or coupled with stable traits and/or task appraisals. The results indicated that the use of raw data and the feature set that combined keystroke/timing features with task appraisals and stable traits, yielded accuracies that were 11% to 38% above random guessing and generalized to new individuals. Applications of our affect detector for intelligent interfaces that provide engagement support during writing are discussed.
- Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater® V. 2. The Journal of Technology, Learning and Assessment, 4(3).Google Scholar
- Bleha, S., Slivinsky, C., & Hussien, B. (1990). Computer-access security systems using keystroke dynamics. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 12(12), 1217--1222. Google ScholarDigital Library
- Calvo, R. A., & D'Mello, S. K. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1), 18--37. doi: 10.1109/T-AFFC.2010.1 Google ScholarDigital Library
- Cole, J. S., & Gonyea, R. M. (2010). Accuracy of selfreported SAT and ACT test scores: Implications for research. Research in Higher Education, 51(4), 305319. doi: 10.1007/s11162-009-9160-9.Google ScholarCross Ref
- Crawford, H. (2010). Keystroke dynamics: Characteristics and opportunities. In Privacy Security and Trust (PST), 2010 Eighth Annual International Conference on (pp. 205--212). IEEE.Google ScholarCross Ref
- Craig, S., D'Mello, S., Witherspoon, A., & Graesser, A. (2008). Emote aloud during learning with AutoTutor: Applying the facial action coding system to cognitiveaffective states during learning. Cognition & Emotion, 22(5), 777--788.Google ScholarCross Ref
- Craig, S., Graesser, A., Sullins, J., & Gholson, J. (2004). Affect and learning: An exploratory look into the role of affect in learning. Journal of Educational Media, 29, 241--250.Google ScholarCross Ref
- Cunningham, A. E., & Stanovich, K. E. (1997). Early reading acquisition and its relation to reading experience and ability 10 years later. Developmental Psychology, 33(6), 934--945.Google ScholarCross Ref
- Daly, J., & Miller, M. (1975). The empirical development of an instrument to measure writing apprehension. Research in the Teaching of English 9(3), 242--249.Google Scholar
- D'Mello, S., & Graesser, A. (2011). The half-life of cognitive-affective states during complex learning. Cognition & Emotion, 25(7), 1299--1308.Google ScholarCross Ref
- D'Mello, S., & Graesser, A. (2010). Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Modeling and User-adapted Interaction, 20(2), 147--187. Google ScholarDigital Library
- D'Mello, S., & Mills, C. (in review). Emotions during emotional and non-emotional writing.Google Scholar
- Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6(3-4), 169--200.Google ScholarCross Ref
- Epp, C., Lippold, M., & Mandryk, R. L. (2011). Identifying emotional states using keystroke dynamics. In Proceedings of the 2011 annual conference on Human factors in computing systems (pp. 715--724). ACM. Google ScholarDigital Library
- Foltz, P. W., Laham, D., & Landauer, T. K. (1999). The intelligent essay assessor: Applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1(2).Google Scholar
- Gunetti, D., & Picardi, C. (2005). Keystroke analysis of free text. ACM Transactions on Information and System Security (TISSEC), 8(3), 312--347. Google ScholarDigital Library
- Joyce, R., & Gupta, G. (1990). Identity authentication based on keystroke latencies. Communications of the ACM, 33(2), 168--176. Google ScholarDigital Library
- K., & Litman, D. J. (2011). Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor. Speech Communication, 53(9-10), 1115--1136. doi: 10.1016/j.specom.2011.02.006 Google ScholarDigital Library
- Khanna, P., & Sasikumar, M. (2010). Recognising Emotions from Keyboard Stroke Pattern. International Journal of Computer Applications IJCA, 11(9), 24--28.Google Scholar
- McNamara, D. S., Raine, R., Roscoe, R., Crossley, S., Jackson, G. T., Dai, J., & Graesser, A. C. (2012). The Writing-Pal: Natural language algorithms to support intelligent tutoring on writing strategies. Applied natural language processing and content analysis: Identification, investigation, and resolution. Hershey, PA: IGI Global.Google Scholar
- Metallinou, A., Wollmer, M., Katsamanis, A., Eyben, F., Schuller, B., & Narayanan, S. (in press). ContextSensitive Learning for Enhanced Audiovisual Emotion Classification. IEEE Transactions on Affective Computing. Google ScholarDigital Library
- Mills, C., & D'Mello, S. K. (2012). Emotions during writing on topics that align or misalign with personal beliefs. In S. Cerri, W. Clancey, G. Papadourakis & K. Panourgia (Eds.), Proceedings of the 11th International Conference on Intelligent Tutoring Systems (pp. 638--639). Berlin Heidelberg: Springer-Verlag. Google ScholarDigital Library
- NAEP. (2007). The Nation's Report Card: Writing 2007.Google Scholar
- Nijholt, Anton, & Desney S. Tan (2010). BrainComputer Interfaces: Applying our Minds to Human-Computer Interaction. Springer. Google ScholarDigital Library
- Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2), 1--135. Google ScholarDigital Library
- Pantic, M., & Patras, I. (2006). Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transactions on Systems, Man, and Cybernetics, Part B., 36(2), 433--449. doi: 10.1109/tsmcb.2005.859075. Google ScholarDigital Library
- Pantic, M., & Rothkrantz, L. (2003). Toward an affect-sensitive multimodal human-computer interaction. {Review}. Proceedings of the IEEE, 91(9), 1370--1390. doi: 10.1109/jproc.2003.817122.Google ScholarCross Ref
- Picard, R. (1997). Affective Computing. Cambridge, Mass: MIT Press. Google ScholarDigital Library
- Picard, R. (2010). Affective Computing: From Laughter to IEEE. IEEE Transactions on Affective Computing, 1(1), 11--17. Google ScholarDigital Library
- Rosenberg, E., & Ekman, P. (1994). Coherence between expressive and experiential systems in emotion. Cognition & Emotion, 8(3), 201--229.Google ScholarCross Ref
- Valstar, M. F., Mehu, M., Jiang, B., Pantic, M., Scherer, K., Jiang, B., Valstar, M., Pantic, M., Valstar, M., & Jiang, B. (in press). Meta-Analysis of the First Facial Expression Recognition Challenge. IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics.Google Scholar
- Vizer, L. M., Zhou, L., & Sears, A. (2009). Automated stress detection using keystroke and linguistic features: An exploratory study. International Journal of HumanComputer Studies, 67(10), 870--886. Google ScholarDigital Library
- Wade-Stein, D., & Kintsch, E. (2004). Summary Street: Interactive computer support for writing. Cognition and Instruction, 22(3), 333--362.Google ScholarCross Ref
- Witten, I. H., Frank, E., Trigg, L., Hall, M., Holmes, G. & Cunningham, S. J. (1999). Weka: Practical machine learning tools and techniques with Java implementations. Hamilton, New Zealand: University of Waikato, Department of Computer Science. Google ScholarDigital Library
- Zeng, Z., Pantic, M., Roisman, G., & Huang, T. (2009). A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), 39--58. Google ScholarDigital Library
Index Terms
- Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits
Recommendations
Identifying emotional states using keystroke dynamics
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThe ability to recognize emotions is an important part of building intelligent computers. Emotionally-aware systems would have a rich context from which to make appropriate decisions about how to interact with the user or adapt their system response. ...
The Complex Relationship Between Empathy, Engagement and Boredom
ECCE '16: Proceedings of the European Conference on Cognitive ErgonomicsIn human computer interactions --- especially gaming --- the role of empathy has been mooted as a necessary prerequisite for higher levels of engagement and immersion. More recently other forms of engagement, including intellectual/cognitive engagement, ...
Dynamics of Student Cognitive-Affective Transitions During a Mathematics Game
Researchers of interactive learning environments have grown increasingly interested in designing these systems to become more responsive to differences in students' cognitive-affective states. They believe that the detection of and adaptation to student ...
Comments