Jiajia Li
Grace Ngai
ABSTRACT
Affective computing in human-computer interaction research enables computers to understand human affects or emotions to provide better service. In this paper, we investigate the detection of human attention useful in intelligent e-learning applications. Our principle is to use only ubiquitous hardware available in most computer systems, namely, webcam and mouse. Information from multiple modalities is fused together for effective human attention detection. We invite human subjects to carry out experiments in reading articles being subjected to different kinds of distraction to induce different attention levels. Machine-learning techniques are applied to identify useful features to recognize human attention level. Our results indicate improved performance with multimodal inputs, suggesting an interesting affective computing direction.
- Calvo, R. and Mello, S. Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1):18--37, 2010. Google Scholar
Digital Library
- Picard, R. Affective Computing. The MIT Press, 1997. Google ScholarDigital Library
- Zeng, Z., Pantic, M., Roisman, G. and Huang, T. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1):39--58, 2009. Google ScholarDigital Library
- Oviatt, S. Multimodal interfaces. Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, L. Erlbaum Assoc. Inc., pp. 286--304, 2007. Google ScholarDigital Library
- Cunningham, D. W., Kleiner, M., Bülthoff, H. H. and Wallraven, C. The components of conversational facial expressions. In Proc. of Symposium on Applied Perception in Graphics and Visualization, ACM, pp. 143--150, 2004. Google ScholarDigital Library
- Ekman, P. Universals and cultural differences in facial expression of emotion. In Proc. of Nebraska Symposium on Motivation, pp. 207--283, 1972.Google Scholar
- Ambadar, Z., Cohn, J. F. and Reed, L. I. All smiles are not created equal: Morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. Journal of Nonverbal Behavior, 33(1):17--34, 2009.Google ScholarCross Ref
- Anderson, A., Christenson, S., Sinclair, M. and Lehr, C. Check and connect: The importance of relationships for promoting engagement with school. Journal of School Psychology, 42:95--113, 2004.Google ScholarCross Ref
- Yamauchi, T. Mouse trajectories and state anxiety: Feature selection with random forest. In Proc. of ACII, pp. 399--404, IEEE, 2013. Google ScholarDigital Library
- Tsoulouhas, G., Georgiou, D. and Karakos, A. Detection of learners' affective state based on mouse movements. Journal of Computing, 3(11):9--18, 2011.Google Scholar
- Bixler, R. and D'Mello, S. Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits. In Proc. of IUI, pp. 225--, ACM, 2013. Google ScholarDigital Library
- Bolt, R. Put-that-there: Voice and gesture at the graphics interface. ACM SIGGRAPH Computer Graphics, 14(3):262--270, 1980. Google ScholarDigital Library
- Saragih, J. M., Lucey, S. and Cohn, J. F. Deformable model fitting by regularized landmark man-shift. International Journal of Computer Vision, 91(2):200--215, 2011. Google ScholarDigital Library
- Gross, R., Matthews, I., Cohn, J., Kanade, T. and Baker, S. Multi-PIE. In Proc. of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 1--8, 2008.Google Scholar
- Xiong, X. and De la Torre, F. Supervised descent method and its applications to face alignment. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 532--539, 2013. Google ScholarDigital Library
- Tsalakanidou, F. and Malassiotis, S. Real-time 2D+3D facial action and expression recognition. Pattern Recognition, 43(5):1763--1775, 2010. Google ScholarDigital Library
- Talavera, L. An evaluation of filter and wrapper methods for feature selection in categorical clustering. In Proc. of International Symposium on Intelligent Data Analysis, pp. 440--451, 2005. Google ScholarDigital Library
- Stroupe, G. Comprehension and time difference across no-music, lyrical music, and non-lyrical music groups. Department of Sociology, Emory University, 2005.Google Scholar
- Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A. and Movellan, J. R. The faces of engagement: Automatic recognition of student engagement from facial expressions. IEEE Transactions on Affective Computing, 5(1):86--98, 2014.Google ScholarCross Ref
- Fu, Y., Leong, H. V., Ngai, G., Huang, M. X. and Chan, S. C. F. Physiological mouse: Towards an emotion-aware mouse. Universal Access in Information Society: An International Journal, Springer.Google Scholar
Index Terms
- Multimodal human attention detection for reading
Recommendations
Multimodal human attention detection for reading from facial expression, eye gaze, and mouse dynamics
Affective computing has recently become an important area in human-computer interaction research. Techniques have been developed to enable computers to understand human affects or emotions, in order to predict human intention more precisely and provide ...
StressClick: Sensing Stress from Gaze-Click Patterns
MM '16: Proceedings of the 24th ACM international conference on MultimediaStress sensing is valuable in many applications, including online learning crowdsourcing and other daily human-computer interactions. Traditional affective computing techniques investigate affect inference based on different individual modalities, such ...
Moment-to-Moment Detection of Internal Thought during Video Viewing from Eye Vergence Behavior
MM '19: Proceedings of the 27th ACM International Conference on MultimediaInternal thought refers to the process of directing attention away from a primary visual task to internal cognitive processing. It is pervasive and closely related to primary task performance. As such, automatic detection of internal thought has ...
Comments