Abstract
This article examines how emerging pervasive computing and affective computing technologies might enhance the adoption of ICT in e-Learning which takes place in the home and wider city environment. In support of this vision we describe two cutting edge ICT environments which combine to form a holistic connected future learning environment. The first is the iSpace, a specialized digital-home test-bed that represents the kind of high-tech, context aware home-based learning environment we envisage future learners using, the second a sophisticated pervasive e-Learning platform that typifies the educational delivery platform our research is targeting. After describing these environments we then present our research that explores how emotion evolves during the learning process and how to leverage emotion feedback to provide adaptive e-Learning system. The motivation driving this work is our desire to improve the performance of the educational experience by developing learning systems that recognize and respond appropriately to emotions exhibited by learners. Finally we report on the results about the emotion recognition from physiological signals which achieved a best-case accuracy rate of 86.5% for four types of learning emotion. To the best of our knowledge, this is the first report on emotion detection by data collected from close-to-real-world learning sessions. We also report some finding about emotion evolution during learning, which are still not enough to validate Kort’s learning spiral model.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bassili, J. N. (1979). Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37, 2049–2058.
Burleson, W., Picard, R. W., Perlin, K., & Lippincott, J. (2004). A platform for affective agent research, 3rd International Conference on Autonomous Agents and Multi Agent Systems. New York: ACM.
Callaghan, V., Clark, G., Colley, M., Hagras, H., Chin, J. S. Y., & Doctor, F. (2004). Inhabited intelligent environments. BT Technology Journal, 22(3), 233–247, (215).
CSEA (2001). The international affective picture system: Digitized photographs: CENTER for the Study of Emotion and Attention, University of Florida.
Chang, C. C., & Lin, C. J. (2007). LIBSVM—a library for support vector machines.
Clarke, G., & Callaghan, V. (2007). Ubiquitous computing, informatization, urban structures and density. Built Environment Journal, 33(2), 196–212.
Clynes, M. (1977). Sentics: The touch of the emotions. New York: Prism Pr Ltd.
CNNIC (2007). Statistical Survey Report on The Internet Development in China: China Internet Network Information Center.
Conati, C., & Zhou, X. (2002). Modeling students’ emotions from cognitive appraisal in educational games, the 6th International Conference on Intelligent Tutoring Systems. Biarritz, France: Springer.
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., et al. (2001). Emotion recognition in human–computer interaction. IEEE Sinal Process Magazine, 18, 32–80.
Craig, S. D., Graesser, A. C., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250.
Duda, R. O., & Hart, P. E. (1973). Pattern classification and scene analysis. New York: Wiley.
Ekman, P., Levenson, R. W., & Friesen, W. V. (1983). Automatic nervous system activity distinguishes among emotions. Science, 221, 1208–1210.
Fagerberg, P., Ståhl, A., & Höök, K. (2004). eMoto: Emotionally engaging interaction. Personal and Ubiquitous Computing, 8(5), 377–381.
Fowler, C. J. H. (1977). The role of arousal in memory and attention. London: University of London.
Haag, A., Goronzy, S., Schaich, P., & Williams, J. (2004). Emotion Recognition Using Bio-Sensors: First Step Towards an Automatic System. Paper presented at the Affective Dialogue Systems, Tutorial and ResearchWorkshop, Kloster Irsee, Germany.
Healey, J. (2000). Wearable and automotive systems for affect recognition from physiology. Cambridge, MA: MIT.
Isen, A. M. (2000). Positive affect and decision making. In M. Lewis, & J. Haviland (Eds.), Handbook of emotions (p. 720). Guilford, New York: Guilford.
Katsionis, G., & Virvou, M. (2005). Adapting OCC theory for affect perception in educational software, the International Conference on Human–computer Interaction 2005 (HCI 2005). Las Vegas, Nevada USA.
Keller, J. M., & Suzuki, K. (1988). Use of the ARCS motivation model in courseware design. In D. H. Jonassen (Ed.), Instructional designs for microcomputer courseware. Hillsdale, NJ: Erlbaum.
Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy-building a learning companion, IEEE International Conference on Advanced Learning Technologies. Madison, WI, USA.
Lévesque, J. (2006). BCIA-EEG. Université de Montréal, Canada.
Leon, E., Clarke, G., & Callaghan, V. (2007). A user-independent real-time emotion recognition system for software agents in domestic environments. International Journal of Intelligent Real-time Automation, 20(3), 337–345.
Luo, Q., Wan, L. Y., & Wu, Y. W. (2006). The application of affective computing in e-Learning systems. Open Education Research, 12(3), 80–84, (in Chinese).
Moore, M. G. (2007). The Theory of Transactional Distance. In M. G. Moore (Ed.), The Handbook of Distance Education. (pp. 89–108, 2nd ed.). Mahwah, NJ: Erlbaum.
Morishima, S. (2000). Real-time face Analysis and Synthesis using Neural Networks, IEEE Workshop on Neural Networks for Signal Processing.
NCES. (2006). Homeschooling in the United States: 2003 (No. NCES 2006-042): National Center for Education Statistics.
Nicholson, J., Takahashi, K., & Nakatsu, R. (2000). Emotion recognition in speech using neural networks. Neural Computing and Applications, 9(4), 290–296.
Oppenheim, A. V., Schafer, R. W., & Buck, J. R. (1998). Discrete-time signal processing (2nd edn). New Jersey: Prentice-Hall.
Ortony, A., Clore, G. L., & Collins, A. (1990). The cognitive structure of emotions. Cambridge: Cambridge University Press.
Papert, S. (1996). An exploration in the space of mathematics educations. International Journal of Computers for Mathematical Learning, 1(1), 95–123.
Picard, R. W. (1995). Affective computing. Cambridge, MA: M.I.T.
Picard, R. W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., et al. (2004). Affective learning—a manifesto. BT Technology Journal, 22(4), 253–269.
Picard, R. W., Vyzas, E., & Healey, J. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), 1175–1191.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178.
Shen, L. P., & Shen, R. M. (2005). Ontology-based intelligent learning content recommendation service. International Journal of Continuing Engineering Education and Life-Long Learning, 15(3–6), 308–317.
Sundström, P. (2005). Exploring the affective loop. Stockholm: Stockholm University.
Thomas, S. (2008). Pervasive scale: A model of pervasive, ubiquitous, and ambient learning. IEEE Pervasive Computing, 7(1), 85–88.
Vyzas, E., & Picard, R. W. (1999). Offline and online recognition of emotion expression from physiological data, The Third International Conference on Autonomous Agents. Seattle, WA.
Acknowledgements
Affective e-Learning is a collaborative research project conducted between the Digital Lifestyles Center, University of Essex, UK (http://digital-lifestyles.essex.ac.uk) and e-Learning Center, Shanghai Jiaotong University, China (http://www.dlc.sjtu.edu.cn). During the formation of the research ideas and models, we have enjoyed many discussions with our colleagues whose intellectual contributions we are most grateful for. We would especially like to thank Michael Gardner for his support and encouragement which made the visit to Essex possible, Chris Fowler for the discussion of learning psychology, Enrique Leon for his excellent valence recognition work, Malcolm Lear for the design of the early emotion sensing prototypes and Bernard Horan of Sun Microsystems for use of the wearable computing Sun-SPOT technology. This work was carried out as part of the ‘Research on Context-aware Multi-modal Interaction Model and Key Technologies’ Project Supported by the High-tech Research and Development Program of China (National 863 project) under Grant No. 2007AA01Z157.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shen, L., Callaghan, V. & Shen, R. Affective e-Learning in residential and pervasive computing environments. Inf Syst Front 10, 461–472 (2008). https://doi.org/10.1007/s10796-008-9104-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-008-9104-5