Skip to main content
SpringerLink
Log in

Challenges in the Development and Evaluation of Immersive Digital Educational Games

  • Conference paper
Book cover HCI and Usability for Education and Work (USAB 2008)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5298))

Included in the following conference series:

Abstract

In this paper, we describe a conceptual framework and address the related issues and solutions in the identification of three major challenges for the development and evaluation of Immersive Digital Educational Games (IDEGs). These challenges are (i) advancing adaptive educational technologies to shape learning experience, ensuring the individualization of learning experiences, adaptation to personal aims, needs, abilities and prerequisites; (ii) providing technological approaches to reduce the development costs for IDEGs; by enabling the creation of entirely different stories and games for a variety of different learning domains, each based, more or less, on the same pool of story units; patterns and structures; (iii) developing robust evaluation methodologies for IDEGs by the extension of ISO 9241 to include user satisfaction, motivation and learning progress and other User Experience (UX) attributes. While our research and development is by no means concluded, we believe that we have arrived at a stage where conclusions may be drawn, which will be of considerable use to other researchers in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Albert, D., Lukas, J.: Knowledge spaces: theories, empirical research, and applications. Lawrence Erlbaum Associates, Mahwah (1999)

    Google Scholar 

  2. Holzinger, A., Pichler, A., Almer, W., Maurer, H.: TRIANGLE: A Multi-Media test-bed for examining incidental learning, motivation and the Tamagotchi-Effect within a Game-Show like Computer Based Learning Module Educational Multimedia, Hypermedia and Telecommunication 2001, Association for the Advancement of Computing in Education, Charlottesville (VA), pp. 766–771 (2001)

    Google Scholar 

  3. Mann, B.D., Eidelson, B.M., Fukuchi, S.G., Nissman, S.A., Robertson, S., Jardines, L.: The development of an interactive game-based tool for learning surgical management algorithms via computer. The American Journal of Surgery 183(3), 305–308 (2002)

    Article  Google Scholar 

  4. Cai, Y., Snel, I., Bharathi, B.S., Klein, C., Klein-Seetharaman, J.: Towards biomedical problem solving in a game environment. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J., Zomaya, A.Y. (eds.) ICCS 2003. LNCS, vol. 2659, pp. 1005–1014. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  5. Holzinger, A., Pichler, A., Maurer, H.: Multi Media e-Learning Software TRIANGLE Case-Study: Experimental Results and Lessons Learned. Journal of Universal Science and Technology of Learning 0(0), 61–92 (2006), http://www.justl.org/justl_0_0/multi_media_elearning_software/justl_0_0_0061_0092_holzinger.pdf

    Google Scholar 

  6. de Freitas, S., Oliver, M.: How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers & Education 46(3), 249–264 (2006)

    Article  Google Scholar 

  7. Ebner, M., Holzinger, A.: Successful Implementation of User-Centered Game Based Learning in Higher Education – an Example from Civil Engineering. Computers & Education 49(3), 873–890 (2007)

    Article  Google Scholar 

  8. Robertson, J., Howells, C.: Computer game design: Opportunities for successful learning. Computers & Education 50(2), 559–578 (2008)

    Article  Google Scholar 

  9. Garris, R., Ahlers, R., Driskell, J.E.: Games, Motivation, and Learning: A Research and Practice Model. Simulation & Gaming 33(4), 441–467 (2002)

    Article  Google Scholar 

  10. Prensky, M.: Digital game-based Learning. McGraw Hill, New York (2001)

    Google Scholar 

  11. Merrill, M.D.: First principles of instruction. Etr&D-Educational Technology Research and Development 50(3), 43–59 (2002)

    Article  Google Scholar 

  12. Cordova, D.I., Lepper, M.R.: Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology 88(4), 715–730 (1996)

    Article  Google Scholar 

  13. Holzinger, A.: Computer-aided Mathematics Instruction with Mathematica 3.0. Mathematica in Education and Research 6(4), 37–40 (1997)

    Google Scholar 

  14. Shneiderman, B.: Relate-Create-Donate: a teaching/learning philosophy for the cyber-generation. Computers & Education 31(1), 25–39 (1998)

    Article  Google Scholar 

  15. van Reekum, C.M., Johnstone, T., Banse, R., Etter, A., Wehrle, T., Scherer, K.R.: Psychophysiological responses to appraisal dimensions in a computer game. Cognition & Emotion 18(5), 663–688 (2004)

    Article  Google Scholar 

  16. ELEKTRA: ELEKTRA - Enhanced Learning Experience and Knowledge TRAnsfer (last access: 2008-09-06), http://www.elektra-project.org

  17. Aviram, R., Winer, A., Somech, S., Schellas, Y., Dotan, I., Ronen, Y.: iClass Methodologies: Intelligent Distributed Cognitive-based Open Learning System for Schools (last access: 2008-09-06), http://www.iclass.info/docs/iClass%20Methodologies.pdf

  18. Wouters, P., Tabbers, H.K., Paas, F.: Interactivity in video-based models. Educational Psychology Review 19(3), 327–342 (2007)

    Article  Google Scholar 

  19. Entwistle, N., McCune, V.: The conceptual bases of study strategy inventories. Educational Psychology Review 16(4), 325–345 (2004)

    Article  Google Scholar 

  20. Caprara, G.V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G.M., Barbaranelli, C., Bandura, A.: Longitudinal analysis of the role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. Journal of Educational Psychology 100(3), 525–534 (2008)

    Article  Google Scholar 

  21. Boekaerts, M.: Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students. Learning and Instruction 7(2), 161–186 (1997)

    Article  Google Scholar 

  22. Holzinger, A., Kickmeier-Rust, M., Albert, D.: Dynamic Media in Computer Science Education; Content Complexity and Learning Performance: Is Less More? Educational Technology & Society 11(1), 279–290 (2008)

    Google Scholar 

  23. Kickmeier-Rust, M., Albert, D.: The ELEKTRA ontology model: A learner-centered approach to resource description. LNCS. Springer, Heidelberg (in press)

    Google Scholar 

  24. Holzinger, A., Nischelwitzer, A.: Chameleon Learning Objects: Quasi-Intelligente doppelt adaptierende Lernobjekte: Vom Technologiemodell zum Lernmodell. OCG Journal 30(4), 4–6 (2005)

    Google Scholar 

  25. Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behaviour & Information Technology 25(2), 91–97 (2006)

    Article  Google Scholar 

  26. Law, E., Hvannberg, E.T., Hassenzahl, M.: Proceedings of the Workshop Towards a Unified View of UX (last access: 2008-09-10), http://www.cost294.org/

  27. van Schaik, P., Ling, J.: Modelling user experience with web sites: Usability, hedonic value, beauty and goodness. Interacting with Computers 20(3), 419–432 (2008)

    Article  Google Scholar 

  28. Hassenzahl, M., Beu, A., Burmester, M.: Engineering joy. IEEE Software 18(1), 70 (2001)

    Article  Google Scholar 

  29. Harbich, S., Auer, S.: Rater bias: The influence of hedonic quality on usability questionnaires. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 1129–1133. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  30. Schrepp, M., Held, T., Laugwitz, B.: The influence of hedonic quality on the attractiveness of user interfaces of business management software. Interacting with Computers 18(5), 1055–1069 (2006)

    Article  Google Scholar 

  31. Hassenzahl, M., Platz, A., Burmester, M., Lehner, K.: Hedonic and ergonomic quality aspects determine a software’s appeal. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 201–208 (2000)

    Google Scholar 

  32. Wideman, H.H., Owston, R.D., Brown, C., Kushniruk, A., Ho, F., Pitts, K.C.: Unpacking the potential of educational gaming. Simulation & Gaming 38(1), 10–30 (2007)

    Article  Google Scholar 

  33. Holzinger, A.: Multimedia Basics, Laxmi, New Delhi. Learning. Cognitive Fundamentals of multimedial Information Systems, vol. 2 (2002), www.basiswissen-multimedia.at

  34. Holzinger, A.: Usability Engineering for Software Developers. Communications of the ACM 48(1), 71–74 (2005)

    Article  Google Scholar 

  35. Stephanie, R., Janice Anne, R., Judee, H.: A toolkit for strategic usability: results from workshops, panels, and surveys. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 337–344 (2000)

    Google Scholar 

  36. Holzinger, A.: Application of Rapid Prototyping to the User Interface Development for a Virtual Medical Campus. IEEE Software 21(1), 92–99 (2004)

    Article  Google Scholar 

  37. Bernhaupt, R., Eckschlager, M., Tscheligi, M.: Methods for evaluating games: how to measure usability and user experience in games? In: Proceedings of the international conference on Advances in computer entertainment technology (2007)

    Google Scholar 

  38. Fabricatore, C., Nussbaum, M., Rosas, R.: Playability in action videogames: A qualitative design model. Human-Computer Interaction 17(4), 311–368 (2002)

    Article  Google Scholar 

  39. Desurvire, H., Caplan, M., Toth, J.A.: Using heuristics to evaluate the playability of games. In: CHI 2004, pp. 1509–1512 (2004)

    Google Scholar 

  40. Korhonen, H., Koivisto, E.M.I.: Playability Heuristics for Mobile Multi-Player Games. In: DIMEA 2007, Second International Conference on Digital Interactive Media in Entertainment and Arts (2007)

    Google Scholar 

  41. Röcker, C., Haar, M.: Exploring the usability of videogame heuristics for pervasive game development in smart home enviroments. In: Third International Workshop on Pervasive Game Applications - PerGames 2006, pp. 199–206 (2006)

    Google Scholar 

  42. Mandryk, R.L., Atkins, M.S.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human-Computer Studies 65(4), 329–347 (2007)

    Article  Google Scholar 

  43. van den Haak, M., De Jong, M., Schellens, P.J.: Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behaviour & Information Technology 22(5), 339–351 (2003)

    Article  Google Scholar 

  44. Van den Haak, M.J., de Jong, M.D.T., Schellens, P.J.: Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison. Interacting with Computers 16(6), 1153–1170 (2004)

    Article  Google Scholar 

  45. Csíkszentmihályi, M.: Finding flow: The Psychology of engagement with everyday life. Basic Books, London (1998)

    Google Scholar 

  46. de Lera, E., Garreta-Domingo, M.: Ten emotion heuristics: Guidelines for assessing the user’s affective dimension easily and cost effectively. In: Proceedings of HCI 2007 The 21st British HCI Group Annual Conference, University of Lancaster, September 3 - 7 (2007)

    Google Scholar 

  47. Stickel, C., Fink, J., Holzinger, A.: Enhancing Universal Access – EEG based Learnability Assessment. In: Stephanidis, C. (ed.) HCI 2007. LNCS, vol. 4556, pp. 813–822. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  48. Stickel, C., Holzinger, A., Ebner, M.: Useful Oblivion Versus Information Overload in e-Learning Examples in the Context of Wiki Systems. In: Luzar-Stiffler, V., Dobric, V.H., Bekic, Z. (eds.) Proceedings of the ITI 2008 30th International Conference on Information Technology Interface, June 23-26, pp. 171–176 (2008)

    Google Scholar 

  49. Bernhaupt, R., IJsselsteij, W., Mueller, F., Tscheligi, M., Wixon, D.R.: Evaluating user experience games CHI 2008 extended abstracts on Human factors in computing systems, pp. 3905–3908 (2008)

    Google Scholar 

  50. Lockee, B., Moore, M., Burton, J.: Measuring success: Evaluation strategies for distance education. Educause Quaterly 25(1), 20–26 (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Leicester, LE1 7RH, Leicester, UK

    Effie Lai-Chong Law

  2. Department of Psychology Cognitive Science Section, Graz University, A-8010, Graz, Austria

    Michael D. Kickmeier-Rust & Dietrich Albert

  3. Institute for Medical Informatics, Statistics & Documentation (IMI) Research Unit HCI4MED, Medical University Graz, A-8036, Graz, Austria

    Andreas Holzinger

Authors
  1. Effie Lai-Chong Law
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael D. Kickmeier-Rust
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dietrich Albert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andreas Holzinger
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute for Medical Informatics, Statistics and Documentation (IMI) Research Unit HCI4MED, Medical University Graz (MUG), Auenbruggerplatz 2/V, 8036, Graz, Austria

    Andreas Holzinger

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Law, E.LC., Kickmeier-Rust, M.D., Albert, D., Holzinger, A. (2008). Challenges in the Development and Evaluation of Immersive Digital Educational Games. In: Holzinger, A. (eds) HCI and Usability for Education and Work. USAB 2008. Lecture Notes in Computer Science, vol 5298. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89350-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-89350-9_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-89349-3

  • Online ISBN: 978-3-540-89350-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation