Skip to main content
SpringerLink
Log in

Multimodal Serious Games Technologies for Cultural Heritage

  • Chapter
  • First Online:
Mixed Reality and Gamification for Cultural Heritage

Abstract

This chapter describes how multimodal serious games can create an immersive experience to enhance the visitor’s experience. The creation of more engaging digital heritage exhibitions by seamlessly integrating technologies to provide a multimodal virtual and augmented reality experience is presented. In engaging exhibitions, participants can switch between different modes of exploring the physical artefacts at the museum and can explore these artefacts further through serious games, user interfaces, virtual reality and augmented reality. Different types of interaction paradigms are also illustrated. Moreover, a framework for multimodal cultural heritage is proposed based on the above technologies. Finally, future research directions for creating new opportunities for scientific research are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S. Arnab, P. Petridis, I. Dunwell, S. de Freitas, in Touching Artefacts in an Ancient World on a Browser-Based Platform, ed. by Y. Xiao, T. Amon, R. Muffoletto. Proceedings of the IADIS International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing. International Association for Development of the Information Society (IADIS), 2010.

    Google Scholar 

  2. D. Kolb, Experiential Learning: Experience as the Source of Learning and Development (Prentice Hall, Inc., New Jersey, 1984)

    Google Scholar 

  3. S. de Freitas, T. Neumann, The use of ‘exploratory learning’ for supporting immersive learning in virtual environments. Comput. Educ. 52(2), 343–352 (2009)

    Article  Google Scholar 

  4. S. Warburton, in Defining a Framework for Teaching Practices Inside Virtual Immersive Environments: The Tension Between Control and Pedagogical Approach. Proceedings of RELive ‘08 Conference, 2008

    Google Scholar 

  5. B. Parker, F. Myrick, A critical examination of high-fidelity human patient simulation within the context of nursing pedagogy. Nurse Educ. Today 29(3), 322–329 (2009)

    Article  Google Scholar 

  6. J. McGonigal, Reality Is Broken: Why Games Make us Better and How They Can Change the World (Jonathan Cape, London, 2008)

    Google Scholar 

  7. R. Garris, R. Ahlers, J.E. Driskell, Games, motivation, and learning: a research and practice model. Simul. Gaming 33, 441–467 (2002)

    Article  Google Scholar 

  8. E.F. Anderson, L. McLoughlin et al., in Serious Games in Cultural Heritage. Proceedings of the 10th VAST International Symposium on Virtual Reality, Archaeology and Cultural Heritage—STARs Session, Eurographics, Malta, pp. 29–48, 2009

    Google Scholar 

  9. J. Hamari, J. Koivisto, H. Sarsa, in Does Gamification Work?—A Literature Review of Empirical Studies on Gamification. Proceedings of the 47th Hawaii International Conference on System Sciences, Hawaii, USA, 2014

    Google Scholar 

  10. M. Zyda, From visual simulation to virtual reality to games. IEEE Comput. 38(9), 25–32 (2005)

    Article  Google Scholar 

  11. B. Sawyer, in Serious Games: Improving Public Policy Through Game-Based Learning and Simulation. Whitepaper for the Woodrow Wilson International Center for Scholars, 2002

    Google Scholar 

  12. D. Panzoli, A. Qureshi et al., Levels of interaction (LoI): a model for scaffolding learner engagement in an immersive environment. Intelligent tutoring systems. Lect. Notes Comput. Sci. 6095, 393–395 (2010)

    Article  Google Scholar 

  13. S. de Freitas, Game for change. Nature 470(7334), 330–331 (2011)

    Article  Google Scholar 

  14. P.D. Mautone, V.A. Spiker, M.R. Karp, in Using Serious Game Technology to Improve Aircrew Training. Proceedings of the Interservice/Industry Training, Simulation and Education Conference (IITSEC) December 2008, Orlando, Florida, 2008

    Google Scholar 

  15. J. McGonigal, Reality Is Broken: Why Games Make us Better and How They Can Change the World (Jonathan Cape, London, 2011)

    Google Scholar 

  16. S. Deterding, R. Khaled et al., Gamification: Towards a Definition. CHI 2011 Gamification Workshop Proceedings, Vancouver, BC, Canada, 2011

    Google Scholar 

  17. S. Deterding, The lens of intrinsic skill atoms: a method for gameful design. Human Comput. Interact. 30, 294–335 (2015)

    Article  Google Scholar 

  18. O. Korn, Industrial Playgrounds: How Gamification Helps to Enrich Work for Elderly or Impaired Persons in Production. Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 313–316, 2012

    Google Scholar 

  19. Gartner Group, Press Release. Gartner Says by 2015, More Than 50 Percent of Organizations that Manage Innovation Processes Will Gamify Those Processes (2011), Available from http://www.gartner.com/newsroom/id/1629214

  20. G. Zichermann, C. Cunningham, Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps, 1st edn. (O’Reilly Media, Sebastopol, CA, 2011)

    Google Scholar 

  21. K. Werbach, D. Hunter, For the Win: How Game Thinking Can Revolutionize Your Business” (Wharton Digital Press, Philadelphia, PA, 2012)

    Google Scholar 

  22. K. Seaborn, D.I. Fels, Gamification in theory and action: a survey. Int. J. Human Comput. Stud. 74, 14–31 (2015)

    Article  Google Scholar 

  23. D. Bowman, E. Kruijff, J. LaViola, I. Poupyrev, 3D user interface output hardware, in 3D User Interfaces: Theory and Practice (Addison Wesley, Boston, MA, 2004), pp. 29–86

    Google Scholar 

  24. D. Bowman, E. Kruijff, J. LaViola, I. Poupyrev, 3D user interface input hardware, in 3D User Interfaces: Theory and Practice (Addison Wesley, Boston, MA, 2004), pp. 87–134

    Google Scholar 

  25. D. Bowman, E. Kruijff, J. LaViola, I. Poupyrev, An introduction to 3D user interface design. Presence Teleop. Virt. Environ. 10(1), 96–108 (2001)

    Article  Google Scholar 

  26. K. Herndon, A. Van Dam, M. Gleicher, The challenges of 3D interaction. Siggraph Bull. 26(4), 36–43 (1994)

    Google Scholar 

  27. D. Aliakseyeu, J.B. Martens, S. Subramanian, M. Rauterberg, in Interaction Techniques for Navigation through and Manipulation of 2D and 3D Data. Proceedings of Eight Eurographics Conference on Virtual Environments, 2002

    Google Scholar 

  28. C.A. Wuthrich, in An Analysis and a Model of 3D Interaction Methods and Devices for Virtual Reality. Proceedings of the Eurographics Workshop, 1999

    Google Scholar 

  29. S. Subramanian, D. Aliakseyeu, J.B. Martens, in Empirical Evaluation of Performance in Hybrid 3D and 2D Interfaces. Proceedings of Human Computer Interaction—Interact’03, pp. 916–919, 2003

    Google Scholar 

  30. P. Petridis, K. Mania, D. Pletinckx, M. White, in Usability Evaluation of the EPOCH Multimodal User Interface: Designing 3D Tangible Interactions. Proceedings of the Interactive Technologies and Sociotechnical Systems: Lecture Notes in Computer Science, 2006

    Google Scholar 

  31. P. Petridis, M. White et al., in Exploring and Interacting with Virtual Museums. CAA 2005: The World in Your Eyes, Tomar, Portugal, 2005

    Google Scholar 

  32. J. Packer, R. Ballantyne, Motivational factors and the visitor experience: a comparison of three sites. Curator Mus. J. 45(3), 183–198 (2002)

    Article  Google Scholar 

  33. S. Bayne, J. Ross, Z. Williamson, Objects, subjects, bits and bytes: learning from the digital collections of the National Museums. Mus. Soc. J. 7(2), 110–124 (2009)

    Google Scholar 

  34. N. Rahim, T.S.M.T. Wook, A.M. Zin, in Developing Conceptual Model of Virtual Museum Environment Based on User Interaction Issues, ed. by H.B. Zaman, P. Robinson, M. Petrou, P. Olivier, T.K. Shih. Proceedings of the Second International Conference on Visual Informatics: Sustaining Research And Innovations—Volume Part II (IVIC’11) (Springer, Berlin, Heidelberg, 2011), pp. 253–260

    Google Scholar 

  35. The Virtual Egyptian Antiquity Museum, (1 December 2007). Available from http://www.touregypt.net/museum/

  36. J. Jacobson, in The Virtual Egyptian Temple. ED-MEDIA: Proceedings of the World Conference on Educational Media, Hypermedia and Telecommunications, 2005

    Google Scholar 

  37. A. Santangelo, A. Augello et al., in A Chat-Bot Based Multimodal Virtual Guide for Cultural Heritage Tours. Proceedings of the 2006 World Congress in Computer Science Computer Engineering and Applied Computing, Las Vegas, Nevada, 2006

    Google Scholar 

  38. J. Cosmas, T. Itegaki et al., in 3D MURALE: A Multimedia System for Archaeology. Proceedings of the ACM-SIGGRAPH Conference on Virtual Reality, Archaeology and Cultural Heritage, Athens, Greece, pp. 297–306, 2001

    Google Scholar 

  39. D. Pletinckx, D. Callebaut et al., Virtual-reality heritage presentation at Ename. IEEE Multimedia 7(2), 45–48 (2000)

    Article  Google Scholar 

  40. A. Gaitatzes, D. Christopoulos, G. Papaioannou, in The Ancient Olympic Games: Being Part of the Experience. Proceedings of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, pp. 19–28, 2004

    Google Scholar 

  41. R.T. Azuma, A survey of augmented reality. Presence Teleop. Virt. Environ. 6(4), 355–385 (1997)

    Article  Google Scholar 

  42. D. Stricker, P. Daehne et al., in Design and Development Issues for ARCHEOGUIDE: An Augmented Reality Based Cultural Heritage On-Site Guide. Proceedings of the International Conference on Augmented, Virtual Environments and Three-Dimensional Imaging, IEEE Computer Society, 2001

    Google Scholar 

  43. G. Papagiannakis, M. Ponder et al., in LIFEPLUS: Revival of Life in Ancient Pompeii. Proceedings of Virtual Systems and Multimedia (VSMM 2002), 2002

    Google Scholar 

  44. M. White, N. Mourkoussis et al., in ARCO—An Architecture for Digitization, Management and Presentation of Virtual Exhibitions. Proceedings of the CGI’2004 Conference, Hersonissos, Crete, June 2004 (IEEE Computer Society, Los Alamitos, CA, 2004), pp. 622–625

    Google Scholar 

  45. F. Liarokapis, M. White, Augmented reality techniques for museum environments. Mediterr. J. Comput. Netw. 1(2), 90–96 (2005)

    Google Scholar 

  46. M. Zoellner, A. Pagani et al., in Reality Filtering: A Visual Time Machine in Augmented Reality. Proceedings of the 9th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, Eurographics, Braga, Portugal, 2–5 December, pp. 71–77, 2008

    Google Scholar 

  47. I. Herbst, A.K. Braun, R. McCall, W. Broll, TimeWarp: Interactive Time Travel with a Mobile Mixed Reality Game. Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2008), ACM Press, pp. 235–244, 2008

    Google Scholar 

  48. D.I. Han, T. Jung, A. Gibson, Dublin AR: implementing augmented reality in tourism, in Information and Communication Technologies in Tourism 2014 (Springer, Cham, 2014), pp. 511–523

    Google Scholar 

  49. P. Kourouthanassis, C. Boletsis, C. Bardaki, D. Chasanidou, Tourists responses to mobile augmented reality travel guides: the role of emotions on adoption behavior. Pervasive Mob. Comput. Elsevier 18, 71–87 (2015)

    Google Scholar 

  50. D. Střelák, F. Škola, F. Liarokapis, in Examining User Experiences in a Mobile Augmented Reality Tourist Guide. Proceedings of the 9th International Conference on PErvasive Technologies Related to Assistive Environments (Petra 2016), ACM Press, Corfu Island, Greece, 29 June–1 July, 2016

    Google Scholar 

  51. S. Arnab, P. Petridis, I. Dunwell, S. de Freitas, Enhancing learning in distributed virtual worlds through touch: a browser-based architecture for haptic interaction, in Serious Games and Edutainment Applications, ed. by M. Ma, A. Oikonomou, L.C. Jain (Springer, London, 2011), pp. 149–167

    Chapter  Google Scholar 

  52. T.E. Roden, I. Parberry, D. Ducrest, Toward mobile entertainment: a paradigm for narrative-based audio only games. Science of computer programming. Spec. Issue Aspects Game Program. Elsevier 67(1), 76–90 (2007)

    Google Scholar 

  53. S. Sanei, J.A. Chambers, EEG Signal Processing (Wiley, Hoboken, NJ, 2013)

    Google Scholar 

  54. J. Knezik, M. Drahansky, Simple EEG driven mouse cursor movement. Comput. Recognit. Syst. 2 45, 526–531 (2007)

    Article  Google Scholar 

  55. M. Ninaus, S.E. Kober et al., Neurophysiological methods for monitoring brain activity in serious games and virtual environments: a review. Int. J. Technol. Enhanc. Learn. 6(1), 78–103 (2014)

    Article  Google Scholar 

  56. G. Rebodello-Mendez, I. Dunwell et al., in Assessing Neurosky’s Usability to Detect Attention Levels in an Assessment Exercise. Proceedings of the 13th International Conference on Human-Computer Interaction, Springer, Berlin, Heidelberg. Lecture Notes in Computer Science, vol 5610/2009, San Diego, CA, USA, 19–24 July, pp. 149–158, 2009

    Google Scholar 

  57. A. Vourvopoulos, F. Liarokapis, P. Petridis, in Brain-Controlled Serious Games for Cultural Heritage. Proceedings of the 18th International Conference on Virtual Systems and Multimedia, Virtual Systems in the Information Society, IEEE Computer Society, Milan, Italy, 2–5 September, pp. 291–298, 2012

    Google Scholar 

  58. F. Liarokapis, K. Debattista, A. Vourvopoulos, P. Petridis, A. Ene, Comparing interaction techniques for serious games through brain-computer interfaces: a user perception evaluation study. Entertain. Comput. Elsevier 5(4), 391–399 (2014)

    Article  Google Scholar 

  59. S. Oviatt, Multimodal interfaces, in Handbook of Human-Computer Interaction (Lawrence Erlbaum, New Jersey, 2002)

    Google Scholar 

  60. M. White, P. Petridis, F. Liarokapis, D. Plecinckx, Multimodal mixed reality interfaces for visualizing digital heritage. Int. J. Archit. Comput. (IJAC) Special Iss. Cult Herit Multi-Sci 5(2), 322–337 (2008)

    Google Scholar 

  61. S.A. Brewster, Impact of haptic ‘touching’ technology on cultural applications, in Digital Applications for Cultural Heritage Institutions, ed. by J. Hemsley, V. Cappellini, G. Stanke (Ashgate, Burlington, VT, 2005), pp. 273–284

    Google Scholar 

  62. A. Frisoli, The Museum of Pure Form (2007), Available from http://www.pureform.org/

  63. B. Frischer, The Rome Reborn Project. How Technology Is Helping us to Study History, OpEd (University of Virginia, 2008)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Human–Computer Interaction Laboratory members for their support and inspiration.

Author information

Authors and Affiliations

  1. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Fotis Liarokapis

  2. Business School, Aston University, Birmingham, UK

    Panagiotis Petridis & Daniel Andrews

  3. Murdoch University, Murdoch, WA, Australia

    Sara de Freitas

Authors
  1. Fotis Liarokapis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Panagiotis Petridis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Andrews
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sara de Freitas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fotis Liarokapis .

Editor information

Editors and Affiliations

  1. Digital Heritage Research, Cyprus University of Technology Digital Heritage Research, Limassol, Cyprus

    Marinos Ioannides

  2. University of Geneva , Carouge, Switzerland

    Nadia Magnenat-Thalmann

  3. Department of Computer Science, University of Crete & FORTH Department of Computer Science, Heraklion, Greece

    George Papagiannakis

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Liarokapis, F., Petridis, P., Andrews, D., de Freitas, S. (2017). Multimodal Serious Games Technologies for Cultural Heritage. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds) Mixed Reality and Gamification for Cultural Heritage. Springer, Cham. https://doi.org/10.1007/978-3-319-49607-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49607-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49606-1

  • Online ISBN: 978-3-319-49607-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation