Skip to main content
SpringerLink
Log in

Cross-objects user interfaces for video interaction in virtual reality museum context

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Museums are good places for learning and nowadays many museums are integrating digital media such as video and increasingly moving towards using virtual reality. In the physical world people used to seek information from object surfaces e.g. posters on the wall and this has been used as a metaphor in the virtual reality museum: numerous videos were inhabited within virtual objects and shaped cross-objects user interfaces (COUIs). However, how such interfaces perform for video interactions still needs more investigations. In this study we implemented and investigated COUIs in comparison with the conventional card-style user interfaces and the plain virtual reality user interfaces in the virtual reality museum. The results reported no significant differences in the perceived usability or learning experience between these user interfaces, except the COUIs had a lower level of satisfaction than the card-style user interfaces. However, the COUIs showed greater efficiency with shorter eye fixation durations and higher saccade frequencies, and within these COUIs instances, namely the fully-detached, semi-attached, and fully-attached COUIs, the fully-attached instance was closest to the form of interacting with physical object surfaces and it reported highest efficiency as well. Rationales behind these results and implications generalising for the future design of COUIs, are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Aarts E, Wichert R (2009) Ambient intelligence. In: Technology guide. Springer, pp 244–249

  2. Alexandri E, Tzanavara A (2014) New technologies in the service of museum education. World Transactions on Engineering and Technology Education 12(2):317–320

    Google Scholar 

  3. Baraldi L, Paci F, Serra G, Benini L, Cucchiara R (2015) Gesture recognition using wearable vision sensors to enhance visitors museum experiences. IEEE Sensors J 15(5):2705–2714

    Google Scholar 

  4. Barbieri L, Bruno F, Muzzupappa M (2017) User-centered design of a virtual reality exhibit for archaeological museums. International Journal on Interactive Design and Manufacturing (IJIDeM)

  5. Beer S (2015) Digital heritage museums and virtual museums. Paper presented at the Proceedings of the 2015 Virtual Reality International Conference, Laval, France,

  6. Beer S (2015) Virtual museums: an innovative kind of museum survey. Paper presented at the Proceedings of the 2015 Virtual Reality International Conference, Laval, France

  7. Branch P, Egan G, Tonkin B (1999) Modeling interactive behaviour of a video based multimedia system. In: Communications, 1999. ICC'99. 1999 IEEE International Conference on. IEEE, pp 978–982

  8. Brooke J (1996) SUS-A quick and dirty usability scale. Usability evaluation in industry, pp 189–194

  9. Carrozzino M, Bergamasco M (2010) Beyond virtual museums: experiencing immersive virtual reality in real museums. J Cult Herit 11(4):452–458

    Article  Google Scholar 

  10. Chang C-M, Huang Y (2013) ArtTorchlight: an exploratory way of viewing paintings using handheld projectors and smart phones. Paper presented at the SIGGRAPH Asia 2013 Posters, Hong Kong, Hong Kong

  11. Ciolfi L, Damala A, Hornecker E, Lechner M, Maye L, Petrelli D (2015) Cultural heritage communities: technologies and challenges. In: Proceedings of the 7th international conference on communities and technologies. ACM, pp 149–152

  12. Clini P, Frontoni E, Quattrini R, Pierdicca R (2014) Augmented reality experience: from high-resolution acquisition to real time augmented contents. Advances in Multimedia 2014:18

    Article  Google Scholar 

  13. Cuendet S, Dehler-Zufferey J, Ortoleva G, Dillenbourg P (2015) An integrated way of using a tangible user interface in a classroom. Int J Comput-Support Collab Learn 10(2):183–208

    Google Scholar 

  14. Davis MM, Gabbard JL, Bowman DA, Gracanin D (2016) Depth-based 3D gesture multi-level radial menu for virtual object manipulation. In: Virtual reality (VR), 2016 IEEE. IEEE, pp 169–170

  15. Desai PR, Desai PN, Ajmera KD, Mehta K (2014) A review paper on oculus rift-a virtual reality headset. arXiv preprint arXiv:14081173

    Google Scholar 

  16. Deuschel T, Heuss T, Broomfield C (2014) The design scope of adaptive storytelling in virtual museums. In: Proceedings of the eurographics workshop on graphics and cultural heritage. Eurographics Association, pp 97–106

  17. Díaz P, Bellucci A, Aedo I (2015) Enabling social interaction in the museum through the social display environment. In: Digital heritage. IEEE, pp 345–348

  18. Falk JH, Dierking LD (2016) The museum experience revisited. Routledge

  19. Fasel B, Gool LV (2007) Interactive museum guide: accurate retrieval of object descriptions. Paper presented at the Proceedings of the 4th international conference on adaptive multimedia retrieval: user, context, and feedback, Geneva, Switzerland

  20. Fukkink RG, Tavecchio LW (2010) Effects of video interaction guidance on early childhood teachers. Teach Teach Educ 26(8):1652–1659

    Article  Google Scholar 

  21. Ghinea G, Chen SY (2008) Measuring quality of perception in distributed multimedia: verbalizers vs. imagers. Comput Hum Behav 24(4):1317–1329

    Article  Google Scholar 

  22. Hou H-T, Sheng-Yi W, Peng-Chun L, Yao-Ting S, Lin J-W, Chang K-E (2014) A blended mobile learning environment for museum learning. Journal of Educational Technology & Society 17(2)

  23. Hsu C-H, Wu Y-L, Cheng W-H, Chen Y-J, Hua K-L (2017) HoloTube: a low-cost portable 360-degree interactive autostereoscopic display. Multimed Tools Appl 76(7):9099–9132

    Article  Google Scholar 

  24. Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human factors in computing systems. ACM, pp 234–241

  25. Kim J, Li S-W, Cai CJ, Gajos KZ, Miller RC (2014) Leveraging video interaction data and content analysis to improve video learning. In: Proceedings of the CHI 2014 Learning Innovation at Scale workshop

  26. Kiourt C, Koutsoudis A, Pavlidis G (2016) DynaMus: a fully dynamic 3D virtual museum framework. J Cult Herit 22:984–991

    Article  Google Scholar 

  27. Koskenranta O, Colley A, Häkkilä J (2013) Portable CAVE using a mobile projector. Paper presented at the Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, Zurich, Switzerland

  28. Lee J, Cheon M, Moon S-E, Lee J-S (2016) Peripersonal space in virtual reality: navigating 3D space with different perspectives. In: Proceedings of the 29th annual symposium on user interface software and technology. ACM, pp 207–208

  29. Luyten K, Coninx K (2005) Distributed user interface elements to support smart interaction spaces. In: Multimedia, Seventh IEEE International Symposium on. IEEE, p 8

  30. McAllister G, White GR (2015) Video game development and user experience. In: Game user experience evaluation. Springer, pp 11–35

  31. Nguyen C, DiVerdi S, Hertzmann A, Liu F (2017) Vremiere: in-headset virtual reality video editing. In: CHI 2017, Denver CO USA, May 6–11 2017. Virtual reality. ACM

  32. Pallavicini F, Ferrari A, Zini A, Garcea G, Zanacchi A, Barone G, Mantovani F (2018) What distinguishes a traditional gaming experience from one in virtual reality? An exploratory study. In: Ahram T, Falcão C (eds) advances in human factors in wearable technologies and game design: proceedings of the AHFE 2017 international conference on advances in human factors and wearable technologies, July 17–21, 2017, The Westin Bonaventure Hotel, Los Angeles, California, USA. Springer International Publishing, Cham, pp 225–231

    Google Scholar 

  33. Papaefthymiou M, Plelis K, Mavromatis D, Papagiannakis G (2015) Mobile virtual reality featuring a six degrees of freedom interaction paradigm in a virtual museum application. Technical Report, FORTH-ICS/TR-462. Foundation for Research and Technology–Hellas (FORTH), Institute of Computer Science, Heraklion

    Google Scholar 

  34. Partarakis N, Antona M, Zidianakis E, Stephanidis C (2016) Adaptation and content personalization in the context of multi user museum exhibits. In: AVI*CH, pp 5–10

  35. Procter R, Hartswood M, McKinlay A, Gallacher S (1999) An investigation of the influence of network quality of service on the effectiveness of multimedia communication. In: Proceedings of the international ACM SIGGROUP conference on Supporting group work. ACM, pp 160–168

  36. Ragan ED, Bowman DA, Kopper R, Stinson C, Scerbo S, McMahan RP (2015) Effects of field of view and visual complexity on virtual reality training effectiveness for a visual scanning task. IEEE Trans Vis Comput Graph 21(7):794–807

    Article  Google Scholar 

  37. Rieh SY, Collins-Thompson K, Hansen P, Lee H-J (2016) Towards searching as a learning process: a review of current perspectives and future directions. J Inf Sci 42(1):19–34

    Article  Google Scholar 

  38. Robles-Ortega MD, Feito FR, Jiménez JJ, Segura RJ (2012) Web technologies applied to virtual heritage: an example of an Iberian Art Museum. J Cult Herit 13(3):326–331

    Article  Google Scholar 

  39. Schaper M-M, Santos M, Malinverni L, Pares N (2017) Towards the design of a virtual heritage experience based on the world-as-support interaction paradigm. Paper presented at the Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, Denver, Colorado, USA

  40. Seidel N (2014) Interaction design patterns for design and development of video learning environments. Paper presented at the Proceedings of the 19th European Conference on Pattern Languages of Programs, Irsee, Germany

  41. Sidorakis N, Koulieris GA, Mania K (2015) Binocular eye-tracking for the control of a 3D immersive multimedia user interface. In: Everyday virtual reality (WEVR), 2015 IEEE 1st Workshop on. IEEE, pp 15–18

  42. Stawniak M, Walczak K (2006) Geographical presentation of virtual museum exhibitions. Paper presented at the Proceedings of the 7th International conference on Virtual Reality, Archaeology and Intelligent Cultural Heritage, Nicosia, Cyprus

  43. Styliani S, Fotis L, Kostas K, Petros P (2009) Virtual museums, a survey and some issues for consideration. J Cult Herit 10(4):520–528

    Article  Google Scholar 

  44. Su Y-C, Grauman K (2016) Detecting engagement in egocentric video. In: European conference on computer vision. Springer, pp 454–471

  45. Sylaiou S, Economou M, Karoulis A, White M (2008) The evaluation of ARCO: a lesson in curatorial competence and intuition with new technology. Comput Entertain 6(2):1–18

    Article  Google Scholar 

  46. Toril P, Reales JM, Ballesteros S (2014) Video game training enhances cognition of older adults: a meta-analytic study. Psychol Aging 29(3):706–716

    Article  Google Scholar 

  47. van der Meij H (2017) Reviews in instructional video. Comput Educ 114:164–174

    Article  Google Scholar 

  48. Wang RG, Dong HL, Wu MW, Wu QF (2014) Research and design of digital museum based on virtual reality. In: Advanced materials research. Trans Tech Publ, pp 2516–2520

  49. Weiser M (1993) Some computer science issues in ubiquitous computing. Commun ACM 36(7):75–84

    Article  Google Scholar 

  50. Wibirama S, Nugroho HA, Hamamoto K (2017) Evaluating 3D gaze tracking in virtual space: a computer graphics approach. Entertainment Computing 21:11–17

    Article  Google Scholar 

  51. Wisneski CA (1999) The design of personal ambient displays. Massachusetts Institute of Technology

  52. Wyman SM, Waldo JT, Doherty D (2016) Methods and models for museum learning at the Samuel Dorsky Museum of Art. Journal for Learning Through the Arts 12(1):n1

    Google Scholar 

  53. Yoo B, Han J-J, Choi C, Yi K, Suh S, Park D, Kim C (2010) 3D user interface combining gaze and hand gestures for large-scale display. In: CHI'10 extended abstracts on human factors in computing systems. ACM, pp 3709–3714

  54. Yoshimura Y, Sobolevsky S, Ratti C, Girardin F, Carrascal JP, Blat J, Sinatra R (2014) An analysis of visitors' behavior in the Louvre Museum: a study using Bluetooth data. Environ Plann B Plann Des 41(6):1113–1131

    Article  Google Scholar 

  55. Zhang T, Chowdhery A, Bahl PV, Jamieson K, Banerjee S (2015) The design and implementation of a wireless video surveillance system. In: Proceedings of the 21st annual international conference on mobile computing and networking. ACM, pp 426–438

Download references

Acknowledgements

The authors thank the reviewers for their helpful comments. The research is supported by the funding of “National Key R&D program” (2016YFB1001304).

Author information

Authors and Affiliations

  1. International Design Institute, Zhejiang University, Hangzhou, People’s Republic of China

    Lingyun Sun & Yunzhan Zhou

  2. Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Hangzhou, People’s Republic of China

    Yunzhan Zhou & Xiangdong Li

  3. Department of Computer Science and Systems, Stockholm University, Kista, Sweden

    Preben Hansen

  4. Department of Digital Media, Zhejiang University, Hangzhou, People’s Republic of China

    Weidong Geng & Xiangdong Li

Authors
  1. Lingyun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunzhan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Preben Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weidong Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangdong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangdong Li.

Electronic supplementary material

ESM 1

(GIF 1902 kb)

Appendices

Appendix 1

Questionnaire 1 – Early Experience and Attitudes

  1. 1.

    I have been to museums.

1 (never)---2 --- 3 --- 4 --- 5 (everyday)

  1. 2.

    I think the museum is a good place of learning.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 3.

    I think the exhibition and interaction in the museum help the learning.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 4.

    I have used virtual reality devices to learn some topics.

1 (never)---2 --- 3 --- 4 --- 5 (everyday)

  1. 5.

    I think virtual reality technology will enhance the learning effect in museums.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

Appendix 2

Questionnaire 2 – Perceived Usability (SUS)

  1. 1.

    I am willing to use this VR-based museum for learning.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 2.

    I feel like this museum scene is complicated.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 3.

    I think this museum is easy to use.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 4.

    I need some help to use this museum and related user interfaces.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 5.

    I think this museum and its scenes have well designed functions.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 6.

    I think there are some inconsistent features in this museum.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 7.

    I believe most people can learn to use this museum quickly.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 8.

    I think the use of this museum is troublesome.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 9.

    I am confident I can use this museum.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 10.

    I need to learn something before I can master this virtual reality museum.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

Appendix 3

Questionnaire 3 – learning experience

  1. 1.

    I feel like the use of this virtual reality museum is smooth and satisfied.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 2.

    I feel like the virtual reality museum helps me understand the exhibitions and the environments.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 3.

    I think the virtual reality museum enhances my willingness of learning.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 4.

    I feel like I have learnt some new stuffs from this virtual reality museum and related scenes.

1 (strongly disagree)---2 --- 3 --- 4 --- 5 (strongly agree)

  1. 5.

    Please use keywords to describe the details of the virtual reality museum

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, L., Zhou, Y., Hansen, P. et al. Cross-objects user interfaces for video interaction in virtual reality museum context. Multimed Tools Appl 77, 29013–29041 (2018). https://doi.org/10.1007/s11042-018-6091-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6091-5

Keywords

Search

Navigation