Skip to main content
SpringerLink
Log in

Encoding immersive sessions for online, interactive VR analytics

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Capturing and recording immersive VR sessions performed through HMDs in explorative virtual environments may offer valuable insights on users’ behavior, scene saliency and spatial affordances. Collected data can support effort prioritization in 3D modeling workflow or allow fine-tuning of locomotion models for time-constrained experiences. The web with its recent specifications (WebVR/WebXR) represents a valid solution to enable accessible, interactive and usable tools for remote VR analysis of recorded sessions. Performing immersive analytics through common browsers however presents different challenges, including limited rendering capabilities. Furthermore, interactive inspection of large session records is often problematic due to network bandwidth or may involve computationally intensive encoding/decoding routines. This work proposes, formalizes and investigates flexible dynamic models to volumetrically capture user states and scene saliency during running VR sessions using compact approaches. We investigate image-based encoding techniques and layouts targeting interactive and immersive WebVR remote inspection. We performed several experiments to validate and assess proposed encoding models applied to existing records and within networked scenarios through direct server-side encoding, using limited storage and computational resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://immersive-web.github.io/webxr/.

  2. https://sketchfab.com/.

  3. https://www.w3.org/TR/webxr/.

  4. Axis-Aligned Bounding Box.

  5. http://osiris.itabc.cnr.it/scenebaker/index.php/projects/aton/.

  6. “TourismA” 2018 event - http://www.tourisma.it/home-2/.

  7. comma-separated values, ASCII file.

  8. https://nodejs.org/en/.

  9. https://vr.tobii.com/.

  10. https://sketchfab.com/TheHallwylMuseum.

  11. https://www.raspberrypi.org/products/raspberry-pi-3-model-b-plus/.

  12. https://revealvr.eu/.

References

  • Agus M, Marton F, Bettio F, Gobbetti E (2016) Interactive 3d exploration of a virtual sculpture collection: an analysis of user behavior in museum setting. In: Proceedings of the 13th eurographics workshop on graphics and cultural heritage

  • Antal A et al (2016) A complete workflow from the data collection on the field to the deployment of a virtual museum: the case of virtual sarmizegetusa

  • Barsanti SG, Fanini B et al (2018) The winckelmann300 project: Dissemination of culture with virtual reality at the capitoline museum in rome. Int Arch Photogramm Remote Sens Spat Inf Sci 42(2):371–378. https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2/371/2018/

    Article  Google Scholar 

  • Boletsis C (2017) The new era of virtual reality locomotion: a systematic literature review of techniques and a proposed typology. Multimodal Technol Interact 1(4):24

    Article  Google Scholar 

  • Butcher PW, Roberts JC, Ritsos PD (2016) Immersive analytics with WebVR and Google cardboard. Posters of IEEE VIS

  • Butcher PW, John NW, Ritsos PD (2018) Towards a framework for immersive analytics on the web

  • Chen KB, Ponto K, Sesto ME, Radwin RG (2014) Influence of altered visual feedback on neck movement for a virtual reality rehabilitative system. In: Proceedings of the human factors and ergonomics society annual meeting, vol 58. SAGE Publications Sage, Los Angeles, pp 693–697

  • Cliquet G, Perreira M, Picarougne F, Prié Y, Vigier T (2017) Towards HMD-based immersive analytics. In: Immersive analytics workshop, IEEE VIS 2017

  • Cook KA, Thomas JJ (2005) Illuminating the path: the research and development agenda for visual analytics. Technical report, Pacific Northwest National Lab (PNNL), Richland

  • Dibbern C, Uhr M, Krupke D, Steinicke F (2018) Can WebVR further the adoption of virtual reality? Mensch und Computer 2018-Usability Professionals

  • Dworak D, Pietruszka M (2015) Fast encoding of huge 3D data sets in lossless PNG format. In: New research in multimedia and internet systems. Springer, Berlin, pp 15–24

  • Dwyer T, Marriott K, Isenberg T, Klein K, Riche N, Schreiber F, Stuerzlinger W, Thomas BH (2018) Immersive analytics: an introduction. In: Immersive analytics. Springer, Berlin, pp 1–23

  • Fanini B, d’Annibale E (2016) A framework for compact and improved panoramic VR dissemination. In: Proceedings of the 14th eurographics workshop on graphics and cultural heritage, Eurographics Association, pp 33–42

  • Fanini B, Pescarin S, Palombini A (2019) A cloud-based architecture for processing and dissemination of 3D landscapes online. Digital Applications in Archaeology and Cultural Heritage p. e00100

  • Gu X, Gortler SJ, Hoppe H (2002) Geometry images. ACM Trans Graph (TOG) 21(3):355–361

    Article  Google Scholar 

  • Hadjar H, Meziane A, Gherbi R, Setitra I, Aouaa N (2018) WebVR based interactive visualization of open health data. In: Proceedings of the 2nd international conference on web studies. ACM, pp 56–63

  • Jain AK (2010) Data clustering: 50 years beyond k-means. Pattern Recogn Lett 31(8):651–666

    Article  Google Scholar 

  • Jankowski P, Andrienko N, Andrienko G, Kisilevich S (2010) Discovering landmark preferences and movement patterns from photo postings. Trans GIS 14(6):833–852

    Article  Google Scholar 

  • Knorr S, Ozcinar C, Fearghail CO, Smolic A (2018) Director’s cut-a combined dataset for visual attention analysis in cinematic VR content

  • Limper M, Jung Y, Behr J, Sturm T, Franke T, Schwenk K, Kuijper A (2013) Fast, progressive loading of binary-encoded declarative-3d web content. IEEE Comput Graph Appl 33(5):26–36

    Article  Google Scholar 

  • Löwe T, Stengel M, Förster EC, Grogorick S, Magnor M (2015) Visualization and analysis of head movement and gaze data for immersive video in head-mounted displays. In: Proceedings of the workshop on eye tracking and visualization (ETVIS), vol 1

  • Maclntyre B, Smith TF (2019) Thoughts on the future of WebXR and the immersive web. In: 2018 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct). IEEE, pp 338–342

  • Manku GS, Motwani R (2002) Approximate frequency counts over data streams. In: VLDB’02 proceedings of the 28th international conference on very large databases. Elsevier, Amsterdam, pp 346–357

  • Meghini C, Scopigno R, Richards J, Fanini B et al (2017) Ariadne: a research infrastructure for archaeology. J Comput Cult Herit (JOCCH) 10(3):18

    Google Scholar 

  • Sitzmann V, Serrano A, Pavel A, Agrawala M, Gutierrez D, Masia B, Wetzstein G (2018) Saliency in VR: how do people explore virtual environments? IEEE Trans Vis Comput Graph 24(4):1633–1642

    Article  Google Scholar 

  • Smith T, Henderson J (2008) Attentional synchrony in static and dynamic scenes. J Vis 8(6):773–773

    Article  Google Scholar 

  • Upenik E, Ebrahimi T (2017) A simple method to obtain visual attention data in head mounted virtual reality. In: 2017 IEEE international conference on multimedia & expo workshops (ICMEW). IEEE, pp 73–78

  • Vincent C, Soroli E, Engemann H, Hendriks H, Hickmann M (2018) Tobii or not tobii? assessing the validity of eye tracking data: Challenges and solutions. In: Scandinavian workshop on applied eye tracking (SWAET)

  • Wagner Filho JA, Rey MF, Freitas CM, Nedel L (2018) Immersive visualization of abstract information: an evaluation on dimensionally-reduced data scatterplots. In: Proceedings of the 25th IEEE conference on virtual reality and 3D user interfaces (March 2018), vol 2, p 4

  • Wiggins RH, Davidson HC, Harnsberger HR, Lauman JR, Goede PA (2001) Image file formats: past, present, and future. Radiographics 21(3):789–798

    Article  Google Scholar 

  • Wille M, Adolph L, Grauel B, Wischniewski S, Theis S, Alexander T (2014) Prolonged work with head mounted displays. In: Proceedings of the 2014 ACM international symposium on wearable computers: adjunct program. ACM, pp 221–224

  • Wong PC, Thomas J (2004) Visual analytics. IEEE Comput Graph Appl 5:20–21

    Article  Google Scholar 

Download references

Acknowledgements

Authors want to thank the Hallwyl Museum and their 3D models freely available (CC BY-SA 4.0) on SketchFab; VisionLab at Computer Science department (Sapienza University) and volunteers; D. Ferdani and E. Demetrescu (CNR ISPC/ITABC) for samples of 3D models used for Keys2Rome project (http://keys2rome.eu/); and A. Palombini, I. Cerato and L. Rescic (CNR ISPC/ITABC) for VR installation support during “TourismA 2018” event.

Author information

Authors and Affiliations

  1. CNR ISPC/ITABC, Rome, Italy

    Bruno Fanini

  2. Department of Computer Science, Sapienza University, Rome, Italy

    Bruno Fanini & Luigi Cinque

Authors
  1. Bruno Fanini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luigi Cinque
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno Fanini.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 88877 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fanini, B., Cinque, L. Encoding immersive sessions for online, interactive VR analytics. Virtual Reality 24, 423–438 (2020). https://doi.org/10.1007/s10055-019-00405-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-019-00405-w

Keywords

Search

Navigation