Skip to main content
SpringerLink
Log in

A virtual reality application for augmented panoramic mountain images

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Virtual reality is a powerful interaction mechanism that holds the promise of engaging users, not only for entertainment, but also for social and environmental purposes. In this paper we present PeakLensVR, a virtual reality mobile application that enables users to capture panoramic mountain images with their mobile devices and later visualize such images, enriched with metadata about the peaks visible from the capture point, with a low-end VR device. The application exploits a multi-stage data processing pipeline, which comprises the following steps: (1) the acquisition of a sequence of frames with the mobile phone camera and their annotation with sensor readings captured during the shooting session; (2) the creation of a panoramic image from the acquired frames, with state-of-the art stitching algorithms; (3) the registration of the panoramic image to the mountain skyline in view, by comparing the image skyline with a virtual profile extracted from the NASA SRTM Digital Elevation Model of the Earth; (4) the enrichment of the registered panoramic image with markers and metadata (name, altitude, etc.) of the peaks in view, by querying the OpenStreetMap GIS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. http://vr.google.com/cardboard/.

  2. http://www.samsung.com/global/galaxy/gear-vr/.

  3. http://snowwatch.polimi.it.

  4. http://peaklens.com.

  5. http://www.openstreetmap.org.

  6. https://developers.google.com/streetview/spherical-metadata.

  7. https://www.google.com/streetview.

  8. https://facebook360.fb.com.

  9. https://www.flickr.com/vr.

  10. https://play.google.com/store/apps/details?id=com.google.vr.cyclops.

  11. https://play.google.com/store/apps/details?id=com.vtcreator.android360.

  12. http://www.cipa.jp/std/documents/e/DC-008-2012_E.pdf.

  13. http://www.youtube.com.

  14. http://www.trailmeup.com.

References

  • Baboud L, Čadík M, Eisemann E, Seidel HP (2011) Automatic photo-to-terrain alignment for the annotation of mountain pictures. In: The 24th conference on computer vision and pattern recognition (CVPR). IEEE, pp 41–48

  • Borba EZ, Montes A, Almeida M, Nagamura M, Lopes R, Zuffo MK, Araujo A, Kopper R (2017) ArcheoVR: exploring Itapeva’s archeological site. In: 2017 IEEE virtual reality (VR). IEEE, Los Angeles, pp 463–464

    Chapter  Google Scholar 

  • Boulos MNK, Lu Z, Guerrero P, Jennett C, Steed A (2017) From urban planning and emergency training to Pokémon go: applications of virtual reality GIS (VRGIS) and augmented reality GIS (ARGIS) in personal, public and environmental health. Int J Health Geogr 16:7

    Article  Google Scholar 

  • Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74:59–73

    Article  Google Scholar 

  • Castelletti A, Fedorov R, Fraternali P, Giuliani M (2016) Multimedia on the mountaintop: using public snow images to improve water systems operation. In: Proceedings of the 2016 ACM on multimedia conference. ACM, pp 948–957

  • Chai Q, Liu S (2016a) Shape-optimizing hybrid warping for image stitching. In: 2016 IEEE international conference on multimedia and expo (ICME). pp 1–6

  • Chai Q, Liu S (2016b) Shape-optimizing hybrid warping for image stitching. In: IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  • Coburn JQ, Freeman I, Salmon JL (2017) A review of the capabilities of current low-cost virtual reality technology and its potential to enhance the design process. J Comput Inf Sci Eng 17:031013

    Article  Google Scholar 

  • Farbman Z, Hoffer G, Lipman Y, Cohen-Or D, Lischinski D (2009) Coordinates for instant image cloning. ACM Trans Graph (TOG) 28:67

    Article  Google Scholar 

  • Fedorov R (2017) Exploiting public web content to enhance environmental monitoring. Ph.D. thesis, Politecnico di Milano

  • Fedorov R, Frajberg D, Fraternali P (2016) A framework for outdoor mobile augmented reality and its application to mountain peak detection. In: International conference on augmented reality, virtual reality and computer graphics. Springer, pp 281–301

  • Ferster CJ, Coops NC (2013) A review of earth observation using mobile personal communication devices. Comput Geosci 51:339–349

    Article  Google Scholar 

  • Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24:381–395

    Article  MathSciNet  Google Scholar 

  • Frajberg D, Fraternali P, Torres RN (2017a) Convolutional neural network for pixel-wise skyline detection. In: International conference on artificial neural networks. Springer, pp 12–20

  • Frajberg D, Fraternali P, Torres RN (2017b) Heterogeneous information integration for mountain augmented reality mobile apps. In: International conference on data science and advanced analytics (DSAA). IEEE, pp 313–322

  • Graph cuts versus dynamic programming (2017). http://www.cs.utexas.edu/~grauman/courses/fall2011/handouts/examples/Adrian_demo_with_notes.pdf. Accessed 17 Aug 2017

  • Ha SJ, Koo HI, Lee SH, Cho NI, Kim SK (2007) Panorama mosaic optimization for mobile camera systems. In: IEEE transactions on consumer electronics, vol 53. IEEE

  • Ha SJ, Lee SH, Cho NI, Kim SK, Son B (2008) Embedded panoramic mosaic system using auto-shot interface. In: IEEE transactions on consumer electronics, vol 54. IEEE

  • Huang B, Jiang B, Li H (2001) An integration of gis, virtual reality and the internet for visualization, analysis and exploration of spatial data. Int J Geogr Inf Sci 15:439–456

    Article  Google Scholar 

  • Image blending (2017). http://graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/blending.pdf. Accessed 17 Aug 2017

  • Joly A, Goëau H, Champ J, Dufour-Kowalski S, Müller H, Bonnet P (2016) Crowdsourcing biodiversity monitoring: how sharing your photo stream can sustain our planet. In: Proceedings of the 2016 ACM on multimedia conference. ACM, pp 958–967

  • Keis F, Wiesner K (2014) Participatory sensing utilized by an advanced meteorological nowcasting system. In: International conference on intelligent sensors, sensor networks and information processing (ISSNIP). IEEE, pp 1–6

  • Kim BS, Lee SH, Cho NI (2011a) Real-time panorama canvas of natural images. In: IEEE transactions on consumer electronics, vol 57. IEEE

  • Kim H, Lee K, Jung J, Jung S, Ko S (2011b) A content-aware image stitching algorithm for mobile multimedia devices. IEEE Trans Consum Electron 57(4):1875–1882. https://doi.org/10.1109/TCE.2011.6131166

    Article  Google Scholar 

  • La Salandra A, Fraternali P, Frajberg D (2018) A location-based virtual reality application for mountain peak detection. In: Companion of the web conference 2018 on the web conference 2018. ACM, pp 1206–1214

  • Liu S, Chai Q (2018a) Shape-optimizing and illumination-smoothing image stitching. IEEE Trans Multimed 21(3):690–703. https://doi.org/10.1109/TMM.2018.2864576

    Article  Google Scholar 

  • Liu S, Chai Q (2018b) Shape-optimizing and illumination-smoothing image stitching. In: IEEE transactions on multimedia. IEEE

  • Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  • Lowry CS, Fienen MN (2013) Crowdhydrology: crowdsourcing hydrologic data and engaging citizen scientists. In: GroundWater, vol 51, Wiley Online Library, pp 151–156

  • Maisonneuve N, Stevens M, Niessen ME, Steels L (2009) NoiseTube: measuring and mapping noise pollution with mobile phones. In: Information technologies in environmental engineering. Springer, Berlin, pp 215–228

    Chapter  Google Scholar 

  • Piccolo L, Fernández M, Alani H, Scharl A, Föls M, Herring D (2016) Climate change engagement: results of a multi-task game with a purpose. In: Proceedings of the 1st international workshop on social web for environmental and ecological monitoring. AAAI Publications

  • Poynton C (2001) Yuv and luminance considered harmful: a plea for precise terminology in video. Engineering. https://poynton.ca/PDFs/YUV_and_luminance_harmful.pdf. Accessed 8 Mar 2018

  • Reddy S, Shilton K, Burke J, Estrin D, Hansen M, Srivastava M (2008) Evaluating participation and performance in participatory sensing. In: International workshop on urban, community, and social applications of networked sensing systems, vol 4

  • Samet H (1984) The quadtree and related hierarchical data structures. ACM Comput Surv (CSUR) 16:187–260

    Article  MathSciNet  Google Scholar 

  • Sharma P (2015) Challenges with virtual reality on mobile devices. In: ACM SIGGRAPH 2015 Talks. ACM, p 57

  • Slater M (2014) Grand challenges in virtual environments. In: Frontiers in robotics and AI, vol 1. Frontiers, p 3

  • Steuer J (1992) Defining virtual reality: dimensions determining telepresence. J Commun 42:73–93

    Article  Google Scholar 

  • Sullivan BL, Wood CL, Iliff MJ, Bonney RE, Fink D, Kelling S (2009) ebird: A citizen-based bird observation network in the biological sciences. Biol Conserv 142:2282–2292

    Article  Google Scholar 

  • Survey UG (2009) Shuttle radar topography mission (SRTM): U.s. geological survey fact sheet 2009–3087. http://dds.cr.usgs.gov/srtm/. Accessed 8 Mar 2018

  • Szeliski R (2006) Image alignment and stitching: a tutorial. Found Trends® Comput Graph Vis 2:1–104

    Article  MathSciNet  Google Scholar 

  • Verbree E, Maren GV, Germs R, Jansen F, Kraak MJ (1999) Interaction in virtual world views-linking 3D GIS with VR. Int J Geogr Inf Sci 13:385–396

    Article  Google Scholar 

  • Xiong Y, Pulli K (2009) Gradient domain image blending and implementation on mobile devices. In: International conference on mobile computing, applications, and services. Springer, pp 293–306

  • Xiong Y, Pulli K (2010a) Fast image stitching and editing for panorama painting on mobile phones. In: 2010 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 47–52

  • Xiong Y, Pulli K (2010b) Fast panorama stitching for high-quality panoramic images on mobile phones. In: IEEE transactions on consumer electronics, vol 56. IEEE

  • Zepeda-Mendoza ML, Resendis-Antonio O (2013) Hierarchical agglomerative clustering. In: Dubitzky W, Wolkenhauer O, Cho KH, Yokota H (eds) Encyclopedia of systems biology. Springer, New York, pp 886–887

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Politecnico di Milano, Piazza L.Da Vinci, 32, Milan, Italy

    Antonio La Salandra, Darian Frajberg & Piero Fraternali

Authors
  1. Antonio La Salandra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Darian Frajberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Piero Fraternali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darian Frajberg.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

La Salandra, A., Frajberg, D. & Fraternali, P. A virtual reality application for augmented panoramic mountain images. Virtual Reality 24, 123–141 (2020). https://doi.org/10.1007/s10055-019-00385-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-019-00385-x

Keywords

Search

Navigation