Skip to main content
SpringerLink
Log in

Enabling Interaction with Arbitrary 2D Applications in Virtual Environments

  • Conference paper
  • First Online:
HCI International 2020 - Posters (HCII 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1225))

Included in the following conference series:

Abstract

Virtual environments (VE) provide immersive experiences that allow users to perceive and interact with three dimensional (3D) content. Yet, the 3D applications are often tailored for specific tasks and hardware setups such as a virtual or augmented reality (VR/AR) head mounted display (HMD). This limits the range of available content, because not every standard application (e.g. web browser, calender, office) that is available on a 2D desktop PC can be easily accessed using VR or AR. As commonly used by other systems, we capture arbitrary 2D windows from a desktop PC and display them as a 2D surface in the VE. Furthermore, we provide different techniques that aim to allow easy and fast interaction with the 2D applications in VR. For example, the work bench tool allows to copy and paste subsections of a 2D window. The copied sections are fully interactable and can be moved around and scaled to provide easy access to small menus. With the macro tool, users can create shortcuts to frequently used functions that are executed by cursor presses or hotkeys. The provided tools yield new ways of interacting with 2D interfaces. By allowing the user to easily interact with already available 2D applications, the productivity of VEs increases strongly.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahlström, D., Hitz, M., Leitner, G.: An evaluation of sticky and force enhanced targets in multi target situations. In: Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles, NordiCHI 2006, pp. 58–67. Association for Computing Machinery, New York (2006). https://doi.org/10.1145/1182475.1182482

  2. Andujar, C., Argelaguet, F.: Virtual pads: decoupling motor space and visual space for flexible manipulation of 2D windows within VEs. In: 2007 IEEE Symposium on 3D User Interfaces, March 2007. https://doi.org/10.1109/3DUI.2007.340781

  3. Angus, I.G., Sowizral, H.A.: Embedding the 2D interaction metaphor in a real 3D virtual environment. In: Fisher, S.S., Merritt, J.O., Bolas, M.T. (eds.) Stereoscopic Displays and Virtual Reality Systems II, vol. 2409, pp. 282–293. International Society for Optics and Photonics, SPIE (1995). https://doi.org/10.1117/12.205875

  4. Azuma, R.T.: A survey of augmented reality. Presence: Teleoperators Virtual Environ. 6(4), 355–385 (1997). https://doi.org/10.1162/pres.1997.6.4.355

  5. Barsoum, E., Kuester, F.: WebVR: an interactive web browser for virtual environments. In: Woods, A.J., Merritt, J.O., Bolas, M.T., McDowall, I.E. (eds.) Stereoscopic Displays and Virtual Reality Systems XII, vol. 5664, pp. 540–547. International Society for Optics and Photonics, SPIE (2005). https://doi.org/10.1117/12.582624

  6. Bigscreen, inc.: Bigscreen (2020). https://www.bigscreenvr.com/

  7. Bowman, D.A., Hodges, L.F.: An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D 1997, p. 35-ff. Association for Computing Machinery, New York (1997). https://doi.org/10.1145/253284.253301

  8. Bowman, D.A., Kruijff, E., LaViola, J.J., Poupyrev, I.: An introduction to 3-D user interface design. Presence: Teleoperators Virtual Environ. 10(1), 96–108 (2001). https://doi.org/10.1162/105474601750182342

  9. Facebook Technologies, LLC.: Oculus Dash (2020). https://developer.oculus.com/documentation/native/pc/dg-dash

  10. hecomi: uDesktopDuplication (2016). https://github.com/hecomi/uDesktopDuplication

  11. Hoppe, A.H., Marek, F., van de Camp, F., Stiefelhagen, R.: Virtualtablet: extending movable surfaces with touch interaction. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 980–981, March 2019. https://doi.org/10.1109/VR.2019.8797993

  12. Hoppe, A.H., Otto, L., van de Camp, F., Stiefelhagen, R., Unmüßig, G.: qVRty: virtual keyboard with a haptic, real-world representation. In: Stephanidis, C. (ed.) HCI 2018. CCIS, vol. 851, pp. 266–272. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92279-9_36

    Chapter  Google Scholar 

  13. Jim and OBS Studio Contributors: Open Broadcaster Software (2020). https://obsproject.com/

  14. Lindeman, R.W., Sibert, J.L., Hahn, J.K.: Hand-held windows: towards effective 2D interaction in immersive virtual environments. In: Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pp. 205–212, March 1999. https://doi.org/10.1109/VR.1999.756952

  15. Mazuryk, T., Gervautz, M.: Virtual reality-history, applications, technology and future (1996)

    Google Scholar 

  16. Microsoft Corporation: Desktop Duplication API (2018). https://docs.microsoft.com/en-us/windows/win32/direct3ddxgi/desktop-dup-api

  17. Microsoft Corporation: Windows GDI (2018). https://docs.microsoft.com/en-us/windows/win32/gdi/windows-gdi

  18. Microsoft Corporation: Windows Mixed Reality home (2018). https://docs.microsoft.com/en-us/windows/mixed-reality/navigating-the-windows-mixed-reality-home

  19. Microsoft Corporation: WindowsGraphicsCapture (2018). https://docs.microsoft.com/en-us/uwp/api/windows.graphics.capture

  20. Mine, M.R., Brooks, F.P., Sequin, C.H.: Moving objects in space: exploiting proprioception in virtual-environment interaction. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997, pp. 19–26. ACM Press/Addison-Wesley Publishing Co., USA (1997). https://doi.org/10.1145/258734.258747

  21. Regenbrecht, H., Baratoff, G., Wagner, M.: A tangible AR desktop environment. Comput. Graph. 25(5), 755–763 (2001). https://doi.org/10.1016/S0097-8493(01)00118-2. Mixed realities - beyond conventions

    Article  Google Scholar 

  22. Steinicke, F., Ropinski, T., Hinrichs, K.: VR and laser-based interaction in virtual environments using a dual-purpose interaction metaphor. In: IEEE VR 2005 Workshop Proceedings on New Directions in 3D User Interfaces, pp. 61–64 (2005)

    Google Scholar 

  23. Toyama, S., Al Sada, M., Nakajima, T.: Vrowser: a virtual reality parallel web browser. In: Chen, J.Y., Fragomeni, G. (eds.) Virtual, Augmented and Mixed Reality: Interaction, Navigation, Visualization, Embodiment, and Simulation, pp. 230–244. Springer, Cham (2018)

    Chapter  Google Scholar 

  24. Valve Corporation: Steam VR (2020). https://store.steampowered.com/app/250820/SteamVR

  25. van de Camp, F., Stiefelhagen, R.: Applying force fields to black-box guls using computer vision. In: 2013 1st IEEE Workshop on User-Centered Computer Vision (UCCV), pp. 1–6, January 2013. https://doi.org/10.1109/UCCV.2013.6530799

  26. Virtual Desktop Inc: vrdesktop (2019). https://www.vrdesktop.net

  27. Wobbrock, J.O., Morris, M.R., Wilson, A.D.: User-defined gestures for surface computing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2009, pp. 1083–1092. Association for Computing Machinery, New York (2009). https://doi.org/10.1145/1518701.1518866

Download references

Author information

Authors and Affiliations

  1. Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

    Adrian H. Hoppe & Rainer Stiefelhagen

  2. Fraunhofer IOSB, Fraunhofer Institute of Optronics, System Technologies and Image Exploitation, Karlsruhe, Germany

    Florian van de Camp

Authors
  1. Adrian H. Hoppe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Florian van de Camp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rainer Stiefelhagen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrian H. Hoppe .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Greece

    Constantine Stephanidis

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Margherita Antona

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hoppe, A.H., van de Camp, F., Stiefelhagen, R. (2020). Enabling Interaction with Arbitrary 2D Applications in Virtual Environments. In: Stephanidis, C., Antona, M. (eds) HCI International 2020 - Posters. HCII 2020. Communications in Computer and Information Science, vol 1225. Springer, Cham. https://doi.org/10.1007/978-3-030-50729-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-50729-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-50728-2

  • Online ISBN: 978-3-030-50729-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation