Skip to main content
SpringerLink
Log in

Augmented reality-based remote coaching for fast-paced physical task

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

One popular application of augmented reality (AR) is the real-time guidance and training in which the AR user receives useful information by a remote expert. For relatively fast-paced tasks, presentation of such guidance in a way that the recipient can make immediate recognition and quick understanding can be an especially challenging problem. In this paper, we present an AR-based tele-coaching system applied to the game of tennis, called the AR coach, and explore for interface design guidelines through a user study. We have evaluated the player’s performance for instruction understanding when the coaching instruction was presented in four different modalities: (1) Visual—visual only, (2) Sound—aural only/mono, (3) 3D Sound—aural only/3D and (4) Multimodal—both visual and aural/mono. Results from the experiment suggested that, among the three, the visual-only augmentation was the most effective and least distracting for the given pace of information transfer (e.g., under every 3 s). We attribute such a result to the characteristic of the visual modality to encode and present a lot of information at once and the human’s limited capability in handling and fusing multimodal information at a relatively fast rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Avery B, Sandor C, Thomas BH (2012) Improving spatial perception for augmented reality x-ray vision. In: Proceedings of the IEEE conference on virtual reality. IEEE, pp 79–82

  • Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in augmented reality. Comput Graph Appl IEEE 21(6):34–47

    Article  Google Scholar 

  • Bolt RA (1980) Put-that-there, voice and gesture at the graphics interface. Comput Graph 14(3):262–270

    Article  Google Scholar 

  • Borst J, Taatgen N, Rijn H (2010) The problem state: a cognitive bottleneck in multitasking. J Exp Psychol Learn Mem Cogn 36(2):363–382

    Article  Google Scholar 

  • Byrne M, Anderson J (2001) Serial modules in parallel: the psychological refractory period and perfect time sharing. Psychol Rev 108(4):847–869

    Article  Google Scholar 

  • Damos D (1991) Dual-task methodology: some common problems. In: Damos D (ed) Multiple-task performance. CRC Press, Boca Raton, pp 101–119

    Google Scholar 

  • Fuchs H, Livingston MA, Raskar R, State A, Crawford JR, Rademacher R, Drake SH, Meyer AA (1998) Augmented reality visualization for laparoscopic surgery. In: Proceedings of the MICCAI, pp 934–943

  • Grasso MA, Ebert DS, Finin TW (1998) The integrality of speech in multimodal interfaces. ACM Trans CHI 5(4):303–325

    Google Scholar 

  • Grohn M, Lokki T, Savioja L, Takala T (2001) Some aspects of role of audio in immersive visualization. Proc SPIE 4302:13–22

    Article  Google Scholar 

  • Hauptmann A, McAvinney P (1993) Gestures with speech for graphic manipulation. Int J Man Mach Stud 38(2):231–249

    Article  Google Scholar 

  • Hecht D, Reiner M, Halevy G (2006) Multimodal virtual environments: response times, attention, and presence. Presence Teleoper Virtual Environ 15(5):515–523

    Article  Google Scholar 

  • Jaimes A, Sebe N (2005) Multimodal human computer interaction: a survey. In: IEEE international workshop on human computer interaction

  • Kajastila R, Holsti L, Hämäläinen P (2016) The augmented climbing wall: high-exertion proximity interaction on a wall-sized interactive surface. In: Proceedings of the SIGCHI, pp 758–769

  • Lecuyer A, Mobuchon P, Megard C, Perret J, Andriot C, Colinot JP (2003) HOMERE: a multimodal system for visually impaired people to explore virtual environments. In: Proceedings of IEEE international conference on virtual reality. IEEE, pp 251–257

  • Meera MB, Denise AS, Robert MG (1989) Earcons and icons: their structure and common design principles. Hum Comput Interact 4(1):11–44

    Article  Google Scholar 

  • Narzt W, Pomberger G, Ferscha A, Kolb D, Müller R, Wieghardt J, Hörtner H, Lindinger C (2006) Augmented reality navigation systems. J Univers Access Inf Soc 4(3):177–187

    Article  Google Scholar 

  • Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C (2007) Action-and workflow-driven augmented reality for computer-aided medical procedures. Comput Graph Appl IEEE 27(5):10–14

    Article  Google Scholar 

  • Oviatt S (2002) Multimodal interfaces. In: Jacko J, Sears A (eds) Handbook of human–computer interaction. Lawrence Erlbaum, Mahwah

    Google Scholar 

  • Richard P, Burdea G, Gomez D, Coiffet P (1994) A comparison of haptic, visual and auditive force feedback for deformable virtual objects. In: Proceedings of ICAT. pp 49–62

  • Rubinstein J, Meyer D, Evans J (2001) Executive control of cognitive processes in task switching. J Exp Psychol Hum Percept Perform 27(4):763–797

    Article  Google Scholar 

  • Sallnas E, Grohn K, Sjostrom C (2000) Supporting presence in collaborative environment by haptic force feedback. ACM Trans CHI 7(4):461–476

    Google Scholar 

  • Salvucci D, Tattgen N, Borst J (2009) Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption. In: Proceedings of the SIGCHI, pp 1819–1828

  • Sanna A, Manuri F (2016) A survey on applications of augmented reality. Adv Comput Sci Int J 5(1):18–27

    Google Scholar 

  • Schwald B, Laval BD (2003) An augmented system for training and assistance to maintenance in the industrial context. In: Proceedings of WSCG 2003

  • Strayer D (2007) Multitasking in the automobile. In: Kramer A, Wiegmann D, Kirlik A (eds) Applied attention. Oxford University Press, Oxford, pp 121–133

    Google Scholar 

  • Tatzgern M, Kalkofen D, Schmalstieg D (2013) Dynamic compact visualizations for augmented reality. In: Proceedings of the IEEE conference on virtual reality 2013. IEEE, pp 3–6

  • Tonnis M, Klinker G, Plavsic M (2009) Survey and classification of head-up display presentation principles. In: Proceedings of the international ergonomics association

  • Walker BN, Lindsay J, Nance A, Nakano Y, Palladino DK, Dingler T, Jeon M (2013) Spearcons (speech-based earcons) improve navigation performance in advanced auditory menus. Hum Factor 55(1):157–182

    Article  Google Scholar 

  • Ware C (2012) Information visualization: perception for design, 3rd edn. Morgan Kaufmann, Burlington

    Google Scholar 

  • Zheng XS, Foucault C, da Silva PM, Dasari S, Yang T, Goose S (2015) Eye-wearable technology for machine maintenance: effects of display position and hands-free operation. In: Proceedings of the SIGCHI, pp 2125–2134

Download references

Acknowledgements

This work was supported in part by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIP) (No. R0190-16-2011, Development of Vulnerability Discovery Technologies for IoT Software Security), and also in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government(MSIP) (No.2011-0030079).

Author information

Authors and Affiliations

  1. Digital Experience Laboratory, Korea University, Anam-dong, Seongbuk-gu, Seoul, Korea

    Youngsun Kim, Seokjun Hong & Gerard Jounghyun Kim

Authors
  1. Youngsun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seokjun Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerard Jounghyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerard Jounghyun Kim.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (MP4 58803 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, Y., Hong, S. & Kim, G.J. Augmented reality-based remote coaching for fast-paced physical task. Virtual Reality 22, 25–36 (2018). https://doi.org/10.1007/s10055-017-0315-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-017-0315-2

Keywords

Search

Navigation