Skip to main content
SpringerLink
Log in

Using Virtual Reality to Overcome Legacy Bias in Remote Gesture Elicitation Studies

  • Conference paper
  • First Online:
Human-Computer Interaction (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14011))

Included in the following conference series:

  • 1150 Accesses

Abstract

End-user Gesture Elicitation Studies (GESs) are the cornerstone of gesture design research and are used extensively when designing gesture-controlled interfaces. With the increasing accessibility of consumer-grade immersive devices, GESs are reshaping towards encapsulated studies using VR as a medium. These VR GESs appear to be effective in addressing the lack of ecological validity and systemic bias in typical GES designs. Yet, VR GESs often suffer from legacy bias, a phenomenon where elicited interactions are distorted by prior experiences and require study designs to mitigate them. In this study, we present a VR GES design that embeds three legacy bias reduction techniques: priming, partnering, and production (3Ps), and compare the impact of each technique with four between-group VR GESs. We discuss our design algorithms along with the results of the design evaluation. From the results, we postulate conducting VR studies outside the laboratory with legacy bias mitigation techniques is feasible and further discuss the implications and limitations of running GESs using VR as a medium for overcoming legacy bias.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://unity.com/.

  2. 2.

    https://normcore.io/.

  3. 3.

    Waist packs, enable people to keep their tools/supplies close at hand at all times [2]. In our design, we use this as a metaphor to present the experiment’s controllers to the participant.

  4. 4.

    http://www.igroup.org/pq/ipq/index.php.

References

  1. Ali, A.X., Morris, M.R., Wobbrock, J.O.: I am iron man priming improves the learnability and memorability of user-elicited gestures. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2021)

    Google Scholar 

  2. Allen, J.: Designing desirability in an augmentative and alternative communication device. Univ. Access Inf. Soc. 4(2), 135–145 (2005)

    Article  Google Scholar 

  3. Arendttorp, E.M.N., Rodil, K., Winschiers-Theophilus, H., Magoath, C.: Overcoming legacy bias: Re-designing gesture interactions in virtual reality with a san community in Namibia. In: CHI Conference on Human Factors in Computing Systems, pp. 1–18 (2022)

    Google Scholar 

  4. Balakrishnan, R., Hinckley, K.: Symmetric bimanual interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 33–40 (2000)

    Google Scholar 

  5. Bellucci, A., Zarraonandia, T., Díaz, P., Aedo, I.: Welicit: a wizard of Oz tool for VR elicitation studies. In: Ardito, C., et al. (eds.) INTERACT 2021. LNCS, vol. 12936, pp. 82–91. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85607-6_6

    Chapter  Google Scholar 

  6. Blandford, A., Furniss, D., Makri, S.: Qualitative HCI research: going behind the scenes. Synth. Lect. Hum.-Centered Inf. 9(1), 1–115 (2016)

    Article  Google Scholar 

  7. Brooke, J., et al.: SUS-A quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  8. Buchanan, S., Floyd, B., Holderness, W., LaViola, J.J.: Towards user-defined multi-touch gestures for 3d objects. In: Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces, pp. 231–240 (2013)

    Google Scholar 

  9. Chan, E., Seyed, T., Stuerzlinger, W., Yang, X.D., Maurer, F.: User elicitation on single-hand microgestures. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 3403–3414 (2016)

    Google Scholar 

  10. Cooper, A.: The myth of metaphor. Visual Basic Programmer’s J. 3, 127–128 (1995)

    Google Scholar 

  11. Corporation, V.: Steam hardware and software survey: May 2022 (2022). https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam

  12. Dow, S., Fortuna, J., Schwartz, D., Altringer, B., Schwartz, D., Klemmer, S.: Prototyping dynamics: sharing multiple designs improves exploration, group rapport, and results, pp. 2807–2816 (2011). https://doi.org/10.1145/1978942.1979359

  13. Gilbert, N.: 74 virtual reality statistics you must know in 2021/2022: Adoption, usage and market share (2022). http://financesonline.com/virtual-reality-statistics/

  14. Human Performance Research Group NASA Ames Research Group: NASA TLX Paper and Pencil Version Instruction Manual. Moffett Feild, California (1988)

    Google Scholar 

  15. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Advances in Psychology, vol. 52, pp. 139–183. Elsevier (1988)

    Google Scholar 

  16. Hoff, L., Hornecker, E., Bertel, S.: Modifying gesture elicitation: do kinaesthetic priming and increased production reduce legacy bias? In: Proceedings of the TEI 2016: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 86–91 (2016)

    Google Scholar 

  17. Jenkins, H.: Game design as narrative architecture. Computer 44(3), 118–130 (2004)

    Google Scholar 

  18. Köpsel, A., Bubalo, N.: Benefiting from legacy bias. Interactions 22(5), 44–47 (2015)

    Article  Google Scholar 

  19. Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human-Computer Interaction. Morgan Kaufmann, Burlington (2017)

    Google Scholar 

  20. Lewkowicz, D.J.: The concept of ecological validity: what are its limitations and is it bad to be invalid? Infancy 2(4), 437–450 (2001)

    Article  Google Scholar 

  21. Madeo, R.C.B., Peres, S.M., de Moraes Lima, C.A.: Gesture phase segmentation using support vector machines. Expert Syst. Appl. 56, 100–115 (2016)

    Google Scholar 

  22. Mäkelä, V., et al.: Virtual field studies: conducting studies on public displays in virtual reality. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2020)

    Google Scholar 

  23. McCrindle, M.: Generation Alpha. Hachette, UK (2021)

    Google Scholar 

  24. McMahan, R.P., Gorton, D., Gresock, J., McConnell, W., Bowman, D.A.: Separating the effects of level of immersion and 3d interaction techniques. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 108–111 (2006)

    Google Scholar 

  25. Morris, M.R.: Web on the wall: insights from a multimodal interaction elicitation study. In: Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces, pp. 95–104 (2012)

    Google Scholar 

  26. Morris, M.R., et al.: Reducing legacy bias in gesture elicitation studies. Interactions 21(3), 40–45 (2014)

    Article  Google Scholar 

  27. Morris, M.R., Wobbrock, J.O., Wilson, A.D.: Understanding users’ preferences for surface gestures. In: Proceedings of Graphics Interface 2010, pp. 261–268 (2010)

    Google Scholar 

  28. Nielsen, J.: Time budgets for usability sessions. Useit. com: Jakob Nielsen’s web site 12 (2005)

    Google Scholar 

  29. Norman, D.A.: Design rules based on analyses of human error. Commun. ACM 26(4), 254–258 (1983)

    Article  Google Scholar 

  30. Norman, D.A.: The way i see it signifiers, not affordances. Interactions 15(6), 18–19 (2008)

    Article  Google Scholar 

  31. Norman, D.A.: The Design of Everyday Things. MIT Press, Cambridge (2013)

    Google Scholar 

  32. Ortega, F.R., et al.: Gesture elicitation for 3d travel via multi-touch and mid-air systems for procedurally generated pseudo-universe. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp. 144–153. IEEE (2017)

    Google Scholar 

  33. Perera, M., Gedeon, T., Adcock, M., Haller, A.: Towards self-guided remote user studies-feasibility of gesture elicitation using immersive virtual reality. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2576–2583. IEEE (2021)

    Google Scholar 

  34. Perera, M., Haller, A., Rodríguez Méndez, S.J., Adcock, M.: HDGI: a human device gesture interaction ontology for the internet of things. In: Pan, J.Z., Tamma, V., d’Amato, C., Janowicz, K., Fu, B., Polleres, A., Seneviratne, O., Kagal, L. (eds.) ISWC 2020. LNCS, vol. 12507, pp. 111–126. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-62466-8_8

    Chapter  Google Scholar 

  35. Piumsomboon, T., Clark, A., Billinghurst, M., Cockburn, A.: User-defined gestures for augmented reality. In: Kotzé, P., Marsden, G., Lindgaard, G., Wesson, J., Winckler, M. (eds.) INTERACT 2013. LNCS, vol. 8118, pp. 282–299. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40480-1_18

    Chapter  Google Scholar 

  36. Radiah, R., et al.: Remote VR studies: a framework for running virtual reality studies remotely via participant-owned HMDs. ACM Trans. Comput.-Hum. Interact. (TOCHI) 28(6), 1–36 (2021)

    Article  Google Scholar 

  37. Ratcliffe, J., Ballou, N., Tokarchuk, L.: Actions, not gestures: contextualising embodied controller interactions in immersive virtual reality. In: Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology, pp. 1–11 (2021)

    Google Scholar 

  38. Ratcliffe, J., Soave, F., Bryan-Kinns, N., Tokarchuk, L., Farkhatdinov, I.: Extended reality (XR) remote research: a survey of drawbacks and opportunities. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2021)

    Google Scholar 

  39. Ratcliffe, J., et al.: Remote XR studies: exploring three key challenges of remote XR experimentation. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–4 (2021)

    Google Scholar 

  40. Rey, A., Bellucci, A., Diaz, P., Aedo, I.: A tool for monitoring and controlling standalone immersive HCI experiments. In: The Adjunct Publication of the 34th Annual ACM Symposium on User Interface Software and Technology, pp. 20–22 (2021)

    Google Scholar 

  41. Rivu, R., et al.: Remote vr studies-a framework for running virtual reality studies remotely via participant-owned hmds. arXiv preprint arXiv:2102.11207 (2021)

  42. Ruiz, J., Li, Y., Lank, E.: User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 197–206 (2011)

    Google Scholar 

  43. Ruiz, J., Vogel, D.: Soft-constraints to reduce legacy and performance bias to elicit whole-body gestures with low arm fatigue. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3347–3350 (2015)

    Google Scholar 

  44. Sanchez-Vives, M.V., Slater, M.: From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6(4), 332–339 (2005)

    Article  Google Scholar 

  45. Schacter, D.L., Buckner, R.L.: Priming and the brain. Neuron 20(2), 185–195 (1998)

    Article  Google Scholar 

  46. Schmuckler, M.A.: What is ecological validity? a dimensional analysis. Infancy 2(4), 419–436 (2001)

    Article  Google Scholar 

  47. Schubert, T.W.: The sense of presence in virtual environments: a three-component scale measuring spatial presence, involvement, and realness. Z. für Medienpsychologie 15(2), 69–71 (2003)

    Article  Google Scholar 

  48. Tsandilas, T.: Fallacies of agreement: a critical review of consensus assessment methods for gesture elicitation. ACM Trans. Comput.-Hum. Interact. (TOCHI) 25(3), 1–49 (2018)

    Article  Google Scholar 

  49. Vatavu, R.D., Wobbrock, J.O.: Formalizing agreement analysis for elicitation studies: new measures, significance test, and toolkit. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1325–1334 (2015)

    Google Scholar 

  50. Vatavu, R.D., Zaiti, I.A.: Leap gestures for tv: insights from an elicitation study. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, pp. 131–138 (2014)

    Google Scholar 

  51. Vogiatzidakis, P., Koutsabasis, P.: Gesture elicitation studies for mid-air interaction: a review. Multimodal Technol. Interact. 2(4), 65 (2018)

    Article  Google Scholar 

  52. Vogiatzidakis, P., Koutsabasis, P.: Frame-based elicitation of mid-air gestures for a smart home device ecosystem. In: Informatics, vol. 6, p. 23. Multidisciplinary Digital Publishing Institute (2019)

    Google Scholar 

  53. Warpefelt, H., Verhagen, H.: A model of non-player character believability. J. Gaming Virtual Worlds 9(1), 39–53 (2017)

    Article  Google Scholar 

  54. Wentura, D., Degner, J.: A practical guide to sequential priming and related tasks (2010)

    Google Scholar 

  55. Williams, A.S., Garcia, J., De Zayas, F., Hernandez, F., Sharp, J., Ortega, F.R.: The cost of production in elicitation studies and the legacy bias-consensus trade off. Multimodal Technol. Interact. 4(4), 88 (2020)

    Article  Google Scholar 

  56. Williams, A.S., Garcia, J., Ortega, F.: Understanding multimodal user gesture and speech behavior for object manipulation in augmented reality using elicitation. IEEE Trans. Visual. Comput. Graph. 26(12), 3479–3489 (2020)

    Article  Google Scholar 

  57. Williams, A.S., Ortega, F.R.: Evolutionary gestures: when a gesture is not quite legacy biased. Interactions 27(5), 50–53 (2020)

    Article  Google Scholar 

  58. Wittorf, M.L., Jakobsen, M.R.: Eliciting mid-air gestures for wall-display interaction. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction, pp. 1–4 (2016)

    Google Scholar 

  59. Wu, H., Fu, S., Yang, L., Zhang, X.: Exploring frame-based gesture design for immersive VR shopping environments. Behav. Inf. Technol. 41(1), 96–117 (2022)

    Article  Google Scholar 

  60. Wu, H., et al.: Understanding freehand gestures: a study of freehand gestural interaction for immersive VR shopping applications. Hum.-Centric Comput. Inf. Sci. 9(1), 1–26 (2019)

    Article  Google Scholar 

  61. Wu, H., Wang, Y., Qiu, J., Liu, J., Zhang, X.: User-defined gesture interaction for immersive VR shopping applications. Behav. Inf. Technol. 38(7), 726–741 (2019)

    Article  Google Scholar 

  62. Zhou, X., Williams, A.S., Ortega, F.R.: Towards establishing consistent proposal binning methods for unimodal and multimodal interaction elicitation studies. In: Kurosu, M. (eds.) International Conference on Human-Computer. Interaction Theoretical Approaches and Design Methods, HCII 2022, vol. 13302, pp. 356–368. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05311-5_25

  63. Zimmerman, J., Forlizzi, J.: Research through design in HCI. In: Olson, J.S., Kellogg, W.A. (eds.) Ways of Knowing in HCI, pp. 167–189. Springer, New York (2014). https://doi.org/10.1007/978-1-4939-0378-8_8

    Chapter  Google Scholar 

  64. Zimmerman, J., Forlizzi, J., Evenson, S.: Research through design as a method for interaction design research in HCI. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 493–502 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Australian National University, Canberra, Australia

    Madhawa Perera, Tom Gedeon, Armin Haller & Matt Adcock

  2. Commonwealth Scientific and Industrial Research Organisation, Canberra, Australia

    Madhawa Perera & Matt Adcock

  3. Curtin University, Perth, Australia

    Tom Gedeon

Authors
  1. Madhawa Perera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tom Gedeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Armin Haller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matt Adcock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Madhawa Perera .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  2. Hosei University, Tokyo, Japan

    Ayako Hashizume

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Perera, M., Gedeon, T., Haller, A., Adcock, M. (2023). Using Virtual Reality to Overcome Legacy Bias in Remote Gesture Elicitation Studies. In: Kurosu, M., Hashizume, A. (eds) Human-Computer Interaction. HCII 2023. Lecture Notes in Computer Science, vol 14011. Springer, Cham. https://doi.org/10.1007/978-3-031-35596-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35596-7_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35595-0

  • Online ISBN: 978-3-031-35596-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation