Skip to main content
SpringerLink
Log in

How natural is a natural interface? An evaluation procedure based on action breakdowns

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

This paper describes an issue-based method to evaluate the naturalness of an interface. The method consists of the execution of a series of tasks on that interface, which is subsequently systematically analyzed to identify breakdowns in the users’ actions. The systematic analysis of breakdowns is allowed by the support of video-coding software (The Observer by Noldus). This method is described on its theoretical bases and then applied to the evaluation of a natural interface, a walk-in-place locomotion system for virtual spaces called Superfeet. The procedure is comparative, since Superfeet is compared to two locomotion devices, Superfeet enhanced with headtracker and a more traditional Joypad. The test involves 36 participants (mean age = 23.68, SD = 3.14). The outcomes of the breakdown analysis are illustrated at a progressively finer level of granularity from the amount and length of breakdowns, to the circumstances of the breakdowns, to the type of actions involved in the breakdowns. The potential of this procedure for usability studies is finally synthesized.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. This is called a walk-in-place solution, in which the user does not actually move with respect to the physical room while moving in the virtual environment (as in treadmills systems, [12], or waist-belts connected to a circular frame that can rotate about the center, [29]). In this way, a large virtual environment can be navigated through a natural locomotion device without requiring an equivalent large physical environment in which to move the feet. An alternative would be to cover a large virtual distance by performing real movements on a smaller scale in a similar physical environment (e.g., [5]).

  2. Abandonment is not analyzed here since it seems more related to the nature of the environment than to the quality of the device: in fact, the mean course of actions abandoned is 14.48 in task D, versus 1.15 in Task A, 4.35 in Task B, and no abandonment at all in Task C. This can be attributed to the fact that in Task D, the virtual environment is very articulated and allows participants to take different paths to reach the same point; therefore, abandoning a course of action in favor of an alternative one is a viable solution.

References

  1. Bailey BP, Konstan JA, Carlis JV (2000) Measuring the effects of interruptions on task performance in the user interface. In: Proceedings of IEEE international conference on systems, man, and cybernetics, vol 2, Nashville, TN, pp 757–762, 8–11 November

  2. Barrera S, Takahashi H, Nakajima M (2004) Hands-free navigation methods for moving through a virtual landscape walking interface virtual reality input devices. In: Proceedings of computer graphics international, IEEE, Los Alamitos, pp 388–394

  3. Beckhaus S, Blom KJ, Haringer M (2005) Intuitive, hands-free travel interfaces for virtual environments. In: Bowman D, Fröhlich B, Kitamura Y, Stürzlinger W (eds) Proceedings of IEEE VR2005 workshop on new directions in 3D user interfaces. Shaker Verlag, Aachen, pp 57s–60s

    Google Scholar 

  4. Bodker S (1991) Through the interface: a human activity approach to user interface design. Lawrence Erlbaum Associates, Hillsdale

    Google Scholar 

  5. Bouguila L, Ishii M, Sato M (2002) Realizing a new step-in-place locomotion interface for virtual environment with large display system. In: Proceedings of workshop on virtual environments 2002, Aire-la-Ville, Switzerland, pp 197–207

  6. Bowman DA, Gabbard JL (2002) A survey of usability evaluation in virtual environments: classification and comparison of methods. Presence 11:404–424

    Article  Google Scholar 

  7. Chung S, Hahn JK (1999) Animation of human walking in virtual environments. In: Proceedings of computer animation, Geneva, Switzerland, 26–29 May 1999

  8. Darken RP, Cockaybe WR, Carmein D (1997) The omni-directional treadmill: A locomotion device for virtual worlds. In: Proceedings of UIST, ACM Press, New York, pp 213–221

  9. Hartswood M, Procter R (2000) Design guidelines for dealing with breakdowns and repairs in collaborative work settings. J Hum Comput St 53:91–120

    Article  Google Scholar 

  10. Heath C, Hindmarsh J (2002) Analysing Interaction: video, ethnography and situated conduct. In: May T (ed) Qualitative research in action. Sage, London, pp 99–121

    Google Scholar 

  11. Iqbal ST, Horvitz E (2007) Disruption and recovery of computing tasks: field study, analysis, and directions. In: Proceedings of SIGCHI conference on human factors in computing systems (CHI’07), ACM Press, New York, pp 677–686

  12. Iwata H (1999) Walking about on an infinite floor. In: Proceedings of virtual reality, IEEE Computer Society Press, Houston, pp 286–293

  13. Kapitsa M, Blinnikova I (2003) Task performance under the influence of interruptions. In: Hockey GRJ, Gaillard AWK, Burov O (eds) Operator functional state: the assessment and prediction of human performance degradation in complex tasks. IOS Press, Amsterdam, pp 323–329

    Google Scholar 

  14. Keppel G, Wickens TD (2004) Design and analysis: a researcher’s handbook. Pearson/Prentice Hall, Upper Saddle River

    Google Scholar 

  15. Mackinlay JD, Card SK, Robertson GG (1990) Rapid controlled movement through a virtual 3cI workspace. In: Proceedings of 17th annual conference on computer graphics and interactive techniques, ACM Press, New York, pp 171–176

  16. McDaniel MA, Einstein GO, Graham T, Rall E (2004) Delaying execution of intentions: overcoming the costs of interruptions. App Cognitive Psych 18:533–547

    Article  Google Scholar 

  17. Norman DA, Nielsen J (2010) Gestural interfaces: a step backward in usability. Interactions 17:46–49

    Article  Google Scholar 

  18. Norman DA (2010) Natural user interfaces are not natural. Interactions 17:6–10

    Article  Google Scholar 

  19. Paelke V, Reimann C, Stichling D (2004) Foot-based mobile interaction with games. In: Proceedings of the 2004 ACM SIGCHI international conference on advances in computer entertainment technology, ACM Press, New York, pp 321–324

  20. Pakkanen T, Raisamo R (2004) Appropriateness of foot interaction for non-accurate spatial tasks. In: Proceedings of CHI 2004, ACM Press, New York, pp 1123–1126

  21. Rey Solaz B, Lozano JA, Alcañiz RM, Gamberini L, Calvet M, Kerrigan D, Martino F (2007) Super-feet: a wireless hand-free navigation system for virtual environments. In: Shumaker R (ed) Virtual reality. Springer, Heidelberg, pp 348–357

    Google Scholar 

  22. van Rhijn A, Mulder JD (2006) Spatial input device structure and bimanual object manipulation in virtual environments. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM Press, New York, pp 51–60

  23. Scrivener SAR, Urquijo SP, Palmen HK (1993) The use of breakdown analysis in synchronous CSCW system design. Int J Man Mach St 31:517–534

    Google Scholar 

  24. Spagnolli A, Gamberini L (2005) Action repair in mediated environments: presence displayed through bodily orientation. Paper presented at the second congress of the international society for gesture studies’ Lyon/France, 15–18 June 2005

  25. Spagnolli A, Gamberini L, Gasparini D (2002) Situated breakdown analysis for the evaluation of a virtual environment. Psychnology 1:5–17

    Google Scholar 

  26. Suchman L (1987) Plans and situated actions: the problem of human-machine communication. Cambridge University Press, New York

    Google Scholar 

  27. Tullis T, Albert B (2008) Measuring the user experience. Morgan Kaufman, Burlington

    Google Scholar 

  28. Usoh M, Arthur K, Whitton MC, Bastos R, Steed A, Slater M, Brooks FP (1999) Walking > walking-in-place > flying. In: Proceedings of SIGGRAPH 99, ACM Press, New York, pp 359–364

  29. Ward M, Azuma R, Bennett R, Gottschalk S, Fuchs H (1992) A demonstrated optical tracker with scalable work area for head-mounted display systems. In: Proceedings of the 1992 symposium on interactive 3D graphics, ACM Press, New York, pp 43–52

  30. Welch G, Bishop G, Vicci L, Brumback S, Keller K, Colucci D (1999) The HiBall tracker: high-performance wide-area tracking for virtual and augmented environments. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM, New York, pp 1–10

  31. Wells M, Peterson B, Aten J (1996) The virtual motion controller: a sufficient-motion walking simulator. In: Proceedings of the IEEE virtual reality annual international symposium, pp 1–8

  32. Winograd T, Flores F (1986) Understanding computers and cognition. Ablex Publishing, Norwood

    MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Daniel Kerrigan for his help in the data collection and quantitative analysis; they also would like to thank Merche Calvet for her work in the development of the SuperFeet system. Beatriz Rey Solaz, Mariano Alcañiz, and Josè Antonio Lozano developed Superfeet and the virtual environments for the test, while Luciano Gamberini, Anna Spagnolli, Lisa Prontu, Sarah Furlan, and Francesco Martino carried out the breakdown analysis.

Author information

Authors and Affiliations

  1. Dipartimento di Psicologia Generale, Università di Padova, Via Venezia 8, 35131, Padova, Italy

    Luciano Gamberini, Anna Spagnolli, Lisa Prontu, Sarah Furlan & Francesco Martino

  2. Instituto de Investigación e Innovación en Bioingeniería, Universidad Politécnica de Valencia, Camino de Vera s/n, 46022, Valencia, Spain

    Beatriz Rey Solaz, Mariano Alcañiz & Josè Antonio Lozano

Authors
  1. Luciano Gamberini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Spagnolli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lisa Prontu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarah Furlan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francesco Martino
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Beatriz Rey Solaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mariano Alcañiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Josè Antonio Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna Spagnolli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gamberini, L., Spagnolli, A., Prontu, L. et al. How natural is a natural interface? An evaluation procedure based on action breakdowns. Pers Ubiquit Comput 17, 69–79 (2013). https://doi.org/10.1007/s00779-011-0476-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-011-0476-z

Keywords

Search

Navigation