Skip to main content

How Do Users Manipulate Graphical Icons? An Empirical Study

  • Conference paper
Progress in Gestural Interaction

Abstract

According to [1] pointing one’s finger at a graphical object then at an empty location on the screen while saying “Put this here,” are semiotic gestures, since they contribute to the meaning of the concomitant utterance. On the other hand, dragging one’s fingertip on the surface of the screen may be termed an ‘ergotic’ gesture, inasmuch as it represents an action, namely the drawing of a 2D graphic or the moving of an icon, according to the current context.

We have conducted a Wizard of Oz experiment on the spontaneous use of speech and 2D gestures for interacting with standard graphical software. Overall results [9, 6, 2] indicate that, in such contexts, hand gestures are used either for pointing at objects and locations on the screen or for acting on a 2D representation of the application.

Our study of the subjects’ multimodal expression being completed, we have now focused our analysis on their use of gestures. We aim at defining useful criteria for the design of gestural human-computer interaction. In this paper, we present user profiles that were defined from a thorough analysis of the subjects’ gestures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. C. Cadoz. Le geste canal de communication homme/machine. Technique et Science Informatiques, 13 (1): 31–61, 1994.

    Google Scholar 

  2. N. Carbonell and C. Mignot. Natural multimodal HCI: Experimental results on the use of spontaneous speech and hand gestures. In Multimodal Human-Computer Interaction, ERCIM Workshop Report, pages 97–112. Rocquencourt(F): INRIA, 1994.

    Google Scholar 

  3. N. Carbonell, C. Valot, C. Mignot, and P. Dauchy. Etude empirique: usage du geste et de la parole en situation de communication homme-machine. In ErgoIA’94 Ergonomie et Informatique Avance,Biarritz, 1994. Bayonne(F): IDLS.

    Google Scholar 

  4. J. Cosnier. Communications et langages gestuels. In J. Cosnier, J. Coulon, A. Berrendonner, and C. Orecchioni, editors, Les voix du langage, communications verbales, gestuelles et animales, chapter 4, pages 255–304. Paris: Dunod, 1982.

    Google Scholar 

  5. J. Coutaz and J. Caelen. A taxonomy for multimedia and multimodal user interfaces. In Proceedings of the ERCIM Workshop, pages 143–148, Lisbon Portugal, November 1991.

    Google Scholar 

  6. P. Dauchy, C. Mignot, and C. Valot. Joint speech and gesture analysis — some experimental results on multimodal interface. In EUROSPEECH’93, pages 1315–1318, Berlin, September 1993.

    Google Scholar 

  7. P. Ekman and W. V. Friesen. The repertoire of nonverbal behavior: categories, origins, usage, and coding. Semiotica, 1 (1): 49–98, 1969.

    Google Scholar 

  8. C. Mignot. Usage de la parole et du geste dans les interfaces multimodales — tude exprimentale et modlisation. Doctorat de l’Universit Henri Poincar, Nancy, 1995.

    Google Scholar 

  9. C. Mignot, C. Valot, and N. Carbonell. An experiment study of future ‘natural’ multimodal human-computer interaction. In INTERCHI’93,pages 67–69, Amsterdam, April 1993. New York: ACM Press, Addison Wesley.

    Google Scholar 

  10. B. Rim and L. Schiaratura. Gesture and speech. In R. S. Feldman and B. Rim, editors, Fundamentals of nonverbal behavior, chapter 7, pages 229–238. Cambridge University Press, 1991.

    Google Scholar 

  11. B. Shneiderman. The future of interactive systems and the emergence of direct manipulation. Behaviour and Information Technology, 1 (3): 237–256, 1982.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag London

About this paper

Cite this paper

Robbe, S., Carbonell, N., Dauchy, P. (1997). How Do Users Manipulate Graphical Icons? An Empirical Study. In: Harling, P.A., Edwards, A.D.N. (eds) Progress in Gestural Interaction. Springer, London. https://doi.org/10.1007/978-1-4471-0943-3_16

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0943-3_16

  • Publisher Name: Springer, London

  • Print ISBN: 978-3-540-76094-8

  • Online ISBN: 978-1-4471-0943-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics