Skip to main content
SpringerLink
Log in
Book cover

Cognitive Behavioural Systems pp 405–416Cite as

Coding Hand Gestures: A Reliable Taxonomy and a Multi-media Support

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 7403))

Abstract

A taxonomy of hand gestures and a digital tool (CodGest) are proposed in order to describe different types of gesture used by speaker during speech in different social contexts. It is an exhaustive and mutually exclusive categories system to be shared within the scientific community to study multimodal signals and their contribute to the interaction. Classical taxonomies from gesture literature were integrated within a comprehensive taxonomy, which was tested in five different social contexts and its reliability was measured across them through inter-observer agreement indexes. A multi-media tool was realized as digital support for coding gestures in observational research.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bakeman, R., Gottman, J.M.: Observing interaction. An introduction to sequential analysis, II edn. Cambridge University Press, New York (1997)

    Book  Google Scholar 

  2. Bavelas, J.B., Chovil, N., Lawrie, D.A., Wade, A.: Interactive gestures. Discourse Processes 15, 469–489 (1992)

    Article  Google Scholar 

  3. Ekman, P., Friesen, W.V.: The repertoire of nonverbal behavior. Semiotica 1, 49–98 (1969)

    Google Scholar 

  4. Kendon, A.: Gesture and speech: How they interact. In: Wiemann, J.M., Harrison, R.P. (eds.) Nonverbal Interaction, pp. 13–45. Sage Publications, Beverly Hills (1983)

    Google Scholar 

  5. McNeill, D.: Hand and Mind. The University of Chicago Press, Chicago (1992)

    Google Scholar 

  6. Kendon, A.: Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  7. Krauss, R.M., Chen, Y., Chawla, P.: Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us? Adv. Exp. Soc. Psychol. 28, 389–450 (1996)

    Article  Google Scholar 

  8. Poggi, I.: Iconicity in different types of gestures. Gesture 8, 45–61 (2008)

    Article  Google Scholar 

  9. McNeill, D.: So you think gesture are nonverbal? Psychol. Rev. 92, 350–371 (1985)

    Article  Google Scholar 

  10. Kipp, M., Neff, M., Albrecht, I.: An Annotation Scheme for Conversational Gestures: How to economically capture timing and form. Lang. Resour. Eval. 41(3-4), 325–339 (2007)

    Article  Google Scholar 

  11. Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., Paggio, P.: The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. Lang. Resour. Eval. 41(3-4), 273–287 (2007)

    Article  Google Scholar 

  12. Bakeman, R., Gnisci, A.: Sequential observational methods. In: Eid, M., Dieneer, E. (eds.) Handbook of Multimethod Measurement in Psychology, pp. 127–140. American Psychological Association, Washington (2006)

    Chapter  Google Scholar 

  13. Cohen, J.A.: Coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)

    Article  Google Scholar 

  14. Krippendorff, K.: Estimating the reliability, systematic error, and random error of interval data. Educ. Psychol. Meas. 30, 61–70 (1970)

    Article  Google Scholar 

  15. Kendon, A.: Gestures as illocutionary and discourse structure markers in Southern Italian conversation. J. Pragmatics 23, 247–279 (1995)

    Article  Google Scholar 

  16. McNeill, D., Levy, E.T.: Cohesion and gesture. Discourse Processes 16, 363–386 (1993)

    Article  Google Scholar 

  17. Rosenfeld, H.M.: Instrumental affiliative functions of facial and gestural expressions. J. Pers. Soc. Psychol. 4, 65–72 (1966)

    Article  Google Scholar 

  18. Edelmann, R.J., Hampson, S.: Embarrassment in dyadic interaction. Soc. Behav. Personal. 9, 171–178 (1981)

    Article  Google Scholar 

  19. Duncan, S.: Coding manual (2004), Technical Report availale from www.mcneilllab.uchicago.edu

  20. Maricchiolo, F., Livi, S., Bonaiuto, M., Gnisci, A.: Hand gestures and perceived influence in small group interaction. Span. J. Psychol. 14, 755–764 (2011)

    Article  Google Scholar 

  21. Caso, L., Maricchiolo, F., Bonaiuto, M., Vrij, A., Mann, S.: The impact of deception and suspicion on different hand movements. J. Nonverbal Behav. 30, 1–19 (2006)

    Article  Google Scholar 

  22. Jefferson, G.: On the interactional unpackaging of a “Gloss”. Lang. Soc. 14, 435–466 (1985)

    Article  Google Scholar 

  23. Robinson, B.F., Bakeman, R.: ComKappa: A Windows 95 Program for Calculating Kappa and Related Statistics. Behav. Res. Meth. Ins. C. 30, 731–732 (1998)

    Article  Google Scholar 

  24. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378–382 (1971)

    Article  Google Scholar 

  25. Kita, S.: Cross-cultural variation of speech-accompanying gesture: A review. Lang. Cognitive Proc. 24, 145–167 (2009)

    Article  Google Scholar 

  26. Bonaiuto, M., Gnisci, A., Maricchiolo, F.: Struttura e funzioni dei gesti delle mani durante la conversazione: nodi concettuali e possibili sviluppi futuri. Presentation to “Gesture in the Mediterranean: Recent Research in Southern Europe”, Procida, October 21-23 (2005)

    Google Scholar 

  27. Bonaiuto, M., Maricchiolo, F., Orlacchio, T.: Cultura e gestualità delle mani durante la conversazione: un confronto tra donne native dell’Italia e del Burkina Faso. Presentation to Workshop: “Intersoggettività. Identità e Cultura”, Urbino, October 14-15 (2005)

    Google Scholar 

  28. Beattie, G., Shovelton, H.: Iconic hand gestures and predistability of words in context in spontaneous speech. Brit. J. Psychol. 91, 473–492 (2000)

    Article  Google Scholar 

  29. Beattie, G., Shovelton, H.: An experimental investigation of some properties of individual iconic gestures that mediate their communicative power. Brit J. Psychol. 93, 179–192 (2002)

    Article  Google Scholar 

  30. Contento, S., Stame, S.: Déixis verbale et non verbale dans la construction de l’espace interpersonnel. Dialogue Analisis 5, 427–433 (1997)

    Google Scholar 

  31. Feyereisen, P., Havard, I.: Mental imagery and production of hand gestures while speaking in younger and older adults. J. Nonverbal Behav. 23, 153–171 (1999)

    Article  Google Scholar 

  32. Cronbach, L.J., Gleser, G.C., Nanda, H., Rajaratnam, N.: The dependability of behavioural measurement: Theory of generazability for scores and profiles. Wiley, New York (1972)

    Google Scholar 

  33. Maricchiolo, F., Bonaiuto, M., Gnisci, A.: Hand gestures in speech: studies on their roles in social interaction. In: Mondada, L. (ed.) Proceedings of the 2nd ISGS Conference, Interacting Bodies, Lyon-France, June 15-18. Ecole Normale Supérieure Lettres et Sciences humaines, Lyon (2007)

    Google Scholar 

  34. Calbris, G.: Elements of Meaning in Gesture. John Benjamins Publishing, Amsterdam (2011)

    Google Scholar 

  35. Wiener, M., Devoe, S., Rubinow, S., Geller, J.: Nonverbal behavior and nonverbal communication. Psychol. Rev. 79, 185–214 (1972)

    Article  Google Scholar 

  36. Barakat, R.: Arabic gestures. J. Pop. Cult. 6, 749–792 (1973)

    Article  MathSciNet  Google Scholar 

  37. De Jorio, A.: La mimica degli antichi investigata nel gestire napoletano (Ancient’s mimicry investigated through gesturing Neapolitan). Fibreno, Napoli (1832)

    Google Scholar 

  38. Ricci Bitti, P.E., Poggi, I.A.: Symbolic nonverbal behavior: Talking through gestures. In: Feldman, R.S., Rimé, B. (eds.) Fundamentals of Nonverbal Behavior, pp. 433–457. Cambridge University Press, New York (1991)

    Google Scholar 

  39. Morris, D.: Manwatching. A field guide to human behavior. Andromeda Oxford Limited and Jonathan cape Limited, London (1977)

    Google Scholar 

  40. Haviland, J.B.: Pointing, gesture spaces, and mental maps.. In: McNeill, D. (ed.) Language and Gesture, pp. 13–46. Cambridge University Press, Cambridge (2002)

    Google Scholar 

  41. Rosenfeld, H.M.: Instrumental affiliative functions of facial and gestural expressions. J. Personal. Soc. Psycho. 4, 65–72 (1966)

    Article  Google Scholar 

  42. Freedman, N., Hoffman, S.P.: Kinetic behavior in altered clinical states: approach to objective analysis of motor behavior during clinical interviews. Percept. Motor Skill. 24, 527–539 (1967)

    Article  Google Scholar 

  43. Kimura, D.: Manual activity during speaking. Neuropsychologia 11, 45–55 (1973)

    Article  Google Scholar 

  44. Bull, P., Connelly, G.: Body movement and emphasis in speech. J. Nonverbal Behav. 9, 169–187 (1985)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Studi dei Processi Formativi Culturali e Interculturali nella Società Contemporanea, Università degli Studi Roma Tre, via Milazzo 11b, 00185, Rome, Italy

    Fridanna Maricchiolo

  2. Dipartimento di Psicologia, Seconda Università degli Studi di Napoli, Italy

    Augusto Gnisci

  3. Dipartimento di Psicologia dei Processi di Sviluppo e Socializzazione, Sapienza Università di Roma, Italy

    Marino Bonaiuto

Authors
  1. Fridanna Maricchiolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Augusto Gnisci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marino Bonaiuto
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Psychology, and IIASS, Seconda Università degli Studi di Napoli, Italy

    Anna Esposito

  2. Istituto Nazionale di Geofisica e Vulcanologia, sezione di Napoli Osservatorio Vesuviano, Napoli, Italy

    Antonietta M. Esposito

  3. School of Computing Science, University of Glasgow, Glasgow, UK

    Alessandro Vinciarelli

  4. Laboratory of Acoustics and Speech Communication, Technische Universität Dresden, 01062, Dresden, Germany

    Rüdiger Hoffmann

  5. Dept. of Humanities and Social Sciences, Anatolia College/ACT, P.O. Box 21021, 55510, Pylaia, Greece

    Vincent C. Müller

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Maricchiolo, F., Gnisci, A., Bonaiuto, M. (2012). Coding Hand Gestures: A Reliable Taxonomy and a Multi-media Support. In: Esposito, A., Esposito, A.M., Vinciarelli, A., Hoffmann, R., Müller, V.C. (eds) Cognitive Behavioural Systems. Lecture Notes in Computer Science, vol 7403. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34584-5_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34584-5_36

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34583-8

  • Online ISBN: 978-3-642-34584-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation