Abstract
Non-verbal communication is important in order to maintain fluency of communication. Gestures, facial expressions and eye-gazing function as non-verbal means to convey feedback and provide subtle cues to control and organise conversations. In this paper, verbal and non-verbal feedback are discussed from the point of view of how they contribute to the communicative activity in conversations, especially the type of strategies that the speakers deploy when they aim to construct shared understanding of the tasks and duties in interaction in general. The study concerns conversational data, collected for the purposes of designing and developing more natural interactive systems.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., Paggio, P.: The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. In: Martin, J.C., Paggio, P., Kuenlein, P., Stiefelhagen, R., Pianesi, F. (eds.) Multimodal corpora for modelling human multimodal behaviour. Special issue of the International Journal of Language Resources and Evaluation, vol. 41(3–4), pp. 273–287 (2007)
André, E., Pelachaud, C.: Interacting with Embodied Conversational Agents. In: Jokinen, K., Cheng, F. (eds.) Speech-based Interactive Systems: Theory and Applications. Springer, Heidelberg (2009)
Bennewitz, M., Faber, F., Joho, D., Behnke, S.: Fritz - A Humanoid Communication Robot. In: Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (2007)
Campbell, N.: On the Use of NonVerbal Speech Sounds in Human Communication. In: Esposito, A., Faundez-Zanuy, M., Keller, E., Marinaro, M. (eds.) COST Action 2102. LNCS (LNAI), vol. 4775, pp. 117–128. Springer, Heidelberg (2007)
Campbell, N.: Speech and expression; the value of a longitudinal corpus. In: The 4th International Conference on Language Resources and Evaluation (LREC), pp. 183–186 (2004)
Campbell, N., Ohara, R.: How far can non-verbal information help us follow a conversation? Preliminary experiments with speech-style and gesture tracking. In: Proceedings of ATR Symposium on the Cross-Modal Processing of Faces & Voices. No laughing matter (2005)
Carletta, J.: Announcing the AMI Meeting Corpus. The ELRA Newsletter 11(1), 3–5 (2006)
Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.): Embodied Conversational Agents. MIT Press, Cambridge (2003)
Clark, H.H., Schaefer, E.F.: Contributing to Discourse. Cognitive Science 13, 259–294 (1989)
Douglas, C.E., Campbell, N., Cowie, R., Roach, P.: Emotional speech: Towards a new generation of databases. Speech Communication 40, 33–60 (2003)
Ekman, P.: Universal and cultural differences in facial expression of emotion. In: Cole, J.R. (ed.) Nebraska Symposium on Motivation, pp. 207–283. Nebraska University Press, Lincoln (1972)
Feldman, R.S., Rim, B.: Fundamentals of Nonverbal Behavior. Cambridge University Press, Cambridge (1991)
Jokinen, K.: Constructive Dialogue Management: Speech Interaction and Rational Agents. John Wiley and Sons, Chichester (2009a)
Jokinen, K.: Natural Language and Dialogue Interfaces. In: Stephanidis, C. (ed.) The Universal Access Handbook, Cenveo, ch. 31, pp. 495–506 (2009b)
Jokinen, K.: Non-verbal Feedback in Interactions. In: Tao, J.H., Tan, T.N. (eds.) Affective Information Processing, pp. 227–240. Science+Business Media LLC. Springer, London (2008)
Jokinen, K., Campbell, N.: Non-verbal Information Sources for Constructive Dialogue Management. In: LREC 2008, Marrakech, Marocco (2008)
Jokinen, K., Vanhasalo, M.: Stand-up gestures – Annotation for Communication Management. In: Proceedings of the Multimodal Workshop at Nodalida Conference, Denmark (2009)
Kendon, A.: Gesture: Visible action as utterance. Cambridge University Press, Cambridge (2004)
Koiso, H., Horiuchi, Y., Tutiya, S., Ichikawa, A., Den, Y.: An analysis of turn taking and backchannels based on prosodic and syntactic features in Japanese Map Task dialogs. Language and Speech 41(3-4), 295–321 (1998)
McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2005)
Norros, L., Kaasinen, E., Plomp, J., Rämä, P.: Human-Technology Interaction Research and Design. VTT Roadmap. VTT Industrial Systems, VTT Research Notes 2220. Espoo (2003), http://www.vtt.fi/inf/pdf/tiedotteet/2003/T2220.pdf
Traum, D.: Computational models of grounding in collaborative systems. In: Working Papers of the AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems, pp. 124–131. AAAI, Menlo Park (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jokinen, K. (2009). Gaze and Gesture Activity in Communication. In: Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Intelligent and Ubiquitous Interaction Environments. UAHCI 2009. Lecture Notes in Computer Science, vol 5615. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02710-9_60
Download citation
DOI: https://doi.org/10.1007/978-3-642-02710-9_60
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02709-3
Online ISBN: 978-3-642-02710-9
eBook Packages: Computer ScienceComputer Science (R0)