Abstract
Nonverbal behavior plays a crucial role in human communication and often leads to misunderstandings between people from different cultures, even if they speak the same language fluently. While translation systems are available for verbal communication, translators for nonverbal communication do not exist yet. We present the conceptual design and an early prototype of a real-time gesture translator using body tracking and gesture recognition in avatar-mediated intercultural interactions. It contributes to the ambitious goal of bridging between cultures by translating culture-specific gestures to enhance mutual understanding. Possible applications of the gesture translator are discussed as a facilitating tool for global business meetings and as a means of technology-enhanced conflict resolution and prevention.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A demo video can be found at http://www.youtube.com/watch?v=Wp6VPb2EaFU.
References
Andersen PA, Hecht ML, Hoobler GD, Smallwood M (2003) Nonverbal communication across cultures. Cross-cultural and intercultural communication. Sage Publications, Thousand Oaks, pp 73–90
Archer D (1991) A world of gestures: culture and nonverbal communication (video). Berkley Media LLC, Berkley
Archer D (1997) Unspoken diversity: cultural differences in gestures. Qual Sociol 20(1):79–105
Armstrong N, Wagner M (2003) Field guide to gestures. How to identify and interpret virtually every gesture known to man. Quirk Books, Philadelphia
Assan M, Grobel K (1997) Video-based sign language recognition using hidden Markov models. In: Gesture and sign language in human-computer interaction. Proceedings of international gesture workshop, Bielefeld, Germany
Axtell RE (1998) Gestures: the do’s and taboos of body language around the world. Wiley, New York
Barna LM (1994) Stumbling blocks in intercultural communication. In: Samovar LA, Porter RE (eds) Intercultural communication: a reader, 7th ed. Wadsworth Publishing Company, Belmont, pp 337–346
Cooper H, Holt B, Bowden R (2011) Sign language recognition. In: Moeslund TB, Hilton A, Krüger V, Sigal L (eds) Visual Analysis of Humans: Looking at People. Springer, London, pp 539–562
D’Andrade R (1984) Cultural meaning systems. In: Shweder RA, LeVine RA (eds) Culture theory: essays on mind, self, and emotion. Cambridge University Press, Cambridge
de-Graft Aikins A (2011) Nonverbal communication in everyday multicultural life. In: Hook D, Franks B, Bauer MW (eds) The social psychology of communication. Palgrave Macmillan, New York, pp 67–86
Dreuw P, Rybach D, Deselaers T, Zahedi M, Ney H (2007). Speech recognition techniques for a sign language recognition system. In: Proceedings of Interspeech 2007, Antwerp, Belgium, pp 2513–2516
Ekman P, Friesen WV (1969) The repertoire of nonverbal behavior: categories, origins, usage, and coding. Semiotica 1(1):49–98
Ekman P, Friesen WV (1986) A new pan cultural expression of emotion. Motivation Emotion 10:159–168
Goldin-Meadow S (1999) The role of gesture in communication and thinking. Trends Cognit Sci 3(11):419–429
Hall BJ (1992) Theories of culture and communication. Commun Theory 2(1):50–70
Johnson WL, Marsella S, Mote N, Viljalmsson H, Narayanan S, Choi S (2004) Tactical language training system: supporting the rapid acquisition of foreign language and cultural skills. In: InSTIL/ICALL symposium: NLP and speech technologies in advanced language learning systems, Venice
Kendon A (1982) The study of gesture: some remarks on its history. Semiotic Inquiry 2:45–62
Kendon A (1997) Gesture. Annu Rev Anthropol 26:109–128
Liu S, Volcic Z, Gallois C (2011) Introducing intercultural communication. Global cultures and contexts. Sage, Los Angeles
Maniar N, Bennett E (2007) Designing a mobile game to reduce culture shock. In: International conference on advances in computer games technology, ACM Press, Lisbon, pp 252–253
McNeill D (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, Chicago
Morris D, Collett P, Marsh P, O’Shaughnessy M (1979) Gestures, their origins and distribution. Stein and Day, New York
Ong SCW, Ranganath S (2005) Automatic sign language analysis: a survey and the future beyond lexical meaning. IEEE Trans Pattern Anal Mach Intell 27(6):873–891
Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–285
Salomon G (1981) Communication and education: social and psychological interaction. Sage, Beverly Hills
Schneller R (1989) Intercultural and intrapersonal processes and factors of misunderstanding: Implications for multicultural training. Int J Intercult Relat 13:465–484
Starner T, Pentland A (1995) Visual recognition of American Sign Language using hidden Markov models. In: Proceedings of international workshop on automatic face- and gesture-recognition, Zurich
Steed A, Steptoe W, Oyekoya W, Pece F, Weyrich T et al (2012) Beaming: an asymmetric telepresence system. In: Spatial interfaces. IEEE Computer Graphics and Applications, pp 10–17
Stening BW (1979) Problems in cross-cultural contact: a literature review. Internat J Intercult Relat 3:269–313
Ting-Toomey S (1999) Communicating across cultures. Guilford Press, New York
Turk M (2002) Gesture recognition. In: Hale KS, Stanney KM (eds) Handbook of virtual environment technology: design, implementation, and applications. Lawrence Erlbaum, Mahwah, pp 223–238
Zieren J, Canzler U, Bauer B, Kraiss KF (2006) Sign language recognition. In: Kraiss KF (ed) Advanced man-machine interaction. Signals and communication technology. Springer, Berlin, pp 95–139
Acknowledgments
This research was funded by the European Commission under the ICT work program 2009 (Grant No. FP7-ICT-248620). The first author was supported by a Marie Curie Fellowship from the European Commission (Grant No. FP7-PEOPLE-2009-IEF-254277).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hasler, B.S., Salomon, O., Tuchman, P. et al. Real-time gesture translation in intercultural communication. AI & Soc 32, 25–35 (2017). https://doi.org/10.1007/s00146-014-0573-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-014-0573-4