Skip to main content
SpringerLink
Log in

Seeing or Not Seeing Your Conversational Partner: The Influence of Interaction Modality on Prosodic Entrainment

  • Conference paper
  • First Online:
Speech and Computer (SPECOM 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11096))

Included in the following conference series:

  • 1457 Accesses

Abstract

In speech entrainment research, a less investigated yet crucial aspect for modelling multimodal interactive dialogue systems is the influence of interaction modality, i.e. whether conversational partners who are visible to each other would entrain their speech more with respect to when eye contact is inhibited, or not. In our study, we compared prosodic adaptation behaviour (convergence and synchrony) of the same speaker pairs involved in collaborative game sessions under two conditions: audiovisual vs audio-only interaction. Results provide a complex picture, with a tendency to enhance vocal entrainment when the speech/audio channel is the only one available to conversational partners.

M. Refice—IEEE Life Senior Member.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Levitan, R., Hirschberg, J.: Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions. In: Proceedings of Interspeech 2011, Florence, pp. 28–31 (2011)

    Google Scholar 

  2. de Looze, C., Rauzy, S.: Measuring speakers’ similarity in speech by means of prosodic cues: methods and potential. In: Proceedings of Interspeech 2011, Florence, pp. 1393–1396 (2011)

    Google Scholar 

  3. Pardo, J.S.: On phonetic convergence during conversational interaction. J. Acoust. Soc. Am. 119(4), 2382–2393 (2006)

    Article  Google Scholar 

  4. Delaherce, E., Chetouani, M., Mahdhaoul, A., Saint-Georges, C., Viaux, S., Cohen, D.: Interpersonal synchrony: a survey of evaluation methods across disciplines. IEEE Trans. Affect. Comput. 3(3), 349–365 (2012)

    Article  Google Scholar 

  5. Mukherjee, S., D’Ausilio, A., Nguyen, N., Fadiga, L., Badino, L.: The relationship between F0 synchrony and speech convergence in dyadic interaction. In: Proceedings of Interspeech 2017, Stockholm, pp. 2341–2345 (2017)

    Google Scholar 

  6. Pickering, M.J., Garrod, S.: Toward a mechanistic psychology of dialogue. Behav. Brain Sci. 27, 105–110 (2004)

    Google Scholar 

  7. Giles, H., Coupland, N., Coupland, J.: Accommodation theory: communication, context and consequence. In: Giles, H., Coupland, N., Coupland, J. (eds.) Contexts of Accommodation: Developments in Applied Sociolinguistics, pp. 1–68. CUP, Cambridge (1991)

    Chapter  Google Scholar 

  8. Krauss, R.M., Pardo, J.: Is alignment always the result of automatic priming? Behav. Brain Sci. 27(2), 203–204 (2004)

    Article  Google Scholar 

  9. Lewandowski, N.: Automaticity and consciousness in phonetic convergence. In: Proceedings of the International Workshop “The Listening Talker”, Edinburgh, p. 71 (2012)

    Google Scholar 

  10. Vinciarelli, A., Pantic, M., Bourland, H.: Social signal processing: survey of an emerging domain. Image Vis. Comput. 27, 1743–1759 (2009)

    Article  Google Scholar 

  11. Benus, S.: Social aspects of entrainment in spoken interaction. Cogn. Comput. 6, 802–813 (2014)

    Article  Google Scholar 

  12. Dias, J.W., Rosenblum, L.D.: Visual influences on interactive speech alignment. Perception 40, 1457–1466 (2011)

    Article  Google Scholar 

  13. Dias, J.W., Rosenblum, L.D.: Visual enhancement of alignment in noisy speech. In: Proceedings of ICA 2013, Montreal, pp. 1–6 (2013). (Proceedings of Meetings of Acoustics, vol. 19, p. 060139. JASA)

    Google Scholar 

  14. Schweitzer, K., Walsh, M., Schweitzer, A.: To see or not to see: interlocutor visibility and likeability influence convergence in intonation. In: Proceedings of Interspeech 2017, Stockholm, pp. 919–923 (2017)

    Google Scholar 

  15. Savino, M., Lapertosa, L., Caffò, A., Refice, M.: Measuring prosodic entrainment in italian collaborative game-based dialogues. In: Ronzhin, A., Potapova, R., Németh, G. (eds.) SPECOM 2016. LNCS (LNAI), vol. 9811, pp. 476–483. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-43958-7_57

    Chapter  Google Scholar 

  16. PAGE (Prosodic And Gestural Entrainment in conversational interactions across diverse languages) project funded by the Volkswagen Foundation. http://page.home.amu.edu.pl/

  17. Wilkes-Gibbs, D., Clark, H.H.: Coordinating beliefs in conversation. J. Mem. Lang. 31, 183–194 (1992)

    Article  Google Scholar 

  18. Manson, J.H., Bryant, G.A., Gervais, M.M., Kline, M.A.: Convergence of speech rate in conversation predicts cooperation. Evol. Hum. Behav. 34, 419–426 (2013)

    Article  Google Scholar 

  19. Kim, M., Horton, W.S., Bradlow, A.R.: Phonetic convergence in spontaneous conversation as a function of interlocutor language distance. J. Lab. Phonol. 2, 125–156 (2011)

    Google Scholar 

  20. Levitan, R., Gravano, A., Willson, L., Benus, S., Hirschberg, J., Nenkova, A.: Acoustic-prosodic entrainment and social behaviour. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, pp. 11–19 (2012)

    Google Scholar 

  21. Pardo, J., Gibbons, R., Suppes, A., Krauss, R.: Phonetic convergence in college roommates. J. Phon. 40, 190–197 (2012)

    Article  Google Scholar 

  22. Babel, M.: Evidence for phonetic and social selectivity in spontaneous phonetic imitation. J. Phon. 40, 177–189 (2012)

    Article  Google Scholar 

  23. Namy, L.L., Nygaard, L.C., Sauerteig, D.: Gender differences in vocal accommodation: the role of perception. J. Lang. Soc. Psychol. 21(4), 422–432 (2002)

    Article  Google Scholar 

  24. Boersma, P.: Praat, a system for doing phonetics by computer. Glot Int. 5(9/10), 131–151 (2001)

    Google Scholar 

  25. Eldlund, J., Heldner, M., Hirschberg, J.: Pause and gap length in face-to-face interaction. In: Proceedings of Interspeech 2009, Brighton, pp. 2779–2782 (2009)

    Google Scholar 

  26. Schweitzer, A., Lewandowski, N.: Convergence of articulation rate in spontaneous speech. In: Proceedings of Interspeech 2013, Lyon, pp. 525–529 (2013)

    Google Scholar 

  27. Dale, R., Fusaroli, R., Håkonsson, D.D., Healey, P., Mønster, D., McGraw, J., Mitkidikis, P., Tylen, K.: Beyond synchrony: complementarity and asynchrony in joint action. In: Proceedings of the Annual Meeting of the Cognitive Science Society, Austin, TX, vol. 35, pp. 79–80 (2013)

    Google Scholar 

  28. Watanabe, T.: Human-entrained embodied interaction and communication technology for advanced media society. In: Proceedings of the 16th IEEE International Conference on Robot & Human Interactive Communication, Jieu, pp. 31–36 (2007)

    Google Scholar 

  29. Yoonjeong, L., Gordon Danner, S., Parrell, B., Sungbok, L., Goldstein, L., Byrd, D.: Prosodic convergence during and after a cooperative maze task. In: LabPhon 2015 Conference, Abstract 245, Cornell (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Bari, Bari, Italy

    Michelina Savino & Loredana Lapertosa

  2. New York, USA

    Mario Refice

Authors
  1. Michelina Savino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Loredana Lapertosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Refice
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michelina Savino .

Editor information

Editors and Affiliations

  1. SPIIRAS, St. Petersburg, Russia

    Alexey Karpov

  2. Leipzig University of Telecommunications, Leipzig, Germany

    Oliver Jokisch

  3. Moscow State Linguistic University, Moscow, Russia

    Rodmonga Potapova

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Savino, M., Lapertosa, L., Refice, M. (2018). Seeing or Not Seeing Your Conversational Partner: The Influence of Interaction Modality on Prosodic Entrainment. In: Karpov, A., Jokisch, O., Potapova, R. (eds) Speech and Computer. SPECOM 2018. Lecture Notes in Computer Science(), vol 11096. Springer, Cham. https://doi.org/10.1007/978-3-319-99579-3_59

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-99579-3_59

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-99578-6

  • Online ISBN: 978-3-319-99579-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation