Skip to main content
SpringerLink
Log in

The Harmonia Corpus – A Dialogue Corpus for Automatic Analysis of Phonetic Convergence

  • Conference paper
  • First Online:
Human Language Technology. Challenges for Computer Science and Linguistics (LTC 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12598))

Included in the following conference series:

Abstract

The work presents the creation of a dialogue corpus for analysis and formal evaluation of phonetic convergence in spoken dialogues in human-human and human-machine communication, with the goal of comparing dialogue features at all levels of language use. The Harmonia corpus was created within a project which aims at (1) extracting phonetic features which can be mapped on a synthetic signal, (2) creating dialogue models applicable in a human-machine interaction and (3) practical evaluation of the convergence. For the corpus the following language groups were recorded: 16 pairs of Polish speakers speaking Polish (native speech), 10 pairs of German speakers speaking German (native speech), 12 pairs of German and Polish speakers speaking Polish (non-native speech), and 10 pairs of Polish and German speakers speaking German (non-native speech). The speakers could hear each other, but could not see each other. The recording scenarios consisted of controlled, neutral and expressive tasks and provided over 27 h of speech. This scenario combination is novel and promises to provide an empirical foundation for both linguistic and computational dialogue modelling of both face-to-face and man-machine dialogue.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The German dialogues between the German native speakers and Polish L1/German L2 speakers were recorded at Saarland University by the Phonetic group led by Prof. Dr. Bernd Möbius who was the partner of the Harmonia project.

References

  1. Bachan, J.: Modelling semantic alignment in emergency dialogue. In: Proceedings of 5th Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, Poznań, Poland, 25–27 November 2011, pp. 324–328 (2011)

    Google Scholar 

  2. Bachan, J.: Communicative alignment of synthetic speech. Ph.D. thesis. Institute of Linguistics, Adam Mickiewicz University, Poznań, Poland (2011)

    Google Scholar 

  3. Baumann, S., Grice, M.: The intonation of accessibility. J. Pragmat. 38, 1636–1657 (2006)

    Article  Google Scholar 

  4. Beňuš, Š.: Social aspects of entrainment in spoken interaction. Cogn. Comput. 6(4), 802–813 (2014). https://doi.org/10.1007/s12559-014-9261-4

    Article  Google Scholar 

  5. Boersma, P., Weenink, D.: PRAAT, a system for doing phonetics by computer. Glot Int. 5(9/10), 341–345 (2001)

    Google Scholar 

  6. Carlson, R., Edlund, J., Heldner, M., Hjalmarsson, A., House, D., Skantze, G.: Towards human-like behaviour in spoken dialog systems. In: Proceedings of Swedish Language Technology Conference (SLTC), Gothenburg, Sweden (2006)

    Google Scholar 

  7. Demenko, G. (ed.): Phonetic Convergence in Spoken Dialogues in View of Speech Technology Applications, Akademicka Oficyna Wydawnicza EXIT, Warszawa (in press)

    Google Scholar 

  8. Demenko, G., Bachan, J.: Annotation specifications of a dialogue corpus for modelling phonetic convergence in technical systems. In: Studientexte zur Sprachkommunikation - Proceedings of 28th Conference on Electronic Speech Signal Processing (ESSV), Saarbrücken, Germany, 15–17 March 2017 (2017)

    Google Scholar 

  9. Duran, D., Lewandowski, N.: Cognitive factors in speech production and perception: a socio-cognitive model of phonetic convergence. In: Matešić, M., Memišević, A. (eds.) Language and Mind: Proceedings from the 32nd International Conference of the Croatian Applied Linguistics Society, pp. 15–31. Peter Lang, Berlin (2020)

    Google Scholar 

  10. Edlund, J., Gustafson, J., Heldnera, M., Hjalmarssona, A.: Towards human-like spoken dialogue systems. Speech Commun. 50(8–9), 630–645 (2008)

    Article  Google Scholar 

  11. Edlund, J., Heldner, M., Gustafson, J.: Two faces of spoken dialogue systems. In: Inter-speech 2006. Pittsburgh, PA, USA (2006)

    Google Scholar 

  12. Gessinger, I., Raveh, E., Le Maguer, S., Möbius, B., Steiner, I.: Shadowing synthesized speech – segmental analysis of phonetic convergence. In: ISCA, pp. 3797–3801 (2017)

    Google Scholar 

  13. Gibbon, D., Moore, R., Winski, R.: Handbook of Standards and Resources for Spoken Language Systems. Mouton de Gruyter, Berlin (1997)

    Google Scholar 

  14. Giles, H.: Accent mobility: a model and some data. Anthropol. Linguist. 15, 87–105 (1973)

    Google Scholar 

  15. Giles, H., Coupland, N., Coupland, J.: Accommodation theory: communication, context, and consequence. In: Giles, H., Coupland, N., Coupland, J. (eds.) Contexts of Accommodation: Developments in Applied Sociolinguistics, pp. 1–68. Cambridge University Press (1991)

    Google Scholar 

  16. Gorisch, J., Wells, B., Brown, G.: Pitch contour matching and interactional alignment across turns: an acoustic investigation. Lang. Speech 55, 57–76 (2012)

    Article  Google Scholar 

  17. Hanuka, A.: Underworld. http://www.asafhanuka.com/underground. Accessed 02 Nov 2020

  18. Jankowska, K., Kuczmarski, T., Demenko, G.: Human converging responses to natural speech and synthesized speech. Lingua Posnaniensis (in press)

    Google Scholar 

  19. Lelong, A., Bailly, G.: Study of the phenomenon of phonetic convergence thanks to speech dominoes. In: Esposito, A., Vinciarelli, A., Vicsi, K., Pelachaud, C., Nijholt, A. (eds.) Analysis of Verbal and Nonverbal Communication and Enactment. The Processing Issues. LNCS, vol. 6800, pp. 273–286. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25775-9_26

    Chapter  Google Scholar 

  20. Maleszewski, P.: Analiza iloczasu polskich samogłosek w dialogach (Eng. Analysis of the Polish vowel length in dialogues). MA thesis. Institute of Ethnolinguistics, Adam Mickiewicz University, Poznań, Poland (2020)

    Google Scholar 

  21. Oertel, C., Gustafson, J., Black, A.: On data driven parametric backchannel synthesis for expressing attentiveness in conversational agents. In: Proceedings of Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction (MA3HMI), Satellite Workshop of ICMI 2016 (2016)

    Google Scholar 

  22. Pardo, J.S.: On phonetic convergence during conversational interaction. J. Acoust. Soc. Am. 119, 2382–2393 (2006)

    Article  Google Scholar 

  23. Patoleta, R.: Penis na krzyżu – gdzie przebiegają granice prowokacji? http://robertpatoleta.bloog.pl/id,5640692,title,penis-na-krzyzu-gdzie-przebiegaja-graniceprowokacji,index.html. Accessed 15 Jan 2016

  24. Pickering, M.J., Garrod, G.: Toward a mechanistic psychology of dialogue. Behav. Brain Sci. 27, 169–225 (2004)

    Google Scholar 

  25. Pikus, S.: An analysis of speech alignment in German dialogues between native speakers of German and Polish. MA thesis. Institute of Ethnolinguistics, Adam Mickiewicz University, Poznań, Poland (2020)

    Google Scholar 

  26. Porzel, R., Baudis, M.: The Tao of CHI: towards effective human-computer interaction. In: Dumais, S., Roukos, S. (eds.) HLT-NAACL 2004: Main Proceedings (Boston, Massachusetts, USA, 2–7 May 2004), pp. 209–216. Association for Computational Linguistics (2004)

    Google Scholar 

  27. Porzel, R., Scheffler, A., Malaka, R.: How entrainment increases dialogical efficiency. In: Proceedings of Workshop on Effective Multimodal Dialogue Interfaces, Sydney (2006)

    Google Scholar 

  28. Savino, M., Lapertosa, L., Caffò, A., Refice, M.: Measuring prosodic entrainment in Italian collaborative game-based dialogues. In: Ronzhin, A., Potapova, R., Németh, G. (eds.) SPECOM 2016. LNCS (LNAI), vol. 9811, pp. 476–483. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-43958-7_57

    Chapter  Google Scholar 

  29. Sonar X1 LE. https://www.roland.fi/products/sonar_x1_le/. Accessed 11 Mar 2017

  30. van Engen, K.J., Baese-Berk, M., Baker, R.E., Choi, A., Kim, M., Bradlow, A.R.: The wildcat corpus of native-and foreign-accented English: communicative efficiency across conversational dyads with varying language alignment profiles. Lang. Speech 53(4), 510–540 (2010)

    Article  Google Scholar 

  31. Ward, A., Litman, D.: Automatically measuring lexical and acoustic/prosodic convergence in tutorial dialog corpora. In: Proceedings of the SLaTE Workshop on Speech and Language Technology in Education (2007)

    Google Scholar 

Download references

Acknowledgements

The present study was supported by the Polish National Science Centre, Harmonia project no.: 2014/14/M/HS2/00631, “Automatic analysis of phonetic convergence in speech technology systems” and was conducted in cooperation with the project partner Prof. Bernd Möbius who was the leader of a project “Phonetic convergence in Human-Machine Communication”. More about the Harmonia project can be found at: http://wczt.pl/technologia_mowy/speech_convergence.html

Author information

Authors and Affiliations

  1. Faculty of Modern Languages and Literatures, Adam Mickiewicz University in Poznań, al. Niepodległości 4, 61-874, Poznań, Poland

    Jolanta Bachan & Grażyna Demenko

  2. Poznan Supercomputing and Networking Center, ul. Jana Pawła II 10, 61-139, Poznań, Poland

    Mariusz Owsianny

Authors
  1. Jolanta Bachan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariusz Owsianny
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Grażyna Demenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jolanta Bachan .

Editor information

Editors and Affiliations

  1. Adam Mickiewicz University, Poznań, Poland

    Zygmunt Vetulani

  2. Laboratoire d’Informatique pour la Méca, Orsay, France

    Patrick Paroubek

  3. Adam Mickiewicz University, Poznań, Poland

    Marek Kubis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bachan, J., Owsianny, M., Demenko, G. (2020). The Harmonia Corpus – A Dialogue Corpus for Automatic Analysis of Phonetic Convergence. In: Vetulani, Z., Paroubek, P., Kubis, M. (eds) Human Language Technology. Challenges for Computer Science and Linguistics. LTC 2017. Lecture Notes in Computer Science(), vol 12598. Springer, Cham. https://doi.org/10.1007/978-3-030-66527-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-66527-2_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-66526-5

  • Online ISBN: 978-3-030-66527-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation