Skip to main content
SpringerLink
Log in

Gestures vs. Prosodic Structure in Laboratory Ironic Speech

  • Conference paper
  • First Online:
Speech and Computer (SPECOM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14338))

Included in the following conference series:

  • 456 Accesses

Abstract

The current paper deals with paralinguistic means of irony expression in Russian laboratory speech and their synchronization with different parts of prosodic structure of the utterance. The role that gestures and prosodic characteristics of laboratory speech play in irony perception is also studied. The research is based on the part of the multimodal ironic speech corpus built at the Department of Phonetics of Saint Petersburg State University. The total of 744 audio-visual stimuli were analysed using Praat, Wave Assistant and ELAN software and then corrected manually. The material includes pairs of homonymous ironic and non-ironic target utterances that allowed making a consistent comparison. The results of the acoustic and paralinguistic analyses showed that gestures are synchronized with the nucleus in most of ironic and non-ironic utterances, the head movements prevail in both speech types, the repetitive head movements are more frequent in ironic speech. As opposed to the actors’ speech in films and series, there is no significant correspondence between pitch movement and gesture direction. Series of perceptual experiments, in which single- and multichannel signals were suggested, confirms the hypothesis about prevalence of the visual channel.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Attardo, S., Eisterhold, J., Hay, J., Poggi, I.: Multimodal markers of irony and sarcasm. Humor Int. J. Humor Res. 16(2), 243–260 (2003)

    Google Scholar 

  2. Attardo, S., Pickering, L., Baker, A.A.: Prosodic and multimodal markers of humor in conversation. Pragmatics Cogn. 19(2), 224–247 (2011)

    Google Scholar 

  3. Badin, P., Bailly, G., Revéret, L., Baciu, M., Segebarth, C., Savariaux, C.: Three-dimensional articulatory modeling of tongue, lips and face, based on MRI and video images. J. Phonetics 30(3), 533–553 (2002)

    Google Scholar 

  4. Birdwhistell, R.L.: Kinesics and context: essays on body motion. Communication University of Pennsylvania Press, Philadelphia, PA (1970)

    Google Scholar 

  5. Bolinger, D.: Intonation and gesture. Am. Speech 58, 156–174 (1983)

    Google Scholar 

  6. Braun, A., Schmiedel, A.: The phonetics of ambiguity: a study on verbal irony. Cultures and Traditions of Wordplay and Wordplay Research, De Gruyter, Berlin, pp. 111–136 (2018)

    Google Scholar 

  7. Caucci, G.M., Kreuz, R.J.: Social and paralinguistic cues to sarcasm. Humor: Int. J. Humor Res. 25(1), 1–22 (2012)

    Google Scholar 

  8. Gonsalez-Fuente, S., Escandell-Vidal, V., Prieto, P.: Gestural codas pave the way to the understanfing of verbal irony. J. Pragmatics 90, 26–47 (2015)

    Google Scholar 

  9. Graf, H.P., Cosatto, E., Strom, V., Huang, F.J.: Visual prosody: facial movements accompanying speech. In: Proceedings of AFGR, pp. 96–102 (2002)

    Google Scholar 

  10. Grice, H.P.: Logic and Conversation. Speech Acts [Syntax and Semantics 3]. eds. Peter Cole and Jerry Morgan, Academic Press, New York, pp. 41–58 (1975)

    Google Scholar 

  11. Grishina, E.A.: Russian gesticulation from a linguistic point of view (corpus studies) [Russkaya zhestikulyaciya s lingvisticheskoj tochki zreniya (korpusnye issledovaniya)] - M.: YASK: Yazyki slavyanskoj kul'tury (2017)

    Google Scholar 

  12. Haiman, J.: Talk Is Cheap: Sarcasm, Alienation, and the Evolution of Language. Oxford University Press (1998)

    Google Scholar 

  13. Karpin´ski, M., Jarmołowicz-Nowikow, E., Malisz, Z.: Aspects of gestural and prosodic structure of multimodal utterances in Polish task-oriented dialogues. In: Speech and Language Technology, vol. 11, pp. 113–122 (2009)

    Google Scholar 

  14. Kendon, A.: Some relationships between body motion and speech. In: Seigman, A., Pope, B. (eds.) Studies in Dyadic Communication, pp. 177–216. Pergamon Press, Elmsford (1972)

    Google Scholar 

  15. Kendon, A.: Gesticulation and speech: two aspects of the process of utterance. In: Key, M.R. (ed.) The Relationship of Verbal and Nonverbal Communication, pp. 207–227. Mouton and Co., The Hague (1980)

    Google Scholar 

  16. Kendon, A.: Gesture: Visible Action as Utterance. Cambridge University Press (2004)

    Google Scholar 

  17. Kibrik, A.A.: Multimodal linguistics. A. A. Kibrik // Cognitive research: collection of scientific papers [Mul'timodal'naya lingvistika / A. A. Kibrik // Kognitivnye issledovaniya : sbornik nauchnyh trudov]. vol. 4. – Moscow : “Institut psihologii RAN”, pp. 135–152 (2010)

    Google Scholar 

  18. Kita, S.: The temporal relationship between gesture and speech: A study of Japanese-English bilinguals. Chicago, IL: Department of Psychology, University of Chicago. – 1990 (1990)

    Google Scholar 

  19. Kochetkova, U.E., Evdokimova, V.V., Skrelin, P.A., German, R.D., Novoselova, D.D.: Interplay of visual and acoustic cues of irony perception: a case study of actor’s speech. Artificial Intelligence and Natural Language, V. 1731, Springer Nature, Cham, pp. 82—94 (2022)

    Google Scholar 

  20. Kreuz, R.J., Roberts, R.M.: Two cues for verbal irony: hyperbole and the ironic tone of voice. Metaphor Symbol. Activity 10(1), 21–31 (1995)

    Google Scholar 

  21. Lapaire, J.-R.: Grammar, gesture and cognition: insights from multimodal utterances and applications for gesture analysis. Visnyk of Lviv University. Philology Series 52, 88–103 (2011)

    Google Scholar 

  22. McNeill, D.: Hand and mind: What gestures reveal about thought. University of Chicago Press (1992)

    Google Scholar 

  23. Morris, T., Elshehry, O.: Real-Time Fingertip Detection for Hand Gesture Recognition. Advanced Concepts for Intelligent Vision Systems (2002)

    Google Scholar 

  24. Munhall, K.G., Jones. J.A., Callan, D.E., Kuratate, T., Vatikiotis-Bateson, E.: Visual prosody and speech intelligibility. Psychol. Sci. 15(2), 133–137 (2004)

    Google Scholar 

  25. Nobe, S.: Representational gestures, cognitive rhythms, and acoustic aspects of speech: a network/threshold model of gesture production. Unpublished dissertation, University of Chicago (1996)

    Google Scholar 

  26. Nobe, S.: Where to most spontaneous representational gestures actually occur with respect to speech? In: McNeill, D. (ed.) Language and Gesture Cambridge University Press, pp. 186–198 (2000)

    Google Scholar 

  27. Searle, R.J.: Expression and Meaning: Studies in the Theory of Speech Acts. Cambridge University Press, Cambridge (1979)

    Google Scholar 

  28. Simpson, P.: That’s not Ironic, that’s just Stupid!: towards an eclectic account of the discourse of irony. In: Dynel, M. (eds.) The Pragmatics of Humour across Discourse Domains, John Benjamins, pp. 33–50 (2011)

    Google Scholar 

  29. Sperber, D., Wilson, D.: Relevance: Communication and Cognition. Oxford: Blackwell (1986/1995)

    Google Scholar 

  30. Wagner, P., Malisz, Z., Kopp, S.: Gesture and speech in interaction: an overview. Speech Commun. 57, 209–232 (2014)

    Google Scholar 

  31. Williams, J.A., Burns, E.L., Harmon, E.A.: Insincere utterances and gaze: Eye contact during sarcastic statements. Perceptual and Motor Skills 108, 565–572 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saint Petersburg University, Saint Petersburg, 199034, Russia

    Polina Vasileva, Uliana Kochetkova & Pavel Skrelin

Authors
  1. Polina Vasileva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Uliana Kochetkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pavel Skrelin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uliana Kochetkova .

Editor information

Editors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia

    Alexey Karpov

  2. Koneru Lakshmaiah Education Foundation, Vaddeswaram, India

    K. Samudravijaya

  3. Indian Institute of Information Technology Dharwad, Dharwad, India

    K. T. Deepak

  4. Indian Institute of Technology Dharwad, Dharwad, India

    Rajesh M. Hegde

  5. KIIT Group of Colleges, Gurugram, India

    Shyam S. Agrawal

  6. Indian Institute of Technology Dharwad, Dharwad, India

    S. R. Mahadeva Prasanna

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vasileva, P., Kochetkova, U., Skrelin, P. (2023). Gestures vs. Prosodic Structure in Laboratory Ironic Speech. In: Karpov, A., Samudravijaya, K., Deepak, K.T., Hegde, R.M., Agrawal, S.S., Prasanna, S.R.M. (eds) Speech and Computer. SPECOM 2023. Lecture Notes in Computer Science(), vol 14338. Springer, Cham. https://doi.org/10.1007/978-3-031-48309-7_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-48309-7_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-48308-0

  • Online ISBN: 978-3-031-48309-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation