Skip to main content
SpringerLink
Log in

Cross-Corpus Experiments on Laughter and Emotion Detection in HRI with Elderly People

  • Conference paper
  • First Online:
Book cover Social Robotics (ICSR 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9388))

Included in the following conference series:

Abstract

Social Signal Processing such as laughter or emotion detection is a very important issue, particularly in the field of human-robot interaction (HRI). At the moment, very few studies exist on elderly-people’s voices and social markers in real-life HRI situations. This paper presents a cross-corpus study with two realistic corpora featuring elderly people (ROMEO2 and ARMEN) and two corpora collected in laboratory conditions with young adults (JEMO and OFFICE). The goal of this experiment is to assess how good data from one given corpus can be used as a training set for another corpus, with a specific focus on elderly people voices. First, clear differences between elderly people real-life data and young adults laboratory data are shown on acoustic feature distributions (such as \(F_0\) standard deviation or local jitter). Second, cross-corpus emotion recognition experiments show that elderly people real-life corpora are much more complex than laboratory corpora. Surprisingly, modeling emotions with an elderly people corpus do not generalize to another elderly people corpus collected in the same acoustic conditions but with different speakers. Our last result is that laboratory laughter is quite homogeneous across corpora but this is not the case for elderly people real-life laughter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Vinciarelli, A., Pantic, M., Bourlard, H., Pentland, A.: Social signals, their function, and automatic analysis: a survey. In: Conference on Multimodal Interfaces (ACM), Chania, Greece, pp. 61–68 (2008)

    Google Scholar 

  2. Delaborde, A., Devillers, L.: Use of nonverbal speech cues in social interaction between human and robot: emotional and interactional markers. In: International Workshop on Affective Interaction in Natural Environements (AFFINE), Firenze, Italy (2010)

    Google Scholar 

  3. Breazeal, C.: Emotion and sociable humanoid robots. Human Computer Studies 59, 119–155 (2003)

    Article  Google Scholar 

  4. Scherer, S., Glodek, M., Schwenker, F., Campbell, N., Palm, G.: Spotting laughter in natural multiparty conversations: A comparison of automatic online and offline approaches using audiovisual data. ACM Transactions on Interactive Intelligent Systems (TiiS) 2(1), Article No. 4 (2012). Special Issue on Affective Interaction in Natural Environments

    Google Scholar 

  5. Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Communication 53(9), 1062–1087 (2011). Special Issue on Sensing Emotion and Affect - Facing Realism in Speech Processing

    Article  Google Scholar 

  6. Batliner, A., Steidl, S., Nöth, E.: Laryngealizations and emotions: how many babushkas? In: Proc. Internat. Workshop on Paralinguistic Speech - Between Models and Data (ParaLing’ 07), Saarbrucken, Germany, pp. 17–22 (2007)

    Google Scholar 

  7. Batliner, A., Hacker, C., Steidl, S., Nöth, E., D’Arcy, S., Russell, M., Wong, M.: You stupid tin box - children interacting with the aibo robot: a cross-linguistic emotional speech corpus. In: LREC, Lisbon, Portugal, pp. 171–174 (2004)

    Google Scholar 

  8. Delaborde, A., Tahon, M., Barras, C., Devillers, L.: Affective links in a child-robot interaction. In: LREC, Valletta, Malta (2010)

    Google Scholar 

  9. McKeown, G., Valstar, M., Cowie, R., Pantic, M., Schröder, M.: The semaine database: annotated multimodal records of emotionally coloured conversations between a person and a limited agent. IEEE Transactions on Affective Computing 3(1), 5–17 (2012)

    Article  Google Scholar 

  10. Tahon, M., Delaborde, A., Devillers, L.: Real-life emotion detection from speech in human-robot interaction: experiments across diverse corpora with child and adult voices. In: Interspeech, Firenze, Italia (2011)

    Google Scholar 

  11. Chastagnol, C., Clavel, C., Courgeon, M., Devillers, L.: Designing an emotion detection system for a socially-intelligent human-robot interaction. In: Towards a Natural Interaction with Robots, Knowbots and Smartphones: Putting Spoken Dialog Systems into Practice. Springer (2013)

    Google Scholar 

  12. Sehili, M.A., Yang, F., Leynaert, V., Devillers, L.: A corpus of social interaction between nao and elderly people. In: International Workshop on Emotion, Social Signals, Sentiment & Linked Open Data, Satellite of LREC (2014)

    Google Scholar 

  13. Tahon, M., Delaborde, A., Barras, C., Devillers, L.: A corpus for identification of speakers and their emotions. In: LREC, Valletta, Malta (2010)

    Google Scholar 

  14. Schuller, B., Zhang, Z., Weninger, F., Rigoll, G.: Selecting training data for cross-corpus speech emotion recognition: prototypicality vs. generalization. In: AVIOS Speech Processing, Tel-Aviv, Israel (2011)

    Google Scholar 

  15. Tahon, M., Devillers, L.: Laughter detection for on-line human-robot interaction. In: Interdisciplinary Workshop on Laughter and Non-verbal Vocalisations in Speech, Enschede, Netherlands (2015)

    Google Scholar 

  16. Brendel, M., Zaccarelli, R., Devillers, L.: Building a system for emotions detection from speech to control an affective avatar. In: LREC, Valletta, Malta (2010)

    Google Scholar 

  17. Ververidis, D., Kotropoulos, C.: Emotional speech recognition: Ressources, features and methods. Speech Communication 48(9), 1162–1181 (2006)

    Article  Google Scholar 

  18. Schuller, B., Batliner, A.: Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing. John Wiley & Sons (2013)

    Google Scholar 

  19. Bachorowski, J.-A., Smoski, M.J., Owren, M.J.: The acoustic features of human laughter. Journal of the Acoustical Society of America 110(3), 1581–1597 (2001)

    Article  Google Scholar 

  20. Campbell, N.: Perception of affect in speech - towards an automatic processing of paralinguistic information in spoken conversation. In: International Conference on Spoken Language Processing, Jeju Island, Korea (2004)

    Google Scholar 

  21. Szameitat, D.P., Darwin, C.J., Szameitat, A.J., Wildgruber, D., Alter, K.: Formant characteristics of human laughter. Journal of Voice 25(1), 32–38 (2011)

    Article  Google Scholar 

  22. Devillers, L., Tahon, M., Sehili, M., Delaborde, A.: Inference of human beings’ emotional states from speech in human-robot interactions. International Journal of Social Robotics, Special Issue on Developmental Social Robotics (in press, 2015)

    Google Scholar 

  23. Schröder, M.: Experimental study of affect bursts. Speech Communication 40(1–2), 99–116 (2003). Special Session on Speech and Emotion

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Human-Machine Communication Department, LIMSI-CNRS, 91403, Orsay, France

    Marie Tahon, Mohamed A. Sehili & Laurence Devillers

  2. University Paris-Sorbonne IV, 28 Rue Serpente, 75006, Paris, France

    Laurence Devillers

Authors
  1. Marie Tahon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed A. Sehili
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laurence Devillers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marie Tahon .

Editor information

Editors and Affiliations

  1. Robotics and Computer Vision Lab, ENSTA-ParisTech, Palaiseau, France

    Adriana Tapus

  2. Inst für Informatik, Human-Ctr Multimedi, Universität Augsburg, Augsburg, Bayern, Germany

    Elisabeth André

  3. LIMSI-CNRS, Orsay, France

    Jean-Claude Martin

  4. Robotics and Computer Vision Lab, ENSTA-ParisTech, Palaiseau, France

    François Ferland

  5. LIMSI-CNRS, Orsay, France

    Mehdi Ammi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Tahon, M., Sehili, M.A., Devillers, L. (2015). Cross-Corpus Experiments on Laughter and Emotion Detection in HRI with Elderly People. In: Tapus, A., André, E., Martin, JC., Ferland, F., Ammi, M. (eds) Social Robotics. ICSR 2015. Lecture Notes in Computer Science(), vol 9388. Springer, Cham. https://doi.org/10.1007/978-3-319-25554-5_63

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25554-5_63

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25553-8

  • Online ISBN: 978-3-319-25554-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation