Skip to main content
Springer Nature Link
Log in

The New Italian Audio and Video Emotional Database

  • Chapter
Development of Multimodal Interfaces: Active Listening and Synchrony

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 5967))

  • 2525 Accesses

Abstract

This paper describes the general specifications and characteristics of the New Italian Audio and Video Emotional Database, collected to improve the COST 2102 Italian Audio and Video Emotional Database [28] and to support the research effort of the COST Action 2102: “Cross Modal Analysis of Verbal and Nonverbal Communication” (http://cost2102.cs.stir.ac.uk/). The database should allow the cross-modal analysis of audio and video recordings for defining distinctive, multi-modal emotional features, and identify emotional states from multimodal signals. Emphasis is placed on stimuli selection procedures, theoretical and practical aspects for stimuli identification, characteristics of selected stimuli and progresses in their assessment and validation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bindu, M., Gupta, P., Tiwary, U.: Cognitive Model-Based Emotion Recognition From Facial Expressions For Live Human Computer Interaction. In: Proceedings of the IEEE Symposium on Computational Intelligence and Signal Processing CIISP (2007)

    Google Scholar 

  2. Ververidis, D., Kotropoulos, C.: Emotional Speech Recognition: Resources, Features and Methods. Elsevier Speech Communication 48(9), 1162–1181 (2006)

    Article  Google Scholar 

  3. Sebe, N., Cohen, I., Gevers, T., Huang, T.: Emotion Recognition Based on Joint Visual and Audio Cues. In: Proceedings of the 18th International Conference on Pattern Recognition, ICPR 2006 (2006)

    Google Scholar 

  4. Kamachi, M., Lyons, M., Gyoba, J.: Japanese Female Facial Expression Database, Psychology Department in Kyushu University, http://www.kasrl.org/jaffe.html

  5. Samaria, F., Harter, A.: The ORL Database of Faces. Cambridge University Press, Cambridge, http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

  6. Roth, D., Yang, M., Ahuja, N.: A SNoW-Based Face Detector. In: Advances in Neural Information Processing Systems, pp. 855–861 (2000)

    Google Scholar 

  7. Ryu, H., Chun, S.S., Sull, S.: Multiple Classifiers Approach for Computational Efficiency in Multi-scale Search Based Face Detection. In: Jiao, L., Wang, L., Gao, X.-b., Liu, J., Wu, F. (eds.) ICNC 2006. LNCS, vol. 4221, pp. 483–492. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  8. Schneiderman, H., Kanade, T.: A Statistical Method for 3D Object Detection Applied to Faces and Cars. In: International Conference on Computer and Pattern Recognition, vol. 1, pp. 746–751 (2000)

    Google Scholar 

  9. Esposito, A., Přinosil, J., Smékal, Z.: Combining Features for Recognizing Emotional Facial Expressions in Static Images. In: Esposito, A., Bourbakis, N.G., Avouris, N., Hatzilygeroudis, I. (eds.) HH and HM Interaction. LNCS (LNAI), vol. 5042, pp. 56–69. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  10. Sung, K., Poggio, T.: Example-Based Learning for View-Based Face Detection. IEEE Transaction on Pattern Analyses and Machine Intelligence 20, 39–51 (1998)

    Google Scholar 

  11. Viola, A.P., Jones, M.J.: Robust Real-Time Face Detection. International Journal of Computer Vision 57(2), 137–154 (2004)

    Google Scholar 

  12. Turk, M., Pentland, A.: Face Recognition Using Eigenfaces. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–591 (1991)

    Google Scholar 

  13. Jollife, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)

    Google Scholar 

  14. Fisher, R.A.: The Statistical Utilization of Multiple Measurements. Annali of Eugenics 8, 376–386 (1938)

    Article  MATH  Google Scholar 

  15. Petkov, N., Wieling, M.B.: Gabor Filtering Augmented with Surround Inhibition for Improved Contour Detection by Texture Suppression. Perception 33, 68c (2004)

    Google Scholar 

  16. Kanade, T., Cohn, J.F.: Comprehensive Database for Facial Expression Analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gestures Recognition, Grenoble, France (2000)

    Google Scholar 

  17. Martin, O., Kotsia, I., Macq, B., Pitas, I.: The eNTERFACE05 Audio-Visual Emotion Database. In: Proceedings of ICDEW (2006)

    Google Scholar 

  18. Douglas-Cowie, E., Cowie, R., Schröder, M.: A New Emotion Database: Considerations, Sources and Scope. In: Proceedings of the ISCA Workshop on Speech and Emotion: A Conceptual Framework for Research, Textflow, Belfast, pp. 39–44 (2000)

    Google Scholar 

  19. Grimm, M., Kroschel, K., Narayanan, S.: The Vera Am Mittag German Audio-Visual Emotional Speech Database. 978-1-4244-2571-6/08/$25.00 ©2008 IEEE (2008)

    Google Scholar 

  20. Davis, S., Mermelstein, P.: Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences. IEEE Transactions, on Acoustic, Speech and Signal Processing 28, 357–366 (1980)

    Article  Google Scholar 

  21. Pao, T., Chen, Y., Yeh, J.: Emotion Recognition from Mandarin Speech Signals, Spoken Language Processing. In: International Symposium on Chinese (2004)

    Google Scholar 

  22. Lee, C., Narayanan, M.: Toward Detecting Emotions in Spoken Dialogs. IEEE Trans. Speech and Audio Process 13, 293–303 (2005)

    Article  Google Scholar 

  23. Schuller, B., Rigoll, G., Lang, M.: Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. In: Proceedingss of International Conference on Acoustics, Speech and Signal Processing (ICASSP 2004), pp. 557–560 (2004)

    Google Scholar 

  24. Schuller, B., Rigoll, G., Lang, M.: Hidden Markov Model-Based Speech Emotion Recognition. In: Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2003, Hong Kong, China, vol. 2 (2003)

    Google Scholar 

  25. Hu, H., Xu, M., Wu, W.: GMM Supervector Based SVM with Spectral Features for Speech Emotion Recognition. In: Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP (2007)

    Google Scholar 

  26. Navas, E., Hernáez, I.: Luengo: An Objective and Subjective Study of the Role of Semantics and Prosodic Features in Building Corpora for Emotional TTS. IEEE Transactions on Audio, Speech, and Language Processing 14, 1117–1127 (2006)

    Article  Google Scholar 

  27. Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A Database of German Emotional Speech. In: Proceedings of Interspeech, pp. 1517–1520 (2005)

    Google Scholar 

  28. Esposito, A., Riviello, M.T., Di Maio, G.: The COST 2102 Italian Audio and Video Emotional Database. In: Apolloni, B., Bassis, S., Morabito, C.F. (eds.) Neural Nets WIRN 2009, vol. 204, pp. 51–61 (2009)

    Google Scholar 

  29. Ekman, P.: Facial Expression of Emotion: New Findings. New Questions. Psychological Science 3, 34–38 (1992)

    Article  Google Scholar 

  30. Oatley, K., Jenk, J.M.: Understanding emotions. Blackwell, Oxford (1996)

    Google Scholar 

  31. Banse, R., Scherer, K.: Acoustic profiles in vocal emotion expression. Journal of Personality & Social Psychology 70(3), 614–636 (1996)

    Article  Google Scholar 

  32. Cacioppo, J.T., Berntson, G.G., Larsen, J.T., Poehlmann, K.M.T.A.: Ito: The Psychophysiology of emotion. In: Lewis, J.M., Haviland-Jones, M. (eds.) Handbook of Emotions, 2nd edn., pp. 173–191. Guilford Press, New York (2000)

    Google Scholar 

  33. Ekman, P.P., Friesen, W.V., Hager, J.C.: The facial action coding system, 2nd edn. Research Nexus eBook, Salt Lake City, Weidenfeld & Nicolson, London (2002)

    Google Scholar 

  34. Ekman, P., Friesen, W.V.: Facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  35. Ekman, P., Friesen, W.V.: Manual for the Facial Action Coding System. Consulting Psychologists Press, Palo Alto (1977)

    Google Scholar 

  36. Izard, C.E., Ackerman, B.P.: Motivational, organizational, and regulatory functions of discrete emotions. In: Lewis, J.M., Haviland-Jones, M. (eds.) Handbook of Emotions, 2nd edn., pp. 253–264. Guilford Press, New York (2000)

    Google Scholar 

  37. Izard, C.E.: Innate and universal facial expressions: Evidence from developmental and cross-cultural research. Psychological Bulletin 115, 288–299 (1994)

    Article  Google Scholar 

  38. Izard, C.E., Dougherty, L.M., Hembree, E.A.: A system for identifying affect expressions by holistic judgments. Unpublished manuscript, Available from Instructional Resource Center, University of Delaware (1983)

    Google Scholar 

  39. Izard, C.E.: The maximally discriminative facial movement coding system (MAX). Unpublished manuscript, Available from Instructional Resource Center, University of Delaware (1979)

    Google Scholar 

  40. Scherer, K.R.: Vocal communication of emotion: A review of research paradigms. Speech Communication 40, 227–256 (2003)

    Article  MATH  Google Scholar 

  41. Scherer, K.R., Banse, R., Wallbott, H.G.: Emotion inferences from vocal expression correlate across languages and cultures. Journal of Cross-Cultural Psychology 32, 76–92 (2001)

    Article  Google Scholar 

  42. Scherer, K.R., Banse, R., Wallbott, H.G., Goldbeck, T.: Vocal cues in emotion encoding and decoding. Motivation and Emotion 15, 123–148 (1991)

    Article  Google Scholar 

  43. Scherer, K.R.: Vocal correlates of emotional arousal and affective disturbance. In: Wagner, H., Manstead, A. (eds.) Handbook of social Psychophysiology, pp. 165–197. Wiley, New York (1989)

    Google Scholar 

  44. Atassi, H., Riviello, M.T., Smékal, Z., Hussain, A., Esposito, A.: Emotional Vocal Expressions Recognition using the COST 2102 Italian Database of Emotional Speech. In: Esposito, A., et al. (eds.) Development of Multimodal Interfaces: Active Listening and Synchrony. Lectures Notes in Computer Science, vol. 5967, pp. 406–422. Springer, Heidelberg (2010)

    Google Scholar 

  45. Esposito, A.: The Amount of Information on Emotional States Conveyed by the Verbal and Nonverbal Channels: Some Perceptual Data. In: Stylianou, Y., Faundez-Zanuy, M., Esposito, A. (eds.) COST 277. LNCS, vol. 4391, pp. 249–268. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  46. Esposito, A.: Affect in Multimodal Information. In: Tao, J., Tan, T. (eds.) Affective Information Processing, pp. 211–234. Springer, Heidelberg (2008)

    Google Scholar 

  47. Esposito, A.: The Perceptual and Cognitive Role of Visual and Auditory Channels in Conveying Emotional Information. Cognitive Computation Journal 1(2), 268–278 (2009)

    Article  Google Scholar 

  48. Esposito, A., Riviello, M.T., Bourbakis, N.: Cultural Specific Effects on the Recognition of Basic Emotions: A Study on Italian Subjects. In: Holzinger, A., Miesenberger, K. (eds.) USAB 2009. LNCS, vol. 5889, pp. 135–148. Springer, Heidelberg (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Second University of Naples, Caserta, Italy

    Anna Esposito & Maria Teresa Riviello

  2. International Institute for Advanced Scientific Studies (IIASS), Vietri sul Mare, Italy

    Anna Esposito & Maria Teresa Riviello

Authors
  1. Anna Esposito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Teresa Riviello
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Second University of Naples, and IIASS, Via Pellegrino, 84019, Vietri sul Mare, SA, Italy

    Anna Esposito

  2. Centre for Language and Communication Studies, Trinity College, The University of Dublin, Dublin 2, Ireland

    Nick Campbell  & Carl Vogel  & 

  3. Department of Computing Science & Mathematics, University of Stirling, FK9 4LA, Stirling, Scotland, UK

    Amir Hussain

  4. Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands

    Anton Nijholt

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Esposito, A., Riviello, M.T. (2010). The New Italian Audio and Video Emotional Database. In: Esposito, A., Campbell, N., Vogel, C., Hussain, A., Nijholt, A. (eds) Development of Multimodal Interfaces: Active Listening and Synchrony. Lecture Notes in Computer Science, vol 5967. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12397-9_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12397-9_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12396-2

  • Online ISBN: 978-3-642-12397-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics