Skip to main content
SpringerLink
Log in
Book cover

Italian Forum of Ambient Assisted Living

ForItAAL 2019: Ambient Assisted Living pp 177–190Cite as

Preliminary Study of Brain-Inspired Model for Multimodal Human Behavior Detection in Social Context

  • Conference paper
  • First Online:

  • 371 Accesses

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 725))

Abstract

In a near future, robots will permeate our daily life; indeed, they have the potential to proactively support the senior citizen in tedious and different daily tasks (i.e. cleaning, gaming, walking activity, promote socialization). However, to efficiently cooperate with human-beings, robots should have enhanced human–robot interaction capabilities. This work addresses the challenge of designing a robotic model by simulating the modality human beings interact with each other. The first objective of this work concerns the identification of the social cues which correctly describe the user’s emotional and engagement state during the interaction. Based on selected descriptors, a perceptual system has been proposed to detect elderly’s behavior in a social context. The proposed architecture is inspired to the human brain structure as concern the functionalities modules and analogies in the modules’ location. The proposed perceptual system aims at transforming raw data coming from three kinds of sensors (camera, microphone, and laser) into behavioral patterns by mimicking the abstraction evolution which characterizes the consciousness process of the human brain.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://orbbec3d.com/.

References

  1. United Nations Department of Economic and Social Affairs Population Division: E02 World Population Prospects The 2017 Revision: Key Findings and Advance Tables. World Popul. Prospect. 2017 (2017) https://doi.org/10.1017/CBO9781107415324.004

  2. Cortellessa G, Fracasso F, Sorrentino A, Orlandini A, Bernardi G, Coraci L, De Benedictis R, Cesta A (2017) Enhancing the interactive services of a telepresence robot for AAL: Developments and a psycho-physiological assessment. In: Lecture Notes in Electrical Engineering. https://doi.org/10.1007/978-3-319-54283-6_25

  3. Tapus A, Mataric MJ, Scassellati B (2007) Socially assistive robotics [Grand challenges of robotics]. IEEE Robot Autom Mag 14:35–42. https://doi.org/10.1109/MRA.2007.339605

    Article  Google Scholar 

  4. Kaiser FG, Glatte K, Lauckner M (2019) How to make nonhumanoid mobile robots more likable: employing kinesic courtesy cues to promote appreciation. Appl Ergon 78:70–75. https://doi.org/10.1016/j.apergo.2019.02.004

    Article  Google Scholar 

  5. Vinciarelli A, Pantic M, Bourlard H (2009) Social signal processing: survey of an emerging domain. Image Vis Comput 27:1743–1759. https://doi.org/10.1016/j.imavis.2008.11.007

    Article  Google Scholar 

  6. Frith U, Frith CD (2003) Development and neurophysiology of mentalizing.https://doi.org/10.1098/rstb.2002.1218

  7. Fukuda T, Chen F, Shi Q (2018) Special feature on bio-inspired robotics.https://doi.org/10.3390/app8050817

  8. Tononi G, Sporns O (2003) Measuring information integration. BMC Neurosci 4. https://doi.org/10.1186/1471-2202-4-31

  9. Sporns O (2007) What neuro-robotic models can teach us about neural and cognitive development. Neuroconstructivism Perspect Prospect 2:179–204. https://doi.org/10.1093/acprof:oso/9780198529934.003.0008

    Article  Google Scholar 

  10. Balkenius C, Morén J (2001) Emotional learning: a computational model of the amygdala. Cybern Syst 32:611–636

    Article  MATH  Google Scholar 

  11. Raymundo CR, Johnson CG, Vargas PA (2015) An architecture for emotional and context-aware associative learning for robot companions.https://doi.org/10.1109/ROMAN.2015.7333699

  12. Dağlarlı E, Dağlarlı SF, Günel GÖ, Köse H (2017) Improving human-robot interaction based on joint attention. Appl Intell 47:62–82. https://doi.org/10.1007/s10489-016-0876-x

    Article  Google Scholar 

  13. Al-Qaderi MK, Rad AB (2018) A brain-inspired multi-modal perceptual system for social robots: an experimental realization. IEEE Access 6:35402–35424. https://doi.org/10.1109/ACCESS.2018.2851841

    Article  Google Scholar 

  14. Premack D, Woodruff G (1978) Does the chimpanzee have a theory of mind? Behav Brain Sci 1:515–526. https://doi.org/10.1017/S0140525X00076512

    Article  Google Scholar 

  15. Wiltshire TJ, Warta SF, Barber D, Fiore SM (2017) Enabling robotic social intelligence by engineering human social-cognitive mechanisms. Cogn Syst Res 43:190–207. https://doi.org/10.1016/j.cogsys.2016.09.005

    Article  Google Scholar 

  16. Franchi AM, Mutti F, Gini G (2016) From learning to new goal generation in a bioinspired robotic setup. Adv Robot 30:795–805. https://doi.org/10.1080/01691864.2016.1172732

    Article  Google Scholar 

  17. Jones EG (2012) The thalamus. Springer Science & Business Media, Berlin

    Google Scholar 

  18. Cacioppo JT, Tassinary LG, Berntson G (2007) Handbook of psychophysiology

    Google Scholar 

  19. Damasio AR (1989) The brain binds entities and events by multiregional activation from convergence zones. Neural Comput 1:123–132. https://doi.org/10.1162/neco.1989.1.1.123

    Article  Google Scholar 

  20. Scheflen AE (1964) The significance of posture in communication systems. Psychiatry 27:316–331. https://doi.org/10.1080/00332747.1964.11023403

    Article  Google Scholar 

  21. Darwin C, Prodger P (1998) The expression of the emotions in man and animals.https://doi.org/10.1017/CBO9780511694110

  22. Costa M, Dinsbach W, Manstead ASR, Ricci Bitti PE (2001) Social presence, embarrassment, and nonverbal behavior. J Nonverbal Behav 25, 225–240. https://doi.org/10.1023/A:1012544204986

  23. Ekman P (1993) Facial expression and emotion. Am Psychol. https://doi.org/10.1037/0003-066X.48.4.384

    Article  Google Scholar 

  24. Emery NJ (2000) The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci Biobehav Rev 24:581–604. https://doi.org/10.1016/S0149-7634(00)00025-7

    Article  Google Scholar 

  25. Mason MF, Tatkow EP, Macrae CN (2005) The look of love: gaze shifts and person perception. Psychol Sci 16:236–239. https://doi.org/10.1111/j.0956-7976.2005.00809.x

    Article  Google Scholar 

  26. Scherer KR (2003) Vocal communication of emotion: a review of research paradigms. Speech Commun 40:227–256. https://doi.org/10.1016/S0167-6393(02)00084-5

    Article  MATH  Google Scholar 

  27. Scherer KR (1979) Personality markers in speech. In: Social markers in speech. Cambridge University Press

    Google Scholar 

  28. Glass CR, Merluzzi TV, Biever JL, Larsen KH (1982) Cognitive assessment of social anxiety: development and validation of a self-statement questionnaire. Cognit Ther Res 6:37–55. https://doi.org/10.1007/BF01185725

    Article  Google Scholar 

  29. Cavallo F, Limosani R, Fiorini L, Esposito R, Furferi R, Governi L, Carfagni M (2018) Design impact of acceptability and dependability in assisted living robotic applications. Int J Interact Des Manuf 12. https://doi.org/10.1007/s12008-018-0467-7

  30. D’Onofrio G, Fiorini L, de Mul M, Fabbricotti I, Okabe Y, Hoshino H, Limosani R, Vitanza A, Greco F, Giuliani F, Guiot D, Senges E, Kung A, Cavallo F, Sancarlo D, Greco A (2018) Agile Co-creation for robots and aging (ACCRA) project: new technological solutions for older people. Eur Geriatr Med 1–6. https://doi.org/10.1007/s41999-018-0106-7

  31. Cao Z, Simon T, Wei SE, Sheikh Y (2017) Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). pp 7291–7299. https://doi.org/10.1109/CVPR.2017.143

  32. Viola P, Jones M (2001) Robust real-time object detection. Int J Comput Vis 4, 34–47

    Google Scholar 

  33. Admoni H, Scassellati B (2017) Social eye gaze in human-robot interaction: a review. J Human-Robot Interact 6:25–63. https://doi.org/10.5898/jhri.6.1.admoni

    Article  Google Scholar 

  34. Vrigkas M, Nikou C, Kakadiaris IA (2015) A review of human activity recognition methods. Front Robot AI 2. https://doi.org/10.3389/frobt.2015.00028

  35. Baltrušaitis T, Ahuja C, Morency LP (2019) Multimodal machine learning: a survey and taxonomy.https://doi.org/10.1109/TPAMI.2018.2798607

  36. Srivastava N, Salakhutdinov RR (2012) Multimodal learning with deep boltzmann machines. Adv Neural Inf Process Syst 2222–2230. https://doi.org/10.1109/CVPR.2013.49

  37. Lan T, Sigal L, Mori G (2012) Social roles in hierarchical models for human activity recognition. In: IEEE conference on computer vision and pattern recognition. pp 1354–1361. https://doi.org/10.1109/CVPR.2012.6247821

Download references

Acknowledgements

This work was supported by the ACCRA Project, founded by the European Commission—Horizon 2020 Founding Programme (H2020-SCI-PM14-2016) and National Institute of Information and Communications Technology (NICT) of Japan under grant agreement No. 738251.

Author information

Authors and Affiliations

  1. The BioRobotics Institute, Scuola Superiore Sant’Anna, Viale Rinaldo Piaggio, 34, 56025, Pontedera (PI), Italy

    Alessandra Sorrentino, Laura Fiorini, Gianmaria Mancioppi, Olivia Nocentini & Filippo Cavallo

  2. Department of Industrial Engineering, University of Florence, Via Santa Marta 3, 50139, Florence (FI), Italy

    Filippo Cavallo

Authors
  1. Alessandra Sorrentino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura Fiorini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gianmaria Mancioppi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivia Nocentini
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Filippo Cavallo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandra Sorrentino .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione (DII), Università Politecnica delle Marche, Ancona, Italy

    Andrea Monteriù

  2. Dipartimento di Ingegneria dell’Informazione (DII), Università Politecnica delle Marche, Ancona, Italy

    Alessandro Freddi

  3. Dipartimento di Ingegneria dell’Informazione (DII), Università Politecnica delle Marche, Ancona, Italy

    Sauro Longhi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sorrentino, A., Fiorini, L., Mancioppi, G., Nocentini, O., Cavallo, F. (2021). Preliminary Study of Brain-Inspired Model for Multimodal Human Behavior Detection in Social Context. In: Monteriù, A., Freddi, A., Longhi, S. (eds) Ambient Assisted Living. ForItAAL 2019. Lecture Notes in Electrical Engineering, vol 725. Springer, Cham. https://doi.org/10.1007/978-3-030-63107-9_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63107-9_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63106-2

  • Online ISBN: 978-3-030-63107-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation