Skip to main content
SpringerLink
Log in

Using Emotion Recognition and Temporary Mobile Social Network in On-Board Services for Car Passengers

  • Conference paper
  • First Online:
Smart Cities, Green Technologies, and Intelligent Transport Systems (SMARTGREENS 2022, VEHITS 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1843))

Abstract

In next-generation cars, passengers will have more time for fun and relaxation, as well as the number of unknown passengers traveling together will increase. Thanks to the progress in Artificial Intelligence and Machine Learning techniques, new interaction models could be exploited to develop specialized applications that will be informed of the passengers’ experience. The mood and the emotional state of driver and passengers can be detected, and utilized to improve safety and comfort by taking actions that improve driver and passengers’ emotional state. Temporary Mobile Social Networking (TMSN) is a key functionality that can enhance passengers’ user experience by allowing passengers to form a mobile social group with shared interests and activities for a time-limited period by utilizing their already existing social networking accounts. By minimizing isolation and promoting sociability, TMSN aims to redesign user profiles and interfaces automatically into a group-wise passengers’ profile and a common interface. This work proposes and develops the generation of TMSN-inspired music selection through the Spotify music streaming service. The results obtained are promising and encourage further development toward the concept of in-car entertainment. Finally, we evaluate the performance of lightweight and heavy intelligent models that recognize the emotion of a person from its face, using Raspberry Pi 4 B devices. The results show that it is possible to realize a system with face detector and facial emotion recognition models on edge devices with sufficient performance (Frame per Second) to detect at least emotions expressed through macro-expressions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Athanasopoulou, A., de Reuver, M., Nikou, S., Bouwman, H.: What technology enabled services impact business models in the automotive industry? An explanatory study? Futures 109, 73–83 (2019)

    Article  Google Scholar 

  2. Bilius, L.B., Vatavu, R.D.: A multistudy investigation of drivers and passengers’ gesture and voice input preferences for in-vehicle interactions. J. Intell. Transp. Syst. 25(2), 197–220 (2020)

    Article  Google Scholar 

  3. Connected car report : Opportunities, risk, and turmoil on the road to autonomous vehicles. Strategy (2016). https://www.strategyand.pwc.com/reports/connected-car-2016-study

  4. Rong, Y., Han, C., et al.: Artificial Intelligence Methods in In-Cabin Use Cases: A Survey. IEEE Intelligent Transportation Systems Magazine (2021)

    Google Scholar 

  5. Arena, F., Pau, G., Severino, A.: An overview on the current status and future perspectives of smart cars. Infrastructures. 5, 53 (2020)

    Article  Google Scholar 

  6. Yin, Y., Xia, J., Li, Y., Xu, W., Yu, L.: Group-wise itinerary planning in temporary mobile social network. IEEE Access 7, 83682–83693 (2019)

    Article  Google Scholar 

  7. Aranha, R.V., Corrêa, C.G., Nunes, F.L.: Adapting software with affective computing: a systematic review. IEEE Trans. Affect. Comput. 12(4), 883–899 (2019)

    Article  Google Scholar 

  8. Foglia, P., Zanda, M., Prete, C.A.: Towards relating physiological signals to usability metrics: a case study with a web avatar. WSEAS Trans. Comput. 13, 624 (2014)

    Google Scholar 

  9. Meixner, G.: Retrospective and future automotive infotainment systems—100 years of user interface evolution. In: Meixner, G., Müller, C. (eds.) Automotive User Interfaces, pp. 3–53. Springer International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-49448-7_1

    Chapter  Google Scholar 

  10. Yvkoff, L.: BMW Rolls-Out Its Intelligent Personal Assistant Feature Via Over-The-Air Update. https://www.forbes.com/sites/lianeyvkoff/2019/05/30/bmw-rolls-out-its-intelligent-personal-assistant-feature-via-over-the-air-update/. Accessed Dec 2022

  11. Trends that Will Shape the Future of the Car Industry by 2030. https://www.hyundai.news/eu/stories/12-trends-that-will-shape-the-future-of-the-car-industry-by-2030/. Accessed Dec 2020

  12. Zepf, S., Hernandez, J., et al.: Driver emotion recognition for intelligent vehicles: a survey. ACM Comput. Surv. 53, 1–30 (2020). https://doi.org/10.1145/3388790

    Article  Google Scholar 

  13. Coppola, R., Morisio, M.: Connected car: technologies, issues, future trends. ACM Comput. Surv. (CSUR) 49(3), 1–36 (2016)

    Article  Google Scholar 

  14. Spotify, C.L.: Collaborative Playlist, support.spotify.com/us/ article/collaborative-playlists/. Accessed Dec 2022

    Google Scholar 

  15. Spotify FM, Family Mix, support.spotify.com/us/article/ family-mix/. Accessed Dec 2022

    Google Scholar 

  16. Cimino M.G.C.A., Lazzerini B., Marcelloni F., Castellano G., Fanelli A.M., Torsello M.A.: A collaborative situation-aware scheme for mobile service recommendation. In: Proceedings of the 11th International Conference on Intelligent Systems Design and Applications, pp. 130–135 (2011)

    Google Scholar 

  17. Spotify API, support.spotify.com/us/article/spotify-in-the-car/, accessed Dec 2022

    Google Scholar 

  18. Toisoul, A., Kossaifi, J., Bulat, A., Tzimiropoulos, G., Pantic, M.: Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nat. Mach. Intell. 3(1), 42–50 (2021)

    Article  Google Scholar 

  19. Kuppens, P., Tuerlinckx, F., Russell, J.A., Barrett, L.F.: The relation between valence and arousal in subjective experience. Psychol. Bull. 139(4), 917 (2013)

    Article  Google Scholar 

  20. Lee, Y.-L., Tsung, P.-K., Wu, M.: Technology trend of edge AI. In: 2018 International Symposium on VLSI Design, Automation and Test (VLSI-DAT), pp. 1–2 (2018)

    Google Scholar 

  21. Cimino M.G.C.A., Di Tecco A., Foglia P., et al.: In-car entertainment via group-wise temporary mobile social networking. In: International Conference on Vehicle Technology and Intelligent Transport Systems, VEHITS - Proceedings, pp. 432 – 437 (2022). https://doi.org/10.5220/0011096000003191

  22. MO, Music Ontology, musicontology.com, accessed Dec. 2022

    Google Scholar 

  23. Campanelli, S., Foglia, P., Prete, C.A.: An architecture to integrate IEC 61131–3 systems in an IEC 61499 distributed solution. Comput. Ind. 72, 47–67 (2015)

    Article  Google Scholar 

  24. Ciaramella, A., Cimino, M.G.C.A., Marcelloni, F., Straccia, U.: Combining fuzzy logic and semantic web to enable situation-awareness in service recommendation. In: Bringas, P.G., Hameurlain, A., Quirchmayr, G. (eds.) Database and Expert Systems Applications, pp. 31–45. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15364-8_3

    Chapter  Google Scholar 

  25. Cimino, M.G.C.A., Palumbo, F., Vaglini, G., Ferro, E., Celandroni, N., La Rosa, D.: Evaluating the impact of smart technologies on harbor’s logistics via BPMN modeling and simulation. Inf. Technol. Manage. 18(3), 223–239 (2016). https://doi.org/10.1007/s10799-016-0266-4

    Article  Google Scholar 

  26. Foglia, P., Solinas, M.: Exploiting replication to improve performances of NUCA-based CMP systems. ACM Trans. Embed. Comput. Syst. 13(3s), 1–23 (2014). https://doi.org/10.1145/2566568

    Article  Google Scholar 

  27. Daher, A.W., Rizik, A., Muselli, M., Chible, H., Caviglia, D.D.: Porting rulex machine learning software to the raspberry pi as an edge computing device. In: Saponara, S., DeGloria, A. (eds.) Applications in Electronics Pervading Industry, Environment and Society. LNEE, vol. 738, pp. 273–279. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-66729-0_33

    Chapter  Google Scholar 

  28. Zamir, M., Ali, N., Naseem, A., et al.: A. Face Detection & Recognition from Images & Videos Based on CNN & Raspberry Pi. Computation. 10, 148 (2022)

    Google Scholar 

  29. Süzen, A.A., Duman, B., Şen, B.: Benchmark analysis of Jetson TX2, Jetson Nano and Raspberry PI using Deep-CNN. In: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1–5 (2020)

    Google Scholar 

  30. Raspberry Pi 4 B. https://www.raspberrypi.com/products/raspberry-pi-4-model-b/. Accessed Dec 2022

  31. Raspberry Pi Imager. https://www.raspberrypi.com/software. Accessed Dec 2022

  32. Neural Compute Stick 2. www.intel.com/content/www/us/en/developer/articles/tool/neuralcomputestick.html. Accessed Dec 2022

  33. Intel Distribution of OpenVINO Toolkit. www.intel.com/content/www/us/en/developer/tools/openvinotoolkit/overview.html. Accessed Dec 2022

  34. OpenCV. https://opencv.org/. Accessed Dec 2022

  35. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media Inc., Sebastopol (2008)

    Google Scholar 

  36. De Vitis, G.A., Foglia, P., Prete, C.A.: Row-level algorithm to improve real-time performance of glass tube defect detection in the production phase. IET Image Process. 14, 2911–2921 (2020). https://doi.org/10.1049/iet-ipr.2019.1506

    Article  Google Scholar 

  37. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)

    Article  Google Scholar 

  38. Jin, H., Liu, Q., et al.: Face detection using improved LBP under Bayesian framework. In: Third International Conference on Image and Graphics (ICIG 2004), pp. 306–309. IEEE (2004)

    Google Scholar 

  39. Joseph Redmon, Darknet: Open-Source Neural Networks in C, Darknet, https://pjreddie.com/darknet/. Accessed Dec 2022

  40. Ma, X.: https://github.com/dog-qiuqiu/MobileNet-Yolo. Accessed Dec 2022

  41. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  42. Serengil, S.I., Ozpinar, A.: Hyperextended lightface: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–4. IEEE, October 2021

    Google Scholar 

  43. Bhatti, Y.K., Jamil, A., Nida, N., Yousaf, M.H., Viriri, S., Velastin, S.A.: Facial expression recognition of instructor using deep features and extreme learning machine. Comput. Intell. Neurosci. 2021, 1–17 (2021). https://doi.org/10.1155/2021/5570870

    Article  Google Scholar 

  44. Matsumoto, D., Hwang, H.S.: Reading facial expressions of emotion. Psychol. Sci. Agenda. 25 (2011)

    Google Scholar 

  45. Ekman, P.: Emotions Revealed, 2nd edn. Times Books, New York (2003)

    Google Scholar 

Download references

Acknowledgements

Work partially supported by the Italian Ministry of Education and Research (MIUR) in the framework of: (i) the CrossLab project (Departments of Excellence); (ii) the FoReLab project (Departments of Excellence); (iii) the National Recovery and Resilience Plan in the National Center for Sustainable Mobility MOST/Spoke10. Work partially carried out by the University of Pisa in the framework of the PRA_2022_101 project “Decision Support Systems for territorial networks for managing ecosystem services”. Research partially funded by PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 - "FAIR - Future Artificial Intelligence Research" - Spoke 1 "Human-centered AI", funded by the European Commission under the NextGeneration EU programme.

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Pisa, Pisa, Italy

    Mario G. C. A. Cimino, Antonio Di Tecco, Pierfrancesco Foglia & Cosimo A. Prete

  2. University of Florence, Florence, Italy

    Antonio Di Tecco

Authors
  1. Mario G. C. A. Cimino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Di Tecco
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pierfrancesco Foglia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cosimo A. Prete
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pierfrancesco Foglia .

Editor information

Editors and Affiliations

  1. Siemens AG, Munich, Germany

    Cornel Klein

  2. RWTH Aachen, Aachen, Germany

    Matthias Jarke

  3. Siemens Industry Software Netherlands B.V., Eindhoven, The Netherlands

    Jeroen Ploeg

  4. Maynooth University, Maynooth, Kildare, Ireland

    Markus Helfert

  5. University of Kaiserslautern, Kaiserslautern, Germany

    Karsten Berns

  6. Ford Motor Company, Commerce Township, MI, USA

    Oleg Gusikhin

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cimino, M.G.C.A., Di Tecco, A., Foglia, P., Prete, C.A. (2023). Using Emotion Recognition and Temporary Mobile Social Network in On-Board Services for Car Passengers. In: Klein, C., Jarke, M., Ploeg, J., Helfert, M., Berns, K., Gusikhin, O. (eds) Smart Cities, Green Technologies, and Intelligent Transport Systems. SMARTGREENS VEHITS 2022 2022. Communications in Computer and Information Science, vol 1843. Springer, Cham. https://doi.org/10.1007/978-3-031-37470-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-37470-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-37469-2

  • Online ISBN: 978-3-031-37470-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation