Skip to main content
SpringerLink
Log in

Emotional Internet of Behaviors: A QoE-QoS Adjustment Mechanism

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14050))

Included in the following conference series:

  • 1262 Accesses

Abstract

The Internet of Behaviors (IoB) approach supports developing socio-technical systems based on humans’ goals, characteristics, behaviors, and emotions. This paper shows how emotions and behaviors could impact the quality of software systems. We propose interactive control loops that supervise application and architecture adaptations toward enhancing the system quality of service (QoS) and human quality of experience (QoE). Under the IoB conceptual model, we first show how historical and real emotions could be the source of the design and adaptation of socio-technical systems. We further use a Reinforcement Learning (RL)-based approach as a self-adaptation supervisor of user interfaces (UIs) to users’ emotions. The approach aims to maximize applying the essential adaptations and minimize the unnecessary ones towards users’ QoE. If the control system detects a drop in QoS in emotion-based adaptations or other functions, another level of adaptation reconfigures the architecture towards better quality. We used the emotional IoB approach to develop a mobile application as a recommender system in emergency evacuation training. The app takes users’ facial emotions and positions as input and adapts its UI to impact users’ target emotions and task completion. In addition to UI adaptation, the system supports architecture adaptations to decrease response time if required. The evaluation process confirms the efficiency of the RL in iterations, as well as compared to other possible UI adaptation techniques. The results also show that architecture adaptations positively impact the system performance and users’ emotions and performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abughazala, M.B., Moghaddam, M.T., Muccini, H., Vaidhyanathan, K.: Human behavior-oriented architectural design. In: Biffl, S., Navarro, E., Löwe, W., Sirjani, M., Mirandola, R., Weyns, D. (eds.) ECSA 2021. LNCS, vol. 12857, pp. 134–143. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86044-8_9

    Chapter  Google Scholar 

  2. Alipour, M., Dupuy-Chessa, S., Céret, E.: An emotion-oriented problem space for ui adaptation: from a literature review to a conceptual framework. In: 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 1–8. IEEE (2021)

    Google Scholar 

  3. Alipour, M., Dupuy-Chessa, S., Jongmans, E.: Disaster mitigation using interface adaptation to emotions: a targeted literature review. In: 10th International Conference on the Internet of Things Companion, pp. 1–15 (2020)

    Google Scholar 

  4. Arbib, C., Arcelli, D., Dugdale, J., Moghaddam, M.T., Muccini, H.: Real-time emergency response through performant IoT architectures. In: International Conference on Information Systems for Crisis Response and Management (ISCRAM) (2019)

    Google Scholar 

  5. Arbib, C., Moghaddam, M.T., Muccini, H.: IoT flows: a network flow model application to building evacuation. In: Dell’Amico, M., Gaudioso, M., Stecca, G. (eds.) A View of Operations Research Applications in Italy, 2018. ASS, vol. 2, pp. 115–131. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25842-9_9

    Chapter  Google Scholar 

  6. Arbib, C., Muccini, H., Moghaddam, M.T.: Applying a network flow model to quick and safe evacuation of people from a building: a real case. RSFF 18, 50–61 (2018)

    Google Scholar 

  7. Bannon, L.: Reimagining HCI: toward a more human-centered perspective. Interactions 18(4), 50–57 (2011)

    Google Scholar 

  8. Bannon, L.J.: From human factors to human actors: the role of psychology and human-computer interaction studies in system design. In: Readings in Human-Computer Interaction, pp. 205–214. Elsevier (1995)

    Google Scholar 

  9. Castillo, J.C., et al.: Software architecture for smart emotion recognition and regulation of the ageing adult. Cogn. Comput. 8, 357–367 (2016)

    Article  Google Scholar 

  10. Dugdale, J., Moghaddam, M.T., Muccini: Human behaviour centered design: developing a software system for cultural heritage. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Society, pp. 85–94 (2020)

    Google Scholar 

  11. Ekman, P.: Basic emotions. Handbook of cognition and emotion 98(45–60), 16 (1999)

    Google Scholar 

  12. Ekman, P., et al.: Universals and cultural differences in the judgments of facial expressions of emotion. J. Pers. Soc. Psychol. 53(4), 712 (1987)

    Article  Google Scholar 

  13. Firmenich, S., Garrido, A., Paternò, F., Rossi, G.: User interface adaptation for accessibility. In: Yesilada, Y., Harper, S. (eds.) Web Accessibility. HIS, pp. 547–568. Springer, London (2019). https://doi.org/10.1007/978-1-4471-7440-0_29

    Chapter  Google Scholar 

  14. Hardin, B., McCool, D.: BIM and construction management: proven tools, methods, and workflows. John Wiley & Sons (2015)

    Google Scholar 

  15. Hunziker, S., Laschinger, L., Portmann-Schwarz, S., Semmer, N.K., Tschan, F., Marsch, S.: Perceived stress and team performance during a simulated resuscitation. Intensive Care Med. 37(9), 1473–1479 (2011)

    Article  Google Scholar 

  16. ISO/IEC/IEEE: ISO/IEC/IEEE 42010, systems and software engineering - architecture description (2011)

    Google Scholar 

  17. Kjærgaard, M.B., Kuhrmann, M.: On architectural qualities and tactics for mobile sensing. In: Proceedings of the 11th International ACM SIGSOFT Conference on Quality of Software Architectures, pp. 63–72. Association for Computing Machinery (2015)

    Google Scholar 

  18. Kleinginna, P.R., Kleinginna, A.M.: A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv. Emot. 5(4), 345–379 (1981)

    Article  Google Scholar 

  19. Konar, A., Chakraborty, A.: Emotion recognition: a pattern analysis approach. John Wiley & Sons (2015)

    Google Scholar 

  20. Lenberg, P., Feldt, R., Wallgren, L.G.: Behavioral software engineering: a definition and systematic literature review. J. Syst. Softw. 107, 15–37 (2015)

    Article  Google Scholar 

  21. Lin, B., Cecchi, G., Bouneffouf, D., Reinen, J., Rish, I.: A story of two streams: reinforcement learning models from human behavior and neuropsychiatry. arXiv preprint arXiv:1906.11286 (2019)

  22. Märtin, C., Rashid, S., Herdin, C.: Designing responsive interactive applications by emotion-tracking and pattern-based dynamic user interface adaptation. In: Kurosu, M. (ed.) HCI 2016. LNCS, vol. 9733, pp. 28–36. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39513-5_3

    Chapter  Google Scholar 

  23. Moghaddam, M.T., Muccini, H., Dugdale, J.: Intelligent building evacuation: from modeling systems to behaviors. In: Scholl, H.J., Holdeman, E.E., Boersma, F.K. (eds.) Disaster Management and Information Technology. Public Administration and Information Technology, vol. 40, pp. 111–129. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20939-0_7

  24. Moghaddam, M.T., Muccini, H., Dugdale, J., Kjægaard, M.B.: Designing internet of behaviors systems. In: 2022 IEEE 19th International Conference on Software Architecture (ICSA), pp. 124–134. IEEE (2022)

    Google Scholar 

  25. Moghaddam, M.T., Rutten, E., Lalanda, P., Giraud, G.: IAS: an IoT architectural self-adaptation framework. In: Jansen, A., Malavolta, I., Muccini, H., Ozkaya, I., Zimmermann, O. (eds.) ECSA 2020. LNCS, vol. 12292, pp. 333–351. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58923-3_22

    Chapter  Google Scholar 

  26. Moghaddam, M.T., Alipour, M., Baun Kjærgaard, M.: User interface and architecture adaption based on emotions and behaviors. In: 2020 IEEE International Conference on Software Architecture (ICSA), pp. 1–10. IEEE (2023)

    Google Scholar 

  27. Moghaddam, M.T., Muccini, H.: Fault-tolerant IoT. In: Calinescu, R., Di Giandomenico, F. (eds.) SERENE 2019. LNCS, vol. 11732, pp. 67–84. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30856-8_5

    Chapter  Google Scholar 

  28. Moghaddam, M.T., Rutten, E., Giraud, G.: Hierarchical control for self-adaptive iot systems a constraint programming-based adaptation approach. In: HICSS-Hawaii International Conference on System Sciences, pp. 1–10 (2022)

    Google Scholar 

  29. Muccini, H., Arbib, C., Davidsson, P., Tourchi Moghaddam, M.: An IoT software architecture for an evacuable building architecture. In: Proceedings of the 52nd Hawaii International Conference on System Sciences (2019)

    Google Scholar 

  30. Muccini, H., Moghaddam, M.T.: IoT architectural styles. In: Cuesta, C.E., Garlan, D., Pérez, J. (eds.) ECSA 2018. LNCS, vol. 11048, pp. 68–85. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00761-4_5

    Chapter  Google Scholar 

  31. Muccini, H., Spalazzese, R., Moghaddam, M.T., Sharaf, M.: Self-adaptive IoT architectures: An emergency handling case study. In: Proceedings of the 12th European Conference on Software Architecture: Companion Proceedings, pp. 1–6 (2018)

    Google Scholar 

  32. Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17(3), 715–734 (2005)

    Article  Google Scholar 

  33. Schirner, G., Erdogmus, D., Chowdhury, K., Padir, T.: The future of human-in-the-loop cyber-physical systems. Computer 46(1), 36–45 (2013)

    Article  Google Scholar 

  34. Schrader, C., Brich, J., Frommel, J., Riemer, V., Rogers, K.: Rising to the challenge: an emotion-driven approach toward adaptive serious games. Serious Games Edutain. Appl. II, 3–28 (2017)

    Article  Google Scholar 

  35. Shteingart, H., Loewenstein, Y.: Reinforcement learning and human behavior. Curr. Opin. Neurobiol. 25, 93–98 (2014)

    Article  Google Scholar 

  36. Sottet, J.-S., et al.: Model-driven adaptation for plastic user interfaces. In: Baranauskas, C., Palanque, P., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4662, pp. 397–410. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74796-3_38

    Chapter  Google Scholar 

  37. Todi, K., Bailly, G., Leiva, L., Oulasvirta, A.: Adapting user interfaces with model-based reinforcement learning. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2021)

    Google Scholar 

  38. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learni. 8(3–4), 279–292 (1992)

    Google Scholar 

  39. Weyns, D., et al.: On patterns for decentralized control in self-adaptive systems. In: de Lemos, R., Giese, H., Müller, H.A., Shaw, M. (eds.) Software Engineering for Self-Adaptive Systems II. LNCS, vol. 7475, pp. 76–107. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-35813-5_4

    Chapter  Google Scholar 

  40. Yechiam, E., Busemeyer, J.R., Stout, J.C., Bechara, A.: Using cognitive models to map relations between neuropsychological disorders and human decision-making deficits. Psychol. Sci. 16(12), 973–978 (2005)

    Article  Google Scholar 

  41. Yigitbas, E., Hottung, A., Rojas, S.M., Anjorin, A., Sauer, S., Engels, G.: Context-and data-driven satisfaction analysis of user interface adaptations based on instant user feedback. Proceed. ACM Hum.-Comput. Interact. 3(EICS), 1–20 (2019)

    Google Scholar 

  42. Zheng, M., Xu, Q., Fan, H.: Modeling the adaption rule in context-aware systems. arXiv preprint arXiv:1609.01614 (2016)

Download references

Acknowledgement

This work is supported by the Innovation Fund Denmark for the project DIREC (9142-00001B).

Author information

Authors and Affiliations

  1. University of Southern Denmark, Odense, 5230, Denmark

    Mina Alipour, Mahyar T. Moghaddam, Tobias Kristensen & Nicolai Krogager Asmussen

  2. IIIT Hyderabad, Hyderabad, 500032, India

    Karthik Vaidhyanathan

Authors
  1. Mina Alipour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahyar T. Moghaddam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karthik Vaidhyanathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tobias Kristensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicolai Krogager Asmussen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mahyar T. Moghaddam .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alipour, M., Moghaddam, M.T., Vaidhyanathan, K., Kristensen, T., Krogager Asmussen, N. (2023). Emotional Internet of Behaviors: A QoE-QoS Adjustment Mechanism. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2023. Lecture Notes in Computer Science(), vol 14050. Springer, Cham. https://doi.org/10.1007/978-3-031-35891-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35891-3_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35890-6

  • Online ISBN: 978-3-031-35891-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation