Skip to main content
SpringerLink
Log in
Book cover

Companion Technology pp 57–78Cite as

Modeling Aspects in Human-Computer Interaction: Adaptivity, User Characteristics and Evaluation

  • Chapter
  • First Online:

  • 829 Accesses

Part of the book series: Cognitive Technologies ((COGTECH))

Abstract

During system interaction, the user’s emotions and intentions shall be adequately determined and predicted to recognize tendencies in his or her interests and dispositions. This allows for the design of an evolving search user interface (ESUI) which adapts to changes in the user’s emotional reaction and the users’ needs and claims.

Here, we concentrate on the front end of the search engine and present two prototypes, one which can be customised to the user’s needs and one that takes the user’s age as a parameter to roughly approximate the user’s skill space and for subsequent system adaptation. Further, backend algorithms to detect the user’s abilities are required in order to have an adaptive system.

To develop an ESUI, user studies with users of gradually different skills have been conducted with groups of young users. In order to adapt the interaction dialog, we propose monitoring the user’s emotional state. This enables monitoring early detection of the user’s problems in interacting with the system, and allows us to adapt the dialog to get the user on the right path. Therefore, we investigate methods to detect changes in the user’s emotional state.

We furthermore propose a user mood modeling from a technical perspective based on a mechanical spring model in PAD-space, which is able to incorporate several psychological observations. This implementation has the advantage of only three internal parameters and one user-specific parameter-pair.

We present a technical implementation of that model in our system and evaluate the principal function of the proposed model on two different databases. Especially on the EmoRecWoz corpus, we were able to show that the generated mood course matched the experimental setting.

By utilizing the user-specific parameter-pair the personality trait extraversion was modeled. This trait is supposed to regulate the individual emotional experiences.

Technically, we present an implementable feature-based, dimensional model for emotion analysis which is able to track and predict the temporal development of emotional reactions in an evolving search user interface, and which is adjustable based on mood and personality traits.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Breadcrumb is a navigation aid that shows a user’s location in a website or Web application [20].

  2. 2.

    https://datamarket.azure.com/dataset/5BA839F1-12CE-4CCE-BF57-A49D98D29A44.

  3. 3.

    A document surrogate summarizes important information about the document for users to be able to judge its relevance without opening the document.

References

  1. Amstad, T.: Wie verständlich sind unsere Zeitungen? Ph.D. Thesis, University of Zurich (1978)

    Google Scholar 

  2. Arapakis, I., Athanasakos, K., Jose, J.M.: A comparison of general vs personalised affective models for the prediction of topical relevance. In: Proceedings of the 33rd ACM SIGIR’10, pp. 371–378 (2010)

    Google Scholar 

  3. Aula, A.: User study on older adults’use of the web and search engines. Univ. Access Inf. Soc. 4(1), 67–81 (2005)

    Article  Google Scholar 

  4. Aula, A., Käki, M.: Less is more in web search interfaces for older adults. First Monday 10(7) (2005)

    Google Scholar 

  5. Becker-Asano, C.: WASABI: affect simulation for agents with believable interactivity. Ph.D. Thesis, University of Bielefeld (2008)

    Google Scholar 

  6. Böck, R., Limbrecht, K., Walter, S., Hrabal, D., Traue, H.C., Glüge, S., Wendemuth, A.: Intraindividual and interindividual multimodal emotion analyses in human-machine-interaction. In: Proceedings of the IEEE CogSIMA, New Orleans, pp. 59–64 (2012)

    Google Scholar 

  7. Böck, R., Limbrecht-Ecklundt, K., Siegert, I., Walter, S., Wendemuth, A.: Audio-based pre-classification for semi-automatic facial expression coding. In: Kurosu, M. (ed.) Human-Computer Interaction. Towards Intelligent and Implicit Interaction. Lecture Notes in Computer Science, vol. 8008, pp. 301–309. Springer, Cham (2013)

    Chapter  Google Scholar 

  8. Carpenter, S.M., Peters, E., Västfjäll, D., Isen, A.M.: Positive feelings facilitate working memory and complex decision making among older adults. Cognit. Emot. 27, 184–192 (2013)

    Article  Google Scholar 

  9. Costa, P.T., McCrae, R.R.: The NEO Personality Inventory Manual. Psychological Assessment Resources, Odessa (1985)

    Google Scholar 

  10. Crawford, J.R., Henry, J.D.: The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. Br. J. Clin. Psychol. 43, 245–265 (2004)

    Article  Google Scholar 

  11. Eickhoff, C., Azzopardi, L., Hiemstra, D., De Jong, F., de Vries, A., Dowie, D., Duarte, S., Glassey, R., Gyllstrom, K., Kruisinga, F., Marshall, K., Moens, S., Polajnar, T., van der Sluis, F.: Emse: initial evaluation of a child-friendly medical search system. In: Proceedings of the 4th ACM Information Interaction in Context Symposium, pp. 282–285 (2012)

    Google Scholar 

  12. Ellsworth, P., Scherer, K.: Appraisal Processes in Emotion, pp. 572–595. Oxford University Press, New York (2003)

    Google Scholar 

  13. Gebhard, P.: ALMA a layered model of affect. In: Proceedings of the 4th ACM AAMAS, Utrecht, pp. 29–36 (2005)

    Google Scholar 

  14. Gossen, T., Nürnberger, A.: Specifics of information retrieval for young users: a survey. Inf. Process. Manag. 49(4), 739–756 (2013)

    Article  Google Scholar 

  15. Gossen, T., Nitsche, M., Nürnberger, A.: Knowledge journey: a web search interface for young users. In: Proceedings of the Symposium on Human-Computer Interaction and Information Retrieval (HCIR’12), pp. 1:1–1:10. ACM, Cambridge (2012)

    Google Scholar 

  16. Gossen, T., Nitsche, M., Nürnberger, A.: Evolving search user interfaces. In: Proceedings of EuroHCIR 2013 Workshop. Dublin, Ireland (2013)

    Google Scholar 

  17. Gossen, T., Nitsche, M., Vos, J., Nürnberger, A.: Adaptation of a search user interface towards user needs - a prototype study with children & adults. In: Proceedings of the Symposium on Human-Computer Interaction and Information Retrieval (HCIR’13), Vancouver, BC (2013)

    Google Scholar 

  18. Gossen, T., Kotzyba, M., Nürnberger, A.: Knowledge journey exhibit: towards age-adaptive search user interfaces. In: Hanbury, A., Kazai, G., Rauber, A., Fuhr, N. (eds.) Advances in Information Retrieval. Lecture Notes in Computer Science, vol. 9022, pp. 781–784. Springer, Cham (2015)

    Google Scholar 

  19. Hartmann, K., Siegert, I., Glüge, S., Wendemuth, A., Kotzyba, M., Deml, B.: Describing human emotions through mathematical modelling. In: Proceedings of the 7th MATHMOD, Vienna, pp. 463–468 (2012)

    Google Scholar 

  20. Hearst, M.: Search User Interfaces. Cambridge University Press, Cambridge (2009)

    Book  Google Scholar 

  21. Hutchinson, H., Druin, A., Bederson, B., Reuter, K., Rose, A., Weeks, A.: How do I find blue books about dogs? The errors and frustrations of young digital library users. In: Proceedings of the 11th International Conference on Human-Computer Interaction (2005)

    Google Scholar 

  22. Jansen, M., Bos, W., van der Vet, P., Huibers, T., Hiemstra, D.: TeddIR: tangible information retrieval for children. In: Proceedings of the 9th ACM International conference on Interaction Design and Children, Barcelona, 2010, pp. 282–285

    Google Scholar 

  23. Kotzyba, M., Deml, B., Neumann, H., Glüge, S., Hartmann, K., Siegert, I., Wendemuth, A., Traue, H.C., Walter, S.: Emotion detection by event evaluation using fuzzy sets as appraisal variables. In: Proceedings of the 11th ICCM, Berlin, pp. 123–124 (2012)

    Google Scholar 

  24. Larsen, R.J., Fredrickson, B.L.: Measurement Issues in Emotion Research, pp. 40–60. Russell Sage Foundation, New York (1999)

    Google Scholar 

  25. Larsen, R.J., Ketelaar, T.: Personality and susceptibility to positive and negative emotional states. J. Pers. Soc. Psychol. 61, 132–140 (1991)

    Article  Google Scholar 

  26. Morris, J.D.: SAM: the self-assessment manikin an efficient cross-cultural measurement of emotional response. J. Advert. Res. 35, 63–68 (1995)

    Google Scholar 

  27. Morris, W.N.: Mood: The Frame of Mind. Springer, New York (1989)

    Book  Google Scholar 

  28. Moshfeghi, Y.: Role of emotion in information retrieval. Ph.D. Thesis, University of Glasgow, School of Computing Science (2012)

    Google Scholar 

  29. Moshfeghi, Y., Piwowarski, B., Jose, J.: Handling data sparsity in collaborative filtering using emotion and semantic based features. In: Proceedings of the 34th ACM SIGIR’11, pp. 625–634 (2011)

    Google Scholar 

  30. Nolen-Hoeksema, S., Fredrickson, B., Loftus, G., Wagenaar, W.: Atkinson & Hilgard’s Introduction to Psychology, 15 edn. Cengage Learning EMEA, Hampshire (2009)

    Google Scholar 

  31. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  32. Scherer, K.R.: Appraisal Considered as a Process of Multilevel Sequential Checking, pp. 92–120. Oxford University Press, Oxford (2001)

    Google Scholar 

  33. Siegert, I.: Emotional and user-specific cues for improved analysis of naturalistic interactions. Ph.D. Thesis, Otto von Guericke University Magdeburg (2015)

    Google Scholar 

  34. Siegert, I., Böck, R., Wendemuth, A.: The influence of context knowledge for multimodal annotation on natural material. In: Joint Proceedings of the IVA 2012 Workshops, Santa Cruz, pp. 25–32 (2012)

    Google Scholar 

  35. Siegert, I., Böck, R., Wendemuth, A.: Modeling users’ mood state to improve human-machine-interaction. In: Esposito, A., Esposito, A.M., Vinciarelli, A., Hoffmann, R., Müller, V.C. (eds.) Cognitive Behavioural Systems. Lecture Notes in Computer Science, vol. 7403, pp. 273–279. Springer, Berlin (2012)

    Chapter  Google Scholar 

  36. Siegert, I., Hartmann, K., Glüge, S., Wendemuth, A.: Modelling of emotional development within human-computer-interaction. Kognitive Systeme 1 s.p. (2013)

    Google Scholar 

  37. Siegert, I., Böck, R., Wendemuth, A.: Inter-rater reliability for emotion annotation in human-computer interaction – comparison and methodological improvements. J. Multimodal User Interfaces 8, 17–28 (2014)

    Article  Google Scholar 

  38. Steichen, B., Ashman, H., Wade, V.: A comparative survey of personalised information retrieval and adaptive hypermedia techniques. Inf. Process. Manag. 48(4), 698–724 (2012)

    Article  Google Scholar 

  39. Stober, S., Nürnberger, A.: Adaptive music retrieval–a state of the art. Multimed. Tools Appl. 65, 467–494 (2013)

    Article  Google Scholar 

  40. Truong, K.P., Neerincx, M.A., Leeuwen, D.A.V.: Assessing agreement of observer- and self-annotations in spontaneous multimodal emotion data. In: Proceedings of the INTERSPEECH-2008, Brisbane, pp. 318–321 (2008)

    Google Scholar 

  41. Walter, S., Scherer, S., Schels, M., Glodek, M., Hrabal, D., Schmidt, M., Böck, R., Limbrecht, K., Traue, H.C., Schwenker, F.: Multimodal emotion classification in naturalistic user behavior. In: Jacko, J. (ed.) Human-Computer Interaction. Towards Mobile and Intelligent Interaction Environments. Lecture Notes in Computer Science, vol. 6763, pp. 603–611. Springer, Cham (2011)

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was done within the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG).

Author information

Authors and Affiliations

  1. Data & Knowledge Engineering Group, Otto von Guericke University, 39016, Magdeburg, Germany

    Tatiana Gossen, Andreas Nürnberger & Michael Kotzyba

  2. Cognitive Systems Group, Otto von Guericke University, 39016, Magdeburg, Germany

    Ingo Siegert & Kim Hartmann

  3. Center for Behavioral Brain Sciences, 39118, Magdeburg, Germany

    Andreas Wendemuth

Authors
  1. Tatiana Gossen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ingo Siegert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Nürnberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kim Hartmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Kotzyba
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andreas Wendemuth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tatiana Gossen .

Editor information

Editors and Affiliations

  1. Institute of Artificial Intelligence, Universität Ulm, Ulm, Germany

    Susanne Biundo

  2. Cognitive Systems Group, Institute for Information Technology and Communications (IIKT) and Center for Behavioral Brain Sciences (CBBS), Otto-von-Guericke Universität Magdeburg, Magdeburg, Germany

    Andreas Wendemuth

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Gossen, T., Siegert, I., Nürnberger, A., Hartmann, K., Kotzyba, M., Wendemuth, A. (2017). Modeling Aspects in Human-Computer Interaction: Adaptivity, User Characteristics and Evaluation. In: Biundo, S., Wendemuth, A. (eds) Companion Technology. Cognitive Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-43665-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-43665-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-43664-7

  • Online ISBN: 978-3-319-43665-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation