Skip to main content
SpringerLink
Log in

Prediction of interface preferences with a classifier selection approach

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Interaction in smart environments should be adapted to the users’ preferences, e.g., utilising modalities appropriate for the situation. While manual customisation of a single application could be feasible, this approach would require too much user effort in the future, when a user interacts with numerous applications with different interfaces, such as e.g. a smart car, a smart fridge, a smart shopping assistant etc. Supporting user groups, jointly interacting with the same application, poses additional challenges: humans tend to respect the preferences of their friends and family members, and thus the preferred interface settings may depend on all group members. This work proposes to decrease the manual customisation effort by addressing the cold-start adaptation problem, i.e., predicting interface preferences of individuals and groups for new (unseen) combinations of applications, tasks and devices, based on knowledge regarding preferences of other users. For predictions we suggest several reasoning strategies and employ a classifier selection approach for automatically choosing the most appropriate strategy for each interface feature in each new situation. The proposed approach is suitable for cases where long interaction histories are not yet available, and it is not restricted to similar interfaces and application domains, as we demonstrate by experiments on predicting preferences of individuals and groups for three different application prototypes: recipe recommender, cooking assistant and car servicing assistant. The results show that the proposed method handles the cold-start problem in various types of unseen situations fairly well: it achieved an average prediction accuracy of \(72 \pm 1\,\%\). Further studies on user acceptance of predictions with two different user communities have shown that this is a desirable feature for applications in smart environments, even when predictions are not so accurate and when users do not perceive manual customisation as very time-consuming.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Adomavicius G, Kwon Y (2007) New recommendation techniques for multicriteria rating systems. IEEE Intell Syst 22(3):48–55

    Article  Google Scholar 

  2. Anand D, Bharadwaj K (2010) Adaptive user similarity measures for recommender systems: a genetic programming approach. In: Proceedings of ICCSIT, pp 121–125

  3. Avouac P-A, Lalanda P, Nigay L (2012) Adaptable multimodal interfaces in pervasive environments. In: Proceedings of IEEE consumer communications and networking conference, pp 544–548

  4. Baltrunas L, Ludwig B, Peer S, Ricci F (2012) Context relevance assessment and exploitation in mobile recommender systems. Pers Ubiquitous Comput 16(5):507–526

    Article  Google Scholar 

  5. Berkovsky Sh, Kuflik T, Ricci F (2008) Mediation of user models for enhanced personalisation in recommender systems. UMUAI 18:245–286

    Google Scholar 

  6. Berkovsky Sh, Freyne J (2010) Group-based recipe recommendations: analysis of data aggregation strategies. In: Proceedings RecSys2010, pp 111–118

  7. Blanco-Fernandez Y, Lopez-Nores M, Pazos-Arias JJ, Gil-Solla A, Ramos-Cabrer M (2010) Exploiting digital TV users’ preferences in a tourism recommender system based on semantic reasoning. IEEE Trans Consumer Electron 56(2):904–912

    Article  Google Scholar 

  8. Böhmer M, Bauer G (2010) Exploiting the icon arrangement on mobile devices as information source for context-awareness. Mobile HCI

  9. Carmagnola F, Cena F, Gena C (2011) User model interoperability: a survey. UMUAI 21:285–331

    Google Scholar 

  10. Chen Y-L, Cheng L-C, Chuang C-N (2008) A group recommendation system with consideration of interactions among group members. Expert Syst Appl 34(3):2082–2090

    Article  Google Scholar 

  11. Dey A, Newberger A (2009) Support for context-aware intelligibility and control. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI), pp 859–868

  12. Dumas B, Lalanne D, Oviatt Sh (2009) Multimodal interfaces: a survey of principles, models and frameworks. In: Lalanne D, Kohlas J (eds) Human machine interaction. Springer, Berlin, pp 3–26

  13. Findlater L, Mcgrenere J (2008) Impact of screen size on performance, awareness, and user satisfaction with adaptive graphical user interfaces. In: Proceedings of CHI 08, pp 1247–1256

  14. Fiske AP (1992) The four elementary forms of sociality: framework for a unified theory of social relations’. Psychol Rev 99:689–723

    Google Scholar 

  15. Florins M, Simarro FM, Vanderdonckt J, Michotte B (2006) Splitting rules for graceful degradation of user interfaces. In: Proceedings of IUI, pp 264–266

  16. Gajos K, Weld D, Wobbrock J (2010) Automatically generating personalized user interfaces with Supple. Artif Intell 174:910–950

    Article  Google Scholar 

  17. Gil M, Giner P, Pelechano V (2012) Personalization for unobtrusive service interaction. Pers Ubiquitous Comput 16(5):543–561

    Article  Google Scholar 

  18. Grimes A, Harper R (2008) Celebratory technology: new directions for food research in HCI, CHI

  19. Gulz A, Haake M, Silvervarg A, Sjödén B, Veletsianos G (2011) Building a social conversational pedagogical agent—design challenges and methodological approaches. In: Perez-Marin D, Pascual-Nieto I (eds) Conversational agents and natural language interaction: techniques and effective practices. IGI Global, pp 128–155

  20. van der Heijden H (2003) Ubiquitous computing, user control, and user performance: conceptual model and preliminary experimental design. In: Proceedings of the research symposium on emerging electronic markets, pp 107–112

  21. Horchani M, Caron B, Nigay L, Panaget F (2007) Natural multimodal dialogue systems: a configurable dialogue and presentation strategies component. In: ICMI, pp 291–298

  22. Jameson A, Smyth B (2007) Recommendation to groups. In: Brusilovsky P, Kobsa A, Nejdl W (eds) The adaptive web, pp 596–627

  23. Ko A, Sabourin R, Britto A (2008) From dynamic classifier selection to dynamic ensemble selection. Pattern Recognit 41:1718–1731

    Article  MATH  Google Scholar 

  24. Kong J, Zhang WY, Yu N, Xia XJ (2011) Design of human-centric adaptive multimodal interfaces. Int J Hum Comput Stud 69:854–869

    Article  Google Scholar 

  25. Konstan JA, Riedl J (2012) Recommender systems: from algorithms to user experience. User Model User Adapt Interact 22(1–2):101–123

    Article  Google Scholar 

  26. Limbourg Q, Vanderdonckt J, Michotte B, Bouillon L, López-Jaquero V (2005) Usixml: a language supporting multi-path development of user interfaces. In: Bastide R, Palanque P, Roth J (eds) Engineering human computer interaction and interactive systems, vol 3425 of, Lecture Notes in Computer Science, pp 134–135

  27. Lü L, Medo M, Yeung Ch, Zhang Y-Ch, Zhang Z-K, Zhou T (2012) Recommender systems review article. Phys Rep 519(1):1–49

    Article  Google Scholar 

  28. Luyten K, Thys K, Vermeulen J, Coninx K (2007) A generic approach for multi-device user interface rendering with UIML. Comput Aided Des User Interfaces 175–182

  29. Masthoff J, Gatt A (2006) In pursuit of satisfaction and the prevention of embarrassment: affective state in group recommender systems. User Model User Adapt Interact 16(3–4):281–319

    Article  Google Scholar 

  30. Meskens J, Luyten K, Coninx K (2010) Jelly: a multi-device design environment for managing consistency across devices. In: Proceedings of AVI, pp 289–296

  31. Moreno R, Mayer RE, Spire HA, Lester JC (2001) The case for social agency in computer-based teaching: do students learn more deeply when they interact with animated pedagogical agents? Cognit Instr 19(2):177–213

    Article  Google Scholar 

  32. Nichols J, Myers B, Harris T, Rosenfeld R, Shriver S, Higgins M, Hughes J (2002) Requirements for automatically generating multi-modal interfaces for complex appliances. In: Proceedings of the 4th IEEE conference on multimodal, interfaces, 2002, pp 377–382

  33. Object Management Group. “Meta Object Facility (MOF) 2.0 Query/View/Transformation (QVT)”

  34. Park D, Kim H, Choi I, Kim J (2012) A literature review and classification of recommender systems research. Expert Syst Appl 39(11):10059–10072

    Article  Google Scholar 

  35. Perez-Mateo M, Guitert M (2012) Which social elements are visible in virtual groups? Addressing the categorization of social expressions. Comput Educ 58(4):1234–1246

    Article  Google Scholar 

  36. Reeves LM, Lai J, Larson JA, Oviatt S, Balaji TS, Buisine Sp, Collings P, Cohen P, Kraal B, Martin J-C, McTear M, Raman T, Stanney KM, Su H, Wang QY (2004) Guidelines for multimodal user interface design. Commun ACM 47(1):57–59

  37. Reis T, de Sá M, Carriço L (2008) Multimodal interaction: real context studies on mobile digital artefacts. In: Proceedings of haptic and audio interaction design HAID 2008

  38. Remesal A, Colomina R (2013) Social presence in online small collaborative group work: a socioconstructivist account. Comput Educ 60:357–367

    Google Scholar 

  39. Ricci F (2011) Mobile recommender systems. Intl J Inf Technol Tour 12(3):205–231

    Article  Google Scholar 

  40. Ronkainen S, Koskinen E, Liu Y, Korhonen P (2010) Environment analysis as a basis for designing multimodal and multidevice user interfaces. Hum Comput Interact 25(2):148–193

    Article  Google Scholar 

  41. Rousseau C, Bellik Y, Vernier F, Bazalgette D (2006) A framework for the intelligent multimodal presentation of information. Signal Process 86(12):3696–3713

    Article  MATH  Google Scholar 

  42. Sarter N (2006) Multimodal information presentation: design guidance and research challenges. Int J Ind Ergon 36:439–445

    Article  Google Scholar 

  43. Senot Ch, Kostadinov D, Bouzid M, Picault J, Aghasaryan A Bernier C (2010) Analysis of strategies for building group profiles. In: Proceedings of UMAP

  44. Trewin Sh, Zimmermann G, Vanderheiden G (2003) Abstract user interface representations: how well do they support universal access? In: Proceedings of the conference on universal usability, 2003, pp 77–84

  45. Vildjiounaite E, Kyllönen V, Hannula T, Alahuhta P (2009) Unobtrusive dynamic modelling of TV programme preferences in a finnish household. Multimed Syst 3:143–157

    Article  Google Scholar 

  46. Vildjiounaite E, Kantorovitch J, Kyllönen V, Niskanen I, Hillukkala M, Virtanen K, Vuorinen O, Mäkelä S, Keränen T, Peltola J, Mäntyjärvi J, Tokmakoff A (2011) Designing socially acceptable multimodal interaction in cooking assistants. Proc IUI 11:415–418

    Google Scholar 

  47. Vildjiounaite E, Kyllönen V, Mäntyjärvi J (2011) If their car talks to them, shall a kitchen talk too? Cross-context mediation of interaction preferences. In: Proceedings of EICS 2011, pp 111–116

  48. Wasinger R, Wahlster W (2006) Multi-modal human-environment interaction. In: Aarts E, Encarnação J (eds) True visions. Springer, Berlin, pp 291–306

  49. Safeguards in a world of ambient intelligence. Wright D, Gutwirth S, Friedewald M, Vildjiounaite E, Punie Y (2008) Springer, Berlin

  50. Yu Z, Zhou X, Hao Y, Gu J (2006) TV program recommendation for multiple viewers based on user profile merging. User Model User Adapt Interact 16:63–68

    Article  Google Scholar 

Download references

Acknowledgments

This research was carried out within the Smart Products EU project, Grant Number 231204. We would like to thank all test subjects for their good will and thorough comments, and we would like to thank Steven Luitjens, Wenzhu Zou (Philips Research, The Netherlands) and Niklas Lochschmidt (TUD) for their efforts to conduct the control study.

Author information

Authors and Affiliations

  1. VTT Technical Research Centre of Finland, Oulu, Finland

    Elena Vildjiounaite, Vesa Kyllönen, Ilkka Niskanen & Jani Mäntyjärvi

  2. Technische Universität Darmstadt, Darmstadt, Germany

    Daniel Schreiber & Marcus Ständer

Authors
  1. Elena Vildjiounaite
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Schreiber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vesa Kyllönen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcus Ständer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ilkka Niskanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jani Mäntyjärvi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elena Vildjiounaite.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vildjiounaite, E., Schreiber, D., Kyllönen, V. et al. Prediction of interface preferences with a classifier selection approach. J Multimodal User Interfaces 7, 321–349 (2013). https://doi.org/10.1007/s12193-013-0127-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-013-0127-y

Keywords

Search

Navigation