Abstract
Many usability evaluation methods for interactive adaptive systems exist in the literature. There is not yet an agreement in the adaptive system community about which method is more useful than another in specific evaluation situations. This raises the question, “What is (are) the best evaluation method(s) that need(s) to be used in specific evaluation constraints?” This paper presents possible directions to address this issue by proposing a multi-criteria decision support framework for selecting the appropriate evaluation methods for interactive adaptive systems. The proposed decision support framework is applied to determine the suitable usability evaluation methods for a specific adaptation layer of a given adaptive hypermedia system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jameson, A.: Adaptive interfaces and agents. In: Jacko, J.A., Sears, A. (eds.) The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp. 305–330. Erlbaum, Mahwah (2003)
Paramythis, A., Weibelzahl, S., Masthoff, J.: Layered evaluation of interactive adaptive systems: framework and formative methods. User Model. User-Adap. Inter. 20, 383–453 (2010)
Gena, C., Weibelzahl, S.: Usability engineering for the adaptive web. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization, pp. 720–762 (2007)
Brusilovsky, P., Karagiannidis, C., Sampson, D.: The benefits of layered evaluation of adaptive applications and services. In: Weibelzahl, S., Chin, D.N., Weber, G. (eds.) Empirical Evaluation of Adaptive Systems, Proceedings of Workshop at the Eighth International Conference on User Modeling, UM2001, pp. 1–8 (2001)
Totterdell, P., Boyle, E.: The evaluation of adaptive systems. In: Browne, D., Totterdell, P., Norman, M. (eds.) Adaptive User Interfaces, pp. 161–194. Academic Press, London (1990)
Van Velsen, L., Van der Geest, T., Klaassen, R., Steehouder, M.: User-centered evaluation of adaptive and adaptable systems: a literature review. Knowl. Eng. Rev. 23, 261–281 (2008)
Mulwa, C., Lawless, S., Sharp, M., Wade, V.: The evaluation of adaptive and user-adaptive systems: a review. Int. J. Knowl. Web Intell. 2, 138–156 (2011)
Gena, C.: Methods and techniques for the evaluation of user-adaptive systems. Knowl. Eng. Rev. 20, 1–37 (2005)
Dhouib, A., Trabelsi, A., Kolski, C., Neji, M.: A classification and comparison of usability evaluation methods for interactive adaptive systems. In: 9th International Conference on Human System Interactions, Portsmouth, UK, pp. 246–251 (2016)
Ferré, X., Bevan, N., Escobar, T.A.: UCD method selection with usability planner. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (2010)
Ferre, X., Bevan, N.: Usability planner: a tool to support the process of selecting usability methods. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6949, pp. 652–655. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23768-3_105
Dhouib, A., Trablesi, A., Kolski, C., Neji, M.: An approach for the selection of evaluation methods for interactive adaptive systems using analytic hierarchy process. In: IEEE Tenth International Conference on Research Challenges in Information Science, 1–3 June, Grenoble, France, pp. 1–10 (2016)
Karagiannidis, C., Sampson, D.G.: Layered evaluation of adaptive applications and services. In: Brusilovsky, P., Stock, O., Strapparava, C. (eds.) AH 2000. LNCS, vol. 1892, pp. 343–346. Springer, Heidelberg (2000). doi:10.1007/3-540-44595-1_43
Manouselis, N., Karagiannidis, C., Sampson, D.G.: Layered evaluation for data discovery and recommendation systems an initial set of principles. In: IEEE 14th International Conference on Advanced Learning Technologies, pp. 518–519 (2014)
Gupta. A., Grover, P.: Proposed evaluation framework for adaptive hypermedia systems. In: 3rd Workshop on Empirical Evaluation of Adaptive Systems, Eindhoven University of Technology, The Netherlands, pp. 161–171 (2004)
Tarpin-Bernard, F., Marfisi-Schottman, I., Habieb-Mammar, H.: AnAmeter: the first steps to evaluating adaptation. In: 6th Workshop on User-Centred Design and Evaluation of Adaptive Systems at UMAP2009, Trento, Italy, pp. 11–20 (2009)
Mulwa, C., Wade, V.: A web-based evaluation framework for supporting novice and expert evaluators of adaptive e-Learning systems. In: International Conference on e-Technologies and Business on the Web (EBW201z3), Thailand, pp. 62–67 (2003)
Saaty, T.L.: The Analytic Hierarchy Process. McGraw-Hill International, New York (1980)
Omkarprasad, S.V., Kumar, S.: Analytic hierarchy process: an overview of applications. Eur. J. Oper. Res. 169, 1–29 (2006)
Saaty, T.L.: Decision making with the analytic hierarchy process. Int. J. Serv. Sci. 1, 83–98 (2008)
Krueger, R., Casey, M.: Focus Groups: A Practical Guide for Applied Research, 4th edn. Sage Publications, Los Angeles (2009)
Nielsen, J.: Heuristic evaluation. In: Nielsen, J., Mack, R.L. (eds.) Usability Inspection Methods, pp. 25–64. Wiley, New York (1994)
Mahatody, T., Sagar, M., Kolski, C.: State of the art on the cognitive walkthrough method, its variants and evolutions. Int. J. Hum. Comput. Interact. 26(8), 741–785 (2010)
Masthoff, J.: The user as wizard: a method for early involvement in the design and evaluation of adaptive systems. In: 5th Workshop on User- Centred Design and Evaluation of Adaptive Systems at AH 2006, Dublin, Ireland, pp. 460–469 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Dhouib, A., Trabelsi, A., Kolski, C., Neji, M. (2017). A Multi-Criteria Decision Support Framework for Interactive Adaptive Systems Evaluation. In: Benferhat, S., Tabia, K., Ali, M. (eds) Advances in Artificial Intelligence: From Theory to Practice. IEA/AIE 2017. Lecture Notes in Computer Science(), vol 10350. Springer, Cham. https://doi.org/10.1007/978-3-319-60042-0_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-60042-0_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60041-3
Online ISBN: 978-3-319-60042-0
eBook Packages: Computer ScienceComputer Science (R0)