Skip to main content
Springer Nature Link
Log in

Usability Engineering for the Adaptive Web

  • Chapter
The Adaptive Web

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 4321))

  • 17k Accesses

Abstract

This chapter discusses a usability engineering approach for the design and the evaluation of adaptive web-based systems, focusing on practical issues. A list of methods will be presented, considering a user-centered approach. After having introduced the peculiarities that characterize the evaluation of adaptive web-based systems, the chapter describes the evaluation methodologies following the temporal phases of evaluation, according to a user-centered approach. Three phases are distinguished: requirement phase, preliminary evaluation phase, and final evaluation phase. Moreover, every technique is classified according to a set of parameters that highlight the practical exploitation of that technique. For every phase, the appropriate techniques are described by giving practical examples of their application in the adaptive web. A number of issues that arise when evaluating an adaptive system are described, and potential solutions and workarounds are sketched.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Alepis, E., Virvou, M.: User Modelling: An Empirical Study for Affect Perception Through Keyboard and Speech in a Bi-modal User Interface. In: Wade, V., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 338–341. Springer, Heidelberg (2006)

    Google Scholar 

  2. Alfonseca, E., Rodriguezi, P.: Modelling Users’ Interests and Needs for an Adaptive On-line Information System. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 76–82. Springer, Heidelberg (2003)

    Google Scholar 

  3. Altman, D., Gore, S., Gardner, M., Pocock, S.: Statistical Guidelines for Contributors to Medical Journals. British Medical Journal 286, 1489–1493 (1983)

    Google Scholar 

  4. Anderson, J.R., Boyle, C.F., Yost, G.: The Geometry Tutor. In: 9th International Joint Confenrence on AI, pp. 1–7 (1985)

    Google Scholar 

  5. Ardissono, L., Gena, C., Torasso, P., Bellifemine, F., Chiarotto, A., Difino, A., Negro, B.: User Modeling and Recommendation Techniques for Personalized Electronic Program Guides. In: Ardissono, L., Kobsa, A., Maybury, M. (eds.) Personalization and user-adaptive interaction in digital tv, pp. 30–36. Kluwer Academic Publishers, Dordrecht (2004)

    Google Scholar 

  6. Bakeman, R., Gottman, J.M.: Observing Behavior: An Introduction to Sequential Analysis. Cambridge University Press, Cambridge (1986)

    Google Scholar 

  7. Bares, W.H., Lester, J.C.: Cinematographic User Models for Automated Realtime Camera Control in Dynamic 3D Environments. In: Jameson, A., Paris, C., Tasso, C. (eds.) User modeling: Proceedings of the Sixth International Conference, UM’97, pp. 215–226. Springer, New York (1997)

    Google Scholar 

  8. Barker, T., Jones, S., Britton, C., Messer, D.: The Use of a Co-operative Student Model of Learner Characteristics to Configure a Multimedia Application. User Modeling and User-adaptive Interaction 12(2), 207–241 (2002)

    MATH  Google Scholar 

  9. Baudisch, P., Brueckner, L.: TV Scout: Lowering the Entry Barrier to Personalized TV Program Recommendation. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 58–68. Springer, Heidelberg (2002)

    Google Scholar 

  10. Beck, J.E., Jia, P., Sison, J., Mostow, J.: Predicting Student Help-request Behavior in an Intelligent Tutor for Reading. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 303–312. Springer, Heidelberg (2003)

    Google Scholar 

  11. Beck, J.E., Jia, P., Sison, J., Mostow, J.: Assessing Student Proficiency in a Reading Tutor That Listens. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 323–327. Springer, Heidelberg (2003)

    Google Scholar 

  12. Beck, J.E., Stern, M., Woolf, B.P.: Using the Student Model to Control Problem Difficulty. In: Jameson, A., Paris, C., Tasso, C. (eds.) User modeling: Proceedings of the Sixth International Conference, UM’97, pp. 277–288. Springer, New York (1997)

    Google Scholar 

  13. Bednarik, R.: Potentials of Eye-Movement Tracking in Adaptive Systems. In: Weibelzahl, S., Paramythis, A., Masthoff, J. (eds.) Proceedings of the Fourth Workshop on Empirical Evaluation of Adaptive Systems, held at the 10th International Conference on User Modeling UM2005, Edinburgh, pp. 1–8 (2005)

    Google Scholar 

  14. Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., Pitkin, R., Rennie, D., Schultz, K., Simel, D., Stroup, D.: Improving the Quality of Reporting Randomized Trials (the CONSORT Statement). Journal of the American Medical Association 276(8), 637–639 (1996)

    Google Scholar 

  15. Bellotti, V., MacLean, A.: Design Space Analysis (DSA), http://www.mrc-cbu.cam.ac.uk/amodeus/summaries/DSAsummary.html

  16. Bental, D., Cawsey, A., Pearson, J., Jones, R.: Does Adapted Information Help Patients with Cancer? In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 288–291. Springer, Heidelberg (2003)

    Google Scholar 

  17. Benyon, D.: Adaptive Systems: A Solution to Usability Problems. User Modeling and User-adaptive Interaction 3, 65–87 (1993)

    Google Scholar 

  18. Beyer, H., Holtzblatt, K.: Contextual Design: Defining Customer-Centered Systems. Morgan Kaufmann Publishers, San Francisco (1998)

    Google Scholar 

  19. Billsus, D., Pazzani, M.: Learning Collaborative Information Filters. In: Proceedings of the International Conference on Machine Learning, Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  20. Brunstein, A., Jacqueline, W., Naumann, A., Krems, J.F.: Learning Grammar with Adaptive Hypertexts: Reading or Searching? In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, Springer, Heidelberg (2002)

    Google Scholar 

  21. Brusilovsky, P., Eklund, J.: A study of user-model based link annotation in educational hypermedia. Journal of Universal Computer Science, Special Issue on Assessment Issues for Educational Software 4(4), 429–448 (1998)

    Google Scholar 

  22. Brusilovsky, P., Karagiannidis, C., Sampson, D.: The benefits of layered evaluation of adaptive applications and services. In: Weibelzahl, S., Chin, D.N., Weber, G. (eds.) Empirical Evaluation of Adaptive Systems. Proceedings of Workshop At the Eighth International Conference on User Modeling, UM2001, pp. 1–8 (2001)

    Google Scholar 

  23. Brusilovsky, P., Millán, E.: User Models for Adaptive Hypermedia and Adaptive Educational Systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. LNCS, vol. 4321, pp. 3–53. Springer, Heidelberg (2007)

    Google Scholar 

  24. Brusilovsky, P., Pesin, L.: Adaptive navigation support in educational hypermedia: An evaluation of the ISIS-tutor. Journal of Computing and Information Technology 6(1), 27–38 (1998)

    Google Scholar 

  25. Brusilovsky, P., Sosnovsky, S., Yudelson, M.: Addictive Links: The Motivational Value of Adaptive Link Annotation in Educational Hypermedia. In: Wade, V., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 51–60. Springer, Heidelberg (2006)

    Google Scholar 

  26. Bull, S.: User Modelling and Mobile Learning. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 383–387. Springer, Heidelberg (2003)

    Google Scholar 

  27. Bunt, A.: User Modeling to support user customization. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 499–501. Springer, Heidelberg (2005)

    Google Scholar 

  28. Burke, R.: Hybrid Recommender Systems: Survey and Experiments. User Modeling and User-Adapted Interaction 12(4), 331–370 (2002)

    MATH  Google Scholar 

  29. Burke, R.: Hybrid Web Recommender Systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. LNCS, vol. 4321, pp. 377–408. Springer, Heidelberg (2007)

    Google Scholar 

  30. Cena, F., Gena, C., Modeo, S.: How to communicate recommendations? Evaluation of an adaptive annotation technique. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 1030–1033. Springer, Heidelberg (2005)

    Google Scholar 

  31. Chapanis, A.: Evaluating usability. In: Shackel, B., Richardson, S.J. (eds.) Human Factors for Informatics Usability, pp. 359–395. Cambridge University Press, Cambridge (1991)

    Google Scholar 

  32. Chin, D.N.: Empirical evaluation of user models and user-adapted systems. User Modeling and User-Adapted Interaction 11(1-2), 181–194 (2001)

    MATH  Google Scholar 

  33. Cohen, J.: Statistical power analysis for the behavioral sciences (revised edn.). Academic Press, New York (1977)

    Google Scholar 

  34. Conati, C., Merten, C., Muldner, K.: Exploring Eye Tracking to Increase Bandwidth. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 357–366. Springer, Heidelberg (2005)

    Google Scholar 

  35. Console, L., Gena, C., Torre, I.: Evaluation of an On-vehicle Adaptive Tourist Service. In: Weibelzahl, S., Paramythis, A. (eds.) Proceedings of the Second Workshop on Empirical Evaluation of Adaptive Systems, held at the 9th International Conference on User Modeling UM2003, Pittsburgh, pp. 51–60 (2003)

    Google Scholar 

  36. Courage, C., Baxter, K.: Understanding Your Users: A Practical Guide to User Requirements Methods, Tools, and Techniques. Morgan Kaufmann, San Francisco (2005)

    Google Scholar 

  37. Cugini, J., Scholtz, J.: VISVIP: 3D Visualization of Paths Through Web Sites. In: Proceedings of the International Workshop on Web-Based Information Visualization, pp. 259–263 (1999)

    Google Scholar 

  38. Czarkowski, M.: Evaluating Scrutable Adaptive Hypertext. In: Weibelzahl, S., Paramythis, A., Masthoff, J. (eds.) Proceedings of the Fourth Workshop on Empirical Evaluation of Adaptive Systems, held at the 10th International Conference on User Modeling UM2005, Edinburgh, pp. 37–46 (2005)

    Google Scholar 

  39. Del Missier, F., Ricci, F.: Understanding Recommender Systems: Experimental Evaluation Challenges. In: Weibelzahl, S., Paramythis, A. (eds.) Proceedings of the Second Workshop on Empirical Evaluation of Adaptive Systems, held at the 9th International Conference, pp. 31–40 (2003)

    Google Scholar 

  40. Diaper, D. (ed.): Task Analysis for Human-computer Interaction. Ellis Horwood, Chicester (1989)

    Google Scholar 

  41. Dimitrova, V.: Using Dialogue Games to Maintain Diagnostic Interactions. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 117–121. Springer, Heidelberg (2003)

    Google Scholar 

  42. Dimitrova, V., Self, J., Brna, P.: Applying Interactive Open Learner Models to Learning Technical Terminology. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 148–157. Springer, Heidelberg (2001)

    Google Scholar 

  43. Dix, A., Finlay, J., Abowd, G., Beale, R.: Human Computer Interaction, 2nd edn. Prentice Hall, Englewood Cliffs (1998)

    Google Scholar 

  44. Dumas, J.S., Redish, J.C.: A Practical Guide To Usability Testing. Ablex Publishing, Greenwich (1999)

    Google Scholar 

  45. Encarnação, L.M., Stoev, S.L.: Application-independent Intelligent User Support System Exploiting Action-sequence Based User Modeling. In: Kay, J. (ed.) User modeling: Proceedings of the Seventh International Conference, UM’99, pp. 245–254. Springer, Heidelberg (1999)

    Google Scholar 

  46. Farzan, R., Brusilovsky, P.: Social Navigation Support in a Course Recommendation System. In: Wade, V., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 91–100. Springer, Heidelberg (2006)

    Google Scholar 

  47. Fink, J., Kobsa, A., Nill, A.: Information Provision for All Users, Including Disabled and Elderly People. New Review of Hypermedia and Multimedia 4, 163–188 (1998)

    Google Scholar 

  48. Fischer, G., Ye, Y.: Personalizing Delivered Information in a Software Reuse Environment. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 178–187. Springer, Heidelberg (2001)

    Google Scholar 

  49. Gena, C.: Designing TV Viewer Stereotypes for an Electronic Program Guide. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 247–276. Springer, Heidelberg (2001)

    Google Scholar 

  50. Gena, C.: Evaluation Methodologies and User Involvement in User Modeling and Adaptive Systems. Unpub. Phd thesis, Università degli Studi di Torino, Italy (2003)

    Google Scholar 

  51. Gena, C.: Methods and Techniques for the Evaluation of User-adaptive Systems. The Knowledge Engineering Review 20(1), 1–37 (2005)

    Google Scholar 

  52. Gena, C., Ardissono, L.: Intelligent Support to the Retrieval of Information About Hydric Resources. In: De Bra, P., Nejdl, W. (eds.) AH 2004. LNCS, vol. 3137, pp. 126–135. Springer, Heidelberg (2004)

    Google Scholar 

  53. Gena, C., Perna, A., Ravazzi, M.: E-tool: a Personalized Prototype for Web Based Applications. In: de Waard, D., Brookhuis, K., Moraal, J., Toffetti, A. (eds.) Human Factors in Transportation, Communication, Health, and the Workplace, pp. 485–496. Shaker Publishing (2001)

    Google Scholar 

  54. Gena, C., Torre, I.: The Importance of Adaptivity to Provide On-Board Services. A Preliminary Evaluation of an Adaptive Tourist Information Service on Board Vehicles. Applied Artificial Intelligence Journal, Special Issue on Mobile A.I.  (2004)

    Google Scholar 

  55. Good, N., Schafer, J.B., Konstan, J.A., Botchers, A., Sarwar, B.M., Herlocker, J.L., Ricdl, J.: Combining Collaborative Filtering with Personal Agents for Better Recommendations. In: Proceedings of the Sixteenth National Conference on Artificial Intelligence, pp. 439–446 (1999)

    Google Scholar 

  56. Gould, J.D., Lewis, C.: Designing for Usability – Key Principles and What Designers Think. In: Human Factors in Computing Systems, CHI ’83 Proceedings, pp. 50–53. ACM Press, New York (1983)

    Google Scholar 

  57. Goy, A., Ardissono, L., Petrone, G.: Personalization in E-Commerce Applications. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. LNCS, vol. 4321, pp. 485–520. Springer, Heidelberg (2007)

    Google Scholar 

  58. Greenbaum, J., Kyng, M. (eds.): Design At Work: Cooperative Design of Computer Systems. Lawrence Erlbaum Associates, Hillsdale (1991)

    Google Scholar 

  59. Greenbaum, T.L.: The Handbook of Focus Group Research, 2nd edn. Lexington Books, New York (1998)

    Google Scholar 

  60. Habieb-Mammar, H., Tarpin-Bernard, F., Prévôt, P.: Adaptive Presentation of Multimedia Interface Case Study: Brain Story Course. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 15–24. Springer, Heidelberg (2003)

    Google Scholar 

  61. Hara, Y., Tomomune, Y., Shigemori, M.: Categorization of Japanese TV Viewers Based on Program Genres They Watch. In: Ardissono, L., Kobsa, A., Maybury, M. (eds.) Personalization and user-adaptive interaction in digital TV, Kluwer Academic Publishers, Dordrecht (2004)

    Google Scholar 

  62. Herder, E.: Utility-Based Evaluation of Adaptive Systems. In: Proceedings of the Second Workshop on Empirical Evaluation of Adaptive Systems, held at the 9th International Conference on User Modeling UM2003, Pittsburgh, pp. 25–30 (2003)

    Google Scholar 

  63. Herder, E., Weinreich, H., Obendorf, H., Mayer, M.: Much to Know About History. In: Wade, V., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 283–287. Springer, Heidelberg (2006)

    Google Scholar 

  64. Höök, K.: Evaluating the Utility and Usability of an Adaptive Hypermedia System. In: Proceedings of 1997 International Conference on Intelligent User Interfaces, pp. 179–186. ACM, New York (1997)

    Google Scholar 

  65. Höök, K.: Steps to Take Before IUIs Become Real. Journal of Interacting with Computers 12(4), 409–426 (2000)

    Google Scholar 

  66. Iglezakis, D.: Is the ACT-value a Valid Estimate for Knowledge? An Empirical Evaluation of the Inference Mechanism of an Adaptive Help System. In: Weibelzahl, S., Paramythis, A., Masthoff, J. (eds.) Proceedings of the Fourth Workshop on Empirical Evaluation of Adaptive Systems, held at the 10th International Conference on User Modeling UM2005, Edinburgh, pp. 19–26 (2005)

    Google Scholar 

  67. Ikehara, S., Chin, D.N., Crosby, M.E.: A Model for Integrating an Adaptive Information Filter Utilizing Biosensor Data to Assess Cognitive Load. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 208–212. Springer, Heidelberg (2003)

    Google Scholar 

  68. International Organisation for Standardisation ISO 9241-11:1998 Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), Part 11 Guidance on Usability (1998), http://www.iso.org/

  69. International Organisation for Standardisation ISO/TS 16071:2003 Ergonomics of Human-system Interaction - Guidance on Software Accessibility. Technical Specification (2003), http://www.iso.org/

  70. Jackson, T., Mathews, E., Lin, D., Olney, A., Graesser, A.: Modeling Student Performance to Enhance the Pedagogy of AutoTutor. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 368–372. Springer, Heidelberg (2003)

    Google Scholar 

  71. Jameson, A., Schäfer, R., Weis, T., Berthold, A., Weyrath, T.: Making Systems Sensitive to the User’s Changing Resource Limitations. Knowledge-Based Systems 12(8), 413–425 (1999)

    Google Scholar 

  72. Jameson, A., Schwarzkopf, E.: Pros and Cons of Controllability: An Empirical Study. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 193–202. Springer, Heidelberg (2002)

    Google Scholar 

  73. Jameson, A.: Adaptive Interfaces and Agents. In: The Human-Computer Interaction Handbook, pp. 316–318. Lawrence Erlbaum Associates, New Jersey (2003)

    Google Scholar 

  74. Jameson, A.: User Modeling Meets Usability Goals. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 1–3. Springer, Heidelberg (2005)

    Google Scholar 

  75. Juvina, I., Herder, E.: The Impact of Link Suggestions on User Navigation and User Perception. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 483–492. Springer, Heidelberg (2005)

    Google Scholar 

  76. Karagiannidis, C., Sampson, D.: Layered Evaluation of Adaptive Applications and Services. In: Brusilovsky, P., Stock, O., Strapparava, C. (eds.) AH 2000. LNCS, vol. 1892, pp. 343–346. Springer, Heidelberg (2000)

    Google Scholar 

  77. Katz-Haas, R.: Ten Guidelines for User-Centered Web Design. Usability Interface 5(1), 12–13 (1998)

    Google Scholar 

  78. Kay, J.: Learner Control. User Modeling and User-Adapted Interaction, Tenth. Anniversary Special Issue 11(1-2), 111–127 (2001)

    MATH  Google Scholar 

  79. Keppel, G.: Design and Analysis: A Researcher’s Handbook. Prentice-Hall, Englewood Cliffs (1991)

    Google Scholar 

  80. Keppel, G., Saufley, W.H., Tokunaga, H.: Introduction to Design and Analysis, A Student’s Handbook, 2nd edn. Prentice-Hall, Englewood Cliffs (1998)

    Google Scholar 

  81. Kirby, M.A.R.: CUSTOM Manual Dpo/std/1.0. Huddersfield: HCI Research Centre, University of Huddersfield (1991)

    Google Scholar 

  82. Krogsæter, M., Oppermann, R., Thomas, C.G.: A User Interface Integrating Adaptability and Adaptivity. In: Oppermann, R. (ed.) Adaptive user support, pp. 97–125. Lawrence Erlbaum, Hillsdale (1994)

    Google Scholar 

  83. Krug, S.: Don’t Make Me Think! A Common Sense Approach to Web Usability. New Riders Publishing, Indianapolis (2000)

    Google Scholar 

  84. Krüger, A., Heckmann, D., Kruppa, M., Wasinger, R.: Adaptive Mobile Guides. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. LNCS, vol. 4321, pp. 521–549. Springer, Heidelberg (2007)

    Google Scholar 

  85. Kuflik, T., Shapira, B., Elovici, Y., Maschiach, A.: Privacy Preservation Improvement By Learning Optimal Profile Generation Rate. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 168–177. Springer, Heidelberg (2003)

    Google Scholar 

  86. Kumar, A.: Evaluating Adaptive Generation of Problems in Programming Tutors – Two Studies. In: Proc. of the Fifth Workshop on User-Centred Design and Evaluation of Adaptive Systems, pp. 450–459 (2006)

    Google Scholar 

  87. Kurhila, J., Miettinen, M., Nokelainen, P., Tirri, H.: EDUCO - A Collaborative Learning Environment Based on Social Navigation. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 242–252. Springer, Heidelberg (2002)

    Google Scholar 

  88. Lang, T., Secic, M.: How to Report Statistics in Medicine: Annotated Guidelines for Authors, Editors and Reviewers. American College of Physicians, Philadelphia (1997)

    Google Scholar 

  89. Lavie, T., Meyer, J., Bengler, K., Coughlin, J.F.: The Evaluation of In-Vehicle Adaptive Systems. In: Weibelzahl, S., Paramythis, A., Masthoff, J. (eds.) Proceedings of the Fourth Workshop on Empirical Evaluation of Adaptive Systems, held at the 10th International Conference on User Modeling UM2005, Edinburgh, pp. 9–18 (2005)

    Google Scholar 

  90. Lee, J., Lai, K.-Y.: What’s in Design Rationale? Human-Computer Interaction: special issue on design rationale 6(3-4), 251–280 (1991)

    Google Scholar 

  91. Litman, D.J., Pan, S.: Predicting and Adapting to Poor Speech Recognition in a Spoken Dialogue System. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence, Austin, TX, pp. 722–728 (2000)

    Google Scholar 

  92. Magnini, B., Strapparava, C.: Improving User Modelling with Content-based Techniques. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, Springer, Heidelberg (2001)

    Google Scholar 

  93. Magoulas, G.D., Chen, S.Y., Papanikolaou, K.A.: Integrating Layered and Heuristic Evaluation for Adaptive Learning Environments. In: Weibelzahl, S., Paramythis, A. (eds.) Proceedings of the Second Workshop on Empirical Evaluation of Adaptive Systems, held at the 9th International Conference on User Modeling UM2003, Pittsburgh, pp. 5–14 (2003)

    Google Scholar 

  94. Martin, B., Mitrovic, T.: WETAS: A Web-Based Authoring System for Constraint-Based ITS. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 396–405. Springer, Heidelberg (2002)

    Google Scholar 

  95. Martin, B., Mitrovic, T.: The Effect of Adapting Feedback Generality in ITS E-Learning and Personalization. In: Wade, V., Ashman, H., Smyth, B. (eds.) AH 2006. LNCS, vol. 4018, pp. 192–202. Springer, Heidelberg (2006)

    Google Scholar 

  96. Masthoff, J.: The Evaluation of Adaptive Systems. In: Patel, N.V. (ed.) Adaptive evolutionary information systems, pp. 329–347. Idea Group publishing, Hershey (2002)

    Google Scholar 

  97. Masthoff, J.: The User As Wizard. In: Proc. of the Fifth Workshop on User-Centred Design and Evaluation of Adaptive Systems, pp. 460–469 (2006)

    Google Scholar 

  98. Matsuo, Y.: Word Weighting Based on User’s Browsing History. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 35–44. Springer, Heidelberg (2003)

    Google Scholar 

  99. Maulsby, D., Greenberg, S., Mander, R.: Prototyping an Intelligent Agent Through Wizard of Oz. In: ACM SIGCHI Conference on Human Factors in Computing Systems, Amsterdam, The Netherlands, May, pp. 277–284. ACM Press, New York (1993)

    Google Scholar 

  100. Maybury, M., Brusilovsky, P.: The Adaptive Web. Communications of the ACM 45 (2002)

    Google Scholar 

  101. McCreath, E., Kay, J.: IEMS: Helping Users Manage Email. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 263–272. Springer, Heidelberg (2003)

    Google Scholar 

  102. McNee, S.M., Lam, S.K, Konstan, J.A., Riedl, J.: Interfaces for Eliciting New User Preferences in Recommender Systems. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 178–187. Springer, Heidelberg (2003)

    Google Scholar 

  103. Mitrovic, A.: Investigating Students’ Self-assessment Skills. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 247–250. Springer, Heidelberg (2001)

    Google Scholar 

  104. Mitrovic, A., Martin, B.: Evaluating the Effects of Open Student Models on Learning. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 296–305. Springer, Heidelberg (2002)

    Google Scholar 

  105. Mobasher, R.: Data Mining for Web Personalisation. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. LNCS, vol. 4321, pp. 90–136. Springer, Heidelberg (2007)

    Google Scholar 

  106. Mobasher, R., Cooley, R., Srivastava, J.: Automatic Personalization Based on Web Usage Mining. Communications of the ACM 43(8) (2000)

    Google Scholar 

  107. Monk, A., Wright, P., Haber, J., Davenport, L.: Improving Your Human Computer Interface: A Practical Approach. BCS Practitioner Series. Prentice-Hall International, Hemel Hempstead (1993)

    Google Scholar 

  108. Müller, C., Großmann-Hutter, B., Jameson, A., Rummer, R., Wittig, F.: Recognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 24–33. Springer, Heidelberg (2001)

    Google Scholar 

  109. Nielsen, J.: Usability Engineering. Academic Press, Boston (1993)

    MATH  Google Scholar 

  110. Nielsen, J.: Why You Only Need to Test With 5 Users. In: Alertbox (2000), http://www.useit.com/alertbox/20000319.html

  111. Nielsen, J.: Risks of Quantitative Studies. In: Alertbox (2004), http://www.useit.com/alertbox/20040301.html

  112. Nielsen, J., Mack, R.L. (eds.): Usability Inspection Methods. John Wiley & Sons, New York (1994)

    Google Scholar 

  113. Nielsen, J., Molich, R.: Heuristic Evaluation of User Interfaces. In: Proceedings of CHI ’90, Seattle, Washington, pp. 249–256 (1990)

    Google Scholar 

  114. Norman, D.A., Draper, S.W.: User Centered System Design: New Perspective on HCI. Lawrence Erlbaum, Hillsdale (1986)

    Google Scholar 

  115. Paramythis, A., Totter, A., Stephanidis, C.: A Modular Approach to the Evaluation of Adaptive User Interfaces. In: Weibelzahl, S., Chin, D., Weber, G. (eds.) Proceedings of the First Workshop on Empirical Evaluation of Adaptive Systems, Sonthofen, Germany, pp. 9–24 (2001)

    Google Scholar 

  116. Paramythis, A., Weibelzahl, S.: A Decomposition Model for the Layered Evaluation of Interactive Adaptive Systems. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 438–442. Springer, Heidelberg (2005)

    Google Scholar 

  117. Paganelli, L., Paterno’, F.: Intelligent Analysis of User Interactions with Web Applications. In: Proceedings of the 2002 International Conference on Intelligent User Interfaces, ACM Press, New York (2002)

    Google Scholar 

  118. Pazzani, M.J., Billsus, D.: Content-based Recommendation Systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization. LNCS, vol. 4321, pp. 324–341. Springer, Heidelberg (2007)

    Google Scholar 

  119. Perkowitz, M., Etzioni, O.: Adaptive Web Sites. Communications of the ACM 43(8) (2000)

    Google Scholar 

  120. Pierrakos, D., Paliouras, G., Papatheodorou, C., Spyropoulos, C.D.: Web Usage Mining As a Tool for Personalization: a Survey. International Journal of User Modeling and User-Adapted Interaction 13(4), 311–372 (2003)

    Google Scholar 

  121. Pirolli, P., Fu, W.T.: SNIF-ACT: A Model of Information Foraging on the World Wide Web. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 45–54. Springer, Heidelberg (2003)

    Google Scholar 

  122. Pohl, W.: LaboUr-machine Learning for User Modeling. In: Smith, M.J., Salvendy, G., Koubek, R.J. (eds.) Design of computing systems: Social and ergonomic considerations, vol. B, pp. 27–30. Elsevier, Amsterdam (1997)

    Google Scholar 

  123. Pohl, W.: User-adapted Interaction, User Modeling, and Machine Learning. In: Timm, U.J., Rössel, M. (eds.) Proceedings of the sixth german workshop on adaptivity and user modeling in interactive systems, ABIS98, Erlangen (1998)

    Google Scholar 

  124. Polson, P.G., Lewis, C., Rieman, J., Wharton, C.: Cognitive Walkthroughs: A Method for Theory- Based Evaluation of User Interfaces. International Journal of Man-Machine Studies 36, 741–773 (1992)

    Google Scholar 

  125. Preece, J., Rogers, Y., Sharp, H., Benyon, D.: Human-computer Interaction. Addison-Wesley, Reading (1994)

    Google Scholar 

  126. Resnick, P., Varian, H.R.: Special Issue on Recommender Systems. Communications of the ACM 40 (1997)

    Google Scholar 

  127. Ritter, F.E., Shadbolt, N., Elliman, D., Young, R., Gobet, F., Baxter, G.: Techniques for Modeling Human and Organizational Behaviour in Synthetic Environments: A Supplementary Review. Wright-Patterson Air Force Base, OH: Human Systems Information Analysis Center (2002)

    Google Scholar 

  128. Rizzo, P., Lee, I.H., Shaw, E., Johnson, W.L., Wang, N., Mayer, R.E.: A Semi-Automated Wizard of Oz Interface for Modeling Tutorial Strategies. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 174–178. Springer, Heidelberg (2005)

    Google Scholar 

  129. Rosenfeld, L., Morville, P.: Information Archtecture. O’Reilly, Sebastopol (1998)

    Google Scholar 

  130. Rubin, J.: Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, 1st edn. John Wiley & Sons, Chichester (1994)

    Google Scholar 

  131. Jr. Santos, E.H., Nguyen, Q., Zhao, E.: Pukinskis: Empirical Evaluation of Adaptive User Modeling in a Medical Information Retrieval Application. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 292–296. Springer, Heidelberg (2003)

    Google Scholar 

  132. Sarwar, B.M., Konstan, J.A., Borchers, A., Herlocker, J., Miller, B., Riedl, J.: Using Filtering Agents to Improve Prediction Quality in the GroupLens Research Collaborative Filtering System. In: Proceeding of the ACM Conference on Computer Supported Cooperative Work (CSCW), Seattle, WA, pp. 439–446. ACM Press, New York (1998)

    Google Scholar 

  133. Scarano, V., Barra, M., Maglio, P., Negro, A.: GAS: Group Adaptive Systems. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 47–57. Springer, Heidelberg (2002)

    Google Scholar 

  134. Sears, A., Shneiderman, B.: Split Menus: Effectively Using Selection Frequency to Organize Menus. ACM Transactions on Computer-Human Interaction 1(1), 27–51 (1994)

    Google Scholar 

  135. Semeraro, G., Ferilli, S., Fanizzi, N., Abbattista, F.: Learning Interaction Models in a Digital Library Service. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 44–53. Springer, Heidelberg (2001)

    Google Scholar 

  136. Shardanand, U., Maes, P.: Social Information Filtering for Automating ”Word of Mouth”. In: Proceedings of CHI-95, Denver, CO, pp. 210–217 (1995)

    Google Scholar 

  137. Smyth, B., Cotter, P.: Personalized Adaptive Navigation for Mobile Portals. In: Proceedings of the 15th European Conference on Artificial Intelligence - Prestigious Applications of Intelligent Systems, Lyons, France (2002)

    Google Scholar 

  138. Specht, M., Kobsa, A.: Interaction of Domain Expertise and Interface Design. In: Adaptive Educational Hypermedia, Workshops on Adaptive Systems and User Modeling on the World Wide Web at WWW-8, Toronto, Canada, and Banff, Canada (1999)

    Google Scholar 

  139. Spiliopoulou, M.: Web Usage Mining for Web Site Evaluation. Communications of the ACM 43(8) (2000)

    Google Scholar 

  140. Spradley, J.: Participant Observation. Wadsworth Publishing (1980)

    Google Scholar 

  141. Stephanidis, C.: Adaptive Techniques for Universal Access. User Modeling and User-Adapted Interaction 11(Issue 1-2), 159–179 (2001)

    MATH  Google Scholar 

  142. Strauss, A.L., Corbin, J.M.: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE, Thousand Oaks (1998)

    Google Scholar 

  143. Tasso, C., Omero, P.: La Personalizzazione Dei Contenuti Web. Franco Angeli, Milano (2002)

    Google Scholar 

  144. Tobar, C.M.: Yet Another Evaluation Framework. In: Weibelzahl, S., Paramythis, A. (eds.) Proceedings of the Second Workshop on Empirical Evaluation of Adaptive Systems, held at the 9th International Conference on User Modeling UM2003, Pittsburgh, pp. 15–24 (2003)

    Google Scholar 

  145. Totterdell, P., Boyle, E.: The Evaluation of Adaptive Systems. In: Browne, D., Totterdell, P., Norman, M. (eds.) Adaptive User Interfaces, pp. 161–194. Academic Press, London (1990)

    Google Scholar 

  146. Travassos, G.H., Barros, M.O.: Contributions of In Virtuo and In Silico Experiments for the Future of Empirical Studies in Software Engineering. In: Jedlitschka, A., Ciolkowski, M. (eds.) Proceedings of the Second Workshop on the Future of Empirical Studies in Software Engineering, pp. 109–121. Roman Castles, Italy (2003)

    Google Scholar 

  147. Uruchurtu, E., MacKinnon, L., Rist, R.: User Cognitive Style and Interface Design for Personal, Adaptive Learning. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 154–163. Springer, Heidelberg (2005)

    Google Scholar 

  148. van Solingen, R., Berghout, E.: The Goal/question/metric Method: A Practical Guide for Quality Improvement of Software Development. McGraw-Hill, London (1999)

    Google Scholar 

  149. van Barneveld, J., van Setten, M.: Involving users in the design of user interfaces for TV recommender systems. 3rd Workshop on Personalization in Future TV, Associated with UM03, Johnstown, PA (2003)

    Google Scholar 

  150. Web Accessibility Initiative W3C - Evaluating Web Sites for Accessibility, http://www.w3.org/WAI/eval/

  151. Web Accessibility Initiative W3C - Web Content Accessibility Guidelines 1.0, http://www.w3c.org/TR/WAI-WEBCONTENT

  152. Weibelzahl, S.: Evaluation of Adaptive Systems. In: Bauer, M., Gmytrasiewicz, P.J., Vassileva, J. (eds.) UM 2001. LNCS (LNAI), vol. 2109, pp. 292–294. Springer, Heidelberg (2001)

    Google Scholar 

  153. Weibelzahl, S.: Evaluation of Adaptive Systems. Dissertation. University of Trier, Germany (2003)

    Google Scholar 

  154. Weibelzahl, S.: Problems and pitfalls in the evaluation of adaptive systems. In: Chen, S., Magoulas, G. (eds.) Adaptable and Adaptive Hypermedia Systems, pp. 285–299. IRM Press, Hershey (2005)

    Google Scholar 

  155. Weibelzahl, S., Lauer, C.U.: Framework for the Evaluation of Adaptive CBRSystems. In: Reimer, U., Schmitt, S., Vollrath, I. (eds.) Proceedings of the 9th German Workshop on Case-Based Reasoning (GWCBR01), Aachen, pp. 254–263. Shaker (2001)

    Google Scholar 

  156. Weibelzahl, S., Weber, G.: A database of empirical evaluations of adaptive systems. In: Klinkenberg, R., Rüping, S., Fick, A., Henze, N., Herzog, C., Molitor, R., Schröder, O. (eds.) Proceedings of Workshop Lernen - Lehren - Wissen - Adaptivität (LLWA 01), Research Report in Computer Science Nr. 763, University of Dortmund, pp. 302–306 (2001)

    Google Scholar 

  157. Weibelzahl, S., Jedlitschka, A., Ayari, B.: Eliciting Requirements for a Adaptive Decision Support System through Structured User Interviews. In: Proc. of the Fifth Workshop on User-Centred Design and Evaluation of Adaptive Systems, pp. 470–478 (2006)

    Google Scholar 

  158. Weibelzahl, S., Weber, G.: Evaluating the inference mechanism of adaptive learning systems. In: Brusilovsky, P., Corbett, A.T., de Rosis, F. (eds.) UM 2003. LNCS, vol. 2702, pp. 154–162. Springer, Heidelberg (2003)

    Google Scholar 

  159. Whiteside, J., Bennett, J., Holtzblatt, K.: Usability Engineering: Our Experience and Evolution. In: Helander, M. (ed.) Handbook of Human-Computer Interaction, pp. 791–817. North-Holland, New York (1988)

    Google Scholar 

  160. Wilkinson, L. and Task Force on Statistical Inference: Statistical methods in psychology journals: Guidelines and explanations. American Psychologist 54(8), 594–604 (1999)

    Google Scholar 

  161. Wingender, E. (ed.): In Silico Biology. An international journal on computational molecular biology. IOS Press, Amsterdam (1998)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Informatica, Università di Torino, Corso Svizzera 185, Torino, Italy

    Cristina Gena

  2. School of Informatics, National College of Ireland, Mayor Street, Dublin, Ireland

    Stephan Weibelzahl

Authors
  1. Cristina Gena
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephan Weibelzahl
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Peter Brusilovsky Alfred Kobsa Wolfgang Nejdl

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this chapter

Cite this chapter

Gena, C., Weibelzahl, S. (2007). Usability Engineering for the Adaptive Web. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds) The Adaptive Web. Lecture Notes in Computer Science, vol 4321. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72079-9_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-72079-9_24

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-72078-2

  • Online ISBN: 978-3-540-72079-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics