Skip to main content
SpringerLink
Log in

Experience with usability evaluation of e-learning systems

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Progress in the field of e-learning has been slow, with related problems mainly associated with the poor design of e-learning systems. Moreover, because of a depreciated importance of usability, usability studies are not very frequent. This paper reports the experience with the usability assessment of intelligent learning and teaching systems which are based on TEx-Sys model and are intended to enhance the process of knowledge acquisition in daily classroom settings. The applied scenario-based usability evaluation, as a combination of behaviour and opinion based measurements, enabled to quantify usability in terms of users’ (teachers’ and students’) performance and satisfaction. According to the achieved results, the main directions for interface redesign are offered. The acquired experience indicates that useful usability assessments with a significant identification of interface limitations can be performed quite easily and quickly. On the other hand, it raised a series of questions which, in order to be clarified, require further comprehensive research, the more so if the employment of universal design within e-learning context is considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ardito, C., Costabile M.F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., Rossano, V.: An approach to usability evaluation of e-learning applications. Univ. Access Inf. Soc. 4, 270–283 (2006)

    Article  Google Scholar 

  2. Bevan, N.: Usability issues in Web Site Design. In: Proceedings of the HCI international ‘97. San Francisco, USA, 24–30 August 1997. Elsevier Press, Amsterdam (1997)

  3. Bevan, N., Macleod M.: Usability measurement in context. Behav. Inf. Technol. 13, 132–145 (1994)

    Article  Google Scholar 

  4. Burns, H., Capps, C.: Foundations of intelligent tutoring systems: an introduction. In: Polson M., Richardson J. (eds.) Foundations of Intelligent Tutoring Systems, pp. 1–18. Lawrence Erlbaum Associates Publishers, Mahwah (1988)

    Google Scholar 

  5. Carbonel, R.: AI in CAI: an artificial-intelligence approach to computer-assisted instruction. IEEE Trans. Man Mach. Syst. MMS 11(4), 190–202 (1970)

    Article  Google Scholar 

  6. Carroll, J., Rosson, M.: Usability specifications as a tool in iterative development. In: Harston R. (eds.) Advances in Human–computer Interaction 1. Ablex, Northwood, pp 1–28 (1985)

    Google Scholar 

  7. Collins, A., Neville, P., Bielaczyc, K.: The role of different media in designing learning environments. Int. J. Artif. Intell. Educ. 11, 144–162 (2000)

    Google Scholar 

  8. Costabile, M., Marisco, M., Lanzilotti, R., Plantamura, V., Roselli, T.: On the usability evaluation of E-learning applications. In: Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS’05)––Track 1, vol. 01. 6.2 (2005)

  9. De Marisco, M., Kimani, S., Mirabella, V., Norman, K.L., Catarci, T.: A proposal toward the development of accessible e-learning content by human involvement. Univ. Access. Inf. Soc. 5, 150–169 (2006)

    Article  Google Scholar 

  10. Dringus, L.: An iterative usability evaluation procedure for interactive online courses. J. Int. Instr. Dev. 7, 10–14 (1995)

    Google Scholar 

  11. Faulkner, L.: Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav. Res. Methods Instrum. Comput. 35, 379–383 (2003)

    Google Scholar 

  12. Feldstein, M.: What is ‘‘Usable’’ e-Learning? eLearn Magazine. ACM Press, New York. http://www.elearnmag.org (2002)

  13. Fleischmann, A.: The electronic teacher: The social impact of intelligent tutoring systems in education. http://www4.in.tum.de/~fleischa/papers/its.html (2000)

  14. Fletcher J.: Evidence for learning from technology––assisted instruction. In: O’Neil, H., Perez, R. (eds.) Technology Applications in Education: A Learning View, pp. 79–99. Lawrence Erlbaum Associates, New Jersey (2003)

    Google Scholar 

  15. Granić, A.: Foundation of adaptive interfaces for computerized educational systems. Ph.D. Dissertation. Faculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, Croatia (in Croatian) (2002)

  16. Harper, B., Norman, K.: Improving user satisfaction: The questionnaire for user interaction satisfaction Version 5.5. In: Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, Virginia Beach, VA, pp. 224–228 (1993)

  17. HFRG, Human Factors Research Group. Software Usability Measurement Inventory, SUMI. http://www.ucc.ie/hfrg/questionnaires/sumi

  18. Holzinger, A.: Usability engineering for software developers. Commun. ACM, 48(1), 71–74 (2005)

    Article  Google Scholar 

  19. Holzinger, A.: Application of rapid prototyping to the user interface development for a virtual medical campus. IEEE Softw. 21(1), 92–99 (2004)

    Article  Google Scholar 

  20. Hsi, S., Soloway, E.: Learner-centered design: Addressing, finally, the unique needs of learners. In: Computer Human Interaction ‘98 (CHI ‘98) conference summary, Los Angeles, USA, pp. 211–212. ACM Press, New York (1998)

  21. ISO (1998) Ergonomic requirements for office work with visual display terminals (VDTs)––part 11: Guidance on usability (ISO 9241–11:1998(E)). Geneva, Switzerland

  22. Jones, M., Okey, J.: Interface design for computer-based learning environments. http://ddi.cs.uni-potsdam.de/HyFISCH/Multimedia/Learning/InterfaceDesignJones.htm (1995)

  23. Kinshuk, P.A., Russell D.: Intelligent and adaptive systems. In: Collis B., Adelsberger H., Pawlowski J. (eds) Handbook on Information Technologies for Education and Training, pp. 79–92. Springer, Berlin (2001)

    Google Scholar 

  24. Levi, M., Conrad, F.: Usability testing of World Wide Web sites. Research Papers of Bureau of Labour Statistics. http://stats.bls.gov/ore/htm_papers/st960150.htm (1998)

  25. Lewis, J.: IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7(1), 57–78 (1995)

    Article  Google Scholar 

  26. Macleod, M., Bowden, R., Bevan, N., Curson, I.: The MUSiC performance measurement method. Behav. Inf. Technol. 16, 279–293 (1997)

    Article  Google Scholar 

  27. Murray, T.: Authoring intelligent tutoring systems: An analysis of the state of the art. Int. J. Artif. Intell. Educ. 10, 98–129 (1999)

    Google Scholar 

  28. Nielsen, J.: Jacob Nielsen on e-learning. Elearningpost. http://www.elearningpost.com/features/archives/001015.asp (2001)

  29. Nielsen J.: Why you only need to test with 5 users. Alertbox March. http://www.useit.com/alertbox/20000319.html (2000)

  30. Nielsen, J.: Usability Engineering. Academic Press, Boston (1993)

    MATH  Google Scholar 

  31. Notess M.: Usability, User Experience and Learner Experience. eLearn Magazine. ACM Press, New York. http://www.elearnmag.org (2001)

  32. Nute, D.: Knowledge representation. In: Shapiro S. (ed): Encyclopedia of Artificial Intelligence, pp 743–869. Wiley, London (1992)

    Google Scholar 

  33. Pantazis, C.:(2001) Executive summary: A vision of E-learning for America’s workforce. Report of the commission on technology and adult learning, ASTD. http://www.learningcircuits.org/2001/aug2001/pantazis.html (1992)

  34. Parlangeli, O., Marchigiani, E., Bagnara, S.: Multimedia systems in distance education: Effects of usability on learning. Interact. Comput. 12, 37–49 (1999)

    Article  Google Scholar 

  35. Patel A, Kinshuk (1996) Knowledge characteristics: Reconsidering the design of intelligent tutoring systems. In: Behrooz, A. (ed.) Knowledge Transfer, pp. 190–197. pAce, London

  36. Preece, J., Rogers Y., Sharp H., Benyon D., Holland S., Carey T.: Human–computer Interaction. Addison-Wesley, Wokingham (1994)

    Google Scholar 

  37. Quigley, A.: Usability-tested E-learning. Not Until the Market Requires It. eLearn Magazine. ACM Press, New York. http://www.elearnmag.org (2002)

  38. Quillian, R.: Semantic memory. In: Minsky M. (eds.) Semantic Information Processing, pp. 216–270. MIT Press, Cambridge (1968)

    Google Scholar 

  39. Rickel, J.: Intelligent computer-aided instruction: A survey organized around system components. IEEE Trans. Syst. Man Cybern. 19(1), 40–57 (1989)

    Article  Google Scholar 

  40. Rosić, M.: Establishing of distance education systems within the information infrastructure. M.Sc. Thesis. Faculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, Croatia (in Croatian) (2000)

  41. Shackel B.: Usability––context, framework, design and evaluation. In: Shackel B., Richardson S. (eds.) Human Factors for Informatics Usability. Cambridge University Press, Cambridge, 21–38 (1991)

    Google Scholar 

  42. Sherwood, R.: Models of computer use in school settings. In: Kinzer, Ch., Sherwood R., Bransford J. (eds.) Computer Strategies for Education: Foundations and Content-area Applications, pp. 105–130. Merrill Publishing Company, Columbus (1986)

    Google Scholar 

  43. Shneiderman, B.: Designing the User Interface. Strategies for Effective Human–computer Interaction, 3rd edn. Addison-Wesley, Reading (1998)

    Google Scholar 

  44. SIGCHI (2001) Notes from E-learning Special Interest Group (SIG) Discussion at CHI 2001. http://www.elearning.org/

  45. Sloan, D., Kelly, B., Heath, A., Petrie, H., Fraser, H., Phipps, L.: Contextual Web accessibility––maximizing the benefit of accessibility guidelines. WWW 2006 Edinburgh, Scotland 22–26 May 2006. In: Conference Proceedings, Special Interest Tracks, Posters and Workshops (CD ROM). http://www.ukoln.ac.uk/web-focus/papers/w4a-2006/ (2006)

  46. Squires, D., Preece, J.: Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interact. Comput. 11, 467–483 (1999)

    Article  Google Scholar 

  47. Squires, D., Preece, J.: Usability and learning: Evaluating the potential of educational software. Comput. Educ. 27, 15–22 (1996)

    Article  Google Scholar 

  48. Stankov, S.: Isomorphic model of the system as the basis of teaching control principles in the intelligent tutoring system. Ph.D. Dissertation, Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, University of Split (in Croatian) (1997)

  49. Stephanidis, C., Salvendy, G., Akoumianakis, D., Bevan, N., Brewer, J., Emiliani, P.L, Galetsas, A., Haataja, S., Iakovidis, I., Jacko, J., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Stary, C., Vanderheiden, G., Weber, G., Ziegler, J.: Towards an information society for all: an international R&D agenda. Int. J. Hum. Comput. Interact. 10(2), 107–134 (1998)

    Article  Google Scholar 

  50. Zaharias, P.: E-learning design quality: A holistic conceptual framework. In: Encyclopedia of Distance Learning, vol. II. Idea Group Inc., Hershey (2005)

  51. Zaharias, P.: Usability and E-learning. The Road Towards Integration. eLearn Magazine. ACM Press, New York. http://www.elearnmag.org (2004)

  52. Zaharias, P., Vassilopoulou, K., Poulymenakou, A.: Designing on-line learning courses: Implications for usability. Sci. J. Appl. Inf. Technol. 1(1) (2002)

  53. WAMMI consortium (2002), WAMMI––web usability questionnaire. http://www.wammi.com

  54. Woolf, B.: AI in education. In: Shapiro I., Stuart C. (Eds.) Artificial Intelligence––Encyclopedias, pp. 434–444. Wiley, New York (1992)

    Google Scholar 

  55. Wrigh, P., Monk, A.: A cost-effective evaluation method for use by designers. Int. J. Man Mach. Stud. 35(6), 891–912 (1991)

    Article  Google Scholar 

Download references

Acknowledgments

Author would like to acknowledge support for the work presented from the project 177-0361994-1998 Usability and Adaptivity of Interfaces for Intelligent Authoring Shells funded by the Ministry of Science, Education and Sports of the Republic of Croatia.

Author information

Authors and Affiliations

  1. Faculty of Science, University of Split, Nikole Tesle 12, 21000, Split, Croatia

    Andrina Granić

Authors
  1. Andrina Granić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrina Granić.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Granić, A. Experience with usability evaluation of e-learning systems. Univ Access Inf Soc 7, 209–221 (2008). https://doi.org/10.1007/s10209-008-0118-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-008-0118-z

Keywords

Search

Navigation