Skip to main content
SpringerLink
Log in

QUEST e-learning portal: applying heuristic evaluation, usability testing and eye tracking

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

In this study, the development of an e-learning portal for university students is presented. The portal is based on responsive web design which is accessible from anywhere, at any time and on any device. A complementary user experience evaluation methodology consisting of heuristic evaluation, usability testing, user experience questionnaire and eye tracking has been applied to get a holistic understanding and identification of the potential usability problems in the portal. The methodology highlights the relative efficacy of each method. The results suggest that the heuristic evaluation identified more usability problems than the usability testing. However, few severe usability problems were identified in the usability testing and most of them were not spotted in the heuristic evaluation. For additional insights, the participants’ objective visual responses were tracked using an eye tracker (Tobii X-30 Hz); the eye tracking data were also helpful in revealing the usability issues within the portal. The aggregated heat maps were helpful in identifying the complexity of the interface elements of the portal. The identified problems were fixed in the final version of the portal. Afterward, another usability test was carried out to measure the effectiveness and efficiency of the portal. The students’ perceptions regarding both versions were also analyzed through the questionnaire. According to the results, the e-learning portal seems to be efficient and effective and the students were satisfied. The proposed methodology resulted in increasing the satisfaction of the students which can lead to the maximum utilization of the e-learning portal. It is recommended that by integrating these evaluation methods, many usability problems can be identified which can then be fixed for achieving maximum user acceptance of the e-learning portals and the similar web portals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. C Flavián M Guinalíu R Gurrea 2006 The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inf. Manag. 43 1 1 14

    Article  Google Scholar 

  2. Hussain, Z., Milchrahm, H., Shahzad, S., Slany, W., Tscheligi, M., Wolkerstorfer, P.: Integration of extreme programming and user-centered design: lessons learned. In: International Conference on Agile Processes and Extreme Programming in Software Engineering, pp. 174–179. Springer, Berlin (2009)

  3. J Wang P Antonenko M Celepkolu Y Jimenez E Fieldman A Fieldman 2019 Exploring relationships between eye tracking and traditional usability testing data. Int. J. Hum. Comput. Interact. 35 6 483 494

    Article  Google Scholar 

  4. Andrews, K. Evaluation Comes in Many Guises: CHI 2008 Workshop on Beyond Time and Errors: Novel Evaluation Methods for Information Visualization (BELIV’08), Florence, Italy (2008)

  5. A Holzinger 2005 Usability engineering methods for software developers. Commun. ACM 48 1 71 74

    Article  Google Scholar 

  6. Rohrer, C.: When to use which user-experience research methods. Nielsen Norman Group. https://www.nngroup.com/articles/which-ux-research-methods/ (2014). Accessed 10 May 2020

  7. Albert, W., Tullis, T.: Measuring the user experience: collecting, analyzing, and presenting usability metrics. Newnes (2013)

  8. MW Jaspers 2009 A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int. J. Med. Inform. 78 5 340 353

    Article  Google Scholar 

  9. BA Kumar MS Goundar 2019 Usability heuristics for mobile learning applications. Edu. Inf. Technol. 24 2 1819 1833

    Article  Google Scholar 

  10. P Zaharias P Koutsabasis 2011 Heuristic evaluation of e-learning courses: a comparative analysis of two e-learning heuristic sets. Campus Wide Inf. Syst. 29 1 45 60

    Article  Google Scholar 

  11. DP Puspitasari A Tarigan 2019 Analysis of user interface and user experience usability on Arsitag.com mobile version using heuristic evaluation method. Int. J. Comput. Sci. Softw. Eng. 8 9 211 213

    Google Scholar 

  12. Maguire, M., Isherwood, P.: A comparison of user testing and heuristic evaluation methods for identifying website usability problems. In: International Conference of Design, User Experience, and Usability, pp. 429–438. Springer, Cham (2018)

  13. Jeffries, R., Miller, J.R., Wharton, C., Uyeda, K.: User interface evaluation in the real world: a comparison of four techniques. In: Robertson, S.P., Olson, G., Olson, J. (eds.) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 119–124. ACM, New York (1991)

  14. L Hasan A Morris S Probets 2012 A comparison of usability evaluation methods for evaluating e-commerce websites. Behav. Inf. Technol. 31 7 707 737

    Article  Google Scholar 

  15. HR Hartson TS Andre RC Williges 2001 Criteria for evaluating usability evaluation methods. Int. J. Hum. Comput. Interact. 13 4 373 410

    Article  Google Scholar 

  16. WS Tan D Liu R Bishu 2009 Web evaluation: heuristic evaluation vs. user testing. Int. J. Ind. Ergon. 39 4 621 627

    Article  Google Scholar 

  17. PT Thankam V Monaco H Thambuganipalle T Schleyer 2009 Comparative study of heuristic evaluation and usability testing methods. Stud. Health Technol. Inform. 143 322 327

    Google Scholar 

  18. F Limin G Salvendy L Turley 2010 Effectiveness of user testing and heuristic evaluation as a function of performance classification. Behav. Inf. Technol. 21 2 137 143

    Google Scholar 

  19. S Kim H Kim S Han 2013 A development of learning widget on m-learning and e-learning environments. Behav. Inf. Technol. 32 2 190 202

    Article  Google Scholar 

  20. K Al-Omar 2017 Evaluating the internal and external usability attributes of e-learning websites in Saudi Arabia. Adv. Comput. Int. J. (ACIJ) 8 3/4 1 12

    Article  Google Scholar 

  21. IKR Arthana IMA Pradnyana GR Dantes 2019 Usability testing on website wadaya based on ISO 9241–11. J. Phys. Conf. Ser. IOP Publ. 1165 1 1 8

    Google Scholar 

  22. AM Valenti 2019 Usability testing for a community college library website. Libr. Hi Tech News 36 1 1 8

    Article  MathSciNet  Google Scholar 

  23. I Mahmud F Sadia M Rahman S Ahmed D Islam 2020 Web usability test in 60 seconds: a theoretical foundation and empirical test. Indones. J. Electr. Eng. Comput. Sci. 17 1 398 403

    Article  Google Scholar 

  24. Kaya, A., Ozturk, R., & Gumussoy, C.A.: Usability measurement of mobile applications with system usability scale (SUS). In: Industrial Engineering in the Big Data Era, pp. 389–400. Springer, Cham (2019)

  25. P Kortum M Sorber 2015 Measuring the usability of mobile applications for phones and tablets. Int. J. Hum. Comput. Interact. 31 8 518 529

    Article  Google Scholar 

  26. PT Kortum A Bangor 2013 Usability ratings for everyday products measured with the system usability scale. Int. J. Hum. Comput. Interact. 29 2 67 76

    Article  Google Scholar 

  27. AM Lund 2001 Measuring usability with the USE questionnaire. Usability Interface 8 2 3 6

    Google Scholar 

  28. Bai, A., Stray, V., Mork, H.: What methods software teams prefer when testing web accessibility. Adv. Hum. Comput. Interact. (2019)

  29. Jamil, A., Faisal, C.M.N., Habib, M.A., Jabbar, S., Ahmad, H.: The experimental investigation of the influence of gender in enjoyment, ease of use, satisfaction, and usefulness during interaction in gaming environment. In: Proceedings of the XX International Conference on Human Computer Interaction, pp. 1–4 (2019)

  30. AN Fajar 2019 E-learning implementation using user experience questionnaire. J. Phys. Conf. Ser. IOP Publ. 1367 1 1 6

    Google Scholar 

  31. Paredes, R.K., Hernandez, A.A.: Measuring the quality of user experience on web services: A case of university in the Philippines. In: IEEE 9th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM), pp. 1–6. IEEE (2017)

  32. Demirci, H.M., Berberoğlu, M.: User experience over time with personal assistants of mobile banking application in Turkey. In: International Conference on Intelligent Human Systems Integration, pp. 646–652. Springer, Cham (2020)

  33. Q Wang S Yang M Liu Z Cao Q Ma 2014 An eye-tracking study of website complexity from cognitive load perspective. Decis. Support Syst. 62 1 10

    Article  Google Scholar 

  34. Pavani, M.L., Prakash, A.B., Koushik, M.S., Amudha, J., Jyotsna, C.: Navigation through eye-tracking for human–computer interface. In: Information and Communication Technology for Intelligent Systems, pp. 575–586. Springer, Singapore (2019)

  35. Eghdam, A., Forsman, J., Falkenhav, M., Lind, M., Koch, S.: Combining usability testing with eye-tracking technology: evaluation of a visualization support for antibiotic use in intensive care. In: MIE, pp. 945–949 (2011)

  36. DT Knoepfle JTY Wang CF Camerer 2009 Studying learning in games using eye-tracking. J. Eur. Econ. Assoc. 7 2–3 388 398

    Article  Google Scholar 

  37. Auinger, A., Nedbal, D., Hochmeier, A., Holzinger, A.: User-centric usability evaluation for enterprise 2.0 platforms: a complementary multi-method approach. In: Proceedings of the International Conference on e-Business, pp. 1–6. IEEE (2011)

  38. Sivaji, A., Downe, A.G., Mazlan, M.F., Soo, S.T., Abdullah, A.: Importance of incorporating fundamental usability with social & trust elements for e-commerce website. In: 2011 International Conference on Business, Engineering and Industrial Applications, pp. 221–226. IEEE (2011)

  39. Nielsen, J.: 10 Usability heuristics for user interface design. https://www.nngroup.com/articles/ten-usability-heuristics (1994). Accessed 10 May 2020

  40. Goh, K.N., Chen, Y.Y., Lai, F.W., Daud, S.C., Sivaji, A., Soo, S.T.: A comparison of usability testing methods for an e-commerce website: a case study on a Malaysia online gift shop. In: 10th International Conference on Information Technology: New Generations, pp. 143–150. IEEE (2013)

  41. Sivaji, A., Abdullah, M.R., Downe, A.G., Ahmad, W.F.W.: Hybrid usability methodology: integrating heuristic evaluation with laboratory testing across the software development lifecycle. In: 10th International Conference on Information Technology: New Generations, pp. 375–383. IEEE (2013)

  42. Pipan, M., Blažič, B.J.: Systematic usability and user experience evaluation as a basis for the redesign of an e-learning platform. In: The International Conference on E-Learning in the Workplace, New York, USA. www.icelw.org (2012)

  43. H Cho PY Yen D Dowding JA Merrill R Schnall 2018 A multi-level usability evaluation of mobile health applications: a case study. J. Biomed. Inform. 86 79 89

    Article  Google Scholar 

  44. Lehong, S., Van Biljon, J., Sanders, I.: Open-distance electronic learning environments: supervisors' views on usability. In: 2019 Conference on Information Communications Technology and Society (ICTAS), pp. 1–7. IEEE (2019)

  45. A Groth D Haslwanter 2016 Efficiency, effectiveness, and satisfaction of responsive mobile tourism websites: a mobile usability study. Inf. Technol. Tour. 16 2 201 228

    Article  Google Scholar 

  46. L Deng MS Poole 2010 Affect in web interfaces: a study of the impacts of web page visual complexity and order. MIS Q. 34 4 711 730

    Article  Google Scholar 

  47. J Sweller 1988 Cognitive load during problem solving: effects on learning. Cogn. Sci. 12 2 257 285

    Article  Google Scholar 

  48. T Erenler H Hale 2018 Heuristic evaluation of e-learning. Int. J. Organ. Leadersh. 7 195 210

    Article  Google Scholar 

  49. Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256 (1990)

  50. C Kadilar H Cingi 2003 Ratio estimators in stratified random sampling. Biometr. J. J. Math. Methods Biosci. 45 2 218 225

    MathSciNet  MATH  Google Scholar 

  51. Arain, A.A., Hussain, Z., Rizvi, W.H., Vighio, M.S.: Evaluating usability of M-learning application in the context of higher education institute. In: International Conference on Learning and Collaboration Technologies, pp. 259–268. Springer, Cham (2016)

  52. T Mahatody M Sagar C Kolski 2010 State of the art on the cognitive walkthrough method, its variants and evolutions. Int. J. Hum. Comput. Interact. 26 8 741 785

    Article  Google Scholar 

  53. HB Santoso M Schrepp R Isal AY Utomo B Priyogi 2016 Measuring user experience of the student-centered e-learning environment. J. Educ. Online 13 1 58 79

    Google Scholar 

  54. Schrepp, M., Hinderks, A., Thomaschewski, J.: Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In: International Conference of Design, User Experience, and Usability, pp. 383–392. Springer, Cham (2014)

  55. Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Symposium of the Austrian HCI and Usability Engineering Group, pp. 63–76. Springer, Berlin (2008)

  56. M Rauschenberger M Schrepp MP Cota S Olschner J Thomaschewski 2013 Efficient measurement of the user experience of interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language version. Int. J. Artif. Intell. Interact. Multimed. 2 1 39 45

    Google Scholar 

  57. R Bidin A Kamaruddin NC Pa 2019 Analysis and development of customer billing telephony system. Baghdad Sci. J. 16 2-SI 428 435

    Google Scholar 

  58. V Davidavičienė J Raudeliūnienė R Viršilaitė 2019 User experience evaluation and creativity stimulation with augmented reality mobile applications. Creativity Stud. 12 1 34 48

    Article  Google Scholar 

  59. Devy, N.P.I.R., Wibirama, S., Santosa, P.I.: Evaluating user experience of english learning interface using user experience questionnaire and system usability scale. In: 1st International Conference on Informatics and Computational Sciences (ICICoS), pp. 101–106. IEEE (2017)

  60. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J.: Eye tracking: A comprehensive guide to methods and measures. OUP Oxford. (2011)

  61. Rizvi, W.H.: Brand visual eclipse (BVE): when the brand fixation spent is minimal in relation to the celebrity. In: Information Systems and Neuroscience, pp. 295–303. Springer, Cham (2020)

  62. LJ Cronbach 1951 Coefficient alpha and the internal structure of tests. Psychometrika 16 3 297 334

    Article  Google Scholar 

  63. M Schrepp A Hinderks J Thomaschewski 2017 Construction of a benchmark for the user experience questionnaire (UEQ). IJIMAI 4 4 40 44

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Quaid-E-Awam University of Engineering, Science and Technology, Nawabshah, Pakistan

    Baqar Ali Zardari, Zahid Hussain, Aijaz Ahmed Arain & Muhammad Saleem Vighio

  2. Institute of Business Administration, Karachi, Pakistan

    Wajid H. Rizvi

Authors
  1. Baqar Ali Zardari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zahid Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aijaz Ahmed Arain
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wajid H. Rizvi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Muhammad Saleem Vighio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zahid Hussain.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical approval

The research study has been approved by the university’s Advanced Studies and Research Board. The research has been conducted according to ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zardari, B.A., Hussain, Z., Arain, A.A. et al. QUEST e-learning portal: applying heuristic evaluation, usability testing and eye tracking. Univ Access Inf Soc 20, 531–543 (2021). https://doi.org/10.1007/s10209-020-00774-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-020-00774-z

Keywords

Search

Navigation