Skip to main content
SpringerLink
Log in

eModeration: The Validation of a User Experience Evaluation Framework

  • Conference paper
  • First Online:
Innovative Technologies and Learning (ICITL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11003))

Included in the following conference series:

Abstract

The effective, efficient and secure moderation of examination scripts can become challenging when the moderators are geographically dispersed. Despite innovative technological developments and the potential benefits, the use of eModeration (online moderation of examination scripts) in the South African Higher Education Institutions (HEIs) context is limited. Various factors contribute to the adoption of eModeration in HEIs, ranging from human factors to technical issues and organisational resistance to change. The focus of this study is on the human factors involved in eModeration (specifically the user experience) and the research is guided by the following question: What are the most important constructs in evaluating a user experience evaluation framework for eModeration within the context of Higher Education Institutions in South Africa? The research uses a design science research methodology, which includes the design, development and testing of a User Experience Evaluation Framework for eModeration, and the focus of this paper is on the validation phase. The data generation methods include interviews with management from two South African HEIs. The contribution of this paper is to propose a validated framework for evaluating eModeration in higher education and demonstrate how that can be used by managers in HEIs to guide the selection and evaluation of eModeration systems towards facilitating more effective learning environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Guàrdia, L., Crisp, G., Alsina, I.: Trends and challenges of e-assessment to enhance student learning in higher education. In: Innovative Practices for Higher Education Assessment and Measurement, pp. 295–415. IGI Global Hershey, PA, USA (2017)

    Google Scholar 

  2. Van Staden, C.J.: IT moderation going green! In: Proceedings of SAICSIT 2010 Conference, pp. 1–3. UNISA Production Printers, Bela Bela, Limpopo, South Africa (2010)

    Google Scholar 

  3. Bridge, P., Appleyard, R.: A comparison of electronic and paper based assignment submission and feedback. Br. J. Educ. Technol. 39(4), 644–650 (2008)

    Article  Google Scholar 

  4. Salmon, G.: E-tivities: The Key to Active Online Learning. Taylor & Francis Group, New York (2013)

    Google Scholar 

  5. Greatorex, J.: Moderated e-portfolio evaluation. Evaluation and Validation Assessment Directorate, UCLES (2004)

    Google Scholar 

  6. Van Staden, C.J., Van Biljon, J.A., Kroeze, J.H.: eModeration: towards a user experience evaluation framework. In: SAICSIT 2015 Proceedings of the 2015 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists. ICPS ACM, Article 39, Stellenbosch, Western Cape, South Africa (2015)

    Google Scholar 

  7. Van Staden, C.J., Van Biljon, J.A., Kroeze, J.H.: Using a user experience evaluation framework for eModeration. In: Olugbara, O., Millham, R., Heukleman, D. (eds.) Proceedings of IEEE ICTAS 2017, pp. 15–21, Durban, South Africa (2017)

    Google Scholar 

  8. Morgan, A.: eModeration: contextualising online learning in undergraduate nurse education. Asian J. Nurs. 11(1), 48–53 (2008)

    Google Scholar 

  9. Wichmann, A., Giemza, A., Hoppe, U., Collide-Group: Effects of awareness support on moderating multiple parallel E-discussions. Computer Supported Collaborative Learning Practices CSCL2009. ISLS, pp. 646–650 (2009)

    Google Scholar 

  10. Adie, L.: An investigation into online moderation. Assess. Matters 3, 5–27 (2011)

    Google Scholar 

  11. Adie, L., Lloyd, M., Beutel, D.: Identifying discourses of moderation in higher education. J. Assess. Eval. High. Educ. 38(8), 968–977 (2013)

    Article  Google Scholar 

  12. Adie, L.: Towards a theoretical framework for online professional discussions. J. Learn. Des. JLD 7(3), 54–66 (2014)

    Google Scholar 

  13. Grainger, P., Adie, L., Weir, K.: Quality assurance of assessment and moderation discourses involving sessional staff. J. Assess. Eval. High. Educ. 41(4), 548–559 (2016)

    Article  Google Scholar 

  14. MGI. Assessment Policy, Midrand (2010)

    Google Scholar 

  15. Van Staden, C.J., Van Biljon, J.A., Kroeze, J.H.: Adopting eModeration: understanding the user experience in the organisation. In: 8th European Conference on IS Management and Evaluation, ECIME 2014 Conference, pp. 356–365. Ghent, Belgium (2014)

    Google Scholar 

  16. ISODIS9241-210. Ergonomics of human systems interaction – Part 210: Human-centred design for interactive systems (formerly known as 13407). ISO (2010)

    Google Scholar 

  17. Hassenzahl, M., Tractinsky, N.: User experience – a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)

    Article  Google Scholar 

  18. Roto, V.: Web browsing on mobile phones – characteristics of user experience. Doctoral dissertation. University of Technology, Helsinki (2006)

    Google Scholar 

  19. Wimmer, B., Wöckl, B., Leitner, M., Tscheligi, M.: Measuring the dynamics of User Experience in short interaction sequences. In: Proceedings of Nordic Computer-Human Interaction CHI 2010, pp. 825–828. ACM, Reykjavik (2010)

    Google Scholar 

  20. Law, E.: The measurability and predictability of User Experience. In: Conference Proceedings of EICS 2011, pp. 1–10, Pisa, ACM, New York (2011)

    Google Scholar 

  21. Van Schaik, P., Van Aranyi, G.: Research white paper model-based user experience. Teesside University (2014)

    Google Scholar 

  22. Van Der Peijl, J., Klein, J., Grass, C., Freudenthal, A.: Design for risk control: the role of usability engineering in the management of use-related risks. J. Biomed. Inform. 45, 795–812 (2012)

    Article  Google Scholar 

  23. Preece, J., Sharp, H., Roger, Y.: Interaction Design. 3rd edn. Wiley, Chichester (2011)

    Google Scholar 

  24. McCarthy, J., Wright, J.: Technology as Experience. MIT Press, Massachusetts (2007)

    Google Scholar 

  25. Väätäjä, H., Koponen, T., Roto, V.: Developing practical tools for user experience evaluation: a case from Mobile News Journalism. In: Proceedings of ECCE 2009 European Conference on Cognitive Ergonomics beyond the Product – Understanding Activity and User Experience in Ubiquitous Environments, Article 23. ACM, Helsinki (2009)

    Google Scholar 

  26. Schulze, K., Krömker, H.A.: A framework to measure User Experience of interactive online products. In: Proceedings of MB 2010 Conference, Article 14. ACM, New York (2010)

    Google Scholar 

  27. Ouma, S.: M-health user experience framework for the public healthcare sector. Ph.D. dissertation, Nelson Mandela Metropole University NMMU (2013)

    Google Scholar 

  28. Hassenzahl, M.: User experience (UX): towards an experiential perspective on product quality. In: Proceedings of the 20th International Conference of the Association Francophone d’Interaction Homme, pp. 11–15. Machine Metz, Proceedings of IHM (2008)

    Google Scholar 

  29. Väänänen-Vainio-Mattila, K., Wäljas, M.: Development of evaluation heuristics for web service User Experience. In: Proceedings of Computer-Human Interaction CHI EA 2009, 27th International Conference on Extended Abstracts on Human Factors in Computing Systems, pp. 3679–3684. ACM, Boston (2009)

    Google Scholar 

  30. Hevner, A.R., March, S.T., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75–105 (2004)

    Article  Google Scholar 

  31. Aier, S., Fischer, C.: Criteria for progress of information systems design theories. Inf. Syst. E-Bus. Manage. 9(1), 133–172 (2011)

    Article  Google Scholar 

  32. Sonnenberg, C., Vom Brocke, J.: Evaluations in the science of the artificial – reconsidering the build-evaluate pattern in Design Science Research. Des. Sci. Res. Inf. Syst. Adv. Theory Pract. 7286, 381–397 (2012)

    Google Scholar 

  33. Braun, V., Clarke, V.: Using thematical analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006)

    Article  Google Scholar 

  34. Gregor, S., Jones, D.: The anatomy of a design theory. J. Assoc. Inf. Syst. 8(5), 312–335 (2007)

    Google Scholar 

  35. Cresswell, J.W.: A concise Introduction to Mixed Methods Research. SAGE Publications, London (2015)

    Google Scholar 

Download references

Acknowledgements

The work is based on the research supported by the South African Research Chairs Initiative of the Department of Science and Technology and National Research Foundation of South Africa (Grant No. 89564). The University of South Africa (UNISA) is acknowledged for granting a period of research and development leave to the third author during which time the paper was completed. Any opinion, finding and conclusion or recommendation expressed in this material is that of the authors, and neither the NRF nor UNISA accepts any liability in this regard.

Author information

Authors and Affiliations

  1. School of Computing, University of South Africa, Science Campus, 28 Pioneer Avenue, Florida Park, Roodepoort, 1709, South Africa

    Corne J. van Staden, Judy A. van Biljon & J. H. Kroeze

Authors
  1. Corne J. van Staden
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Judy A. van Biljon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. H. Kroeze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Corne J. van Staden .

Editor information

Editors and Affiliations

  1. National Yunlin University of Science and Technology, Yunlin, Taiwan

    Ting-Ting Wu

  2. Kun Shan University, Tainan City, Taiwan

    Yueh-Min Huang

  3. Nanjing Normal University, Nanjing, China

    Rustam Shadiev

  4. University of North Texas, Denton, TX, USA

    Lin Lin

  5. University of Ljubljana, Ljubljana, Slovenia

    Andreja Istenič Starčič

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

van Staden, C.J., van Biljon, J.A., Kroeze, J.H. (2018). eModeration: The Validation of a User Experience Evaluation Framework. In: Wu, TT., Huang, YM., Shadiev, R., Lin, L., Starčič, A. (eds) Innovative Technologies and Learning. ICITL 2018. Lecture Notes in Computer Science(), vol 11003. Springer, Cham. https://doi.org/10.1007/978-3-319-99737-7_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-99737-7_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-99736-0

  • Online ISBN: 978-3-319-99737-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation