Sabda Norman Hayat
ABSTRACT
Usability evaluation is an important element which has used to add on insights related to the usability problem of an application. This study aims to identify application's usability problems and to compare the effectiveness of four evaluation methods which has used on an academic portal mobile application: usability testing, interviews, surveys, and heuristic evaluation. The data has collected from users who are in the student category and expert evaluator. The number detail of respondents used are usability testing (N = 10), interviews (N = 10), surveys (N = 110), and heuristic evaluation (N = 3). The four methods identified a total of 44 usability problems: 45% using heuristic evaluation, 24% using surveys, 17% using interviews and 14% using usability testing, resulting into a few similar findings. Then, The problems are categorized using Usability Taxonomy Problem (UPT) which has divided into 5 categories with details of 17 categories of visualness, 6 categories of language, 3 categories of manipulation, 11 categories of task-mapping and 7 others including the category of task-facilitation. The results of this study are capable to prove that the four methods are complementary, each method provides a unique insight to improve the usability of the application user interface. Both researchers recommend using a multi-method approach when evaluating the usability of an application due to it could provide a more comprehensive representation of usability issues.
- TIK UB, Gapura Universitas Brawijaya, 2017. [Online]. Available: https://gapura.ub.ac.id/. [Accessed Feb. 10, 2020].Google Scholar
- S. S. Nathan, A. Hussain, and N. L. Hashim, "Objective Measurements Analysis for Usability Evaluation of Mobile Applications for Deaf People," Journal of Telecommunication, Electronic and Computer Engineering, vol. 9, no. 2--12, pp. 25--28, 2017. [Online]. Available: https://journal.utem.edu.my/index.php/jtec/article/view/2766/1824/. [Accessed Feb. 6, 2020].Google Scholar
- T. D. Susanto, A. I. Prasetyo, and H. M. Astuti, "Web Usability Evaluation on BloobIS Website by Using Hallway Usability Testing Method and ISO 9241:11," Journal of Physics: Conference Series, vol. 974, 2018. [Online]. Available: https://www.researchgate.net/publication/323922615_Web_usability_evaluation_on_BloobIS_website_by_using_hallway_usability_testing_method_and_ISO_924111/. [Accessed Feb. 4, 2020].Google ScholarCross Ref
- M. A. Jabar, U. A. Usman, and A. Awal, "Assessing the Usability Of University Websites From Users' Perspective," Australian Journal of Basic and Applied Sciences, vol. 7, no. 10, pp. 98--111, 2013. [Online]. Available: http://ajbasweb.com/old/ajbas/2013/August/98-111.pdf/. [Accessed Feb. 4, 2020].Google Scholar
- P. Mathur, and S.V. Chande, "Usability Testing Methods for Mobile Learning Applications," International Journal of Computer Science and Mobile Applications, vol. 5, issues. 10, pp. 1--9, October 2017. [Online]. Available: http://ijcsma.com/publications/october2017/V5I1001.pdf/. [Accessed Feb. 10, 2020].Google Scholar
- F. Paz, and J.A Pow-sang, "A Systematic Mapping Review of Usability Evaluation Methods for Software Development Process," International Journal of Software Engineering and Its Applications, vol. 10, no. 1, pp. 165--178, 2016. [Online]. Available: https://www.researchgate.net/publication/297764945_A_systematic_mapping_review_of_usability_evaluation_methods_for_software_development_process/ [Accessed Feb. 10, 2020].Google ScholarCross Ref
- Nielsen, J, Usability 101: Introduction to Usability, 2012. [Online]. Available: https://www.nngroup.com/articles/usability-101-introduction-to-usability/. [Accessed Feb. 6, 2020].Google Scholar
- L. Kantner, D.H. Sova, and S. Rosenbaum, "Alternative Methods for Field Usability Research," SIGDOC 2003 Proceedings, 2003. [Online]. Available: https://www.researchgate.net/publication/220961501_Alternative_methods_for_field_usability_research [Accessed Feb. 10, 2020]. Google ScholarDigital Library
- J. Rubin, and D. Chisnell, Handbook of usability testing: How to Plan, Design, and Conduct Effective Test, 2nd ed. Reading, Indianapolis: Wiley Publishing, Inc. [E-book] Available: https://books.google.co.id/. [Accessed Feb. 10, 2020]. Google ScholarDigital Library
- M. Allen et al., "Heuristic Evaluation of Paper-based Web Pages: A simplified Inspection Usability Methodology," Journal of Biomedical Informatics, vol. 39, pp. 412--423, November 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1532046405001085 [Accessed Feb. 6, 2020]. Google ScholarDigital Library
- J. Nielsen, and R, Molich, "Heuristic Evaluation of User Interfaces," Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249--256, April 1990. [Online]. Available: https://dl.acm.org/doi/10.1145/97243.97281 [Accessed Feb. 6, 2020]. Google ScholarDigital Library
- M.F. Walji et al., "Are Three Methods Better than One? A Comparative Assessment of Usability Evaluation Methods in an EHR," Journal of Biomedical Informatics, vol. 83, pp. 361--367, January 2014. [Online]. Available: https://www.researchgate.net/publication/260030008_Are_Three_Methods_Better_Than_One_A_Comparative_Assessment_of_Usability_Evaluation_Methods_in_an_EHR [Accessed Feb. 10, 2020].Google Scholar
- M. Georgsson, and N. Staggers, "An Evaluation of Patients' Experienced Usability of A Diabetes MHealth System Using A Multi-method Approach," Journal of Biomedical Informatics, vol. 59, pp. 115--129, November 2015. [Online]. Available:https://www.sciencedirect.com/science/article/pii/S1532046415002762 [Accessed Feb. 10, 2020]. Google ScholarDigital Library
- S. Ssemugabi, and M.R Villiers, "Effectiveness of Heuristic Evaluation in Usability Evaluation of Elearning Applications in Higher Education," Research Article-SACJ, no. 45, Juli 2010. [Online]. Available: https://www.researchgate.net/publication/290139198_Effectiveness_of_heuristic_evaluation_in_usability_evaluation_of_elearning_applications_in_higher_educ/ [Accessed Feb. 6, 2020].Google Scholar
- S. L. Keenan, "Product Usability and Process Improvement Based on Usability Problem Classification," Ph.D. dissertation, Virginia Polytechnic Institute and State University., Virginia, 1996. [Online]. https://vtechworks.lib.vt.edu/bitstream/handle/10919/39100/LD5655.V856_1996.K446.pdf?sequence=1&isAllowed=y [Accessed May. 10, 2020]. Google ScholarDigital Library
- Nielsen, J, How Many Test Users in a Usability Study? World Leaders in Research-Based User Experience, 2012. [Online]. Available: https://www.nngroup.com/articles/how-many-test-users/ [Access Feb. 6, 2020]Google Scholar
- C.G. Sevilla et al., Research Methods, Revised ed. Reading, Manila: Rex Book Store, 1992.Google Scholar
- Sugiyono, Metodologi Penelitian Kuantitatif Kualitatif dan R&D, Bandung: Alfabeta, 2011.Google Scholar
- A. Lund, "Measuring Usability with the USE Questionnaire," Usability Interface, 8(2), 3-6, January 2001. [Online]. Available: https://www.researchgate.net/publication/230786746_Measuring_Usability_with_the_USE_Questionnaire/ [Accessed Feb. 6, 2020].Google Scholar
- Nielsen, J, How to Conduct a Heuristic Evaluation, 1994. [Online]. Available: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/ [Accessed Feb. 6, 2020].Google Scholar
- J. A. Monroy, "Study on Heuristic Usabilty Evaluation for Mobile Application," Master thesis, Universidad Politecnica De Madrid., Madrid, 2015. [Online]. http://oa.upm.es/37202/1/EMSE-2015-08-Jorge_Avil%C3%A9s_Monroy.pdf [Accessed May. 8, 2020].Google Scholar
Index Terms
- A comparative analysis of usability evaluation methods of academic mobile application: are four methods better?
Recommendations
A comparative study of two usability evaluation methods using a web-based e-learning application
SAICSIT '07: Proceedings of the 2007 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countriesUsability evaluation of e-learning applications is a maturing area, which addresses interfaces, usability and interaction from human-computer interaction (HCI) and pedagogy and learning from education. The selection of usability evaluation methods (UEMs)...
An evaluation of patients' experienced usability of a diabetes mHealth system using a multi-method approach
Display Omitted Multi-method, structured and standardized approaches are needed in mHealth usability research.A novel application of the FA method and UPT was used to structure, code and classify problems.Clearly defined, classified usability problems ...
E-Government Usability Evaluation: Insights from A Systematic Literature Review
ICSIM '19: Proceedings of the 2nd International Conference on Software Engineering and Information ManagementE-Government aims to deliver benefits to government and citizens by improving transparency, efficiency, trust, and citizen participation. However, e-government initiatives face several barriers. One of them is poor usability. To advance quality of ...
Comments