Abstract
This study proposes the use of Think Aloud and Focus Group methods for evaluating the usability of the Intelligent Police Patrolling System (I-Pat). The study was conducted with twenty-one students of computer engineering. The study included two evaluation methods. First, the application of Think Aloud using audio recordings, image capturing and questionnaires. Second, the application of a Focus Group for brainstorming. The total number of the usability problems identified was fifteen. Comprehensiveness (46%) and layout (43%) problems were the most frequently found. The study showed that the problems encountered were due to the lack of understanding of the system’s functions, so it is recommended increasing the users’ levels of knowledge about the system. The application of these methods caused the students to find a greater number of errors than when applying a single method, allowing them to generate a report with usability improvements according to the reported errors.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
ISO Standards: 9126 standard (2018). https://www.cse.unsw.edu.au/cs3710/PMmaterials/
ISTQB: International software testing qualifications board, advanced level syllabus usability tester (2016). https://www.istqb.org/
Brhel, M., Meth, H., Maedche, A., Werder, K.: Exploring principles of user-centered agile software development: a literature review. Inf. Softw. Technol. 61, 163–181 (2015)
Pilco, H., Sanchez-Gordon, S., Calle-Jimenez, T., Rybarczyk, Y., Jadán, J., Villarreal, S., Esparza, W., Acosta-Vargas, P., Guevara, C., Nunes, I.L.: Analysis and improvement of the usability of a tele-rehabilitation platform for hip surgery patients. In: International Conference on Applied Human Factors and Ergonomics, pp. 197–209. Springer (2018)
Maniega-Legarda, D.: Aplicación de criterios de usabilidad en sitios web: consejos y pautas para una correcta interpretación. Observatorio TIC: REBIUN Red de Bibliotecas Universitarias (2006)
User Experience Professionals’ Association: The usability body of knowledge (2018). https://www.usabilitybok.org/think-aloud-testing
Guan, Z., Lee, S., Cuddihy, E., Ramey, J.: The validity of the stimulated retrospective think-aloud method as measured by eye tracking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1253–1262. ACM (2006)
Olmsted-Hawala, E.L., Murphy, E.D., Hawala, S., Ashenfelter, K.T.: Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data dissemination web sites for usability. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2381–2390. ACM (2010)
McDonald, S., McGarry, K., Willis, L.M.: Thinking-aloud about web navigation: the relationship between think-aloud instructions, task difficulty and performance. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, pp. 2037–2041. SAGE Publications, Los Angeles (2013)
Hertzum, M., Hansen, K.D., Andersen, H.H.: Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload? Behav. Inf. Technol. 28(2), 165–181 (2009)
Barnum, C.M.: Usability testing essentials: ready, set… test!. Elsevier, Amsterdam (2010)
Van Den Haak, M.J., De Jong, M.D., Schellens, P.J.: Constructive interaction: an analysis of verbal interaction in a usability setting. IEEE Trans. Prof. Commun. 49(4), 311–324 (2006)
Makri, S., Blandford, A., Cox, A.L.: This is what im doing and why: methodological reflections on a naturalistic think-aloud study of interactive information behaviour. Inf. Process. Manage. 47(3), 336–348 (2011)
Nørgaard, M., Hornbæk, K.: What do usability evaluators do in practice? An explorative study of think-aloud testing. In: Proceedings of the 6th Conference on Designing Interactive Systems, pp. 209–218. ACM (2006)
Molich, R., Ede, M.R., Kaasgaard, K., Karyukin, B.: Comparative usability evaluation. Behav. Inf. Technol. 23(1), 65–74 (2004)
Gulliksen, J., Boivie, I., Persson, J., Hektor, A., Herulf, L.: Making a difference: a survey of the usability profession in Sweden. In: Proceedings of the Third Nordic-Conference on Human-Computer Interaction, pp. 207–215. ACM (2004)
Nyumba, T.O., Wilson, K., Derrick, C.J., Mukherjee, N.: The use of focus group discussion methodology: Insights from two decades of application in conservation. Methods Ecol. Evol. 9(1), 20–32 (2018)
Plummer-D’Amato, P.: Focus group methodology part 2: considerations for analysis. Int. J. Ther. Rehabil. 15(3), 123–129 (2008)
Onwuegbuzie, A.J., Dickinson, W.B., Leech, N.L., Zoran, A.G.: A qualitative framework for collecting and analyzing data in focus group research. Int. J. Qual. Methods 8(3), 1–21 (2009)
Kontio, J., Lehtola, L., Bragge, J.: Using the focus group method in software engineering: obtaining practitioner and user experiences. In: Proceedings. 2004 International Symposium on Empirical Software Engineering, ISESE 2004, pp. 271–280. IEEE (2004)
Sunikka, A.: Usability evaluation of the helsinki school of economics website. Master’s thesis (2004)
Nielsen, J.: The use and misuse of focus groups. IEEE Softw. 14(1), 94–95 (1997)
Obeso, M.E.A., Lovelle, J.M.C., Prieto, A.B.M.: Metodología de medición y evaluación de la usabilidad en sitios web educativos. Universidad de Oviedo (2005)
Guevara, C., Jadan, J., Zapata, C., Martínez, L., Pozo, J., Manjarres, E.: Model of dynamic routes for intelligent police patrolling. In: Multidisciplinary Digital Publishing Institute Proceedings, vol. 2, p. 1214 (2018)
Habibi, M.R.M., Khajouei, R., Eslami, S., Jangi, M., Ghalibaf, A.K., Zangouei, S., et al.: Usability testing of bed information management system: a think-aloud method. J. Adv. Pharm. Technol. Res. 9(4), 153 (2018)
Harper, B.D., Norman, K.L.: Improving user satisfaction: the questionnaire for user interaction satisfaction version 5.5. In: Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, pp. 224–228 (1993)
Arh, T., Blazic, B.J.: A case study of usability testing-the sumi evaluation approach of the educanext portal. WSEAS Trans. Inf. Sci. Appl. 5(2), 175–181 (2008)
Brooke, J., et al.: Sus-a quick and dirty usability scale. Usabil. Eval. Ind. 189(194), 4–7 (1996)
Nielsen, J.: Severity ratings for usability problems. Pap. Essays 54, 1–2 (1995)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Calle-Jimenez, T. et al. (2020). Improving Usability with Think Aloud and Focus Group Methods. A Case Study: An Intelligent Police Patrolling System (I-Pat). In: Nunes, I. (eds) Advances in Human Factors and Systems Interaction. AHFE 2019. Advances in Intelligent Systems and Computing, vol 959. Springer, Cham. https://doi.org/10.1007/978-3-030-20040-4_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-20040-4_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20039-8
Online ISBN: 978-3-030-20040-4
eBook Packages: EngineeringEngineering (R0)