Georgios Melissourgos
Christos Katsanos
ABSTRACT
Structural design of shared information environments is rather challenging. Card sorting constitutes a valuable method towards this end. However, there is a lack of free software tools for conducting card sorts, especially from a distance, and analysing the collected data. This paper presents CardSorter, an open source, web-based, card sorting tool. Currently, CardSorter offers functionality for running open card sorts online and analysing the collected data in real time. In a user testing study involving 108 participants, CardSorter was found to be from Good to Excellent in terms of perceived usability. A heuristic evaluation of the tool involving 25 teams of novice evaluators identified issues for further improvement.
- Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what individual SUS Scores mean: adding an adjective rating scale. J. Usability Stud.4 (3), 114–123.Google Scholar
Digital Library
- Stefano Bussolon, Barbara Russi, and Fabio Del Missier. 2006. Online card sorting: as good as the paper version. In Proceedings of the 13th European conference on Cognitive ergonomics: trust and control in complex socio-technical systems (ECCE ’06), Association for Computing Machinery, New York, NY, USA, 113–114.Google ScholarDigital Library
- Miranda G. Capra. 2005. Factor analysis of card sort data: an alternative to hierarchical cluster analysis. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, HFES, Santa Monica, CA, 691–695.Google ScholarCross Ref
- Barbara S. Chaparro, Veronica D. Hinkle, and Shannon K. Riley. 2008. The usability of computerized card sorting: a comparison of three applications by researchers and end users. J. Usability Stud.4 (1), 31–48.Google ScholarDigital Library
- Christos Katsanos, Nikolaos Tselios, and Nikolaos Avouris. 2008. AutoCardSorter: designing the information architecture of a web site using latent semantic analysis. In Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, CHI 2008, ACM, Florence, Italy, 875–878.Google ScholarDigital Library
- Christos Katsanos, Nikolaos Tselios, and Nikolaos Avouris. 2008. Automated semantic elaboration of web site information architecture. Interact Comput.20 (6), 535–544.Google ScholarDigital Library
- Christos Katsanos, Nikolaos Tselios, Nikolaos Avouris, Stavros Demetriadis, Ioannis Stamelos, and Lefteris Angelis. 2019. Cross-study reliability of the open card sorting method. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA ’19), ACM, New York, NY, USA, LBW2718:1–LBW2718:6.Google ScholarDigital Library
- Christos Katsanos, Nikolaos Tselios, and Michalis Xenos. 2012. Perceived usability evaluation of learning management systems: a first step towards standardization of the System Usability Scale in Greek. In2012 16th Panhellenic Conference on Informatics, 302–307.Google ScholarDigital Library
- Alexandros Liapis, Nikos Karousos, Christos Katsanos, and Michalis Xenos. 2014. Evaluating user's emotional experience in HCI: the PhysiOBS approach. In Human-Computer Interaction. Advanced Interaction Modalities and Techniques (Lecture Notes in Computer Science), Springer Verlag, 758–767.Google Scholar
- Alexandros Liapis, Christos Katsanos, Nikos Karousos, Michalis Xenos, and Theofanis Orphanoudakis. 2020. User experience evaluation: a validation study of a tool-based approach for automatic stress detection using physiological signals. Int. J. Hum. Comput. Interact.Google Scholar
- Alexandros Liapis, Christos Katsanos, Dimitris G. Sotiropoulos, Nikos Karousos, and Michalis Xenos. 2017. Stress in interactive applications: analysis of the valence-arousal space based on physiological signals and self-reported data. Multimed. Tools Appl.76 (4), 5051–5071.Google ScholarDigital Library
- Jacob Nielsen. 1995. How to conduct a heuristic evaluation. Retrieved from http://www.useit.com/papers/heuristic/heuristic_evaluation.htmlGoogle Scholar
- Jakob Nielsen and Thomas K. Landauer. 1993. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT ’93 and CHI ’93 conference on Human factors in computing systems, ACM, Amsterdam, The Netherlands, 206–213.Google Scholar
- Konstantina Orfanou, Nikolaos Tselios, and Christos Katsanos. 2015. Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale. The International Review of Research in Open and Distributed Learning 16 (2), 227–246.Google ScholarCross Ref
- Sione Paea and Ross Baird. 2018. Information Architecture (IA): Using multidimensional scaling (MDS) and K-Means clustering algorithm for analysis of card sorting Data. J. Usability Stud.13 (3), 138–157.Google ScholarDigital Library
- Celeste Paul. 2008. A modified delphi approach to a new card sorting methodology. J. Usability Stud.4 (1), 7–30.Google ScholarDigital Library
- Celeste Paul. 2014. Analyzing card-sorting data using graph visualization. J. Usability Stud.9 (3), 87–104.Google ScholarDigital Library
- Eric Raymond. 1999. The cathedral and the bazaar. Know. Techn. Pol.12 (3), 23–49.Google ScholarCross Ref
- Carol Righi, Janice James, Michael Beasley, Donald L. Day, Jean E. Fox, Jennifer Gieber, Chris Howe, and Laconya Ruby. 2013. Card sort analysis best practices. J. Usability Stud.8 (3), 69–89.Google ScholarDigital Library
- Louis Rosenfeld, Peter Morville, and Jorge Arango. 2015. Information architecture: For the Web and beyond (4 edition ed.). O'Reilly Media, Sebastopol, CA.Google Scholar
- Donna Spencer. 2009. Card sorting: designing usable categories. Rosenfeld Media, Brooklyn, N.Y.Google Scholar
Recommendations
Test-Retest Reliability of the Open Card Sorting Method
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsOpen card sorting is a popular method for designing information architectures. However, there is a lack of empirical evidence on the validity and reliability of the method. This paper explores the test-retest reliability of open card sorting. To this end,...
Validity of the Open Card Sorting Method for Producing Website Information Structures
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsWhen it comes to creating website information structures, open card sorting is the main approach used. However, scientific research for the method's validity is lacking. This paper explores the validity of open card sorting for website structural ...
Effect of Sense of Direction on Open Card Sorts for Websites
CHI Greece 2021: CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI ChapterCard sorting is one widely used method to design website information architectures based on an understanding of their users’ mental models. This paper studies the effect of participants’ sense of direction on card sorting results. To this end, a two-...
Comments