Skip to main content
SpringerLink
Log in

Effect of Self-efficacy on Open Card Sorts for Websites

  • Conference paper
  • First Online:
Human Interface and the Management of Information: Visual and Information Design (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13305))

Included in the following conference series:

Abstract

Card sorting is a popular way for creating website information architectures based on users’ mental models. This paper explores the effect of participants’ self-efficacy on card sorting results. A two-phase study was carried out. The first phase involved 40 participants rating their self-efficacy on a standardized scale, followed by an open card sort experiment. The median self-efficacy score was used to split the open card sort data into two groups: one for low and one for high participants’ self-efficacy. These two datasets were analyzed following state-of-the-art techniques for open card sort data analysis, which resulted in two information architectures for the eshop. In the second phase, two functional prototypes were first created for the eshop, one for each information architecture of the first phase. Subsequently, 30 participants interacted with both prototypes in a user testing study. This paper found that users interacting with the information architecture produced by open card sort participants with low self-efficacy made statistically significantly more correct first clicks, significantly less time to find content items, rated the tasks as significantly easier, and provided higher perceived usability ratings compared to when they interacted with the information architecture produced by users with high self-efficacy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Rosenfeld, L., Morville, P., Arango, J.: Information architecture: For the Web and beyond. O’Reilly Media, Sebastopol (2015)

    Google Scholar 

  2. Spencer, D.: Card sorting: designing usable categories. Rosenfeld Media, Brooklyn (2009)

    Google Scholar 

  3. Albert, W., Tullis, T.S.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann (2013)

    Google Scholar 

  4. Paul, C.: Analyzing card-sorting data using graph visualization. J. Usability Stud. 9, 87–104 (2014)

    Google Scholar 

  5. Wood, J., Wood, L.: Card sorting: Current practices and beyond. J. Usability Stud. 4, 1–6 (2008)

    Google Scholar 

  6. Adamides, G., Christou, G., Katsanos, C., Xenos, M., Hadzilacos, T.: Usability guidelines for the design of robot teleoperation: A taxonomy. IEEE Trans. Hum. Mach. Syst. 45, 256–262 (2015). https://doi.org/10.1109/THMS.2014.2371048

    Article  Google Scholar 

  7. Kappel, K., Tomitsch, M., Költringer, T., Grechenig, T.: Developing user interface guidelines for DVD menus. In: Extended Abstracts of the 2006 CHI Conference on Human Factors in Computing Systems, pp. 177–182. ACM, New York (2006). https://doi.org/10.1145/1125451.1125490

  8. Zaphiris, P., Ghiawadwala, M., Mughal, S.: Age-centered research-based web design guidelines. In: Extended Abstracts of the 2005 CHI Conference on Human Factors in Computing Systems, pp. 1897–1900. ACM, New York (2005). https://doi.org/10.1145/1056808.1057050

  9. Katsanos, C., Tselios, N., Avouris, N.: AutoCardSorter: Designing the information architecture of a web site using latent semantic analysis. In: Proceedings of the 2008 CHI Conference on Human Factors in Computing Systems, pp. 875–878. ACM, Florence (2008). https://doi.org/10.1145/1357054.1357192

  10. Katsanos, C., Tselios, N., Avouris, N.: Automated semantic elaboration of web site information architecture. Interact. Comput. 20, 535–544 (2008). https://doi.org/10.1016/j.intcom.2008.08.002

    Article  Google Scholar 

  11. Katsanos, C., Tselios, N., Goncalves, J., Juntunen, T., Kostakos, V.: Multipurpose public displays: Can automated grouping of applications and services enhance user experience? Int. J. Hum. Comput. Interact. 30, 237–249 (2014). https://doi.org/10.1080/10447318.2013.849547

    Article  Google Scholar 

  12. Cassidy, B., Antani, D.S., Read, J.C.C.: Using an open card sort with children to categorize games in a mobile phone application store. In: Proceedings of the 2013 CHI Conference on Human Factors in Computing Systems, pp. 2287–2290. ACM, New York (2013). https://doi.org/10.1145/2470654.2481315

  13. Seifi, H., Oppermann, M., Bullard, J., MacLean, K.E., Kuchenbecker, K.J.: Capturing experts’ mental models to organize a collection of haptic devices: affordances outweigh attributes. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, p. 13. ACM, New York (2020). https://doi.org/10.1145/3313831.3376395

  14. Dorn, B., Guzdial, M.: Learning on the job: Characterizing the programming knowledge and learning strategies of web designers. In: Proceedings of the 2010 CHI Conference on Human Factors in Computing Systems, Atlanta, pp. 703–712 (2010). https://doi.org/10.1145/1753326.1753430

  15. Jeong, R., Chiasson, S.: “Lime”, “open lock”, and “blocked”: children’s perception of colors, symbols, and words in cybersecurity warnings. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, p. 14. ACM, New York (2020). https://doi.org/10.1145/3313831.3376611

  16. Kelley, C., Lee, B., Wilcox, L.: Self-tracking for mental wellness: Understanding expert perspectives and student experiences. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 629–641. ACM, New York (2017). https://doi.org/10.1145/3025453.3025750

  17. Nielsen, J.: Card Sorting: How many users to test. http://www.useit.com/alertbox/20040719.html

  18. Tullis, T., Wood, L.: How many users are enough for a card-sorting study? In: Proceedings of the 2004 Conference on Usability Professionals Association (UPA), Minneapolis (2004)

    Google Scholar 

  19. Tullis, T., Wood, L.: How can you do a card-sorting study with LOTS of cards? In: Proceedings of the 2004 Conference on Usability Professionals Association (UPA), Minneapolis (2004)

    Google Scholar 

  20. Pampoukidou, S., Katsanos, C.: Test-retest reliability of the open card sorting method. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. Article330:1–Article330:7. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3411763.3451750

  21. Katsanos, C., Tselios, N., Avouris, N., Demetriadis, S., Stamelos, I., Angelis, L.: Cross-study reliability of the open card sorting method. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. LBW2718:1–LBW2718:6. ACM, New York (2019). https://doi.org/10.1145/3290607.3312999

  22. Harper, M.E., Jentsch, F., Van Duyne, L.R., Smith-Jentsch, K., Sanchez, A.D.: Computerized card sort training tool: Is it comparable to manual card sorting? In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 2049–2053. SAGE Publications Inc. (2002). https://doi.org/10.1177/154193120204602512

  23. Petrie, H., Power, C., Cairns, P., Seneler, C.: Using card sorts for understanding website information architectures: Technological, methodological and cultural issues. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6949, pp. 309–322. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23768-3_26

    Chapter  Google Scholar 

  24. Bussolon, S., Russi, B., Missier, F.D.: Online card sorting: As good as the paper version. In: Proceedings of the 13th European Conference on Cognitive Ergonomics: Trust and Control in Complex Socio-Technical Systems, pp. 113–114. Association for Computing Machinery, New York (2006). https://doi.org/10.1145/1274892.1274912

  25. Chaparro, B.S., Hinkle, V.D., Riley, S.K.: The usability of computerized card sorting: A comparison of three applications by researchers and end users. J. Usability Stud. 4, 31–48 (2008)

    Google Scholar 

  26. Melissourgos, G., Katsanos, C.: CardSorter: Towards an open source tool for online card sorts. In: Proceedings of the 24th Pan-Hellenic Conference on Informatics, pp. 77–81. ACM, New York (2020). https://doi.org/10.1145/3437120.3437279

  27. Nawaz, A.: A Comparison of card-sorting analysis methods. In: Proceedings of the 10th Asia Pacific Conference on Computer-Human Interaction, APCHI 2012, pp. 583–592. ACM Press (2012)

    Google Scholar 

  28. Righi, C., et al.: Card sort analysis best practices. J. Usability Stud. 8, 69–89 (2013)

    Google Scholar 

  29. Paea, S., Baird, R.: Information Architecture (IA): Using multidimensional scaling (MDS) and K-Means clustering algorithm for analysis of card sorting data. J. Usability Stud. 13, 138–157 (2018)

    Google Scholar 

  30. Capra, M.G.: Factor analysis of card sort data: An alternative to hierarchical cluster analysis. In: Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, pp. 691–695. HFES, Santa Monica (2005)

    Google Scholar 

  31. Zafeiriou, G., Katsanos, C., Liapis, A.: Effect of sense of direction on open card sorts for websites. In: Proceedings of the CHI Greece 2021: 1st International Conference of the ACM Greek SIGCHI Chapter, pp. 1–8. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3489410.3489416

  32. Jawahar, I.M., Elango, B.: The effect of attitudes, goal setting and self-efficacy on end user performance. J. End User Comput. 13, 40–45 (2001). https://doi.org/10.4018/joeuc.2001040104

    Article  Google Scholar 

  33. Igbaria, M., Iivari, J.: The effects of self-efficacy on computer usage. Omega 23, 587–605 (1995). https://doi.org/10.1016/0305-0483(95)00035-6

    Article  Google Scholar 

  34. Schwarzer, R., Jerusalem, M.: Generalized self-efficacy scale. In: Proceedings of the Measures in Health Psychology: A User’s Portfolio Causal and Control Beliefs, pp. 35–37. NFER-NELSON, Windsor (1995)

    Google Scholar 

  35. Scholz, U., Gutiérrez Doña, B., Sud, S., Schwarzer, R.: Is General self-efficacy a universal construct? Eur. J. Psychol. Assess. 18, 242–251 (2002). https://doi.org/10.1027//1015-5759.18.3.242

    Article  Google Scholar 

  36. Dong, J., Martin, S., Waldo, P.: A user input and analysis tool for information architecture. In: Extended Abstracts of the 2001 CHI Conference on Human Factors in Computing Systems, pp. 23–24. ACM, Seattle (2001). https://doi.org/10.1145/634067.634085

  37. Sauro, J., Dumas, J.S.: Comparison of three one-question, post-task usability questionnaires. In: Proceedings of the 2009 CHI Conference on Human Factors in Computing Systems, pp. 1599–1608. ACM, New York (2009). https://doi.org/10.1145/1518701.1518946

  38. Brooke, J.: SUS: A “quick and dirty” usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry. Taylor and Francis, London (1996)

    Google Scholar 

  39. Bangor, A., Kortum, P., Miller, J.: An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 24, 574–594 (2008). https://doi.org/10.1080/10447310802205776

    Article  Google Scholar 

  40. Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS Scores mean: Adding an adjective rating scale. J. Usability Stud. 4, 114–123 (2009)

    Google Scholar 

  41. Tullis, T., Stetson, J.: A comparison of questionnaires for assessing website usability. In: Proceedings of the 2004 Conference Usability Professionals Association (UPA), pp. 7–11 (2004)

    Google Scholar 

  42. Katsanos, C., Tselios, N., Xenos, M.: Perceived usability evaluation of learning management systems: A first step towards standardization of the system usability scale in greek. In: Proceedings of the 2012 16th Panhellenic Conference on Informatics, pp. 302–307 (2012). https://doi.org/10.1109/PCi.2012.38

  43. Orfanou, K., Tselios, N., Katsanos, C.: Perceived usability evaluation of learning management systems: Empirical evaluation of the system usability scale. Int. Rev. Res. Open Distrib. Learn. 16, 227–246 (2015). https://doi.org/10.19173/irrodl.v16i2.1955

  44. Field, A.P.: Discovering statistics using SPSS. SAGE, Los Angeles (2009)

    MATH  Google Scholar 

  45. Sauro, J.: 10 Things to Know About the Single Ease Question (SEQ), http://www.measuringu.com/blog/seq10.php.

  46. Cohen, J.: A power primer. Psychol. Bull. 112, 155–159 (1992)

    Article  Google Scholar 

  47. Pajares, F.: Current directions in self-efficacy research. Adv. Motiv. Achiev. 10, 1–49 (1997)

    Google Scholar 

  48. Ormrod, J.E.: Human Learning. Pearson, Upper Saddle River (2007)

    Google Scholar 

  49. Hegarty, M., Richardson, A.E., Montello, D.R., Lovelace, K., Subbiah, I.: Development of a self-report measure of environmental spatial ability. Intell. 30, 425–447 (2002). https://doi.org/10.1016/S0160-2896(02)00116-2

    Article  Google Scholar 

  50. Liapis, A., Katsanos, C., Xenos, M., Orphanoudakis, T.: Effect of personality traits on UX evaluation metrics: A study on usability issues, valence-arousal and skin conductance. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. LBW2721:1-LBW2721:6. ACM, New York (2019). https://doi.org/10.1145/3290607.3312995

  51. Liapis, A., Katsanos, C., Karousos, N., Xenos, M., Orphanoudakis, T.: User experience evaluation: A validation study of a tool-based approach for automatic stress detection using physiological signals. Int. J. Hum. Comput. Interact. 37, 470–483 (2021). https://doi.org/10.1080/10447318.2020.1825205

    Article  Google Scholar 

  52. Liapis, A., Katsanos, C., Karousos, N., Sotiropoulos, D., Xenos, M., Orphanoudakis, T.: Stress heatmaps: A fuzzy-based approach that uses physiological signals. In: Marcus, A., Rosenzweig, E. (eds.) HCII 2020. LNCS, vol. 12202, pp. 268–277. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49757-6_19

    Chapter  Google Scholar 

  53. Liapis, A., Katsanos, C., Sotiropoulos, D., Xenos, M., Karousos, N.: Recognizing emotions in human computer interaction: Studying stress using skin conductance. In: Abascal, J., Barbosa, S., Fetter, M., Gross, T., Palanque, P., Winckler, M. (eds.) INTERACT 2015. LNCS, vol. 9296, pp. 255–262. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22701-6_18

    Chapter  Google Scholar 

  54. Liapis, A., Karousos, N., Katsanos, C., Xenos, M.: Evaluating user’s emotional experience in HCI: The PhysiOBS approach. In: Kurosu, M. (ed.) HCII 2014. LNCS, vol. 8511, pp. 758–767. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07230-2_72

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece

    Christos Katsanos & Georgia Zafeiriou

  2. Department of Electrical and Computer Engineering, University of Peloponnese, Patras, Greece

    Alexandros Liapis

  3. School of Science and Technology, Hellenic Open University, Patras, Greece

    Alexandros Liapis

Authors
  1. Christos Katsanos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgia Zafeiriou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexandros Liapis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christos Katsanos .

Editor information

Editors and Affiliations

  1. Tokyo University of Science, Tokyo, Saitama, Japan

    Sakae Yamamoto

  2. Tokyo City University, Tokyo, Japan

    Hirohiko Mori

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Katsanos, C., Zafeiriou, G., Liapis, A. (2022). Effect of Self-efficacy on Open Card Sorts for Websites. In: Yamamoto, S., Mori, H. (eds) Human Interface and the Management of Information: Visual and Information Design. HCII 2022. Lecture Notes in Computer Science, vol 13305. Springer, Cham. https://doi.org/10.1007/978-3-031-06424-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06424-1_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06423-4

  • Online ISBN: 978-3-031-06424-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation