Abstract
Pairwise similarity judgments and card sorting methodologies are different ways of generating data for similarity matrices used in various analyses such as multidimensional scaling and cluster analysis. Pairwise similarity judgments are considered the gold standard methodology, but can be cumbersome for large numbers of stimuli given the geometric increase in number of judgments necessary to fill the matrix. Card sorting methods provide a more expedient means of gathering this information, although they typically generate only binary data. Nonetheless, aggregated matrices generated from card sorts approximate pairwise similarity matrices. The current study used pairwise similarity and card sorting results from two existing studies that used the same stimuli to determine the optimal number of participants needed in a card sorting task to approximate the similarity matrix of pairwise data collection. In these studies, approximately 10–15 participants provided optimal estimation of the similarity matrix, with minimal increases for higher numbers of participants.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Clatworthy, J., Buick, D., Hankins, M., Weinman, J., & Horne, R. (2005). The use of reporting and cluster analysis in health psychology: a review. British Journal of Health Psychology, 10(3), 329–358.
Danilova, D., Diekhoff, G. M., & Vandehey, M. A. (2013). A multidimensional scaling analysis of male body perception in men with muscle dysmorphia: “The Adonis Complex”. International Journal of Men’s Health, 12(2), 83–105.
Hout, M. C., Papesh, M. H., & Goldinger, S. D. (2012). Multidimensional scaling. Wiley Interdisciplinary Reviews (WIREs): Cognitive Science, 4, 93–103.
Lantz, E., Diekhoff, G., & Carlston, D. (2014). How many participants are needed for card sort data collection? Poster presented at the southwestern psychological association. TX: San Antonio.
Reed, G., Roberts, M., Keeley, J., Hooppell, C., Matsumoto, C., Sharan, P., Robles, R., Carvalho, H., Wu, C., Leal-Leturia, I., Flanagan, E., Correia, J., Ayuso-Mateos, J., Mari, J., Xiao, Z., Evans, S., Saxena, S., & Medina-Mora, M. (2013). Mental health professionals’ natural taxonomies of mental disorders: implications for clinical utility of ICD-11 and DSM-5. Journal of Clinical Psychology, 69, 1191–1212.
Roberts, M., Reed, G., Medina-Mora, M., Keeley, J., Sharan, P., Johnson, D., Mari, J., Ayuso-Mateos, J., Gureje, O., Xiao, Z., Maruta, T., Khoury, B., Robles, R., & Saxena, S. (2012). A global clinicians’ map of mental disorders to improve ICD-11: analysing meta-structure to enhance clinical utility. International Review of Psychiatry, 24(6), 578–590.
Tullis, T., & Wood, L. (June, 2004). How many users are enough for a card-sorting study? Paper presented at the usability professional association. MN: Minneapolis.
Tullis, T., & Wood, L. (June, 2005). How can you do a card-sorting study with LOTS of cards? Paper presented at the usability professional association. MN: Minneapolis.
Whaley, A., & Longoria, R. (2009). Preparing card Sort data for multidimensional scaling analysis in social psychological research: a methodological approach. The Journal of Social Psychology, 149(1), 105–115.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lantz, E., Keeley, J.W., Roberts, M.C. et al. Card Sorting Data Collection Methodology: How Many Participants Is Most Efficient?. J Classif 36, 649–658 (2019). https://doi.org/10.1007/s00357-018-9292-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-018-9292-8