Abstract
Quantitative Ethnography (QE) is an emerging methodological approach that combines ethnographic and statistical tools to analyze both Big Data and smaller data to study human behavior and interactions. This paper presents a methodological scoping review of 60 studies employing QE approaches with an intention to characterize and establish where the boundaries of QE might and should be in order to establish the identity of the field. The key finding is that QE researchers have enough commonality in their approach to the analysis of human behavior with a strong focus on grounded analysis, the validity of codes and consistency between quantitative models and qualitative analysis. Nonetheless, in order to reach a larger audience, the QE community should attend to a number of conceptual and methodological issues (e.g. interpretability). We believe that the strength of work from individual researchers reported in this review and initiatives such as the recently established International Society for Quantitative Ethnography (ISQE) can present a powerful force to shape the identity of the QE community.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, B., Chui, M., Manyika, J.: Are you ready for the era of ‘big data’. McKinsey Q. 4(1), 24–35 (2011)
Shaffer, D.W.: Quantitative Ethnography. Cathcart Press, Madison (2017)
Wu, B., Hu, Y., Ruis, A., Wang, M.: Analysing computational thinking in collaborative programming: a quantitative ethnography approach. J. Comput. Assist. Learn. 35(3), 421–434 (2019)
Shaffer, D.W., Collier, W., Ruis, A.: A tutorial on epistemic network analysis: analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Anal. 3(3), 9–45 (2016)
Misfeldt, M., Spikol, D., Bruun, J., Saqr, M., Kaliisa, R., Ruis, A., Eagan, B.: Quantitative ethnography as a framework for network analysis–a discussion of the foundations for network approaches to learning analysis. In: LAK 2010 Companion Proceedings (2020)
Levac, D., Colquhoun, H., O’Brien, K.K.: Scoping studies: advancing the methodology. Implement. Sci. 5(1) (2010). https://doi.org/10.1186/1748-5908-5-69
Munn, Z., Peters, M.D., Stern, C., Tufanaru, C., McArthur, A., Aromataris, E.: Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med. Res. Methodol. 18(1) (2018). https://doi.org/10.1186/s12874-018-0611-x
Gee, J.P.: Discourse, small d, big D. In: Tracy, K., Sandel, T., Ilie, C. (eds.) The International Encyclopedia of Language and Social Interaction, pp. 1–5 (2015)
Geertz, C.: Deep play: notes on the Balinese cockfight. In: Crothers, L., Lockhart, C. (eds.) The Interpretation of Cultures. Selected Essays, pp. 412–453. Palgrave Macmillan, New York (1973)
Wise, A.F., Shaffer, D.W.: Why theory matters more than ever in the age of big data. J. Learn. Anal. 2(2), 5–13 (2015)
Shaffer, D.W., Ruis, A.R.: Epistemic network analysis: a worked example of theory-based learning analytics. In: Lang, C., Siemens, G., Wise, A.F., Gašević, D. (eds.) Handbook of Learning Analytics, pp. 175–187. Society for Learning Analytics Research (2017)
Rupp, et al.: Modeling learning progressions in epistemic games with epistemic network analysis: principles for data analysis and generation. In: LeaPS 2009 Proceedings (2009)
Arastoopour Irgens, G., Shaffer, D.W.: Measuring social identity development in epistemic games. In: CSCL 2013 Proceedings, pp. 42–48 (2013)
Frey, K.S., Kwak-Tanquay, S., Nguyen, H.A., Onyewuenyi, A.C., Strong, Z.H., Waller, I.A.: Adolescents’ views of third-party vengeful and reparative actions. In: ICQE 2019 Proceedings, pp. 89–105 (2019)
D’Angelo, A.L.D., Ruis, A.R., Collier, W., Shaffer, D.W., Pugh, C.M.: Evaluating how residents talk and what it means for surgical performance in the simulation lab. Am. J. Surg. 220(1), 37–43 (2020)
Eagan, B.R., Rogers, B., Pozen, R., Marquart, C., Shaffer, D.W.: rhoR: Rho for inter-rater reliability (Version 1.1.0) (2016)
Shaffer, D.W., et al.: The nCoder: A Technique for Improving the Utility of Inter-Rater Reliability Statistics. Epistemic Games Group Working Paper 2015-01 (2015)
Cai, Z., Siebert-Evenstone, A., Eagan, B., Shaffer, D.W., Hu, X., Graesser, A.C.: nCoder+: a semantic tool for improving recall of nCoder coding. In: ICQE 2019 Proceedings, pp. 41–54 (2019)
Zörgő, S., Peters, G.J.Y.: Epistemic network analysis for semi-structured interviews and other continuous narratives: Challenges and insights. In: ICQE 2019 Proceedings, pp. 267–277 (2019)
Buckingham Shum, S., Echeverria, V., Martinez-Maldonado, R.: The Multimodal Matrix as a quantitative ethnography methodology. In: ICQE 2019 Proceedings, pp. 26–40 (2019)
Major, L., Warwick, P., Rasmussen, I., Ludvigsen, S., Cook, V.: Classroom dialogue and digital technologies: a scoping review. Educ. Inf. Technol. 23(5), 1995–2028 (2018)
Espino, D., Lee, S., Eagan, B., Hamilton, E.: An initial look at the developing culture of online global meet-ups in establishing a collaborative, STEM media-making community. In: CSCL 2019 Proceedings, pp. 608–611 (2019)
Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.): Advances in Quantitative Ethnography: First International Conference, ICQE 2019, Madison, WI, USA, 20–22 October 2019, Proceedings, vol. 1112. Springer Nature (2019)
Sullivan, S., et al.: Using epistemic network analysis to identify targets for educational interventions in trauma team communication. Surgery 163(4), 938–943 (2018)
Wooldridge, Abigail R., Haefli, R.: Using epistemic network analysis to explore outcomes of care transitions. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 245–256. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_21
Shaffer, D.W.: Big data for thick description of deep learning. In: Millis, K., Long, D., Magliano, J., Wiemer, K. (eds.) Deep Comprehension, pp. 265–277. Routledge, New York (2018)
Arastoopour Irgens, G., et al.: Modeling and measuring high school students’ computational thinking practices in science. J. Sci. Educ. Tech. 29(1), 137–161 (2020)
Ferreira, R., Kovanović, V., Gašević, D., Rolim, V.: Towards combined network and text analytics of student discourse in online discussions. In: AIED 2018 Proceedings, pp. 111–126 (2018)
Lund, K., Quignard, M., Shaffer, D.W.: Gaining insight by transforming between temporal representations of human interaction. J. Learn. Anal. 4(3), 102–122 (2017)
Nash, P., Shaffer, D.W.: Mentor modeling: the internalization of modeled professional thinking in an epistemic game. J. Comput. Assist. Learn. 27(2), 173–189 (2011)
Bauer, E., et al.: Using ENA to analyze pre-service teachers’ diagnostic argumentations: a conceptual framework and initial applications. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 14–25. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_2
Hu, S., Torphy, K.T., Chen, Z., Eagan, B.: How do US teachers align instructional resources to the common core state standards: a case of Pinterest. In: SMSociety 2018 Proceedings, pp. 315–319 (2018)
Pratt, S.M.: A mixed methods approach to exploring the relationship between beginning readers’ dialog about their thinking and ability to self-correct oral reading. Read. Psychol. 41(1), 1–43 (2020)
Karumbaiah, S., Baker, R.S., Barany, A., Shute, V.: Using epistemic networks with automated codes to understand why players quit levels in a learning game. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 106–116. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_9
Peters-Burton, E.E.: Outcomes of a self-regulated learning curriculum model. Sci. Educ. 24(7–8), 855–885 (2015)
Swiecki, Z., Ruis, A.R., Farrell, C., Shaffer, D.W.: Assessing individual contributions to collaborative problem solving: a network analysis approach. Comput. Hum. Behav. 104 105876 (2020)
Zhang, S., Liu, Q., Cai, Z.: Exploring primary school teachers’ technological pedagogical content knowledge (TPACK) in online collaborative discourse: an epistemic network analysis. Br. J. Educ. Technol. 50(6), 3437–3455 (2019)
Siebert-Evenstone, A., Arastoopour Irgens, G., Collier, W., Swiecki, Z., Ruis, A.R., Shaffer, D.W.: In search of conversational grain size: Modelling semantic structure using moving stanza windows. J. Learn. Anal. 4(3), 123–139 (2017)
Melzner, N., Greisel, M., Dresel, M., Kollar, I.: Using Process Mining (PM) and Epistemic Network Analysis (ENA) for Comparing Processes of Collaborative Problem Regulation. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 154–164. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_13
Gašević, D., Joksimović, S., Eagan, B.R., Shaffer, D.W.: SENS: network analytics to combine social and cognitive perspectives of collaborative learning. Comput. Hum. Behav. 92, 562–577 (2019)
Swiecki, Z., Shaffer, D.W.: iSENS: an integrated approach to combining epistemic and social network analyses. In: LAK 2010 Proceedings, pp. 305–313 (2020)
Bagley, E., Shaffer, D.W.: Epistemic mentoring in virtual and face-to-face environments. In: ICLS 2012 Proceedings, pp. 256–260 (2012)
Svarovsky, G.N.: Exploring complex engineering learning over time with epistemic network analysis. J-PEER 1(2), 4 (2011)
Rolim, V., Ferreira, R., Lins, R.D., Gašević, D.: A network-based analytic approach to uncovering the relationship between social and cognitive presences in communities of inquiry. Internet High. Educ. 42, 53–65 (2019)
Ognjanović, I., Gašević, D., Dawson, S.: Using institutional data to predict student course selections in higher education. Internet High. Educ. 29, 49–62 (2016)
Shaffer, D.W.: QE-COVID data challenge. Why QE? [White paper]. (2020). https://sites.google.com/wisc.edu/qe-covid-data-challenge/why-qe
Wu, B., Hu, Y., Ruis, A., Wang, M.: Analysing computational thinking in collaborative programming: a quantitative ethnography approach. J. Comput. Assist. Learn. 35(3), 421–434 (2019)
Swiecki, Z., Shaffer, D.W.: Toward a taxonomy of team performance visualization tools. In: ICLS 2018 Proceedings, pp. 144–151 (2018)
Swiecki, Z., Marquart, C., Sachar, A., Hinojosa, C., Ruis, A.R., Shaffer, D.W.: Designing an Interface for sharing quantitative ethnographic research data. In: ICQE 2019 Proceedings, pp. 334–341 (2019)
Herder, T., et al.: Supporting teachers’ intervention in students’ virtual collaboration using a network based model. In: LAK 2008 Proceedings, pp. 21–25 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix I
Appendix I
List of studies included in the scoping review but not cited in the main text (n = 36)
Authors | Authors |
---|---|
Andrist, Collier, Gleicher, Mutlu, Shaffer (2015) | Orrill, Shaffer (2012) |
Andrist, Ruis, Shaffer (2018) | Peters-Burton (2015) |
Arastoopour Irgens, Shaffer, Swiecki, Ruis, Chesler (2015) | Peters-Burton, Parrish, Mulvey (2019) |
Arastoopour Irgens, Shaffer (2015) | Phillips, Kovanović, Mitchell, Gašević (2019) |
Barany, Foster (2019) | Ruis, Rosser, Nathwani, Beams, Jung, Pugh (2019) |
Bressler, Bodzin, Eagan, Tabatabai (2019) | Ruis, Rosser, Quandt-Walle, Nathwani, Shaffer, Pugh (2018) |
Brown, Nagar, Orrill, Weiland, Burke (2016) | Ruis, Siebert-Evenstone, Pozen, Eagan, Shaffer (2019) |
Cai, Eagan, Dowell, Pennebaker, Shaffer, Graesser (2017) | Shah, Foster, Talafian, Barany (2019) |
Chesler, Ruis, Collier, Swiecki, Arastoopour Irgens, Shaffer (2015) | Siebert-Evenstone, Shaffer (2019) |
Csanadi, Eagan, Shaffer, Kollar, Fischer (2017) | Sinclair, Ferreira, Gašević, Lucas, Lopez (2019) |
Espino, Lee, Van Tress, Baker, Hamilton (2020) | Sung, Cao, Ruis, Shaffer (2019) |
Fisher, Hirshfield, Siebert-Evenstone, Arastoopour Irgens, Koretsky (2016) | Talafian, Shah, Barany, Foster (2019) |
Fougt, Siebert-Evenstone, Eagan, Tabatabai, Misfeldt (2018) | Wakimoto, Sasaki, Hirayama, Mochizuki, Eagan, Yuki, Kato (2019) |
Foster, Shah, Barany, Talafian (2019) | Whitelock-Wainwright, Tsai, Lyons, Kaliff, Bryant, Ryan, Gašević (2020) |
Knight, Arastoopour Irgens, Shaffer, Buckingham Shum, Littleton (2014) | Wooldridge, Carayon, Shaffer, Eagan (2018) |
Lim, Dawson, Joksimović, Gašević (2019) | Yi, Lu, Leng (2019) |
Nachtigall, Sung (2019) | Yue, Hu, Xiao (2019) |
Oner (2020) |
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kaliisa, R., Misiejuk, K., Irgens, G.A., Misfeldt, M. (2021). Scoping the Emerging Field of Quantitative Ethnography: Opportunities, Challenges and Future Directions. In: Ruis, A.R., Lee, S.B. (eds) Advances in Quantitative Ethnography. ICQE 2021. Communications in Computer and Information Science, vol 1312. Springer, Cham. https://doi.org/10.1007/978-3-030-67788-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-67788-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67787-9
Online ISBN: 978-3-030-67788-6
eBook Packages: Computer ScienceComputer Science (R0)