Skip to main content
SpringerLink
Log in

The Multimodal Matrix as a Quantitative Ethnography Methodology

  • Conference paper
  • First Online:
Advances in Quantitative Ethnography (ICQE 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1112))

Included in the following conference series:

Abstract

This paper seeks to contribute to the emerging field of Quantitative Ethnography (QE) by demonstrating its utility to solve a complex challenge in Learning Analytics: the provision of timely feedback to collocated teams and their coaches. We define two requirements that extend the QE concept in order to operationalise it such a design process, namely, the use of co-design methodologies, and the availability of automated analytics workflow to close the feedback loop. We introduce the Multimodal Matrix as a data modelling approach that can integrate theoretical concepts about teamwork with contextual insights about specific work practices, enabling the analyst to map between higher order codes and low-level sensor data, with the option add the results of manually performed analyses. This is implemented in software as a workflow for rapid data modelling, analysis and interactive visualisation, demonstrated in the context of nursing teamwork simulations. We propose that this exemplifies how a QE methodology can underpin collocated activity analytics, at scale, with in-principle applications to embodied, collocated activities beyond our case study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Pozyx developer kit and a multitag-positioning system: https://www.pozyx.io.

  2. 2.

    Laerdal simulation manikins: https://www.laerdal.com/nz/products/simulation-training/emergency-care-trauma/simman-3g.

  3. 3.

    Empatica wristbands: https://www.empatica.com/en-int/research/e4.

References

  1. Alhadad, S.S.J.: Visualizing data to support judgement, inference, and decision making in learning analytics: insights from cognitive psychology and visualization science. J. Learn. Anal. 5(2), 60–85 (2018)

    Article  Google Scholar 

  2. Bakhshinategh, B., Zaiane, O.R., Elatia, S., Ipperciel, D.: Educational data mining applications and tasks: a survey of the last 10 years. Educ. Inf. Technol. 23(1), 537–553 (2018)

    Article  Google Scholar 

  3. Behrens, J.T., Dicerbo, K.E., Foltz, P.W.: Assessment of complex performances in digital environments. Ann. Am. Acad. Polit. Soc. Sci. 683(1), 217–232 (2019)

    Article  Google Scholar 

  4. Bodily, R., Verbert, K.: Trends and issues in student-facing learning analytics reporting systems research. In: Proceedings of the International Learning Analytics and Knowledge Conference, LAK 2017, pp. 309–318 (2017)

    Google Scholar 

  5. Braithwaite, J.J., Watson, D.G., Jones, R., Rowe, M.: A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology 49(1), 1017–1034 (2013)

    Google Scholar 

  6. Buckingham Shum, S., Crick, R.D.: Learning analytics for 21st century competencies. J. Learn. Anal. 3(2), 6–21 (2016)

    Article  Google Scholar 

  7. Chandrasegaran, S., Bryan, C., Shidara, H., Chuang, T.-Y., Ma, K.-L.: TalkTraces: real-time capture and visualization of verbal content in meetings. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2019, pp. 577:571–577:514 (2019)

    Google Scholar 

  8. Chappel, S.E., Verswijveren, S.J.J.M., Aisbett, B., Considine, J., Ridgers, N.D.: Nurses’ occupational physical activity levels: a systematic review. Int. J. Nurs. Stud. 73(August), 52–62 (2017)

    Article  Google Scholar 

  9. Csanadi, A., Eagan, B., Kollar, I., Shaffer, D.W., Fischer, F.: When coding-and-counting is not enough: using epistemic network analysis (ENA) to analyze verbal data in CSCL research. Int. J. Comput.-Support. Collab. Learn. 13(4), 419–438 (2018)

    Article  Google Scholar 

  10. Di Mitri, D., Schneider, J., Klemke, R., Specht, M., Drachsler, H.: Read between the lines: an annotation tool for multimodal data for learning. In: Proceedings of the International Conference on Learning Analytics and Knowledge, LAK 2019, pp. 51–60 (2019)

    Google Scholar 

  11. Echeverria, V.: Designing and validating automated feed-back for collocated teams using multimodal learning analytics. Ph.D. in Learning Analytics, University of Technology Sydney (UTS), Sydney, Australia (in preparation)

    Google Scholar 

  12. Echeverria, V., Martinez-Maldonado, R., Buckingham Shum, S.: Towards collaboration translucence: giving meaning to multimodal group data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2019, p. 39 (2019)

    Google Scholar 

  13. Echeverria, V., Martinez-Maldonado, R., Chiluiza, K., Buckingham Shum, S.: DBCollab: automated feedback for face-to-face group database design. In: Proceedings of the International Conference on Computers in Education, ICCE 2017, pp. 156–165 (2017)

    Google Scholar 

  14. Echeverria, V., Martinez-Maldonado, R., Power, T., Hayes, C., Buckingham Shum, S.: Where is the nurse? Towards automatically visualising meaningful team movement in healthcare education. In: Penstein Rosé, C., et al. (eds.) AIED 2018. LNCS (LNAI), vol. 10948, pp. 74–78. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93846-2_14

    Chapter  Google Scholar 

  15. Gašević, D., Dawson, S., Siemens, G.: Let’s not forget: learning analytics are about learning. TechTrends 59(1), 64–71 (2015)

    Article  Google Scholar 

  16. Gašević, D., Joksimović, S., Eagan, B.R., Shaffer, D.W.: SENS: network analytics to combine social and cognitive perspectives of collaborative learning. Comput. Hum. Behav. 92, 562–577 (2018)

    Article  Google Scholar 

  17. Gibson, A., Martinez-Maldonado, R.: That dashboard looks nice, but what does it mean?: towards making meaning explicit in learning analytics design. In: Proceedings of the Australian Conference on Computer-Human Interaction, OzCHI 2017, pp. 528–532 (2017)

    Google Scholar 

  18. Green, A., Stawicki, S.P., Firstenberg, M.S.: Medical error and associated harm-the the critical role of team communication and coordination. In: Vignettes in Patient Safety, pp. 1–13. IntechOpen, London (2018)

    Google Scholar 

  19. Herder, T., et al.: Supporting teachers’ intervention in students’ virtual collaboration using a network based model. In: Proceedings of the International Conference on Learning Analytics and Knowledge, LAK 2018, pp. 21–25 (2018)

    Google Scholar 

  20. Jivet, I., Scheffel, M., Specht, M., Drachsler, H.: License to evaluate: preparing learning analytics dashboards for educational practice. In: Proceedings of the International Learning Analytics and Knowledge Conference, LAK 2018, pp. 31–40 (2018)

    Google Scholar 

  21. Martinez-Maldonado, R.: I spent more time with that team: making spatial pedagogy visible using positioning sensors. In: Proceedings of the International Conference on Learning Analytics & Knowledge, LAK 2019, pp. 21–25 (2019)

    Google Scholar 

  22. Martinez-Maldonado, R., Dimitriadis, Y., Martinez-Monés, A., Kay, J., Yacef, K.: Capturing and analyzing verbal and physical collaborative learning interactions at an enriched interactive tabletop. Int. J. Comput.-Support. Collab. Learn. 8(4), 455–485 (2013)

    Article  Google Scholar 

  23. Martinez-Maldonado, R., Echeverria, V., Santos, O.C., Dos Santos, A.D.P., Yacef, K.: Physical learning analytics: a multimodal perspective. In: Proceedings of the International Conference on Learning Analytics and Knowledge, LAK 2018, pp. 375–379 (2018)

    Google Scholar 

  24. Martinez-Maldonado, R., Goodyear, P., Kay, J., Thompson, K., Carvalho, L.: An actionable approach to understand group experience in complex, multi-surface spaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2016, pp. 2062–2074 (2016)

    Google Scholar 

  25. Matcha, W., Gasevic, D., Pardo, A.: A systematic review of empirical studies on learning analytics dashboards: a self-regulated learning perspective. IEEE Trans. Learn. Technol. (2019, in press)

    Google Scholar 

  26. Mislevy, R.J., Behrens, J.T., Dicerbo, K.E., Levy, R.: Design and discovery in educational assessment: evidence-centered design, psychometrics, and educational data mining. J. Educ. Data Mining 4(1), 11–48 (2012)

    Google Scholar 

  27. Müller, L., Rivera-Pelayo, V., Kunzmann, C., Schmidt, A.: From stress awareness to coping strategies of medical staff: supporting reflection on physiological data. In: Salah, A.A., Lepri, B. (eds.) HBU 2011. LNCS, vol. 7065, pp. 93–103. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25446-8_11

    Chapter  Google Scholar 

  28. Ochoa, X.: Multimodal learning analytics. In: The Handbook of Learning Analytics, pp. 129–141. SOLAR, Alberta (2017)

    Chapter  Google Scholar 

  29. Ochoa, X., Chiluiza, K., Granda, R., Falcones, G., Castells, J., Guamán, B.: Multimodal transcript of face-to-face group-work activity around interactive tabletops. In: Proceedings of the CROSS-MMLA Workshop on Multimodal Learning Analytics Across Spaces, pp. 1–6 (2018)

    Google Scholar 

  30. Perera, D., Kay, J., Koprinska, I., Yacef, K., Zaïane, O.R.: Clustering and sequential pattern mining of online collaborative learning data. IEEE Trans. Knowl. Data Eng. 21(6), 759–772 (2008)

    Article  Google Scholar 

  31. Prieto-Alvarez, C., Martinez-Maldonado, R., Shum, S.B.: Mapping learner-data journeys: evolution of a visual co-design tool. In: Proceedings of the ACM Australian Computer-Human Interaction Conference, OzCHI 2018, pp. 205–214 (2018)

    Google Scholar 

  32. Prieto-Alvarez, C.G., Martinez-Maldonado, R., Anderson, T.D.: Co-designing learning analytics tools with learners. In: Learning Analytics in the Classroom: Translating Learning Analytics for Teachers, pp. 93–110. Routledge, London (2018)

    Google Scholar 

  33. Sarter, N.B.: Multimodal information presentation: design guidance and research challenges. Int. J. Ind. Ergon. 36(5), 439–445 (2006)

    Article  Google Scholar 

  34. Scherr, R.E., Hammer, D.: Student behavior and epistemological framing: examples from collaborative active-learning activities in physics. Cogn. Instr. 27(2), 147–174 (2009)

    Article  Google Scholar 

  35. Shaffer, D.W.: Epistemic frames for epistemic games. Comput. Educ. 46(3), 223–234 (2006)

    Article  Google Scholar 

  36. Shaffer, D.W.: Quantitative Ethnography. Cathcart Press, Madison (2017)

    Google Scholar 

  37. Shute, V.J., Ventura, M.: Stealth Assessment: Measuring and Supporting Learning in Video Games. MIT Press, Cambridge (2013)

    Book  Google Scholar 

  38. Taylor, S., Jaques, N., Chen, W., Fedor, S., Sano, A., Picard, R.: Automatic identification of artifacts in electrodermal activity data. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2015, pp. 1934–1937 (2015)

    Google Scholar 

  39. Wise, A., Knight, S., Buckingham Shum, S.: Collaborative learning analytics. In: Cress, U., Rosé, C., Wise, A., Oshima, J. (eds.) International Handbook of Computer-Supported Collaborative Learning. Springer, Cham (in press)

    Google Scholar 

  40. Worsley, M., Blikstein, P.: A multimodal analysis of making. Int. J. Artif. Intell. Educ. 28(3), 385–419 (2018)

    Article  Google Scholar 

  41. Zhang, Z., Sarcevic, A.: Constructing awareness through speech, gesture, gaze and movement during a time-critical medical task. In: Boulus-Rødje, N., Ellingsen, G., Bratteteig, T., Aanestad, M., Bjørn, P. (eds.) ECSCW 2015: Proceedings of the 14th European Conference on Computer Supported Cooperative Work, 19-23 September 2015, Oslo, Norway, pp. 163–182. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20499-4_9

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Connected Intelligence Centre, University of Technology Sydney, PO Box 123, Broadway, Sydney, NSW, 2007, Australia

    Simon Buckingham Shum, Vanessa Echeverria & Roberto Martinez-Maldonado

  2. Escuela Superior Politécnica del Litoral, ESPOL, Campus Gustavo Galindo Km 30.5 Vía Perimetral, Guayaquil, Ecuador

    Vanessa Echeverria

Authors
  1. Simon Buckingham Shum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vanessa Echeverria
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roberto Martinez-Maldonado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Buckingham Shum .

Editor information

Editors and Affiliations

  1. University of Wisconsin–Madison, Madison, WI, USA

    Brendan Eagan

  2. University of Copenhagen, Copenhagen, Denmark

    Morten Misfeldt

  3. University of Wisconsin–Madison, Madison, WI, USA

    Amanda Siebert-Evenstone

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Buckingham Shum, S., Echeverria, V., Martinez-Maldonado, R. (2019). The Multimodal Matrix as a Quantitative Ethnography Methodology. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds) Advances in Quantitative Ethnography. ICQE 2019. Communications in Computer and Information Science, vol 1112. Springer, Cham. https://doi.org/10.1007/978-3-030-33232-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-33232-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-33231-0

  • Online ISBN: 978-3-030-33232-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation