Abstract
This paper describes the application of a mixed-evaluation method, published elsewhere, to three different learning scenarios. The method defines how to combine social network analysis with qualitative and quantitative analysis in order to study participatory aspects of learning in CSCL contexts. The three case studies include a course-long, blended learning experience evaluated as the course develops; a course-long, distance learning experience evaluated at the end of the course; and a synchronous experience of a few hours duration. These scenarios show that the analysis techniques and data collection and processing tools are flexible enough to be applied in different conditions. In particular, SAMSA, a tool that processes interaction data to allow social network analysis, is useful with different types of interactions (indirect asynchronous or direct synchronous interactions) and different data representations. Furthermore, the predefined types of social networks and indexes selected are shown to be appropriate for measuring structural aspects of interaction in these CSCL scenarios. These elements are usable and their results comprehensible by education practitioners. Finally, the experiments show that the mixed-evaluation method and its computational tools allow researchers to efficiently achieve a deeper and more reliable evaluation through complementarity and the triangulation of different data sources. The three experiments described show the particular benefits of each of the data sources and analysis techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Borgatti, S., Everett, M., & Freeman, L., (Eds.) (2002). UCINET for Windows: Software for social network analysis. Harvard: Analytic Technologies.
Cho, H., Stefanone, M., & Gay, G. (2002). Social Information Sharing in a CSCL Community. In G. Stahl (Ed.), Computer Support for Collaborative Learning: Foundations of a CSCL community. Proceedings of CSCL 2002 (pp. 43–50). Hillsdale, NY: Lawrence Erlbaum Associates.
Dimitracopoulou, A. (2005). Designing collaborative learning systems: Current trends and future research agenda. In T. Koschmann, D. Suthers & T. Chan (Eds.), Proceedings of computer supported collaborative learning 2005: The next 10 years! (pp. 115–124). Mahwah, NJ: Lawrence Erlbaum Associates.
Frechtling, J., & Sharp, L., (Eds.) (1997). User-Friendly handbook for mixed method evaluations. Directorate for Education and Human Resources Division of Research, Evaluation and Communication, NSF.
Gómez, E., Dimitriadis, Y., Rubia, B., & Martínez, A. (2002). Quest, a telematic tool for automatic management of student questionnaires in educational research. In Proceedings of the 2nd European conference on information technologies in education and citizenship: A critical insight, TIEC 2002, 26–28 June, Barcelona, Spain.
Greene, J., Caracelly, V., & Graham, W. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274.
Harrer, A., Zeini, S., & Pinkwart, N. (2005). The effects of electronic communication support on presence learning scenarios. In T. Koschmann, D. Suthers & T. Chan (Eds.), Proceedings of computer supported collaborative learning 2005: The next 10 years! (pp. 190–194). Mahwah, NJ: Lawrence Erlbaum Associates.
Johnson, R., & Onwuegbuzie, A. (2004). Mixed method research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.
Kaleidoscope (2005). Kaleidoscope network of excellence. http://www.noe-kaleidoscope.org/ (last visit, 11/11/2005).
Koschmann, T., Suthers, D., & Chan, T., (Eds.) (2005). Proceedings of computer supported collaborative learning: The next 10 years!. Mahwah, NJ: Lawrence Erlbaum Associates.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripherical participation. Cambridge, U.K.: Cambridge University Press.
Marcos, J., Martínez, & Dimitriadis, Y. (2005). Towards adaptable interaction analysis tools in CSCL. In Representing and analyzing collaborative interactions, workshop at the 12th international conference on artificial intelligence in education, AIED'2005, Amsterdam, The Netherlands.
Martínez, A., Dimitriadis, Y., Rubia, B., Gómez, E., & de la Fuente, P. (2003a). Combining qualitative evaluation and social network analysis for the study of classroom social interactions. Computers and Education, 41(4), 353–368.
Martínez, A., de la Fuente, P., & Dimitriadis, Y. (2003b). Towards an XML-Based representation of collaborative interaction. In B. Wasson, S. Ludvigsen & U. Hoppe (Eds.), Designing for change in networked environments. Proceedings of the international conference on computer support for collaborative learning 2003. (pp. 379–388). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Martínez, A., Gómez, E., Dimitriadis, Y., Jorrín, I., Rubia, B., & Vega, G. (2005). Multiple case studies to enhance project-based learning in a computer architecture course. IEEE Transactions on Education, 48(3), 482–489.
Nurmela, K., Lehtinen, E., & Palonen, T. (1999). Evaluating CSCL log files by Social Network Analysis. In C. Hoadley & J. Roschelle (Eds.), Proceedings of computer support for collaborative learning 1999 (pp. 434–442). Mahwah, NJ: Lawrence Erlbaum Associates.
Nurmela, K., Palonen, T., Lehtinen, E., & Hakkarainen, K. (2003). Developing tools for analyzing CSCL process. In B. Wasson, S. Ludvigsen & U. Hoppe (Eds.), Computer support for collaborative learning: designing for change in networked environments. Proceedings of CSCL 2003 (pp. 333–342). The Netherlands, Dordecht: Kluwer Academic Publishers.
QSR (1997). QSR, NUD*IST. Software for qualitative data analysis. Thousand Oaks, CA: Scolari.
Reffay, C., & Chanier, T. (2003). How social network analysis can help to measure cohesion in collaborative distance-learning. In B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.), Designing for change in networked learning environments. Proceedings of the international conference on computer support for collaborative learning 2003. (pp. 343–352). Dordrecht, The Netherlands: Kluwer Academic Publishers.
Reyes, P., & Tchounikine, P. (2005). Mining learning groups' activities in forum-type tools. In T. Koschmann, D. Suthers & T. Chan (Eds.), Proceedings of computer supported collaborative learning 2005: The next 10 years! (pp. 509–513). Mahwah, NJ: Lawrence Erlbaum Associates.
Scott, J. (2000). Social network analysis. A handbook (2nd edition). London, U.K: Sage Publications.
Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27, 4–13.
Soller, A., Martínez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: A review of the state of the art technology for supporting collaborative learning. Int. J. on Artificial Intelligence in Education, 15, 261–290.
Stake, R., (Ed.) (1995). The art of case study research. Thousand Oaks, CA: Sage Publications.
TELL Project (2004). Introducing a framework for the evaluation of networked supported collaborative learning, WP1 deliverable, Project number: EAC/61/03/GR009 e-Learning initiative.
Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications. Cambridge: Cambridge University Press.
Wasson, B., Ludvigsen, S., & Hoppe, U., (Eds.) (2003). Computer support for collaborative learning: designing for change in networked environments. Proceedings of CSCL 2003, Dordrecht, The Netherlands: Kluwer Academic Publishers.
Wenger, E. (1998). Communities of practice. Learning, meaning and identity. Cambridge, U.K.: Cambridge University Press.
Wilson, B., & Myers, K. (2000). Situated cognition in Theoretical and Practical Context. In D. Jonassen & S. Land (Eds.), Theoretical foundations of learning environments (pp. 57–88). Mahwah, N.J.: Lawrence Erlbaum Associates.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Martínez, A., Dimitriadis, Y., Gómez-Sánchez, E. et al. Studying participation networks in collaboration using mixed methods. Computer Supported Learning 1, 383–408 (2006). https://doi.org/10.1007/s11412-006-8705-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11412-006-8705-6