Skip to main content
SpringerLink
Log in

Towards the real-time evaluation of collaborative activities: Integration of an automatic rater of collaboration quality in the classroom from the teacher’s perspective

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

This paper presents the integration of a real time evaluation method of collaboration quality in a monitoring application that supports teachers in class orchestration. The method is implemented as an automatic rater of collaboration quality and studied in a real time scenario of use. We argue that automatic and semi-automatic methods which provide assessments of collaboration quality or detect disorders and anomalies in collaborative activities should be further used to assist class monitoring. To that end, we study the routine of teachers when they monitor computer-supported collaborative activities with respect to the size of the classroom and the support of an automatic rater. We focus on the way the rater affects their task and how it can be further used to empower the practice of teachers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Avouris, N., Margaritis, M., & Komis, V. (2004). Modelling interaction during small-group synchronous problem-solving activities: The Synergo approach. In Proceedings of ITS 2004 workshop on designing computational models of collaborative learning interaction (pp. 13–18).

  • Casamayor, A., Amandi, A., & Campo, M. (2009). Intelligent assistance for teachers in collaborative e-learning environments. Computers & Education, 53(4), 1147–1154.

    Article  Google Scholar 

  • Chan, K.S. (2010). TSA: time series analysis. R package version 0.98. URL: http://CRAN.R-project.org/package=TSA.

  • Chan, C. K. K. (2011). Bridging research and practice: Implementing and sustaining knowledge building in Hong Kong classrooms. International Journal of Computer-Supported Collaborative Learning, 6(2), 147–186.

    Article  Google Scholar 

  • Chounta, I.-A., & Avouris, N. (2012). Time series analysis of collaborative activities. In V. Herskovic, H. U. Hoppe, M. Jansen, & J. Ziegler (Eds.), Collaboration and technology (pp. 145–152). Berlin Heidelberg: Springer.

    Chapter  Google Scholar 

  • Chounta, I.-A., & Avouris, N. (2014). It’s all about time: towards the real-time evaluation of collaborative activities. In 14th International Conference on Advanced Learning Technologies, ICALT2014 (pp. 283–285). Athens: IEEE.

  • De Groot, R., Drachman, R., Hever, R., Schwarz, B.B., Hoppe, U., Harrer, A., et al. (2007). Computer supported moderation of e-discussions: the ARGUNAUT approach. In Proceedings of the 8th iternational conference on Computer supported collaborative learning (pp. 168–170). International Society of the Learning Sciences.

  • de los Constantino-González, M. A., & Suthers, D. D. (2007). An approach for coaching collaboration based on difference recognition and participation tracking. In H. U. Hoppe, H. Ogata, & A. Soller (Eds.), The role of technology in CSCL (pp. 87–113). US: Springer.

    Chapter  Google Scholar 

  • Dillenbourg, P. (1999). What do you mean by collaborative learning? Collaborative-Learning: Cognitive and Computational Approaches, 1–19.

  • Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL? (pp. 61–91). Heerlen: Open Universiteit Nederland.

    Google Scholar 

  • Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. In M. S. Khine & I. M. Saleh (Eds.), New science of learning (pp. 525–552). New York: Springer.

    Chapter  Google Scholar 

  • Dillenbourg, P., Järvelä, S., & Fischer, F. (2009). The evolution of research on computer-supported collaborative learning. In Technology-enhanced learning (pp. 3–19). Springer.

  • Duval, E. (2011). Attention please!: Learning analytics for visualization and recommendation. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 9–17). New York, NY, USA: ACM.

  • Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and implementation of a learning analytics toolkit for teachers. Educational Technology & Society, 15(3), 58–76.

    Google Scholar 

  • Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5), 304–317.

    Article  MathSciNet  Google Scholar 

  • Giorgino, T. (2009). Computing and visualizing dynamic time warping alignments in R: the dtw package. Journal of Statistical Software, 31(7), 1–24.

    Article  Google Scholar 

  • Harrer, A., McLaren, B. M., Walker, E., Bollen, L., & Sewall, J. (2006). Creating cognitive tutors for collaborative learning: Steps toward realization. User Modeling and User-Adapted Interaction, 16(3–4), 175–209.

    Article  Google Scholar 

  • Kahrimanis, G., Meier, A., Chounta, I.-A., Voyiatzaki, E., Spada, H., Rummel, N., & Avouris, N. (2009). Assessing collaboration quality in synchronous CSCL problem-solving activities: Adaptation and empirical evaluation of a rating scheme. In Learning in the Synergy of Multiple Disciplines (pp. 267–272). Springer.

  • Kahrimanis, G., Avouris, N., & Komis, V. (2011a). Interaction analysis as a tool for supporting collaboration: An overview. In T. Daradoumis, S. Caballé, A. A. Juan, & F. Xhafa (Eds.), Technology-enhanced systems and tools for collaborative learning scaffolding (pp. 93–114). Berlin Heidelberg: Springer.

    Chapter  Google Scholar 

  • Kahrimanis, G., Chounta, I.-A., & Avouris, N. (2011b). Rating dimensions of collaboration quality in synchronous collaborating dyads: findings and interpretations. In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), Connecting Computer-Supported Collaborative Learning to Policy and Practice: CSCL2011 Conference Proceedings (Vol. 1, pp. 446–453). Hong Kong, China.

  • Kahrimanis, G., Chounta, I.-A., & Avouris, N. (2012). Validating empirically a rating approach for quantifying the quality of collaboration. In Intelligent adaptation and personalization techniques in computer-supported collaborative learning (pp. 295–310). Springer.

  • Martínez-Monés, A., Harrer, A., & Dimitriadis, Y. (2011). An interaction-aware design process for the integration of interaction analysis into mainstream CSCL practices. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL (pp. 269–291). US: Springer.

    Chapter  Google Scholar 

  • Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J.I., & Dimitriadis, Y. (2014). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology.

  • Schümmer, T., Strijbos, J.-W., & Berkel, T. (2005). A new direction for log file analysis in CSCL: Experiences with a spatio-temporal metric. In Proceedings of th 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years! (pp. 567–576). International Society of the Learning Sciences.

  • Soller, A., Martínez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15(4), 261–290.

    Google Scholar 

  • Spada, H., Meier, A., Rummel, N., & Hauser, S. (2005). A new method to assess the quality of collaborative process in CSCL. In Proceedings of th 2005 Conference on Computer Support for Collaborative Learning: Learning 2005: The Next 10 Years! (pp. 622–631). Taipei, Taiwan: International Society of the Learning Sciences.

  • Stahl, G. (2002). Rediscovering CSCL. In Cscl (Vol. 2, pp. 169–181).

  • Stahl, G. (2007). Meaning making in CSCL: Conditions and preconditions for cognitive processes by groups. In Proceedings of the 8th Iternational Conference on Computer Supported Collaborative Learning (pp. 652–661). New Brunswick, New Jersey, USA: International Society of the Learning Sciences.

  • Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. Cambridge Handbook of the Learning Sciences, 2006.

  • Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42.

    Article  Google Scholar 

  • Team, R. C., & others. (2012). R: a language and environment for statistical computing.

  • Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2013). Teacher interventions in a synchronous, co-located CSCL setting: Analyzing focus, means, and temporality. Computers in Human Behavior, 29(4), 1377–1386.

    Article  Google Scholar 

  • Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers & Education.

  • Voyiatzaki, E., & Avouris, N. (2014). Support for the teacher in technology-enhanced collaborative classroom. Education and Information Technologies, 19(1), 129–154.

    Article  Google Scholar 

  • Voyiatzaki, E., Polyzos, P., & Avouris, N. (2008). Teacher tools in a networked learning classroom: monitor, view and interpret interaction data. In Proceedings of the 6th Networked Learning Conference (pp. 849–850). Halkidiki, Greece.

  • Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46(1), 71–95.

    Article  Google Scholar 

  • Wichmann, A., Giemza, A., Hoppe, U., & Krauß, M. (2009). Effects of awareness support on moderating multiple parallel E-discussions. In Proceedings of the 9th International Conference on Computer Supported Collaborative Learning - Volume 1 (pp. 646–650). Rhodes, Greece: International Society of the Learning Sciences.

  • Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. The Internet and Higher Education, 10(1), 15–25.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Patras, Ypatias Str., 26500, Rio Patras, Greece

    Irene-Angelica Chounta & Nikolaos Avouris

Authors
  1. Irene-Angelica Chounta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nikolaos Avouris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irene-Angelica Chounta.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chounta, IA., Avouris, N. Towards the real-time evaluation of collaborative activities: Integration of an automatic rater of collaboration quality in the classroom from the teacher’s perspective. Educ Inf Technol 21, 815–835 (2016). https://doi.org/10.1007/s10639-014-9355-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-014-9355-3

Keywords

Search

Navigation