Skip to main content

Advertisement

Springer Nature Link
Log in

Promoting Students’ Collective Cognitive Responsibility through Concurrent, Embedded and Transformative Assessment in Blended Higher Education Courses

  • Original research
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

This study investigates different instructional designs to promote students’ collective cognitive responsibility for Knowledge Building in blended university courses. Using an iterative, design-based research methodology with reference to the conjecture mapping technique, the blended learning design of an undergraduate educational psychology course was refined over three years in three design iterations. The iterations differed substantially in the embodiment of the Concurrent Embedded and Transformative Assessment Knowledge Building principle that engaged students in knowledge assessments and strategy assessments of their community’s work. The design of the knowledge assessment involved face-to-face small group and whole class discussions in all three iterations. In the first and second iterations, students also worked online by writing individual reflections and contributing to a community portfolio. The design of the strategy assessment changed in each iteration. In the first iteration, the students’ strategy assessment took place in face-to-face discussions; in the second iteration, students contributed to an online community portfolio; and in the third iteration, the strategy assessment took place in an online community portfolio and face-to-face discussions before beginning the course and in the online community portfolio in the middle of the course. Collective cognitive responsibility was analyzed in terms of productive and informative participation, interdependence between participants, self-regulation skills. The results show that the second iteration’s design was most effective for fostering the students' collective cognitive responsibility, showing an increase in productive participation and self-regulation skills in the first part of the course and also an increase in the interdependence of participants during the course. Some implications concerning the relationship between the implementation of the CETA principle and Knowledge Building are identified for future directions of inquiry and for blended learning environments design.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. The sum of the percentage is higher than 100% because some students indicated more than one problem time or the study time allotment). Three students did not post any messages.

  2. Time 1 corresponds to the metacognitive reflection after the module 1 for the 1st and the 2nd iteration and before module 1 for the 3rd iteration; Time 2 correspond to the metacognitive reflection after module 2 for all the iterations.

References

  • Alamri, H. A., Watson, S., & Watson, W. (2020). Learning technology models that support personalization within blended learning environments in higher education. TechTrends, 65(1), 62–78. https://doi.org/10.1007/s11528-020-00530-3

    Article  Google Scholar 

  • Alexander, B., Ashford-Rowe, K., Barajas-Murphy, N., Dobbin, G., Knott, J., McCormack, M., Pomerantz, J., Seilhamer, R., & Weber, N. (2019). NMC horizon report: 2019 higher education edition. CO: EDUCAUSE.

    Google Scholar 

  • Alstete, J., & Beutell, N. (2004). Performance indicators in online distance learning courses: A study of management education. Quality Assurance in Education, 12(1), 6–14. https://doi.org/10.1108/09684880410517397

    Article  Google Scholar 

  • Anthony, B., Jr., Kamaludin, A., Romli, A., Raffei, A. F. M., Phon, D. N. A. E., Abdullah, A., Ming, G. L., Shukor, N. A., Nordin, M. S., & Baba, S. (2019). Exploring the role of blended learning for teaching and learning effectiveness in institutions of higher learning: An empirical investigation. Education and Information Technologies, 24(6), 3433–3466. https://doi.org/10.1007/s10639-019-09941-z

    Article  Google Scholar 

  • Anthony, B., Jr., Kamaludin, A., Romli, A., Raffei, A. F., Phon, D. N. A. E., Abdullah, A., Shukor, N. A., Nordin, M. S., & Baba, S. (2020a). The International Journal of Information and Learning Technology, 37(4), 179–196. https://doi.org/10.1108/ijilt-02-2020-0013

    Article  Google Scholar 

  • Anthony, B., Jr., Kamaludin, A., Romli, A., Raffei, A. F. M., Phon, D. N. A. E., Abdullah, A., & Ming, G. L. (2020b). Blended learning adoption and implementation in higher education: A theoretical and systematic review. Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-020-09477-z

    Article  Google Scholar 

  • Ashton, J., & Newman, L. (2006). An unfinished symphony: 21st century teacher education using knowledge-creating heutagogies. British Journal of Educational Technology, 37(6), 825–884.

    Article  Google Scholar 

  • Balboni, G., Perrucci, V., Cacciamani, S., & Zumbo, B. D. (2018). Development of a scale of sense of community in university online courses. Distance Education, 39(3), 317–333. https://doi.org/10.1080/01587919.2018.1476843

    Article  Google Scholar 

  • Beaudoin, M. (2003). Learning or lurking? Tracking the invisible online student. The Internet and Higher Education, 5, 147–155. https://doi.org/10.1016/S1096-7516(02)00086-6

    Article  Google Scholar 

  • Bereiter, C. (2002). Education and mind in the knowledge age. Erlbaum.

    Google Scholar 

  • Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and implications of expertise. Open Court.

    Google Scholar 

  • Burtis, J. (1998). Analytic toolkit for knowledge forum. Centre for Applied Cognitive Science: The Ontario Institute for Studies in Education/University of Toronto.

  • Cacciamani, S. (2017). Experiential learning and knowledge building in higher education: An application of the progressive design method. Journal of E-Learning and Knowledge Society, 13(1), 27–38.

    Google Scholar 

  • Cacciamani, S., Cesareni, D., Perrucci, V., Balboni, G., & Khanlari, A. (2019). Effects of social tutor on sense of community in online university courses. British Journal of Educational Technology, 50(4), 1171–1784. https://doi.org/10.1111/bjet.12656

    Article  Google Scholar 

  • Cacciamani, S., Cesareni, D., Martini, F., Ferrini, T., & Fujita, N. (2012). Influence of participation, facilitator styles, and metacognitive reflection on knowledge building in online university courses. Computers and Education, 58(3), 874–884. https://doi.org/10.1016/j.compedu.2011.10.019

    Article  Google Scholar 

  • Cesareni, D., Cacciamani, S., & Fujita, N. (2016). Role taking and knowledge building in a blended university course. International Journal of Computer-Supported Collaborative Learning, 11(1), 9–39. https://doi.org/10.1007/s11412-015-9224-0

    Article  Google Scholar 

  • Cesareni, D., Albanese, O., Cacciamani, S., Castelli, S., De Marco, B., Fiorilli, C., Luciani, M., Mancini, I., Martini, F., & Vanin, L. (2008). Tutorship style and knowledge building in an online community: cognitive and metacognitive aspects. In B. M. Varisco (Ed.), Psychological pedagogical and sociological models for learning and assessment in virtual communities (pp. 13–56). Polimetrica.

    Google Scholar 

  • Chan, C. K. K., & Chan, Y. Y. (2011). Students’ views of collaboration and online participation in knowledge forum. Computers and Education, 57(1), 1445–1457. https://doi.org/10.1016/j.compedu.2010.09.003

    Article  Google Scholar 

  • Chen, B., & Hong, H.-Y. (2016). Schools as knowledge-building organizations: Thirty years of design research. Educational Psychologist, 51(2), 266–288. https://doi.org/10.1080/00461520.2016.1175306

    Article  Google Scholar 

  • Chen, B., Ma, L., Matsuzawa, Y., & Scardamalia, M. (2015). The development of productive vocabulary in Knowledge Building: A longitudinal study. In O. Lindwall, P. Häkkinen, T. Koschman, P. Tchounikine, & S. Ludvigsen. (Eds.), Exploring the material conditions of learning: The computer supported collaborative learning (CSCL) conference 2015, Volume 1 (pp. 443–450). Gothenburg, Sweden: International Society of the Learning Sciences.

  • Chiu, M. M., & Fujita, N. (2014). Statistical discourse analysis: A method for modeling online discussion processes. Journal of Learning Analytics, 1(3), 61–83. https://doi.org/10.18608/jla.2014.13.5

    Article  Google Scholar 

  • Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Learning Sciences, 13(1), 15–42. https://doi.org/10.1207/s15327809jls1301_2

    Article  Google Scholar 

  • Conrad, R.-M., & Donaldson, J. A. (2011). Engaging the online learner: Activities and resources for creative instruction. Jossey-Bass.

    Google Scholar 

  • De Marco, B., & Albanese, O. (2009). Le competenze autoregolative dell’attività di studio in comunità virtuali. Qwerty-Open and Interdisciplinary Journal of Technology, Culture and Education, 4(2), 123–139.

    Google Scholar 

  • Di Donato, N. C. (2013). Effective self-and co-regulation in collaborative learning groups: An analysis of how students regulate problem solving of authentic interdisciplinary tasks. Instructional Science, 41(1), 25–47. https://doi.org/10.1007/s11251-012-9206-9

    Article  Google Scholar 

  • Dunn, T. J., & Kennedy, M. (2019). Technology enhanced learning in higher education: Motivations, engagement, and academic achievement. Computers and Education, 137, 104–113. https://doi.org/10.1016/j.compedu.2019.04.004

    Article  Google Scholar 

  • Fabbri, L., Giampaolo, M., & Capaccioli, M. (2021). Blended learning and transformative processes: A model for didactic development and innovation. In D. Burgos, P. Ducange, P. Limone, L. Perla, P. Picero, & C. M. Stracke (Eds.), Bridges and mediation in higher distance education (pp. 214–225). Springer International Publishing. https://doi.org/10.1007/978-3-030-67435-9

    Chapter  Google Scholar 

  • Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013). Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist, 48(1), 56–66.

    Article  Google Scholar 

  • Fujita, N. (2020). Transforming online teaching and learning: towards learning design informed by the information science and learning sciences. Information and Learning Sciences. https://doi.org/10.1108/ILS-04-2020-0124

    Article  Google Scholar 

  • Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering the transformative potential in higher education. The Internet and Higher Education, 7, 95–105. https://doi.org/10.1016/j.iheduc.2004.02.001

    Article  Google Scholar 

  • Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6

    Article  Google Scholar 

  • Gutiérrez-Braojos, C., Montejo-Gámez, J., Ma, L., Chen, B., Muñoz de Escalona-Fernández, M., Scardamalia, M., & Bereiter, C. (2018). Exploring collective cognitive responsibility through the emergence and flow of forms of engagement in a knowledge building community. In L. Daniela (Ed.), Didactics of smart pedagogy (pp. 213–232). Cham: Springer. https://doi.org/10.1007/978-3-030-01551-0_11

    Chapter  Google Scholar 

  • Harasim, L. (2012). Learning theory and online technologies. Routledge.

    Book  Google Scholar 

  • Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2010). Student contribution in asynchronous online discussion: A review of the research and empirical exploration. Instructional Science, 38, 571–606. https://doi.org/10.1007/s11251-008-9087-0

    Article  Google Scholar 

  • Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer conferences. The Journal of the Learning Sciences, 14(4), 567–589. https://doi.org/10.1207/s15327809jls1404_4

    Article  Google Scholar 

  • Hrastinski, S. (2009). A theory of online learning as online participation. Computers and Education, 52(1), 78–82. https://doi.org/10.1016/j.compedu.2008.06.009

    Article  Google Scholar 

  • Lan, Y. F., Lin, P. C., & Hung, C. L. (2012). An approach to encouraging and evaluating learner’s knowledge contribution in web-based collaborative learning. Journal of Educational Computing Research, 47(2), 107–135. https://doi.org/10.2190/EC.47.2.a

    Article  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  Google Scholar 

  • Lee, C. Y. (2020). How to improve the effectiveness of blended learning of pharmacology and pharmacotherapy? A case study in pharmacy program. Technology, Knowledge and Learning, 25(4), 977–988. https://doi.org/10.1007/s10758-020-09447-5

    Article  Google Scholar 

  • Lee, E. Y. C., Chan, C. K. K., & Van Aalst, J. (2006). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning, 1(1), 57–87. https://doi.org/10.1007/s11412-006-6844-4

    Article  Google Scholar 

  • Lee, M. J. W., McLoughlin, C., & Chan, A. (2008). Talk the talk: Learner-generated podcasts as catalysts for knowledge creation. British Journal of Educational Technology, 39(3), 501–521. https://doi.org/10.1111/j.1467-8535.2007.00746.x

    Article  Google Scholar 

  • Lee, I. A., & Preacher, K. J. (2013, September). Calculation for the test of the difference between two dependent correlations with no variable in common [Computer software]. Retrieved from http://quantpsy.org.

  • Mayer, R. E., Fiorella, L., & Stull, A. (2020). Five ways to increase the effectiveness of instructional video. Educational Technology Research and Development, 68(2020), 837–852. https://doi.org/10.1007/s11423-020-09749-6

    Article  Google Scholar 

  • McLoughlin, C., & Lee, M. J. (2010). Personalised and self-regulated learning in the Web 2.0 era: International exemplars of innovative pedagogy using social software. Australasian Journal of Educational Technology, 26(1), 28–43.

    Article  Google Scholar 

  • Narciss, S., Proske, A., & Koerndle, H. (2007). Promoting self-regulated learning in web-based learning environments. Computers in Human Behavior, 23(3), 1126–1144. https://doi.org/10.1016/j.chb.2006.10.006

    Article  Google Scholar 

  • Nguyen, N., Muilu, T., Dirin, A., & Alamäki, A. (2018). An interactive and augmented learning concept for orientation week in education. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-018-0118-x

    Article  Google Scholar 

  • Paavola, S., & Hakkarainen, K. (2005). The knowledge creation metaphor: An emergent epistemological approach to learning. Science and Education, 14, 535–557. https://doi.org/10.1007/s11191-004-5157-0

    Article  Google Scholar 

  • Perrucci, V., Coscarelli, A., Balboni, G., & Cacciamani, S. (2012). Preliminary validation of the scale of sense of community in online course. World Journal on Educational Technology, 4(2), 126–136.

    Google Scholar 

  • Preece, J., Nonnecke, B., & Andrews, D. (2004). The top five reasons for lurking: Improving community experience for everyone. Computers in Human Behavior, 20, 201–223. https://doi.org/10.1016/j.chb.2003.10.015

    Article  Google Scholar 

  • Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18–36. https://doi.org/10.1080/10508406.2013.778204

    Article  Google Scholar 

  • Sansone, N., Ligorio, M. B., & Buglass, S. L. (2016). Peer e-tutoring: Effects on students’ participation and interaction style in online courses. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2016.1190296

    Article  Google Scholar 

  • Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In B. Smith (Ed.), Liberal education in a knowledge society (pp. 67–98). Open Court.

    Google Scholar 

  • Scardamalia, M. (2004). CSILE/Knowledge Forum®. In Education and technology: An encyclopedia (pp. 183-192). Santa Barbara: ABC-CLIO.

  • Scardamalia, M., & Bereiter, C. (2014). Knowledge building and knowledge creation. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 397–417). Cambridge University Press.

    Chapter  Google Scholar 

  • Scardamalia, M., & Bereiter, C. (2010). A brief history of knowledge building. Canadian Journal of Learning and Technology. https://doi.org/10.21432/T2859M

    Article  Google Scholar 

  • Scardamalia, M., Bransford, J., Kozma, B., & Quellmalz, E. (2012). New assessments and environments for knowledge building. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 231–300). Springer. https://doi.org/10.1007/978-94-007-2324-5_5

    Chapter  Google Scholar 

  • Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior, 78, 397–407. https://doi.org/10.1016/j.chb.2017.06.030

    Article  Google Scholar 

  • Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27, 4–13. https://doi.org/10.3102/0013189X027002004

    Article  Google Scholar 

  • Siemens, G. (2011). In 1st International Conference on Learning Analytics and Knowledge, Banff, Alberta, February 27-March 1, 2011. Retrieved from https://tekri.athabascau.ca/analytics/.

  • Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245–251. https://doi.org/10.1037/0033-2909.87.2.245

    Article  Google Scholar 

  • Strijbos, J. W., & Weinberger, A. (2010). Emerging and scripted roles in computer-supported collaborative learning. Computers in Human Behavior, 26, 491–494. https://doi.org/10.1016/j.chb.2009.08.006

    Article  Google Scholar 

  • Teplovs, C. (2008). The knowledge space visualizer: a tool for visualizing online discourse. In Paper presented at the common framework for CSCL interaction analysis workshop, international conference of the learning sciences. Utrecht: NL.

  • Teplovs, C. (2010). Visualization of knowledge spaces to enable concurrent, embedded and transformative input to knowledge building processes. Unpublished doctoral dissertation, University of Toronto, Toronto, ON. Retrieved from http://hdl.handle.net/1807/24893.

  • Teplovs, C., & Scardamalia, M. (2007). Visualizations for knowledge building assessment. Paper presented at the Institute for Knowledge Innovation and Technology Summer Institute 2007. Retrieved from  https://ikit.org/SummerInstitute2007/Highlights/SI2007_papers/48_Teplovs.pdf.

  • Teplovs, C., Donoahue, Z., Scardamalia, M., & Philip, D. (2007). Tools for concurrent, embedded, and transformative assessment of knowledge building processes and progress. In C. A. Chinn, G. Erkens, & S. Puntambekar (Eds.), Proceedings of the 8th international conference on computer-supported collaborative learning (CSCL’ 07) (pp. 721–723). International Society of the Learning Sciences.

  • The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005

    Article  Google Scholar 

  • Van Aalst, J., & Chan, C. K. K. (2007). Student-directed assessment of knowledge building using electronic portfolios. The Journal of the Learning Sciences, 16(2), 175–220. https://doi.org/10.1080/10508400701193697

    Article  Google Scholar 

  • Van Laer, S., & Elen, J. (2018). Adults’ self-regulatory behaviour profiles in blended learning environments and their implications for design. Technology, Knowledge and Learning, 25(3), 509–539. https://doi.org/10.1007/s10758-017-9351-y

    Article  Google Scholar 

  • Vatrapu, R., Teplovs, C., Fujita, N., & Bull, S. (2011). Toward visual analytics for teachers’ dynamic diagnostic pedagogical decision-making. In LAK’11 proceedings of the 1st international conference on learning analytics and knowledge (pp.93–98). Banff, AB: ACM. https://doi.org/10.1145/2090116.2090129

  • Yang, S., Carter, R. A., Zhang, L., & Hunt, T. (2021). Emanant themes of blended learning in K-12 educational environments: Lessons from the every student succeeds Act. Computers and Education. https://doi.org/10.1016/j.compedu.2020.104116

    Article  Google Scholar 

  • Zhang, J., Scardamalia, M., Reeve, R., & Messina, R. (2009). Designs for collective cognitive responsibility in knowledge-building communities. The Journal of the Learning Sciences, 18(1), 7–44. https://doi.org/10.1080/10508400802581676

    Article  Google Scholar 

  • Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3–17. https://doi.org/10.1207/s15326985ep2501_2

    Article  Google Scholar 

  • Zimmerman, B. J. (1998). Developing self-fulfilling cycles of academic regulation: An analysis of exemplary instructional models. In D. H. Schunk & B. J. Zimmerman (Eds.), Self-regulated learning: From teaching to self-reflective practice (pp. 1–19). Guilford Press.

    Google Scholar 

  • Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–39). Academic Press.

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Human and Social Sciences Department, University of Valle D’Aosta, Strada Cappuccini, 2A, 11100, Aosta, Italy

    Stefano Cacciamani & Vittore Perrucci

  2. Office of Open Learning, University of Windsor, 401 Sunset Avenue, Windsor, ON, Canada

    Nobuko Fujita

Authors
  1. Stefano Cacciamani
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Vittore Perrucci
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Nobuko Fujita
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

SC designed the research and worked on coding and data analysis. He coordinated the co-authoring of the manuscript; contributed to the Introduction, Method, and Results sections; wrote the sections Discussion and Conclusions; and contributed to revision of the entire manuscript. VP contributed to design the research and worked on the coding and data analysis. He wrote the sections Method and Results, and contributed to revision of the entire manuscript. NF contributed to the Introduction and Methods sections. She revised the entire manuscript as a developmental English editor.

Corresponding author

Correspondence to Stefano Cacciamani.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cacciamani, S., Perrucci, V. & Fujita, N. Promoting Students’ Collective Cognitive Responsibility through Concurrent, Embedded and Transformative Assessment in Blended Higher Education Courses. Tech Know Learn 26, 1169–1194 (2021). https://doi.org/10.1007/s10758-021-09535-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-021-09535-0

Keywords