Abstract
Students’ performance in human-to-human and human-to-agent collaborative problem solving assessment task is investigated in this paper. A secondary data analysis of the research reported by Rosen and Tager (2013) was conducted in order to investigate the comparability of the opportunities for conflict situations in human-to-human and human-to-agent settings. The major research question in this study was whether the semi-standardized human-to-human collaboration provides the student with adequate conflict opportunities in which the student can showcase collaborative skills. This is in comparison to the conflict opportunities in a similar standardized human-to-agent setting. While one type of assessment task was explored in this study, the aim of this paper is to offer strategies for evaluating complex collaborative problem solving process data generally. A process analysis of the chats and actions of the students showed that in human-to-agent group the students encounter significantly more conflict situations than in the human-to-human group. However, it was found that students in human-to-human setting were engaged in significantly more situations in which the partner proposed different solutions for a problem. The paper concludes with recommendations for collaborative problem solving assessment development, and proposes further research directions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94, 416–427.
Barth, C. M., & Funke, J. (2010). Negative affective environments improve complex solving performance. Cognition and Emotion, 24(7), 1259–1268.
Bereiter, C. (2002). Education in a knowledge society. In B. Smith (Ed.), Liberal education in a knowledge society (pp. 11–33). Chicago, IL: Open Court.
Biswas, G., Jeong, H., Kinnebrew, J. S., Sulcer, B., & Roscoe, A. R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology-Enhanced Learning, 5(2), 123–152.
Chen, G., & Chiu, M. (2008). Online discussion processes: Effects of earlier messages’ evaluations, knowledge content, social cues and personal information on later messages. Computers & Education, 50(3), 678–692.
Dillenbourg, P. (Ed.). (1999). Collaborative learning: Cognitive and computational approaches. Amsterdam, NL: Pergamon, Elsevier Science.
Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1995). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.), Learning in humans and machines: Towards an interdisciplinary learning science. Oxford: Elsevier.
Dillenbourg, P., & Traum, D. (2006). Sharing solutions: Persistence and grounding in multi-modal collaborative problem solving. The Journal of the Learning Sciences, 15(1), 121–151.
Graesser, A., Foltz, P., Rosen, Y., Forsyth, C., & Germany, M. (in press). Assessing collaborative problem solving. In B. Csapo, J. Funke, & A. Schleicher (Eds.), The nature of problem solving. OECD Series.
Graesser, A. C., Jeon, M., & Dufty, D. (2008). Agent technologies designed to facilitate interactive knowledge construction. Discourse Processes, 45, 298–322.
Greiff, S., Holt, D., & Funke, J. (2013). Perspectives on problem solving in educational assessment: Analytical, interactive, and collaborative problem solving. The Journal of Problem Solving, 5(2), 71–91.
Griffin, P., Care, E., & McGaw, B. (2012). The changing role of education and schools. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching 21st century skills (pp. 1–15). Heidelberg: Springer.
Halpern, D., Millis, K., Graesser, A., Butler, H., Forsyth, C., & Cai, Z. (2012). Operation ARA: A computerized learning game that teaches critical thinking and scientific reasoning. Thinking Skills and Creativity, 7, 93–100.
Israel, J., & Aiken, R. (2007). Supporting collaborative learning with an intelligent web-based system. Paper presented at the Artificial Intelligence in Education conference, Los Angeles, CA.
Keefer, M., Zeitz, C., & Resnick, L. (2000). Judging the quality of peer-led student dialogues. Cognition and Instruction, 18(1), 53–81.
King, A. (1999). Discourse patterns for mediating peer learning. In A. O’Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 87–116). Mahwah, NJ: Erlbaum.
Kreijns, C., Kirschner, P., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335–353.
Mayer, R. E., & Wittrock, M. C. (1996). Problem-solving transfer. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 47–62). New York: Macmillan Library Reference USA, Simon & Schuster Macmillan.
McNamara, D. S., O’Reilly, T., Rowe, M., Boonthum, C., & Levinstein, I. B. (2007). iSTART: A web-based tutor that teaches self-explanation and metacognitive reading strategies. In D. S. McNamara (Ed.), Reading comprehension strategies: Theories, interventions, and technologies (pp. 397–421). Mahwah, NJ: Erlbaum.
Millis, K., Forsyth, C., Butler, H., Wallace, P., Graesser, A., & Halpern, D. F. (2011). Operation ARIES!: A serious game for teaching scientific inquiry. In M. Ma, A. Oikonomou, & L. Jain (Eds.), Serious games and edutainment applications (pp. 169–195). London: Springer.
Mitchell, R., & Nicholas, S. (2006). Knowledge creation in groups: The value of cognitive diversity, transactive memory and open-mindedness norms. The Electronic Journal of Knowledge Management, 4(1), 64–74.
National Research Council (2011). Assessing 21st century skills. Washington, DC: National Academies Press.
OECD. (2013). PISA 2015 collaborative problem solving framework. OECD Publishing.
O’Neil, H. F, Jr, & Chuang, S. H. (2008). Measuring collaborative problem solving in low-stakes tests. In E. L. Baker, J. Dickieson, W. Wulfeck, & H. F. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 177–199). Mahwah, NJ: Lawrence Erlbaum Associates.
Pellegrino, J. W., & Hilton, M. L. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, DC: National Academy Press.
Pritchard, A. (2010). Psychology for the classroom: Constructivism and social learning. Hoboken, NJ: Taylor & Francis.
Puntambekar, S., Erkens, G., & Hmelo-Silver, C. (2011). Analyzing interactions in CSCL: Methods, approaches and issues. New York: Springer.
Rimor, R., Rosen, Y., & Naser, K. (2010). Complexity of social interactions in collaborative learning: The case of online database environment. Interdisciplinary Journal of E-Learning and Learning Objects, 6, 355–365.
Roberts, A., & Nason, R. (2011). Nobody says no: Student self-censorship in a collaborative knowledge building activity. Journal of Learning Design, 4(4), 56–68.
Rosen, Y. (2014). Thinking tools in computer-based assessment: Technology enhancements in assessments for learning. Educational Technology, 54(1), 30–34.
Rosen, Y. (in press). Assessing collaborative problem solving through computer agent technologies. In M. Khosrow-Pour (Ed.), Encyclopedia of information science and technology (3rd ed). Hershey, PA: Information Science Reference, IGI Global.
Rosen, Y., & Rimor, R. (2012). Teaching and assessing problem solving in online collaborative environment. In R. Hartshorne, T. Heafner, & T. Petty (Eds.), Teacher education programs and online learning tools: Innovations in teacher preparation (pp. 82–97). Hershey, PA: Information Science Reference, IGI Global.
Rosen, Y., & Tager, M. (2013). Computer-based assessment of collaborative problem-solving skills: Human-to-agent versus human-to-human approach. Boston, MA: Pearson Education.
Rosen, Y., Wolf, I., & Tager, M. (in press). How to assess collaborative problem solving skills in computer-based settings? Paper presented at International Society for Technology in Education, Atlanta, GA.
Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66(2), 181–221.
Salomon, G. (2002). Technology and pedagogy: Why don’t we see the promised revolution? Educational Technology, 42(2), 71–75.
Scardamalia, M. (Ed.). (2002). Collective cognitive responsibility for the advancement of knowledge. Chicago, IL: Open Court.
Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press.
Uline, C., Tschannen-Moran, M., & Perez, L. (2003). Constructive conflict: How controversy can contribute to school improvement. Teachers College Record, 105(5), 782–816.
Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.
Webb, N. M., Ender, P., & Lewis, S. (1986). Problem-solving strategies and group processes in small groups learning computer programming. American Educational Research Journal, 23(2), 243–261.
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46, 71–95.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rosen, Y. Comparability of Conflict Opportunities in Human-to-Human and Human-to-Agent Online Collaborative Problem Solving. Tech Know Learn 19, 147–164 (2014). https://doi.org/10.1007/s10758-014-9229-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10758-014-9229-1