Abstract
The goal of this research is to provide an overview of CSCL methodological practices. CSCL is a vibrant interdisciplinary research field where several different theoretical and methodological traditions converge. Given the diversity of theoretical and methodological traditions that co-exist in CSCL, it is important to document the kinds and range of methodological practices and examine how they are related to the diverse theoretical perspectives in the field. In the current study, we examined CSCL research methodology in terms of (1) research designs, (2) research settings, (3) data sources, and (4) analysis methods. We then examined how these dimensions are related to the theoretical frameworks of the research. A content analysis was carried out based on empirical CSCL studies published in seven leading journals of the field during 2005–2009. The analysis identified the dominant CSCL research practices. We found that the modal CSCL study used descriptive designs that were carried out in classroom settings, typically collected questionnaires and/or analyzed the data quantitatively. CSCL research methods, however, were also quite diverse and eclectic, as researchers used range of data collection and analysis practices. Methodological practices were influenced by the theoretical framework of the research. A cluster analysis examined how these practices co-varied and revealed four distinctive method-theory clusters. Remaining methodological challenges of the field are discussed along with suggestions to move the field toward meaningful synthesis.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Studies that used more than one design (N = 1) or research settings (N = 8) were excluded from the analysis.
References
Alpers, G. W., Winzelberg, A. J., Classen, C., Roberts, H., Dev, P., Koopman, C., et al. (2005). Evaluation of computerized text analysis in an Internet breast cancer support group. Computers in Human Behavior, 21, 361–376.
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25.
Ares, N. (2008). Cultural practices in networked classroom learning environments. International Journal of Computer-Supported Collaborative Learning, 3, 301–326.
Arnseth, H. C., & Ludvigsen, S. (2006). Approaching institutional contexts: Systemic versus dialogic research in CSCL. International Journal of Computer-Supported Collaborative Learning, 1, 167–185.
Baker, M., Andriessen, J., Lund, K., van Amelsvoort, M., & Quignard, M. (2007). Rainbow: A framework for analyzing computer-mediated pedagogical debates. International Journal of Computer-Supported Collaborative Learning, 2, 315–357.
Barab, S. A., & Kirshner, D. (2001). Methodologies for capturing learner practices occurring as part of dynamic learning environments. Journal of the Learning Sciences, 10, 5–6.
Barab, S. A., & Squire, K. D. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14.
Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human Development, 49(4), 193.
Beers, P., Boshuizen, H. P. A., Kirschner, P., & Gijselaers, W. H. (2007). The analysis of negotiation of common ground in CSCL. Learning and Instruction, 17, 427–435.
Berge, O., & Fjuk, A. (2006). Understanding the roles of online meetings in a net-based course. Journal of Computer Assisted Learning, 22, 13–23.
Blin, F., & Munro, M. (2008). Why hasn’t technology disrupted academics’ teaching practices? Understanding resistance to change through the lens of activity theory. Computers and Education, 50, 475–490.
Borrego, M. (2007). Conceptual difficulties experienced by engineering faculty becoming engineering education researchers. Journal of Engineering Education, 96(2), 91–102.
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178.
Bryman, A. (1984). The debate about quantitative and qualitative research: A question of method or epistemology. The British Journal of Sociology, 35(1), 75–92.
Calinski, T., & Harabasz, J. (1974). A dendrite method for cluster analysis. Communications in Statistics-Theory and Methods, 3(1), 1–27.
Chen, W., & Hirschheim, R. (2004). A paradigmatic and methodological examination of information systems research from 1991 to 2001. Information Systems Journal, 14, 197–235.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315.
Chi, M. T. H. (2009). Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1, 73–105.
Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471–533.
Cho, K., & Schunn, C. (2007). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers and Education, 48, 409–426.
Cobb, P., & Jackson, K. (2008). The consequences of experimentalism in formulating recommendations for policy and practice in mathmatics education. Educational Researcher, 37(9), 573–581.
Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). Handbook of research synthesis and meta-analysis (2nd ed.). New York: Russell Sage.
Council, N. R. (2002). Scientific culture and educational research. Washington: National Academies of Press.
Cress, U. (2008). The need for considering multilevel analysis in CSCL research—an appeal for the use of more advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 3, 69–84.
Cress, U., Barquero, B., Schwain, S., & Hesse, F. W. (2007). Improving quality and quantity of contributions: Two models for promoting knowledge exchange with shared database. Computers and Education, 49, 423–440.
Davies, A., Fidler, D., & Gorbis, M. (2011). Future work skills 2020. Palo Alto: Institue for the future.
De Laat, M., Lally, V., Lipponen, L., & Simons, R.-J. (2007). Online teaching in networked learning communities: A multi-method approach to studying the role of the teacher. Instructional Science, 35, 257–286.
De Lisi, R., & Golbeck, S. L. (1999). Implications of Piagetian theory for peer learning. In A. M. O’Donnell & A. King (Eds.), Cognitive perspectives on peer learning. Mahwah: Erlbaum.
De Wever, B., Schellens, T., Valcke, M., & van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: a review. Computers and Education, 46, 6–28.
De Wever, B., van Keer, H., Schellens, T., & Valcke, M. (2007). Applying multi-level modelling to content analysis data: Methodological issues in the study of role assignment in asynchronous discussion groups. Learning and Instruction, 17, 436–447.
Dillenbourg, P., Jarvela, S., & Fischer, F. (2009). The evolution of research on computer-supported collaborative learning. In N. Balacheff (Ed.), Technology-enhance learning. New York: Springer.
Doise, W., Mugny, G., & Perret-Clermont, A. (1975). Social interaction and the development of cognitive operations. European Journal of Social Psychology, 5(3), 367–383.
Dori, Y. J., & Belcher, J. (2005). How does technology-enabled active learning affect undergraduate students’ understanding of electromagnetism concepts? Journal of the Learning Sciences, 14(2), 243–279.
Dringus, L. P., & Ellis, T. (2005). Using data mining as a strategy for assessing asynchronous discussion forum. Computers and Education, 45, 141–160.
Dyke, G., Lund, K., & Girardot, J.-J. (2009). Tatiana: An environment to support the CSCL analysis process. In Proceedings of the 8th International Conference of the Learning Sciences, International Society of the Learning Sciences.
Ellis, R. A., Goodyear, P., Prosser, M., & O’Hara, A. O. (2006). How and what university students learn through online and face-to-face discussion: conceptions, intentions and approaches. Journal of Computer Assisted Learning, 22(4), 244–256.
Engeström, R. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14(1), 133–156.
Erkens, G., & Janssen, J. (2008). Automatic coding of dialogue acts in collaboration protocols. International Journal of Computer-Supported Collaborative Learning, 3, 447–470.
Everitt, B. S., Landau, S., Leese, M., & Stahl, D. (2011). Cluster analysis (Wiley series in probability and statistics) (5th ed.). West Sussex: Wiley.
Finch, H. (2005). Comparison of distance measures in cluster analysis with dichotomous data. Journal of Data Science, 3(1), 85–100.
Fuks, H., Pimentel, M., & de Lucena, C. J. P. (2006). R-U-Typing-2-Me? Evolving a chat tool to increase understanding in learning activities. International Journal of Computer-Supported Collaborative Learning, 1, 117–142.
Gee, J. P., & Green, J. L. (1998). Discourse analysis, learning, and school practice: A methodological study. Review of Research in Education, 23, 119–169.
Glaser, B. G., & Straus, A. (1967). The discovery of grounded theory: Strategies for qualitative research. New Brunswick: Aldine transaction.
Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age web 2.0 and classroom research: what path should we take now? Educational Researcher, 38(4), 246–259.
Greeno, J. G. (2006). Authorative, accountable positioning and connected, general knowing: progressive themes in understanding transfer. Journal of the Learning Sciences, 15(4), 537–547.
Guribye, F., & Wasson, B. (2002). The ethnography of distributed collaborative learning. In Proceedings of the Conference on Computer Support for Collaborative Learning: Foundations for a CSCL Community. International Society of the Learning Sciences.
Hew, K. F., Kale, U., & Kim, N. (2007). Past research in instructional technology: Results of a content analysis of empirical studies published in three prominent instructional technology journals from the year 2000 through 2004. Journal of Educational Computing Research, 36(3), 269–300.
Hewitt, J., & Brett, C. (2007). The relationship between class size and online activity patterns in asynchronous computer conferencing environments. Computers and Education, 49, 1258–1271.
Hmelo-Silver, C. E. (2003). Analyzing collaborative knowledge construction: Multiple methods for integrated understanding. Computers and Education, 41, 397–420.
Howley, I., Kumar, R., Mayfield, E., Dyke, G., & Rosé, C. P. (2013). Gaining insights from sociolinguistic style analysis for redesign of conversational agent based support for collaborative learning Productive multivocality in the analysis of group interactions (pp. 477–494): Springer.
Hrastinski, S., & Keller, C. (2007). An examination of research approaches that underlie research on educational technology: A review from 2000 to 2004. Journal of Educational Computing Research, 36(2), 175–190.
Hummel, H. G. K., Burgos, D., Tattersall, F., Brouns, F., Kurvers, H., & Koper, R. (2005). Encouraging contributions in learning networks using incentive mechanisms. Journal of Computer Assisted Learning, 21, 355–365.
Hutchins, E. (1995). Cognition in the wild. Cambridge: The MIT Press.
Jacobs, N., & McFarlane, A. (2005). Conferences as learning communities: Some early lessons in using ‘back-channel’ technologies at an academic conference-distributed intelligence or divided attention? Journal of Computer Assisted Learning, 21, 317–329.
Jeong, H. (2013). Verbal data analysis for understanding interactions. In C. Hmelo-Silver, A. M. O’Donnell, C. Chan, & C. Chinn (Eds.), The international handbook of collaborative learning (pp. 168–183). London: Routledge.
Jeong, H., & Hmelo-Silver, C. E. (2010a). Technology use in CSCL: A content meta-analysis. In Proceedings of the 43rd Hawaiian International Conference on System Science. IEEE.
Jeong, H., & Hmelo-Silver, C. E. (2010b). An overview of CSCL methodologies. In Proceedings of the 9th International Conference of the Learning Sciences, International Society of the Learning Sciences.
Jeong, H., & Hmelo-Silver, C. E. (2011). A portrait of CSCL methodologies. In Proceedings of the 10th International Conference of the Learning Sciences. International Society of the Learning Sciences.
Jeong, H., & Hmelo-Silver, C. E. (2012). Technology supports in CSCL. In Proceedings of the 11th International Conference of the Learning Sciences. International Society of the Learning Sciences.
Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in collaborative problem solving. Computers and Education, 51, 279–296.
Johnson, R., & Christensen, L. (2008). Educational research: Quantitative, qualitative, and mixed approaches. Thousand Oaks: Sage.
Johnson, R., & Onwuegbuzie, A. J. (2004). Mixed method research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.
Jordan, B., & Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the Learning Science, 4(1), 39–103.
Kelly, A. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 115–128.
Keppel, G., & Wickens, T. D. (2004). Design and analysis. Upper Saddle River: Prentice Hall.
Koschmann, T. (2013). Conversation analysis and collaborative learning. In C. Hmelo-Silver, A. M. O’Donnell, C. Chan, & C. Chinn (Eds.), International handbook of collaborative learning. London: Taylor and Francis.
Koschmann, T., & LeBaron, C. D. (2003). Reconsidering common ground: Examining Clark’s contribution theory in the OR. In K. Kuutti, E. Karsten, G. Fitzpatrick, P. Dourish, & K. Schmidt (Eds.), ECSCW 2003 (pp. 81–98). Amsterdam: Kluwer Academic Publishing.
Krauss, R. M., & Fussell, S. R. (1990). Mutual knowledge and communicative effectiveness. In J. Galegher, R. E. Kraut, & C. Edigo (Eds.), Intellectual teamwork: Social and technological foundations of cooperative work. Hillsdale: Erlbaum.
Lee, E. Y. C., Chan, C. K. K., & van Aalst, J. (2006). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning, 1, 57–87.
Levine, J. M., & Thompson, L. (1996). Conflict in groups. In E. T. Higgins & A. Kruglanski (Eds.), Social psychology: Handbook of basic principles. New York: The Guilford Press.
Lim, C. P., & Barnes, S. (2005). A collective case study of the use of ICT in economics courses: A sociocultural approach. Journal of the Learning Sciences, 14(4), 489–526.
Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 30–32.
Markett, C., Sanchez, I. A., Weber, S., & Tangney, B. (2006). Using short message service to encourage interactivity in the classroom. Computers and Education, 46, 280–293.
Martin, T., & Sherin, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: an introduction to the special issue. Journal of the Learning Sciences, 22(4), 511–520. doi:10.1080/10508406.2013.840466.
Martinez, A., Dimitriadis, Y., & Fuente, P. d. l. (2003). Interaction analysis for formative evaluation in CSCL. In M. Llamas, M. J. Fernández & L. E. Anido (Eds.), Computers and Education. Towards a Lifelong Learning Society (pp. 227–238): Kluwer Academic.
Martinez, A., Dimitriadis, Y., Gomez-Sanchez, E., Rubia-Avi, B., Jorrin-Abellan, I., & Marcos, J. A. (2006). Studying participation networks in collaborating using mixed methods. International Journal of Computer-Supported Collaborative Learning, 1, 383–408.
Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research, 1(2). Art. 20, http://nbn-resolving.de/urn:nbn:de:0114-fqs0002204.
McCarthy, C., Bligh, J., Jennings, K., & Tangney, B. (2005). Virtual collaborative learning environments for music networked drumsteps. Computers and Education, 44, 173–195.
Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2, 63–86.
Miyake, N. (2006). Computer supported collaborative learning. In R. Andrew & C. Haythornwaite (Eds.), Sage handbook of e-learning research. London: Sage.
Morken, E. M., Divitini, M., & Haugalokken. (2007). Enriching spaces in practice-based education to support collaboration while mobile: the case of teacher education. Journal of Computer Assisted Learning, 23, 300–311.
Morrow, R. A., & Brown, D. B. (1994). Deconstructing the conventional discourse of methodology. Critical theory and methodology. Thousand Oaks: Sage Publication.
Munneke, L., Andriessen, J., Kanselaar, G., & Kirschner, P. (2007). Supporting interactive argumentation: influence of representational tools on discussing a wicked problem. Computers in Human Behavior, 23, 1072–1088.
Naidu, S., & Jarvela, S. (2006). Analyzing CMC for what? Computers and Education, 46, 96–103.
Neuendorf, K. A. (2002). The content analysis: Guidebook. London: Thousand Oaks.
Pintrich, P. R. (1999). The role of motivation in prompting and sustaining self-regulated learning. International Journal of Educational Research, 31, 459–470.
Plomp, T., & Nieveen, N. (2007). An introduction to educational design research. In Proceedings of the Seminar Conducted at the East China Normal University. Netherlands: SLO-Netherlands Institute for Curriculum Development.
Powell, W. W., Koput, K. W., & Smith-Doerr, L. (1996). Interorganizational collaboration and the locus of innovation: networks of learning in biotechnology. Administrative Science Quarterly, 41(1), 116–145.
Puntambekar, S. (2013). Mixed methods for analyzing collaborative learning. In C. Hmelo-Silver, A. M. O’Donnell, C. Chan, & C. Chinn (Eds.), The international handbook of collaborative learning. London: Taylor and Francis.
Raffleff. (2007). The reliability of content analysis of computer conference communication. Computers and Education, 49, 230–242.
Rick, M., & Guzdial, M. (2006). Situating coweb: A scholarship of application. International Journal of Computer-Supported Collaborative Learning, 1, 89–115.
Robertson, J., & Howells, C. (2008). Computer game design: Opportunities for successful learning. Computers and Education, 50, 559–578.
Rogoff, B. (1998). Cognition as a collaborative process. In W. Damon (Ed.), Handbook of child psychology (pp. 679–744). New York: Wiley.
Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle case study and tutorial. Computers and Education, 51, 368–384.
Rosé, C., Wang, Y.-C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., et al. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3, 237–271.
Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201–241.
Sacks, H. (1992). Lectures on conversation. Malden: Blackwell.
Salomon, G. (Ed.). (1993). Dsitributed cognitions: Psychological and educational considerations. Cambridge: Cambridge University Press.
Sandoval, W. A. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23, 18–36.
Schmid, E. C. (2008). Potential pedagogical benefits and drawbacks of multimedia use in the English language classroom equipped with interactive whiteboard technology. Computers and Education, 51, 1553–1568.
Schwarz, B. B., & Glassner, A. (2007). The role of floor control and of ontology in argumentative activities with discussion-based tools. International Journal of Computer-Supported Collaborative Learning, 2, 449–478.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York: Houghton Mifflin Company.
Shavelson, R. J. (1996). Statistical reasoning for the behavioral sciences (3rd ed.). Boston: Allyn & Bacon.
Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25–28.
Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychological Review, 84(2), 127–190.
Shih, M., Feng, J., & Tsai, C.-C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers and Education, 51, 955–967.
Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge: MIT Press.
Stahl, G. (2013). Learning across levels. International Journal of Computer-Supported Collaborative Learning, 8(1), 2–11.
Stahl, G., Koschmann, T., & Suthers, D. D. (2006). Computer-supported collaborative learning: A historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences. New York: Cambridge University Press.
Straus, A., & Corbin, J. (1990). Basics of qualitative research. Newbury Park: Sage publication.
Strijbos, J., & Stahl, G. (2007). Methodological issues in developing a multi-dimensional coding procedure for small-group chat communication. Learning and Instruction, 17, 394–404.
Strijbos, J., Martens, R. L., Prins, F. J., & Jochems, W. M. G. (2006). Content analysis: What are they talking about? Computers and Education, 46, 29–48.
Sung, S., Shen, J., & Zhang, D. (2012). Toward a cognitive framework of interdisciplinary understanding. In Proceedings of the 11th International Conference of the Learning Sciences. International Society of the Learning Sciences.
Suthers, D. D. (2006). Technology affordances for intersubjective meaning making: A research agenda for CSCL. International Journal of Computer-Supported Collaborative Learning, 1, 315–337.
Suthers, D. D., Lund, K., Rose, C., Teplovs, C., & Law, N. (2013). Productive multivocality in the analysis of group interactions. Cambridge: MIT Press.
Swinglehurst, D., Russell, J., & Greenhalgh, T. (2008). Peer observation of teaching in the online environment: An action research approach. Journal of Computer Assisted Learning, 24, 383–393.
The Secretary’s Commission on Achieving Necessary Skills. (1991). What work require of schools: A SCANS report for America 2000. Washington: US Department of Labor.
Tulving, E., & Madigan, S. A. (1970). Memory and verbal learning. Annual Review of Psychology, 21, 437–484.
Van der Meij, H., de Vries, B., Boersma, K., Pieters, J., & Wegerif, R. (2005). An examination of interactional coherence in email use in elementary school. Computers in Human Behavior, 21, 417–439.
Van der Pol, J., Van den Berg, B., Admiraal, W. F., & Simons, P. R. J. (2008). The nature, reception, and use of online peer feedback in higher education. Computers and Education, 51(4), 1804–1817.
Van Drie, J., van Boxtel, C., Jaspers, J., & Kanselaar, G. (2005). Effect of representational guidance on domain specific reasoning in CSCL. Computers in Human Behavior, 21, 575–602.
Von Glaserfeld, E. (1987). Learning as a constructive activity. In C. Janvier (Ed.), Problems of representation in the teaching and learning of mathematics (pp. 3–38). Hillsdale: Erlbaum.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge: Harvard University Press.
What Works Clearninghouse. (2008). Procedures and standards handbook (version 2.1) Retrieved February 14, 2013, from http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
Yanchar, S. C., & Williams, D. D. (2006). Reconsidering the compatibility thesis and electicism: Five proposed guidelines for method use. Educational Researcher, 35(9), 3-12.
Yang, Z., & Liu, Q. (2007). Research and development of web-based virtual online classroom. Computers and Education, 48(2), 171–184.
Yukawa, J. (2006). Co-reflection in online learning: Collaborative critical thinking as narrative. International Journal of Computer-Supported Collaborative Learning, 1, 203–228.
Acknowledgments
Preliminary findings from this research were published in Jeong and Hmelo-Silver (2010a, 2011). This research was funded in part by the National Research Foundation of Korea under Grant No. 2009-0068919 awarded to the first author and also by the US National Science Foundation under Grant No. 1249492 awarded to the first two authors. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies. We thank Andrew Walker for his assistance with the cluster analysis.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOC 160 kb)
Rights and permissions
About this article
Cite this article
Jeong, H., Hmelo-Silver, C.E. & Yu, Y. An examination of CSCL methodological practices and the influence of theoretical frameworks 2005–2009. Intern. J. Comput.-Support. Collab. Learn. 9, 305–334 (2014). https://doi.org/10.1007/s11412-014-9198-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11412-014-9198-3