Thiemo Wambsganss
Jan Marco Leimeister
ABSTRACT
Recent advances in Natural Language Processing (NLP) bear the opportunity to analyze the argumentation quality of texts. This can be leveraged to provide students with individual and adaptive feedback in their personal learning journey. To test if individual feedback on students' argumentation will help them to write more convincing texts, we developed AL, an adaptive IT tool that provides students with feedback on the argumentation structure of a given text. We compared AL with 54 students to a proven argumentation support tool. We found students using AL wrote more convincing texts with better formal quality of argumentation compared to the ones using the traditional approach. The measured technology acceptance provided promising results to use this tool as a feedback application in different learning settings. The results suggest that learning applications based on NLP may have a beneficial use for developing better writing and reasoning for students in traditional learning settings.
Supplemental Material
- Rob Abbott, Brian Ecker, Pranav Anand, and Marilyn Walker. 2016. Internet Argument Corpus 2.0: An SQL schema for Dialogic Social Media and the Corpora to go with it. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association (ELRA), Paris, France.Google Scholar
- Ritu Agarwal and Elena Karahanna. 2000. Time Flies When You're Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage. MIS Quarterly 24, 4 (12 2000), 665. DOI: http://dx.doi.org/10.2307/3250951Google ScholarDigital Library
- Ahmet Aker, Alfred Sliwa, Yuan Ma, Ruishen Lui, Niravkumar Borad, Seyedeh Ziyaei, and Mina Ghobadi. 2017. What works and what does not: Classifier and feature analysis for argument mining. In Proceedings of the 4th Workshop on Argument Mining. Association for Computational Linguistics, Copenhagen, Denmark, 91--96. DOI: http://dx.doi.org/10.18653/v1/W17--5112Google ScholarCross Ref
- S. J. Ashford. 1986. Feedback-Seeking in Individual Adaptation : A Resource Perspective. Academy of Management Journal 29, 3 (9 1986), 465--487. DOI: http://dx.doi.org/10.2307/256219Google ScholarCross Ref
- Paul Black and Dylan Wiliam. 2009. Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability 21, 1 (2009), 5--31. DOI: http://dx.doi.org/10.1007/s11092-008--9068--5Google ScholarCross Ref
- Elena Cabrio and Serena Villata. 2014. Towards a Benchmark of Natural Language Arguments. CoRR abs/1405.0941 (2014).Google Scholar
- Glenn Rowe Chris Reed, Raquel Mochales Palau and Marie-Francine Moens. 2008. Language Resources for Studying Argument. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08), Bente Maegaard Joseph Mariani Jan Odijk Stelios Piperidis Daniel Tapias Nicoletta Calzolari (Conference Chair), Khalid Choukri (Ed.). European Language Resources Association (ELRA), Marrakech, Morocco.Google Scholar
- Mike Cohn. 2004. User Stories Applied For Agile Software Development. Technical Report.Google Scholar
- Harris M. Cooper. 1988. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society 1, 1 (1988), 104--126. DOI: http://dx.doi.org/10.1007/BF03177550Google ScholarCross Ref
- R De Groot, R Drachman, R Hever, B Schwartz, U Hoppe, A Harrer, M De Laat, R Wegerif, B M Mclaren, and B Baurens. 2007. Computer Supported Moderation of E-Discussions: the ARGUNAUT Approach. Technical Report. http://www.argunaut.orgGoogle Scholar
- Lingjia Deng and Janyce Wiebe. 2015. MPQA 3.0: An Entity/Event-Level Sentiment Corpus. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Denver, Colorado, 1323--1328.Google ScholarCross Ref
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (10 2018). http://arxiv.org/abs/1810.04805Google Scholar
- Rosalind Driver, Paul Newton, and Jonathan Osborne. 2000. Establishing the norms of scientific argumentation in classrooms. Science Education 84, 3 (5 2000), 287--312. DOI: http://dx.doi.org/10.1002/(SICI)1098--237X(200005)84: 3287::AID-SCE13.0.CO;2-AGoogle ScholarCross Ref
- Judith Eckle-Kohler, Roland Kluge, and Iryna Gurevych. 2015. On the Role of Discourse Markers for Discriminating Claims and Premises in Argumentative Discourse. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, 2236--2242.Google ScholarCross Ref
- Frans H. van Eemeren, Rob Grootendorst, Ralph H. Johnson, Christian Plantin, Charles A. Willard, Rob Grootendorst, Ralph H. Johnson, Christian Plantin, and Charles A. Willard. 1996. Fundamentals of Argumentation Theory. Routledge. DOI: http://dx.doi.org/10.4324/9780203811306Google ScholarCross Ref
- Andrew J. Elliot and Patricia G. Devine. 1994. On the motivational nature of cognitive dissonance: Dissonance as psychological discomfort. Journal of Personality and Social Psychology 67, 3 (1994), 382--394. DOI: http://dx.doi.org/10.1037/0022--3514.67.3.382Google ScholarCross Ref
- Charles Fadel, Maya Bialik, and Bernie Trilling. 2015. Four-dimensional education : the competencies learners need to succeed. 177 pages.Google Scholar
- Leon Festinger. 1962. Cognitive Dissonance. Scientific American 207, 4 (10 1962), 93--106. DOI: http://dx.doi.org/10.1038/scientificamerican1062--93Google ScholarCross Ref
- Frank Fischer, Ingo Kollar, Karsten Stegmann, and Christof Wecker. 2013. Toward a Script Theory of Guidance in Computer-Supported Collaborative Learning. Educational psychologist 48, 1 (1 2013), 56--66. DOI: http://dx.doi.org/10.1080/00461520.2012.748005Google ScholarCross Ref
- J.L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76, 5 (1971), 378--382.Google ScholarCross Ref
- Jürgen Flender, Ursula Christmann, and Norbert Groeben. 1999. Entwicklung und erste Validierung einer Skala zur Erfassung der passiven argumentativ-rhetorischen Kompetenz. Zeitschrift für Differentielle und Diagnostische Psychologie 20, 4 (9 1999), 309--325. DOI: http://dx.doi.org/10.1024//0170--1789.20.4.309Google Scholar
- R Flesch. 1943. Marks of readable style; a study in adult education. Teachers College Contributions to Education 897 (1943), ix + 69--ix + 69.Google Scholar
- Pauline Carolyne Fortes and Abdellatif Tchantchane. 2010. Dealing with large classes: A real challenge. Procedia - Social and Behavioral Sciences 8 (2010), 272--280. DOI: http://dx.doi.org/10.1016/j.sbspro.2010.12.037Google ScholarCross Ref
- Hansjörg Fromm, Thiemo Wambsganss, and Matthias Söllner. 2019. Towards a taxonomy of text mining features. (2019), 1--12.Google Scholar
- Jochen. Gläser and Grit. Laudel. 2010. Experteninterviews und qualitative Inhaltsanalyse : als Instrumente rekonstruierender Untersuchungen. VS Verlag für Sozialwiss. http://www.springer.com/de/book/9783531172385Google Scholar
- Ivan Habernal and Iryna Gurevych. 2015. Exploiting Debate Portals for Semi-Supervised Argumentation Mining in User-Generated Web Discourse. Technical Report. 17--21 pages. https://github.com/habernal/emnlp2015Google Scholar
- John Hattie and Helen Timperley. 2007. The Power of Feedback. Review of Educational Research 77, 1 (2007), 81--112. DOI: http://dx.doi.org/10.3102/003465430298487Google ScholarCross Ref
- Chenn Jung Huang, Shun Chih Chang, Heng Ming Chen, Jhe Hao Tseng, and Sheng Yuan Chien. 2016. A group intelligence-based asynchronous argumentation learning-assistance platform. Interactive Learning Environments 24, 7 (2016), 1408--1427. DOI: http://dx.doi.org/10.1080/10494820.2015.1016533Google ScholarCross Ref
- David H. Jonassen and Bosung Kim. 2010. Arguing to learn and learning to argue: Design justifications and guidelines. Educational Technology Research and Development 58, 4 (2010), 439--457. DOI: http://dx.doi.org/10.1007/s11423-009--9143--8Google ScholarCross Ref
- Barbara Konat, John Lawrence, Joonsuk Park, Katarzyna Budzynska, and Chris Reed. 2016. A Corpus of Argument Networks: Using Graph Properties to Analyse Divisive Issues. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association (ELRA), Paris, France.Google Scholar
- Timothy Koschmann. 1996. Paradigm Shifts and Instructional Technology. Technical Report. 1--23 pages. http://opensiuc.lib.siu.edu/meded_books/4Google Scholar
- Klaus Krippendorff. 1980. Content Analysis: An Introduction to Methodology. Sage Publications, Inc., Beverly Hills, CA. http://www.amazon.ca/gp/product/0803914989/ref=wl_it_dp/702-0885532--1303250?ie=UTF8& coliid=I3UJ8HY4GH9OWF&colid=1DVGN4EKR6AVMGoogle Scholar
- Klaus Krippendorff. 2004. Measuring the Reliability of Qualitative Text Analysis Data. Quality and Quantity 38, 6 (01 Dec 2004), 787--800. DOI: http://dx.doi.org/10.1007/s11135-004--8107--7Google ScholarCross Ref
- Deanna Kuhn. 1992. Thinking as Argument. Harvard Educational Review 62, 2 (7 1992), 155--179. DOI: http://dx.doi.org/10.17763/haer.62.2.9r424r0113t670l1Google ScholarCross Ref
- Deanna. Kuhn. 2005. Education for thinking. Harvard University Press. 209 pages. http://www.hup.harvard.edu/catalog.php?isbn=9780674027459Google Scholar
- Katja Lehmann, Matthias Söllner, and Jan Marco Leimeister. 2016. Design and Evaluation of an IT-based Peer Assessment to Increase Learner Performance in Large-Scale Lectures. (12 2016). https://www.alexandria.unisg.ch/249309/Google Scholar
- Marco Lippi and Paolo Torroni. 2015. Argumentation Mining: State of the Art and Emerging Trends. IJCAI International Joint Conference on Artificial Intelligence 2015-Janua, 2 (2015), 4207--4211. DOI: http://dx.doi.org/10.1145/2850417Google ScholarDigital Library
- Jack Mezirow. 1991. Transformative dimensions of adult learning. Jossey-Bass, San Francisco, CA 94104--1310.Google Scholar
- Raquel Mochales Palau and Aagje Ieven. 2009. Creating an argumentation corpus: do theories apply to real arguments? A case study on the legal argumentation of the ECHR. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Law (ICAIL 2009), Twelfth international conference on artificial intelligence and law (ICAIL 2009)., Barcelona, Spain, 8--12 June 2009. ACM, 21--30.Google Scholar
- E. Michael Nussbaum, Denise L. Winsor, Yvette M. Aqui, and Anne M. Poliquin. 2007. Putting the pieces together: Online argumentation vee diagrams enhance thinking during discussions. International Journal of Computer-Supported Collaborative Learning 2, 4 (11 2007), 479--500. DOI: http://dx.doi.org/10.1007/s11412-007--9025--1Google ScholarCross Ref
- OECD. 2018. The Future of Education and Skills Education 2030. (2018). DOI: http://dx.doi.org/2018-06--15Google Scholar
- Jonathan F. Osborne, J. Bryan Henderson, Anna MacPherson, Evan Szu, Andrew Wild, and Shi Ying Yao. 2016. The development and validation of a learning progression for argumentation in science. Journal of Research in Science Teaching 53, 6 (2016), 821--846. DOI: http://dx.doi.org/10.1002/tea.21316Google ScholarCross Ref
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. Association for Computational Linguistics (ACL), 2227--2237. DOI: http://dx.doi.org/10.18653/v1/n18--1202Google ScholarCross Ref
- Jean Piaget, Terrance Brown, and Kishore Julian Thampy. 1986. The Equilibration of Cognitive Structures: The Central Problem of Intellectual Development. Jean Piaget , Terrance Brown , Kishore Julian Thampy. American Journal of Education 94, 4 (8 1986), 574--577. DOI: http://dx.doi.org/10.1086/443876Google ScholarCross Ref
- Niels Pinkwart, Kevin Ashley, Collin Lynch, and Vincent Aleven. 2009. Evaluating an Intelligent Tutoring System for Making Legal Arguments with Hypotheticals. Technical Report. 401--424 pages. http://iaiedsoc.org/pub/1302/file/19_4_05_Pinkwart.pdfGoogle Scholar
- Eric Ries. 2011. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. (2011). https://books.google.de/books?hl=de&lr=&id= tvfyz-4JILwC&oi=fnd&pg=PA15&dq=lean+startup+ries& ots=8H8ay99lrV&sig=NTC5ybgihYWRr6m9aT0XH-F6Ixc#v= onepage&q=leanstartupries&f=falsehttp://en.wikipedia.org/wiki/Lean_StartupGoogle Scholar
- Roman Rietsche, Kevin Duss, Jan Martin Persch, and Matthias Söllner. 2018. Design and Evaluation of an IT-based Formative Feedback Tool to Foster Student Performance Understanding and Designing Trust in Information Systems View project Future of Collaboration View project. Technical Report. https://www.researchgate.net/publication/329450233Google Scholar
- D. Royce Sadler. 1989. Formative assessment and the design of instructional systems. Instructional Science 18, 2 (6 1989), 119--144. DOI: http://dx.doi.org/10.1007/BF00117714Google ScholarCross Ref
- Christos Sardianos, Ioannis Manousos Katakis, Georgios Petasis, and Vangelis Karkaletsis. 2015. Argument Extraction from News. In Proceedings of the 2nd Workshop on Argumentation Mining. Association for Computational Linguistics, Denver, Colorado, 56--66.Google ScholarCross Ref
- Katharina Scheiter and Peter Gerjets. 2007. Learner control in hypermedia environments. Educational Psychology Review 19, 3 (2007), 285--307. DOI: http://dx.doi.org/10.1007/s10648-007--9046--3Google ScholarCross Ref
- Oliver Scheuer. 2015. Towards adaptive argumentation learning systems. https://www.researchgate.net/publication/298087259Google Scholar
- Oliver Scheuer, Frank Loll, Niels Pinkwart, and Bruce M. McLaren. 2010. Computer-supported argumentation: A review of the state of the art. International Journal of Computer-Supported Collaborative Learning 5, 1 (2010), 43--102. DOI: http://dx.doi.org/10.1007/s11412-009--9080-xGoogle ScholarCross Ref
- Parinaz Sobhani, Diana Inkpen, and Stan Matwin. 2015. From argumentation mining to stance classification. NAACL HLT 2015 (2015), 67--77.Google Scholar
- Elliot Soloway, Mark Guzdial, and Kenneth E Hay. 1994. Learner-Centered Design The Challenge For WC1 In The Xst Century. Interactions (1994), 36--48.Google Scholar
- Yi Song, Michael Heilman, Beata Beigman Klebanov, and Paul Deane. 2014. Applying Argumentation Schemes for Essay Scoring. Technical Report. 69--78 pages. http://acl2014.org/acl2014/W14--21/pdf/W14--2110.pdfGoogle Scholar
- Christian Stab and Iryna Gurevych. 2014a. Annotating Argument Components and Relations in Persuasive Essays. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers ,. 1501--1510. http://www.ukp.tu-darmstadt.deGoogle Scholar
- Christian Stab and Iryna Gurevych. 2014b. Identifying Argumentative Discourse Structures in Persuasive Essays. In Conference on Empirical Methods in Natural Language Processing (EMNLP 2014)(Oct. 2014), Association for Computational Linguistics. 46--56. www.ukp.tu-darmstadt.deGoogle Scholar
- Christian Stab and Iryna Gurevych. 2017a. Parsing Argumentation Structures in Persuasive Essays. Computational Linguistics 43, 3 (9 2017), 619--659. DOI: http://dx.doi.org/10.1162/COLI{_}a{_}00295Google ScholarCross Ref
- Christian Stab and Iryna Gurevych. 2017b. Recognizing Insufficiently Supported Arguments in Argumentative Essays. Technical Report. 980--990 pages. www.ukp.tu-darmstadt.deGoogle Scholar
- Karsten Stegmann, Christof Wecker, Armin Weinberger, and Frank Fischer. 2012. Collaborative argumentation and cognitive elaboration in a computer-supported collaborative learning environment. Instructional Science 40, 2 (2012), 297--323. DOI: http://dx.doi.org/10.1007/s11251-011--9174--5Google ScholarCross Ref
- Pontus Stenetorp, Sampo Pyysalo, and Goran Topi. 2012. BRAT : a Web-based Tool for NLP-Assisted Text Annotation. Figure 1 (2012), 102--107.Google Scholar
- Daniel D Suthers and Christopher D Hundhausen. 2001. European Perspectives on Computer-Supported Collaborative Learning. Technical Report. 577--584 pages. http://lilt.ics.hawaii.edu/papers/2001/Suthers-Hundhausen-Euro-CSCL-2001.pdfGoogle Scholar
- Stephen E. Toulmin. 1984. Introduction to Reasoning.Google Scholar
- Viswanath Venkatesh and Hillol Bala. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences 39, 2 (5 2008), 273--315. DOI: http://dx.doi.org/10.1111/j.1540--5915.2008.00192.xGoogle ScholarCross Ref
- Viswanath Venkatesh, Michael G Morris, Gordon B Davis, and Fred D Davis. 2003. User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly 27, 3 (2003), 425--478.Google ScholarCross Ref
- Jan vom Brocke, Wolfgang Maaß, Peter Buxmann, Alexander Maedche, Jan Marco Leimeister, and Günter Pecht. 2018. Future Work and Enterprise Systems. Business and Information Systems Engineering 60, 4 (2018), 357--366. DOI: http://dx.doi.org/10.1007/s12599-018-0544--2Google ScholarCross Ref
- Jan vom Brocke, Alexander Simons, Kai Riemer, Björn Niehaves, Ralf Plattfaut, and Anne Cleven. 2015. Standing on the Shoulders of Giants: Challenges and Recommendations of Literature Search in Information Systems Research. Communications of the Association for Information Systems 37, 1 (8 2015). DOI: http://dx.doi.org/10.17705/1CAIS.03709Google ScholarCross Ref
- Henning Wachsmuth, Martin Trenkmann, Benno Stein, Gregor Engels, and Tsvetomira Palakarska. 2014. A Review Corpus for Argumentation Analysis. In Proceedings of the 15th International Conference on Computational Linguistics and Intelligent Text Processing - Volume 8404 (CICLing 2014). Springer-Verlag New York, Inc., New York, NY, USA, 115--127.Google ScholarDigital Library
- Marilyn Walker, Jean Fox Tree, Pranav Anand, Rob Abbott, and Joseph King. 2012. A Corpus for Research on Deliberation and Debate. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012). Istanbul, Turkey, 812--817. ACL Anthology Identifier: L12--1643.Google Scholar
- Thiemo Wambsganss and Roman Rietsche. 2019. Towards Designing an Adaptive Argumentation Learning Tool. International Conference on Information Systems (ICIS) 2019, 1--9.Google Scholar
- Armin Weinberger and Frank Fischer. 2006. A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education 46, 1 (2006), 71--95. DOI: http://dx.doi.org/10.1016/j.compedu.2005.04.003Google ScholarDigital Library
Index Terms
- AL: An Adaptive Learning Support System for Argumentation Skills
Recommendations
Designing Adaptive Argumentation Learning Systems Based on Artificial Intelligence
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsArgumentation skills are an omnipresent foundation of our daily communication and thinking. However, the learning of argumentation skills is limited due to the lack of individual learning conditions for students. Within this dissertation, I aim to ...
ArgueTutor: An Adaptive Dialog-Based Learning System for Argumentation Skills
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsTechniques from Natural-Language-Processing offer the opportunities to design new dialog-based forms of human-computer interaction as well as to analyze the argumentation quality of texts. This can be leveraged to provide students with adaptive ...
Improving Students Argumentation Learning with Adaptive Self-Evaluation Nudging
CSCWRecent advantages from computational linguists can be leveraged to nudge students with adaptive self-evaluation based on their argumentation skill level. To investigate how individual argumentation self-evaluation will help students write more ...
Comments