Skip to main content
SpringerLink
Log in

Learning to Give a Complete Argument with a Conversational Agent: An Experimental Study in Two Domains of Argumentation

  • Conference paper
  • First Online:
Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption (EC-TEL 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13450))

Included in the following conference series:

Abstract

This paper reports a between-subjects experiment (treatment group N = 42, control group N = 53) evaluating the effect of a conversational agent that teaches users to give a complete argument. The agent analyses a given argument for whether it contains a claim, a warrant and evidence, which are understood to be essential elements in a good argument. The agent detects which of these elements is missing, and accordingly scaffolds the argument completion. The experiment includes a treatment task (Task 1) in which participants of the treatment group converse with the agent, and two assessment tasks (Tasks 2 and 3) in which both the treatment and the control group answer an argumentative question. We find that in Task 1, 36 out of 42 conversations with the agent are coherent. This indicates good interaction quality. We further find that in Tasks 2 and 3, the treatment group writes a significantly higher percentage of argumentative sentences (task 2: t(94) = 1.73, p = 0.042, task 3: t(94) = 1.7, p = 0.045). This shows that participants of the treatment group used the scaffold, taught by the agent in Task 1, outside the tutoring conversation (namely in the assessment Tasks 2 and 3) and across argumentation domains (Task 3 is in a different domain of argumentation than Tasks 1 and 2). The work complements existing research on adaptive and conversational support for teaching argumentation in essays.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/DANCEcollaborative/.

  2. 2.

    https://doi.org/10.5281/zenodo.6627040.

References

  1. Adamson, D., Dyke, G., Jang, H., Rosé, C.P.: Towards an agile approach to adapting dynamic collaboration support to student needs. Int. J. Artif. Intell. Educ. 24(1), 92–124 (2014)

    Article  Google Scholar 

  2. Afrin, T., Kashefi, O., Olshefski, C., Litman, D., Hwa, R., Godley, A.: Effective interfaces for student-driven revision sessions for argumentative writing. In: Proceedings of the CHI, pp. 1–13 (2021)

    Google Scholar 

  3. Aguiar, E.V.B., Tarouco, L.M.R., Reategui, E.: Supporting problem-solving in Mathematics with a conversational agent capable of representing gifted students’ knowledge. In: IEEE 47th HICSS, pp. 130–137. IEEE (2014)

    Google Scholar 

  4. Aleven, V., McLaughlin, E. A., Glenn, R. A., Koedinger, K.R.: Instruction based on adaptive learning technologies (2017)

    Google Scholar 

  5. Artstein, R., Poesio, M.: Inter-coder agreement for computational linguistics. Comput. Linguist. 34(4), 555–596 (2008)

    Article  Google Scholar 

  6. Chalaguine, L.A., Hunter, A.: A persuasive chatbot using a crowd-sourced argument graph and concerns. Front. AI Appl. 326, 9–20 (2020)

    Google Scholar 

  7. Dawson, V.M., Venville, G.: Teaching strategies for developing students’ argumentation skills about socioscientific issues in high school genetics. Res. Sci. Educ. 40(2), 133–148 (2010). https://doi.org/10.1007/s11165-008-9104-y

    Article  Google Scholar 

  8. Daxenberger, J., Schiller, B., Stahlhut, C., Kaiser, E., Gurevych, I.: ArgumenText: argument classification and clustering in a generalized search scenario. Datenbank-Spektr. 20(2), 115–121 (2020)

    Article  Google Scholar 

  9. Duschl, R.A., Osborne, J.: Supporting and promoting argumentation discourse in science education. Stud. Sci. Educ. 38(1), 39–72 (2002). https://doi.org/10.1080/03057260208560187. Routledge

  10. Erduran, S.: Promoting ideas, evidence and argument in initial science teacher training. Sch. Sci. Rev. 87(321), 45 (2006)

    Google Scholar 

  11. Erduran, S.: Methodological foundations in the study of argumentation in science classrooms. In: Erduran, S., Jiménez-Aleixandre, M.P. (eds.) Argumentation in Science Education. Science & Technology Education Library, vol. 35, pp. 47–69. Springer, Dordrecht (2007). https://doi.org/10.1007/978-1-4020-6670-2_3

    Chapter  Google Scholar 

  12. Erduran, S., Ardac, D., Yakmaci-Guzel, B.: Learning to teach argumentation: case studies of pre-service secondary science teachers. Eur. J. Math. Sci. Technol. Educ. 2(2), 1–14 (2006)

    Google Scholar 

  13. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76(5), 378 (1971)

    Article  Google Scholar 

  14. Flesch, R.: Marks of readable style: a study in adult education, no. 897. Teachers College Contributions to Education (1943)

    Google Scholar 

  15. Georgiou, M., Mavrikaki, E., Halkia, K., Papassideri, I.: Investigating the impact of the duration of engagement in socioscientific issues in developing Greek students’ argumentation and informal reasoning skills. Am. J. Educ. Res. 8(1), 16–23 (2020)

    Google Scholar 

  16. Habernal, I., Gurevych, I.: Argumentation mining in user-generated web discourse. Comput. Linguist. 43(1), 125–179 (2017)

    Article  MathSciNet  Google Scholar 

  17. Jagtap, R., Phulare, K., Kurhade, M., Gawande, K.S.: Healthcare conversational chatbot for medical diagnosis. In: Handbook of Research on Engineering, Business, and Healthcare Applications of Data Science, pp. 401–415. IGI Global (2021)

    Google Scholar 

  18. Kelly, T., Weaver, R.: The goal structuring notation-a safety argument notation. In: Proceedings of the Dependable Systems and Networks Workshop on Assurance Cases, p. 6. Citeseer (2004)

    Google Scholar 

  19. Kuhn, D.: Thinking as argument. Harv. Educ. Rev. 62(2), 155–179 (1992)

    Article  Google Scholar 

  20. Kuhn, D.: Science as argument: implications for teaching and learning scientific thinking. Sci. Educ. 77(3), 319–337 (1993)

    Article  Google Scholar 

  21. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)

    Article  Google Scholar 

  22. Le, D.T., Nguyen, C.T., Nguyen, K.A.: Dave the debater: a retrieval-based and generative argumentative dialogue agent, pp. 121–130 (2018)

    Google Scholar 

  23. Mirzababaei, B., Pammer-Schindler, V.: Developing a conversational agent’s capability to identify structural wrongness in arguments based on Toulmin’s model of arguments. Front. Artif. Intell. 4 (2021). https://doi.org/10.3389/frai.2021.645516. https://www.frontiersin.org/articles/10.3389/frai.2021.645516. ISSN 2624-8212

  24. Mitrovic, A.: An intelligent SQL tutor on the web. Int. J. Artif. Intell. Educ. 13(2–4), 173–197 (2003)

    Google Scholar 

  25. Nakayama, H., Kubo, T., Kamura, J., Taniguchi, Y., Liang, X.: Doccano: text annotation tool for human (2018)

    Google Scholar 

  26. Neto, A.J.M., Fernandes, M.A.: Chatbot and conversational analysis to promote collaborative learning in distance education. In: 2019 IEEE 19th International Conference on Advanced Learning Technologies, vol. 2161, pp. 324–326. IEEE (2019)

    Google Scholar 

  27. Rakshit, G., Bowden, K.K., Reed, L., Misra, A., Walker, M.: Debbie, the debate bot of the future. In: Eskenazi, M., Devillers, L., Mariani, J. (eds.) Advanced Social Interaction with Agents. LNEE, vol. 510, pp. 45–52. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-92108-2_5

    Chapter  Google Scholar 

  28. Suleman, R.M., Mizoguchi, R., Ikeda, M.: A new perspective of negotiation-based dialog to enhance metacognitive skills in the context of open learner models. Int. J. Artif. Intell. Educ. 26(4), 1069–1115 (2016)

    Article  Google Scholar 

  29. Toulmin, S.E.: The Uses of Argument. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  30. Van Eemeren, F.H., Grootendorst, R., Kruiger, T.: Handbook of Argumentation Theory: A Critical Survey of Classical Backgrounds and Modern Studies, vol. 7. De Gruyter Mouton (2019)

    Google Scholar 

  31. Wambsganss, T., Kueng, T., Soellner, M., Leimeister, J.M.: ArgueTutor: an adaptive dialog-based learning system for argumentation skills. In: Proceedings of the CHI, pp. 1–13 (2021)

    Google Scholar 

  32. Wang, W., Arya, D., Novielli, N., Cheng, J., Guo, J.L.: ArguLens: anatomy of community opinions on usability issues using argumentation models. In: Proceedings of the ACM 2020, vol. 20, pp. 1–14 (2020)

    Google Scholar 

  33. Winkler, R., Söllner, M., Leimeister, J.M.: Enhancing problem-solving skills with smart personal assistant technology. Comput. Educ. 165, 104148 (2021)

    Article  Google Scholar 

  34. Wolfbauer, I., Pammer-Schindler, V., Rosé, C.: Rebo junior: analysis of dialogue structure quality for a reflection guidance chatbot. In: Proceedings of the EC-TEL 15th, pp. 14–18 (2020)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the “DDAI” COMET Module within the COMET—Competence Centers for Excellent Technologies Program, funded by the Austrian Federal Ministry (BMK and BMDW), the Austrian Research Promotion Agency (FFG), the province of Styria (SFG) and partners from industry and academia. The COMET Program is managed by FFG.

Author information

Authors and Affiliations

  1. Know-Center GmbH, Graz, Austria

    Behzad Mirzababaei & Viktoria Pammer-Schindler

  2. Graz University of Technology, Graz, Austria

    Viktoria Pammer-Schindler

Authors
  1. Behzad Mirzababaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Viktoria Pammer-Schindler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Behzad Mirzababaei .

Editor information

Editors and Affiliations

  1. Pontificia Universidad Católica de Chile, Santiago, Chile

    Isabel Hilliger

  2. Universidad Carlos III de Madrid, Madrid, Spain

    Pedro J. Muñoz-Merino

  3. KU Leuven, Leuven, Belgium

    Tinne De Laet

  4. Universidad de Valladolid, Valladolid, Spain

    Alejandro Ortega-Arranz

  5. The Open University, Milton Keynes, UK

    Tracie Farrell

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mirzababaei, B., Pammer-Schindler, V. (2022). Learning to Give a Complete Argument with a Conversational Agent: An Experimental Study in Two Domains of Argumentation. In: Hilliger, I., Muñoz-Merino, P.J., De Laet, T., Ortega-Arranz, A., Farrell, T. (eds) Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption. EC-TEL 2022. Lecture Notes in Computer Science, vol 13450. Springer, Cham. https://doi.org/10.1007/978-3-031-16290-9_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16290-9_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16289-3

  • Online ISBN: 978-3-031-16290-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation