Skip to main content
Springer Nature Link
Log in

Empirical Study on Human Evaluation of Complex Argumentation Frameworks

  • Conference paper
  • First Online:
Logics in Artificial Intelligence (JELIA 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11468))

Included in the following conference series:

Abstract

In abstract argumentation, multiple argumentation semantics have been proposed that allow to select sets of jointly acceptable arguments from a given argumentation framework, i.e. based only on the attack relation between arguments. The existence of multiple argumentation semantics raises the question which of these semantics predicts best how humans evaluate arguments. Previous empirical cognitive studies that have tested how humans evaluate sets of arguments depending on the attack relation between them have been limited to a small set of very simple argumentation frameworks, so that some semantics studied in the literature could not be meaningfully distinguished by these studies. In this paper we report on an empirical cognitive study that overcomes these limitations by taking into consideration twelve argumentation frameworks of three to eight arguments each. These argumentation frameworks were mostly more complex than the argumentation frameworks considered in previous studies. All twelve argumentation framework were systematically instantiated with natural language arguments based on a certain fictional scenario, and participants were shown both the natural language arguments and a graphical depiction of the attack relation between them. Our data shows that grounded and CF2 semantics were the best predictors of human argument evaluation. A detailed analysis revealed that part of the participants chose a cognitively simpler strategy that is predicted very well by grounded semantics, while another part of the participants chose a cognitively more demanding strategy that is mostly predicted well by CF2 semantics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Anastasi, A.: Differential Psychology: Individual and Group Differences in Behavior. Macmillan, London (1958)

    Google Scholar 

  2. Augustinova, M.: Falsification cueing in collective reasoning: example of the Wason selection task. Eur. J. Soc. Psychol. 38(5), 770–785 (2008)

    Article  Google Scholar 

  3. Baroni, P., Caminada, M., Giacomin, M.: An introduction to argumentation semantics. Knowl. Eng. Rev. 26(4), 365–410 (2011)

    Article  Google Scholar 

  4. Baroni, P., Caminada, M., Giacomin, M.: Abstract argumentation frameworks and their semantics. In: Baroni, P., Gabbay, D., Giacomin, M., van der Torre, L. (eds.) Handbook of Formal Argumentation, pp. 159–236. College Publications (2018)

    Google Scholar 

  5. Baroni, P., Giacomin, M., Guida, G.: SCC-recursiveness: a general schema for argumentation semantics. Artif. Intell. 168(1), 162–210 (2005)

    Article  MathSciNet  Google Scholar 

  6. Cerutti, F., Tintarev, N., Oren, N.: Formal arguments, preferences, and natural language interfaces to humans: an empirical evaluation. In: Schaub, T., Friedrich, G., O’Sullivan, B. (eds.) Proceedings of the 21st ECAI 2014, pp. 207–212 (2014)

    Google Scholar 

  7. Cramer, M., Guillaume, M.: Directionality of attacks in natural language argumentation. In: Schon, C. (ed.) Proceedings of the Workshop on Bridging the Gap between Human and Automated Reasoning, vol. 2261, pp. 40–46. RWTH Aachen University, CEUR-WS.org (2018). http://ceur-ws.org/Vol-2261/

  8. Cramer, M., Guillaume, M.: Empirical cognitive study on abstract argumentation semantics. Frontiers in Artificial Intelligence and Applications, pp. 413–424 (2018)

    Google Scholar 

  9. Cramer, M., Guillaume, M.: Technical report of “Empirical study on human evaluation of complex argumentation frameworks”. arXiv:1902.10552, February 2019

  10. Cronbach, L.J.: The two disciplines of scientific psychology. Am. Psychol. 12(11), 671–684 (1957)

    Article  Google Scholar 

  11. Dung, P.M.: On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artif. Intell. 77(2), 321–357 (1995)

    Article  MathSciNet  Google Scholar 

  12. Dvorák, W., Gaggl, S.A.: Incorporating stage semantics in the SCC-recursive schema for argumentation semantics. In: Proceedings of the 14th International Workshop on Non-Monotonic Reasoning (NMR 2012) (2012)

    Google Scholar 

  13. Dvořák, W., Gaggl, S.A.: Stage semantics and the SCC-recursive schema for argumentation semantics. J. Log. Comput. 26(4), 1149–1202 (2016)

    Article  MathSciNet  Google Scholar 

  14. Evans, J.S.B., Newstead, S.E., Byrne, R.M.: Human Reasoning: The Psychology of Deduction. Psychology Press, London (1993)

    Google Scholar 

  15. Geil, D.M.M.: Collaborative reasoning: evidence for collective rationality. Think. Reason. 4(3), 231–248 (1998)

    Article  Google Scholar 

  16. Hunter, A., Polberg, S.: Empirical methods for modelling persuadees in dialogical argumentation. In: 29th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 382–389. IEEE (2017)

    Google Scholar 

  17. Modgil, S., Prakken, H.: The ASPIC+ framework for structured argumentation: a tutorial. Argum. Comput. 5(1), 31–62 (2014)

    Article  Google Scholar 

  18. Piaget, J., Smith, L., Brown, T., Campbell, R., Emler, N., Ferrari, D.: Sociological Studies. Routledge, London (1995)

    Google Scholar 

  19. Polberg, S., Hunter, A.: Empirical evaluation of abstract argumentation: supporting the need for bipolar and probabilistic approaches. Int. J. Approx. Reason. 93, 487–543 (2018)

    Article  MathSciNet  Google Scholar 

  20. Rahwan, I., Madakkatel, M.I., Bonnefon, J.-F., Awan, R.N., Abdallah, S.: Behavioral experiments for assessing the abstract argumentation semantics of reinstatement. Cogn. Sci. 34(8), 1483–1502 (2010)

    Article  Google Scholar 

  21. Rahwan, I., Simari, G.R.: Argumentation in Artificial Intelligence, 1st edn. Springer, Boston (2009). https://doi.org/10.1007/978-0-387-98197-0

    Book  Google Scholar 

  22. Rosenfeld, A., Kraus, S.: Providing arguments in discussions on the basis of the prediction of human argumentative behavior. ACM Trans. Interact. Intell. Syst. 6(4), 30:1–30:33 (2016)

    Article  Google Scholar 

  23. Verheij, B.: Two approaches to dialectical argumentation: admissible sets and argumentation stages. Proc. NAIC 96, 357–368 (1996)

    Google Scholar 

  24. Wason, P.C.: Reasoning. In: Foss, B. (ed.) New Horizons in Psychology, pp. 135–151. Penguin Books, Harmondsworth (1966)

    Google Scholar 

  25. Weiner, B.: Theories of Motivation: From Mechanism to Cognition. Markham Psychology Series. Markham Publishing Co., Chicago (1972)

    Google Scholar 

  26. Wu, Y., Caminada, M.: A labelling-based justification status of arguments. Stud. Log. 3(4), 12–29 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Center for Computational Logic, TU Dresden, Dresden, Germany

    Marcos Cramer

  2. Centre for Research in Cognitive Neuroscience (CRCN), Université Libre de Bruxelles, Brussels, Belgium

    Mathieu Guillaume

Authors
  1. Marcos Cramer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathieu Guillaume
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcos Cramer .

Editor information

Editors and Affiliations

  1. University of Calabria, Rende, Italy

    Francesco Calimeri

  2. University of Calabria, Rende, Italy

    Nicola Leone

  3. Mathematics and Computer Science, University of Calabria, Rende, Italy

    Marco Manna

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cramer, M., Guillaume, M. (2019). Empirical Study on Human Evaluation of Complex Argumentation Frameworks. In: Calimeri, F., Leone, N., Manna, M. (eds) Logics in Artificial Intelligence. JELIA 2019. Lecture Notes in Computer Science(), vol 11468. Springer, Cham. https://doi.org/10.1007/978-3-030-19570-0_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-19570-0_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-19569-4

  • Online ISBN: 978-3-030-19570-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics