Skip to main content
SpringerLink
Log in

Abductive Theorem Proving for Analyzing Student Explanations to Guide Feedback in Intelligent Tutoring Systems

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

The Why2-Atlas tutoring system presents students with qualitative physics questions and encourages them to explain their answers through natural language. Although there are inexpensive techniques for analyzing explanations, we claim that better understanding is necessary for use within tutoring systems. In this paper we motivate and describe how the system creates and uses a deeper proof-based representation of student essays in order to provide students with substantive feedback on their explanations. We describe in detail the abductive reasoner, Tacitus-lite+, that we use within the tutoring system. We also discuss evaluation results for an early version of the Why2-Atlas system and a subsequent evaluation of the theorem-proving module. We conclude with the discussion of work in progress and additional future work for deriving more benefits from a proof-based approach for tutoring applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aleven, V. and Koedinger, K. R. (2000) The need for tutorial dialog to support self-explanation, in Building Dialogue System for Tutorial Applications, Papers of the 2000 AAAI Fall Symposium.

  • Aleven, V., Popescu, O. and Koedinger, K. (2002) Pilot-testing a tutorial dialogue system that sup-ports self-explanation, in Proceedings of Intelligent Tutoring Systems Conference, LNCS 2363, Springer, pp. 344–354.

  • Aleven, V., Popescu, O. and Koedinger, K. R. (2001a) Toward tutorial dialog to support self-explanation: Adding natural language understanding to a cognitive tutor, in Proceedings of 10th International Conference on Artificial Intelligence in Education (AI-ED 2001), IOS Press, Amsterdam, pp. 246–255.

    Google Scholar 

  • Aleven, V., Popescu, O. and Koedinger, K. R. (2001b) A tutorial dialogue system with knowledge-based understanding and classification of student explanations, in Working Notes of 2nd IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems.

  • Appelt, D. and Pollack, M. (1992) Weighted abduction for plan ascription, User Modeling and User-Adapted Interaction 2(1-2), 1–25

    Article  Google Scholar 

  • Austin, J. L. (1962) How to Do Things with Words, Oxford University Press, Oxford.

    Google Scholar 

  • Bacchus, F., Tennenberg, J. and Koomen, J. (1989) A non-reified temporal logic, in J. F. Allen, R. Fikes and E. Sandewall (eds.), KR'89: Principles of Knowledge Representation and Reasoning, San Mateo, California, Morgan Kaufmann, pp. 2–10.

    Google Scholar 

  • Bylander, T., Allemang, D., Tanner, M. C. and Josephson, J. R. (1991) The computational complexity of abduction, Artificial Intelligence 49(1-3), 25–60.

    Article  MATH  MathSciNet  Google Scholar 

  • Charniak, E. (1986) A neat theory of marker passing, in Proceedings of the 5th National Conference on Artificial Intelligence (AAAI'86), pp. 584–588.

  • Charniak, E. and Shimony, S. E. (1990) Probabilistic semantics for cost-based abduction, in Proceedings of AAAI-90, pp. 106–111.

  • Charniak, E. and Shimony, S. E. (1994) Cost-based abduction and MAP explanation, Artificial Intelligence 66, 345–374.

    Article  MATH  MathSciNet  Google Scholar 

  • Chi, M. T. H. and Ceci, S. J. (1987) Content knowledge: Its role, representation and restructuring in memory development, Advances in Child Development and Behavior 20, 91–142.

    Google Scholar 

  • Chi, M. T. H., de Leeuw, N., Chiu, M.-H. and LaVancher, C. (1994) Eliciting self-explanations improves understanding, Cognitive Science 18, 439–477.

    Article  Google Scholar 

  • Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T. and Hausmann, R. G. (2001) Learning from human tutoring, Cognitive Science 25(4), 471–533.

    Article  Google Scholar 

  • Cohn, A. G. (1989) Taxonomic reasoning with many-sorted logics, Artificial Intelligence 3, 89–128.

    Google Scholar 

  • Conati, C., Gertner, A. and VanLehn, K. (2002) Using Bayesian networks to manage uncertainty in student modeling, J. User Modeling and User-Adapted Interaction 12(4).

  • de Kleer, J. (1990) Multiple representations of knowledge in a mechanics problem-solver, in D. S. Weld and J. de Kleer (eds.), Readings in Qualitative Reasoning about Physical Systems, San Mateo, California, Morgan Kaufmann, pp. 40–45.

    Google Scholar 

  • Eiter, T. and Gottlob, G. (1993) The complexity of logic-based abduction, in Symposium on Theoretical Aspects of Computer Science, pp. 70–79.

  • Eshghi, K. (1993) A tractable class of abduction problems, in Proceedings 13th International Joint Conference on Artificial Intelligence, Chambery, France, pp. 3–8.

    Google Scholar 

  • Eshghi, K. and Kowalski, R. A. (1989) Abduction compared with negation by failure, in Proceedings of the 6th International Conference on Logic Programming (ICLP '89), pp. 234–254.

  • Forbus, K., Carney, K., Harris, R. and Sherin, B. (2001) A qualitative modeling environment for middle-school students: A progress report, in QR-01.

  • Forbus, K. D. (1997) Using qualitative physics to create articulate educational software, IEEE Expert, pp. 32–41.

  • Frisch, A. M. (1991) The substitutional framework for sorted deduction: Fundamental results on hybrid reasoning, Artificial Intelligence 49(1-3), 161–198.

    Article  MATH  MathSciNet  Google Scholar 

  • Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N. and the TRG (2000) Using latent semantic analysis to evaluate the contributions of students in AutoTutor, Interactive Learning Environments 8, 129–148.

    Article  Google Scholar 

  • Hake, R. R. (1998) Interactive-engagement versus traditional methods: A six-thousand student survey of mechanics test data for introductory physics students, American Journal of Physics 66(4), 64–74.

    Article  Google Scholar 

  • Haugh, B. (1987) Non-standard semantics for the method of temporal arguments, in Proc. of IJCAI'87, pp. 449–454.

  • Hestenes, D., Wells, M. and Swackhamer, G. (1992) Force concept inventory, The Physics Teacher 30, 141–158.

    Google Scholar 

  • Hewitt, P. G. (1998) Conceptual Physics, 8th edn, Addison-Wesley.

  • Hobbs, J., Stickel, M., Appelt, D. and Martin, P. (1993) Interpretation as abduction, Artificial Intelligence 63(1-2), 69–142.

    Article  Google Scholar 

  • Hobbs, J., Stickel, M., Martin, P. and Edwards, D. (1988) Interpretation as abduction, in Proc. 26th Annual Meeting of the ACL, Association of Computational Linguistics, pp. 95–103

  • Horacek, H. (1997) A model for adapting explanations to users' likely inferences, User Modeling and User-Adapted Interaction 7(1), 1–55.

    Article  MathSciNet  Google Scholar 

  • Jordan, P. and VanLehn, K. (2002) Discourse processing for explanatory essays in tutorial applications, in Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue.

  • Jordan, P. W. (2004) Using student explanations as models for adapting tutorial dialogues, in Proceedings of 17th International FLAIRS Conference.

  • Kakas, A., Kowalski, R. A. and Toni, F. (1998) The role of abduction in logic programming, in D. M. Gabbay, C. J. Hogger and J. A. Robinson (eds.), Handbook of Logic in Artificial Intelligence and Logic Programming, Vol. 5, Oxford University Press, pp. 235–324.

  • Kaneiwa, K. and Tojo, S. (2001) An order-sorted resolution with implicitly negative sorts, in Proceedings of the 2001 International Conference on Logic Programming (ICLP'01), LNCS 2237, Springer, pp. 300–314.

  • Keeney, R. and Raiffa, H. (1976) Decisions with Multiple Objectives, Wiley.

  • Landauer, T. K., Foltz, P. W. and Laham, D. (1998) An introduction to latent semantic analysis, Discourse Processes 25, 259–284.

    Article  Google Scholar 

  • Lascarides, A. and Asher, N. (1991) Discourse relations and defeasible knowledge, in 29th Annual Meeting of the Association for Computational Linguistics, pp. 55–62.

  • Lavoie, B. and Rambow, O. (1997) A fast and portable realizer for text generation systems, in Proceedings of the Fifth Conference on Applied Natural Language Processing Chapter of the Association for Computational Linguistics, Washington, DC, pp. 265–268.

  • Leake, D. (1995) Abduction, experience, and goals: A model of everyday abductive explanation, J. Experimental and Theoretical Artificial Intelligence 7, 407–428.

    Google Scholar 

  • McCallum, A. and Nigam, K. (1998) A comparison of event models for naive Bayes text classification, in Proceedings of AAAI/ICML-98 Workshop on Learning for Text Categorization, AAAI Press.

  • McRoy, S. and Hirst, G. (1995) The repair of speech act misunderstandings by abductive inference, Computational Linguistics 21(4), 435–478.

    Google Scholar 

  • Murray, R. C. and VanLehn, K. (2000) DT tutor: A dynamic decision-theoretic approach for optimal selection of tutorial actions, in Proceedings of Intelligent Tutoring Systems Conference,LNCS 1839, Springer, pp. 153–162.

  • Ng, V. and Cardie, C. (2002) Improving machine learning approaches to coreference resolution, in Proceedings of Association for Computational Linguistics 2002.

  • Paul, G. (1993) Approaches to abductive reasoning-An overview, Artificial Intelligence Review 7(2), 109–152.

    Article  Google Scholar 

  • Ploetzner, R. and VanLehn, K. (1997) The acquisition of qualitative physics knowledge during textbook-based physics training, Cognition and Instruction 15(2), 169–205.

    Article  Google Scholar 

  • Poole, D. (1993) Probabilistic Horn abduction and Bayesian networks, Artificial Intelligence 64(1), 81–129.

    Article  MATH  Google Scholar 

  • Rayner, M. and Alshawi, H. (1992) Deriving database queries from logical forms by abductive definition expansion, in Proceedings of the Third Conference of Applied Natural Language Processing, Trento, Italy, pp. 1–8.

    Google Scholar 

  • Rosé, C., Bhembe, D., Roque, A., Siler, S., Srivastava, R. and VanLehn, K. (2002) A hybrid understanding approach for robust selection of tutoring goals, in Proceedings of Intelligent Tutoring Systems Conference, LNCS 2363, Springer, pp. 552–561.

  • Rosé, C., Jordan, P., Ringenberg, M., Siler, S., VanLehn, K. and Weinstein, A. (2001) Interactive conceptual tutoring in Atlas-Andes, in Proceedings of AI in Education 2001 Conference.

  • Rosé, C., Roque, A., Bhembe, D. and VanLehn, K. (2002) An efficient incremental architecture for robust interpretation, in Proceedings of Human Language Technology Conference, San Diego, CA.

  • Santos, Jr., E. and Santos, E. S. (1996) Polynomial solvability of cost-based abduction, Artificial Intelligence 86(1), 157–170

    Article  MathSciNet  Google Scholar 

  • Schmidt-Schauß, M. (1989) Computational Aspects of an Order-Sorted Logic with Term Declarations, Springer.

  • Searle, J. R. (1975) Indirect speech acts, in P. Cole and J. Morgan (eds.), Syntax and Seman-tics 3: Speech Acts, Academic Press. Reprinted in S. Davis (ed.), Pragmatics. A Reader, Oxford University Press, 1991.

  • Selman, B. and Levesque, H. J. (1990) Abductive and default reasoning: A computational core, in Proceedings of AAAI-90, Boston, MA, pp. 343–348.

  • Slotta, J. D., Chi, M. T. and Joram, E. (1995) Assessing students' misclassifications of physics concepts: An ontological basis for conceptual change, Cognition and Instruction 13(3), 373–400.

    Article  Google Scholar 

  • Stickel, M. (1988) A Prolog-like inference system for computing minimum-cost abductive explanations in natural-language interpretation, Technical Report 451, SRI International, 333 Ravenswood Ave., Menlo Park, California.

  • Strube, M., Rapp, S. and Müller, C. (2002) The influence of minimum edit distance on reference resolution, in Proceedings of Empirical Methods in Natural Language Processing Conference.

  • Thomason, R. H., Hobbs, J. and Moore, J. D. (1996) Communicative goals, in K. Jokinen, M. May-bury, M. Zock and I. Zukerman (eds.), Proceedings of the ECAI 96 Workshop Gaps and Bridges: New Directions in Planning and Natural Language Generation.

  • VanLehn, K., Jordan, P., Rosé, C., Bhembe, D., Böttner, M., Gaydos, A., Makatchev, M., Pap-puswamy, U., Ringenberg, M., Roque, A., Siler, S. and Srivastava, R. (2002) The architecture of Why2-Atlas: A coach for qualitative physics essay writing, in Proceedings of Intelligent Tutoring Systems Conference, LNCS 2363, Springer, pp. 158–167.

  • Wahlster, W. and Kobsa, A. (1989) User models in dialogue systems, in A. Kobsa and W. Wahlster (eds.), User Models in Dialogue Systems, Springer Verlag, Berlin, pp. 4–34.

    Google Scholar 

  • Walther, C. (1987) A Many-Sorted Calculus Based on Resolution and Paramodulation, Morgan Kaufmann, Los Altos, California.

    Google Scholar 

  • Weld, D. S. and de Kleer, J. (eds.) (1990) Readings in Qualitative Reasoning about Physical Systems, Morgan Kaufmann, San Mateo, California.

    Google Scholar 

  • Zukerman, I. and Albrecht, D. W. (2001) Predictive statistical models for user modeling, User Modeling and User-Adpated Interaction 11(1-2), 5–18.

    Article  MATH  Google Scholar 

  • Zukerman, I. and George, S. (2002) A minimum message length approach for argument interpretation, in Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue.

  • Zukerman, I. and McConachy, R. (1993) Generating concise discourse that addresses a user's infer-ences, in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers, Inc., pp. 1202–1207.

  • Zukerman, I., McConachy, R. and Korb, K. B. (2000) Using argumentation strategies in automated argument generation, in Proceedings of the 1st International Natural Language Generation Conference, pp. 55–62.

Download references

Author information

Authors and Affiliations

  1. Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA 15260, U.S.A. e-mail

    Maxim Makatchev, Pamela W. Jordan & Kurt VanLehn

Authors
  1. Maxim Makatchev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pamela W. Jordan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kurt VanLehn
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Makatchev, M., Jordan, P.W. & VanLehn, K. Abductive Theorem Proving for Analyzing Student Explanations to Guide Feedback in Intelligent Tutoring Systems. Journal of Automated Reasoning 32, 187–226 (2004). https://doi.org/10.1023/B:JARS.0000044823.50442.cd

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:JARS.0000044823.50442.cd

Search

Navigation