Skip to main content
SpringerLink
Log in

The TPTP Problem Library

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

This paper provides a detailed description of the CNF part of the TPTP Problem Library for automated theorem-proving systems. The library is available via the Internet and forms a common basis for development and experimentation with automated theorem provers. This paper explains the motivations and reasoning behind the development of the TPTP (thus implicitly explaining the design decisions made) and describes the TPTP contents and organization. It also provides guidelines for obtaining and using the library, summary statistics about release v1.2.1, and an overview of the tptp2X utility program. References for all the sources of TPTP problems are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mathematical Subject Classification, American Mathematical Society, 1992.

  2. Allen, J. F., Kautz, H. A., Pelavin, R. N., and Tenenberg, J. D.: Reasoning about Plans, Morgan Kaufmann, 1991.

  3. Anantharaman, S. and Hsiang, J.: Automated proofs of the Moufang identities in alternative rings, Journal of Automated Reasoning 6(1) (1990), 79–110. y

    Google Scholar 

  4. Angshuman, G. and Zhang, H.: Andrews' challenge problem: Clause conversion and solutions, AAR Newsletter 14 (1989), 5–8.

    Google Scholar 

  5. ANL. Argonne National Laboratory Problem Library. Available by anonymous ftp from info.msc.anl.gov, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois.

  6. Astrachan, O. L.: METEOR: Exploring model elimination theorem proving, Journal of Automated Reasoning 13(3) (1994), 283–296.

    Google Scholar 

  7. Barendregt, H. P.: The Lambda Calculus: Its Syntax and Semantics, North-Holland, 1981.

  8. Beckert, B. and Posegga, J.: leanTAP: Lean, tableau-based deduction, Journal of Automated Reasoning 15(3) (1995), 339–358.

    Google Scholar 

  9. Benanav, D.: Recognising unnecessary clauses in resolution based systems, Journal of Automated Reasoning 9(1) (1992), 43–76.

    Google Scholar 

  10. Birkhoff, G. and Bartee, T.: Modern Applied Algebra, McGraw-Hill, 1970.

  11. Birkhoff, G. and MacLane, S.: A Survey of Modern Algebra, Macmillan, 1965.

  12. Bledsoe, W. W.: Non-Resolution Theorem Proving, Artificial Intelligence 9 (1977), 1–35.

    Google Scholar 

  13. Bledsoe, W. W.: Challenge problems in elementary calculus, Journal of Automated Reasoning 6 (1990), 341–359.

    Google Scholar 

  14. Bonacina, M. P.: Problems in Lukasiewicz logic, AAR Newsletter 18 (1991), 5–12.

    Google Scholar 

  15. Bourbaki, N.: Algebra I – Chapters 1–3, Springer-Verlag, 1989.

  16. Bourely, C., Caferra, R., and Peltier, N.: A method for building models automatically. Experiments with an extension of Otter, in A. Bundy (ed.), Proceedings of the 12th International Conference on Automated Deduction, Lecture Notes in Artificial Intelligence 814, Springer-Verlag, 1994, pp. 72–86.

  17. Boyer, R., Lusk, E., McCune, W. W., Overbeek, R., Stickel, M., and Wos, L.: Set theory in first-order logic: Clauses for Godel's axioms, Journal of Automated Reasoning 2(3) (1986), 287–327.

    Google Scholar 

  18. Brushi, M.: The halting problem, AAR Newsletter 17 (1991), 7–12.

    Google Scholar 

  19. Burkholder, L.: A 76th automated theorem proving problem, AAR Newsletter 8 (1987), 6–7.

    Google Scholar 

  20. Carroll, L.: Lewis Carroll's Symbolic Logic, C. N. Potter, 1986.

  21. Chang, C.-L.: The unit proof and the input proof in theorem proving, Journal of the ACM 17(4) (1970), 698–707.

    Google Scholar 

  22. Church, A.: Introduction to Mathematical Logic I, Princeton University Press, 1956.

  23. Curry, H. B. and Feys, R.: Combinatory Logic I, North-Holland, 1958.

  24. Curry, H. B., Hindley, J. R., and Seldin, J. P.: Combinatory Logic II, North-Holland, 1972.

  25. Dewey, M.: Dewey Decimal Classification and Relative Index, 20th edition, Forest Press, 1989.

  26. Fermuller, C.: E-mail to G. Sutcliffe, 1994.

  27. Fermuller, C., Leitsch, A., Tammet, T., and Zamov, N.: Resolution ethods for the Decision Problem, Lecture Notes in Computer Science 679, Springer-Verlag, 1993.

  28. Fleisig, S., Loveland, D. W., Smiley, A. K., and Yarmush, D. L.: An implementation of the model elimination proof procedure, Journal of the ACM 21(1) (1974), 124–139.

    Google Scholar 

  29. Fujita, M., Hasegawa, R., Koshimura, M., and Fujita, H.: Model generation theorem provers on a parallel inference machine, in Proceedings of the International Conference on Fifth Generation Computer Systems, 1992, pp. 357–375.

  30. Fujita, M., Slaney, J., and Bennett, F.: Automatic generation of some results in finite algebra, in R. Bajcsy (ed.), Proceedings of the 13th International Joint Conference on Artificial Intelligence, Morgan Kaufmann, 1993, pp. 52–57.

  31. Genesereth, M. R. and Fikes, R. E.: Knowledge Interchange Format, Version 3.0 Reference Manual, Technical Report Technical Report Logic-92-1, Computer Science Department, Stanford University, 1992.

  32. Glickfield, B. and Overbeek, R.: A foray into combinatory logic, Journal of Automated Reasoning 2(4) (1986), 419–431.

    Google Scholar 

  33. Hähnle, R., Beckert, B., and Gerberding, S.: The many-valued tableau-based theorem prover 3TAP, Technical Report TR 30/94, Fakultät für Informatik, Universät Karlsruhe, Karlsruhe, Germany, 1994.

    Google Scholar 

  34. Harary, F.: Graph Theory, Addison-Wesley, 1969.

  35. Hardy, G. F. and Wright, E.M.: An Introduction to the Theory of Numbers, 5th edition, Oxford University Press, 1992.

  36. Hayes, J. P.: Introduction to Digital Logic Circuit Design, Addison-Wesley, 1993.

  37. Henkin, L.: Completeness in the theory of types, Journal of Symbolic Logic 15 (1950), 81–91.

    Google Scholar 

  38. Henkin, L., Monk, J., and Tarski, A.: Cylindrical Algebras, volume Part 1, North-Holland, 1971.

  39. Hopcroft, J. and Ullman, J.: Introduction to Automata Theory, Languages and Computation, Addison-Wesley, 1979.

  40. Howson, A. G.: A Handbook of Terms used in Algebra and Analysis, Cambridge University Press, 1972.

  41. Jech, T.: LD-algebras, Association for Automated Reasoning Newsletter 22 (1993), 9–12.

    Google Scholar 

  42. Jech, T.: Otter experiments in a system of combinatory logic, Journal of Automated Reasoning 14(3) (1995), 413–426.

    Google Scholar 

  43. Kelley, J. L.: General Topology, D. Van Nostrand, 1955.

  44. Kunen, K.: Single axioms for groups, Journal of Automated Reasoning 9(3) (1992), 291–308.

    Google Scholar 

  45. Lawrence, J. D. and Starkey, J. D.: Experimental tests of resolution based theorem-proving strategies, Technical Report, Computer Science Department, Washington State University, Pullman, USA, 1974.

    Google Scholar 

  46. Leblanc, H.: Alternatives to standard first-oder semantics, in D. Gabbay and F. Guenther (eds), Handbook of Philosophical Logic, volume I, chapter I.3, D. Reidel, 1983, pp. 189–274.

  47. Lee, S.-J. and Plaisted, D. A.: Eliminating duplication with the hyper-linking strategy, Journal of Automated Reasoning 9(1) (1992), 25–42.

    Google Scholar 

  48. Letz, R., Mayr, K., and Goller, C.: Controlled integration of the cut rule into connection tableau calculi. Journal of Automated Reasoning 13(3) (1994), 297–337.

    Google Scholar 

  49. Letz, R., Schumann, J., Bayerl, S., and Bibel, W.: SETHEO: A high-performance theorem prover, Journal of Automated Reasoning 8(2) (1992), 183–212.

    Google Scholar 

  50. Loveland, D. W.: Mechanical theorem proving by model elimination, Journal of the ACM 15(2) (1968), 236–251.

    Google Scholar 

  51. Loveland, D. W.: Theorem-provers combining model elimination and resolution, Machine Intelligence 4 (1969), 73–86.

    Google Scholar 

  52. Luckham, D.: Some tree-paring strategies for theorem proving, Machine Intelligence 3 (1968), 95–112.

    Google Scholar 

  53. Lukasiewicz, J.: Elements of Mathematical Logic, Pergamon Press, 1963.

  54. Lusk, E. and Overbeek, R.: Non-Horn problems, Journal of Automated Reasoning 1(1) (1985), 103–114.

    Google Scholar 

  55. Lusk, E. and Wos, L.: Benchmark problems in which equality plays the major role, Technical Report MCS-P275-1191, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, USA, 1991.

    Google Scholar 

  56. Lusk, E. and Wos, L.: Benchmark problems in which equality plays the major role, in: D. Kapur (ed.), Proceedings of the 11th International Conference on Automated Deduction, Lecture Notes in Artificial Intelligence 607, Springer-Verlag, 1992, pp. 781–785.

  57. Lusk, E. L. and McCune, W. W.: Experiments with ROO, a parallel automated deduction system, in B. Fronhofer and G. Wrightson (eds), Parallelization in Inference Systems, volume 590, 1992, pp. 139–162.

  58. Lusk, E. L. and McCune, W. W.: Uniform strategies: The CADE-11 theorem proving contest, Journal of Automated Reasoning 11(3) (1993), 317–331.

    Google Scholar 

  59. MacLane, S.: Categories for the Working Mathematician, Springer-Verlag, 1971.

  60. Manthey, R. and Bry, F.: SATCHMO: A theorem prover implemented in prolog, in R. Overbeek and E. Lusk (ed.), Proceedings of the 9th International Conference on Automated Deduction, Lecture Notes in Computer Science 310, Springer-Verlag, 1988, pp. 415–434.

  61. McCharen, J. D., Overbeek, R. A., and Wos, L. A.: Problems and experiments for and with automated theorem-proving programs, IEEE Transactions on Computers 25(8) (1976), 773–782.

    Google Scholar 

  62. McCune, W. W.: Challenge equality problems in lattice theory, in Proceedings of the 9th International Conference on Automated Deduction – Argonne, Illinois, USA, May 1988, Springer-Verlag, 1988, pp. 704–709.

  63. McCune, W. W. and Wos, L.: Experiments in automated deduction with condensed detachment, in D. Kapur (ed.), Proceedings of the 11th International Conference on Automated Deduction, Lecture Notes in Artificial Intelligence 607, Springer-Verlag, 1992, pp. 209–223.

  64. McCune, W. W.: Automated discovery of new axiomatizations of the left group and right group calculi, Journal of Automated Reasoning 9(1) (1992), 1–24.

    Google Scholar 

  65. McCune, W. W.: Single axioms for groups and Abelian groups with various operations, Journal of Automated Reasoning 10(1) (1993), 1–13.

    Google Scholar 

  66. McCune, W.W.: Otter 3.0 reference manual and guide, Technical Report ANL-94/6, Argonne National Laboratory, Argonne, USA, 1994.

    Google Scholar 

  67. McCune, W. W. and Lusk, E.: A challenging theorem of Levi, AAR Newsletter 21 (1992), 8.

    Google Scholar 

  68. McCune, W. W. and Wos, L.: Some fixed point problems in combinatory logic,AAR Newsletter 10 (1988), 7–8.

    Google Scholar 

  69. Michie, D., Ross, R., and Shannan, G. J.: G-deduction, Machine Intelligence 7 (1972), 141–165.

    Google Scholar 

  70. Munkres, J. R.: Topology: A First Course, Prentice-Hall, 1975.

  71. von Neumann, J.: Eine Axiomatisierung der Mengenlehre, Journal für die Reine und Angewandte Mathematik 154 (1925), 219–240.

    Google Scholar 

  72. Ohlbach, H. J.: Predicate logic hacker tricks, Journal of Automated Reasoning 1(4) (1985), 435–440.

    Google Scholar 

  73. Ohlbach, H. J. and Schmidt-Schauss, M.: The lion and the unicorn, Journal of Automated Reasoning 1(3) (1985), 327–332.

    Google Scholar 

  74. Otter, The problem collection distributed with the Otter ATP system [McC90].

  75. Overbeek, R.: ATP competition announced at CADE-10, 1990.

  76. Overbeek, R., McCharen, J., and Wos, L.: Complexity and related enhancements for automated theorem-proving programs, Computers and Mathematics with Applications 2 (1976), 1–16.

    Google Scholar 

  77. Pelletier, F. J.: Seventy-five problems for testing automatic theorem provers, Journal of Automated Reasoning 2(2) (1986), 191–216.

    Google Scholar 

  78. Pfenning, F.: Single axioms in the implicational propositional calculus, in: R. Overbeek and E. Lusk (eds), Proceedings of the 9th International Conference on Automated Deduction, Lecture Notes in Computer Science 310, Springer-Verlag, 1988, pp. 710–713.

  79. Plaisted, D. A.: Theorem proving with abstraction, Artificial Intelligence 16 (1981), 47–108.

    Google Scholar 

  80. Plaisted, D. A.: A simplified problem reduction format, Artificial Intelligence 18 (1982), 227–261.

    Google Scholar 

  81. Plaisted, D. A.: Non-Horn clause logic programming without contrapositives, Journal of Automated Reasoning 4(3) (1988), 287–325.

    Google Scholar 

  82. Plaisted, D. A.: The search efficiency of theorem proving strategies, in A. Bundy (ed.), Proceedings of the 12th International Conference on Automated Deduction, Lecture Notes in Artificial Intelligence 814, Springer-Verlag, 1994, pp. 57–71.

  83. Quaife, A.: Automated development of Tarski's geometry, Journal of Automated Reasoning 5(1) (1989), 97–118.

    Google Scholar 

  84. Quaife, A.: Andrews' challenge problem revisited, AAR Newsletter 15 (1990), 3–7.

    Google Scholar 

  85. Quaife, A.: Automated deduction in von Neumann–Bernays–Godel set theory, Journal of Automated Reasoning 8(1) (1992), 91–147.

    Google Scholar 

  86. Quaife, A.: Automated Development of Fundamental Mathematical Theories, Kluwer Academic Publishers, 1992.

  87. Quaife, A.: E-mail to G. Sutcliffe, 1992.

  88. Quine, W. V.: Set Theory and its Logic, Harvard University Press, 1969.

  89. Reboh, R., Raphael, B., Yates, R. A., Kling, R. E., and Velarde, C.: Study of Automatic Theorem Proving Programs. Technical Report Technical Note 72, Artificial Intelligence Center, SRI International, Menlo Park, CA, 1972.

    Google Scholar 

  90. Robinson, J. A.: Theorem proving on the computer, Jounal of the ACM 10(2) (1963), 163–174.

    Google Scholar 

  91. Ross, K. A.: Elementary Analysis: The Theory of Calculus, 2nd edition, Springer-Verlag, 1990.

  92. Schulz, S.: Explanation based learning for distributed equational deduction, Master's thesis, Universität Kaiserslautern, 1995.

  93. Schumann, J., Trapp, N., and van der Koelen, M.: SETHEO/PARTHEO Users Manual, Technical Report SFB Bericht 342/7/90 A, Institut für Informatik, Technische Universität München, Munich, Germany, 1990.

    Google Scholar 

  94. Segre, A. and Elkan, C.: A high-performance explanation-based learning algorithm, Artificial Intelligence 69 (NEED) (1994), 1–50.

    Google Scholar 

  95. SETHEO, AR Research Group, Technische Universität München, Problem Library, No longer available.

  96. Shostak, R. E.: Refutation graphs, Artificial Intelligence 7 (1976), 51–64.

    Google Scholar 

  97. Slagle, J. R.: Automatic theorem proving with renamable and semantic resolution, Journal of the ACM 14(4) (1967), 687–697.

    Google Scholar 

  98. Slaney, J.: E-mail to G. Sutcliffe, 1993.

  99. Slaney, J., Fujita, M., and Stickel, M.: Automated reasoning and exhaustive search: Quasigroup existence problems, Computers and Mathematics with Applications 29(2) (1995), 115–132.

    Google Scholar 

  100. Smullyan, R. M.: To Mock a Mocking Bird and Other Logic Puzzles, Knopf, 1978.

  101. Smullyan, R.M.: What Is the Name of This Book? The Riddle of Dracula and Other Logical Puzzles, Prentice-Hall, 1978.

  102. SPRFN, The problem collection distributed with the SPRFN ATP system.

  103. Stevens, R. L.: Some experiments in nonassociative ring theory with an automated theorem prover, Journal of Automated Reasoning 3(2) (1987), 211–221.

    Google Scholar 

  104. Stickel, M.: E-mail to G. Sutcliffe, 1993.

  105. Stickel, M. E.: A Prolog technology theorem prover, New Generation Computing 2(4) (1984), 371–383.

    Google Scholar 

  106. Stickel, M. E.: Schubert's steamroller problem: Formulations and solutions, Journal of Automated Reasoning 2(1) (1986), 89–101.

    Google Scholar 

  107. Stickel, M. E.: Upside-down meta-interpretation of the model elimination theorem-proving procedure for deduction and abduction, Journal of Automated Reasoning 13(2) (1994), 189–210.

    Google Scholar 

  108. Sutcliffe, G. and Suttner, C. B.: ATP system results for the TPTP problem library (upto TPTP v1.1.3), 1995, Technical Report 95/16, Department of Computer Science, James Cook University, Townsville, Australia; Technical Report AR-95-06, Institut für Informatik, Technische Universität München, Munich, Germany.

    Google Scholar 

  109. Sutcliffe, G. and Suttner, C. B.: Special issue: The CADE-13 ATP system competition, Journal of Automated Reasoning 18(2) (1997).

  110. Suttner, C. B. and Sutcliffe, G.: The TPTP problem library (TPTP v1.2.1), 1996, Technical Report AR-96-02, Institut für Informatik, Technische Universität München, Munich, Germany; Technical Report 96/09, Department of Computer Science, James Cook University, Townsville, Australia.

    Google Scholar 

  111. Tammet, T.: E-mail to Geoff Sutcliffe, 1994.

  112. Tarski, A.: A Decision Method for Elementary Algebra and Geometry, 2nd edition, University of California Press, 1951.

  113. Tarski, A.: What is Elementary Geometry? in L. Henkin (ed.), Proceedings of an International Symposium. The Axiomatic Method with Special Reference to Geometry and Physics, North-Holland, 1959.

  114. Veroff, R.: Group Theory Problems, 1992.

  115. Veroff, R.: Problem set, AAR Newsletter 24 (1994), 8.

    Google Scholar 

  116. Wang, H.: Formalization and automatic theorem-proving, in W. A. Kalenich (ed.), Proceedings of the IFIP Congress, Spartan Books, 1965, pp. 51–58.

  117. Wang, T.-C.: Designing examples for semantically guided hierarchical deduction, in A. Joshi (ed.), Proceedings of the 9th International Joint Conference on Artificial Intelligence, Los Altos, International Joint Conferences on Artificial Intelligence, Inc., 1985, pp. 1201–1207

    Google Scholar 

  118. Wang, T.-C. and Bledsoe, W.W.: Hierarchical deduction, Journal of Automated Reasoning 3(1) (1987), 35–77.

    Google Scholar 

  119. Weidenbach, C.: SPASS – a first-order theorem prover and a CNF-translator, URL http://www.mpi-sb.mpg.de/guide/software/spass.html, 1996.

  120. Weidenbach, C., Gaede, B., and Rock, G.: SPASS & FLOTTER, in M. McRobbie and J. Slaney (eds), Proceedings of the 13th International Conference on Automated Deduction, Lecture Notes in Artificial Intelligence 1104, Springer-Verlag, 1996, pp. 141–145.

  121. Whitehead, A. N. and Russell, B.: Principia Mathematica, Vol. 1, 2nd edition, Cambridge Univ. Press, 1927.

  122. Whitesitt, J. E.: Boolean Algebra and Its Applications, Addison-Wesley, 1961.

  123. Wick, C. and McCune, W.: Automated reasoning about elementary point-set topology, Journal of Automated Reasoning 5 (1989), 239–255.

    Google Scholar 

  124. Wilson, D. S. and Loveland, D.W.: Incorporating relevancy testing in SATCHMO, Technical Report CS-1989-24, Department of Computer Science, Duke University, Durham, USA, 1989.

    Google Scholar 

  125. Wilson, G. A. and Minker, J.: Resolution strategies: A comparative study, IEEE Transactions on Computers C-25(8) (1976), 782–801.

    Google Scholar 

  126. Winker, S.: Generation and verification of finite models and counterexamples using an automated theorem prover answering two open questions, Journal of the ACM 29(2) (1982), 273–284.

    Google Scholar 

  127. Winker, S.: Robbins algebra: Conditions that make a near-Boolean algebra Boolean, Journal of Automated Reasoning 6(4) (1990), 465–489.

    Google Scholar 

  128. Wojciechowski, W. S. and Wojcik, A. S.: Automated design of multi-valued logic circuits by automated theorem proving techniques, IEEE Transactions on Computers C-32(8) (1983), 785–798.

    Google Scholar 

  129. Wojcik, A. S.: Formal design verification of digital systems, in Proceedings of the 20th Design Automation Conference, 1983.

  130. Wos, L.: Unpublished note, Argonne National Laboratory, Argonne, USA, 1965.

  131. Wos, L.: Automated Reasoning – 33 Basic Research Problems, Prentice-Hall, 1988.

  132. Wos, L.: A challenge problem and a recent workshop, AAR Newsletter 13 (1989), 2–8.

    Google Scholar 

  133. Wos, L.: The kernel strategy and its use for the study of combinatory logic, Journal of Automated Reasoning 10(3) (1993), 287–344.

    Google Scholar 

  134. Wos, L.: Challenge in group theory, AAR Newsletter 26 (1994), 3–5.

    Google Scholar 

  135. Wos, L. and McCune, W.: Challenge problems focusing on equality and combinatory logic: Evaluating automated theorem-proving programs, in R. Overbeek and E. Lusk (eds), Proceedings of the 9th International Conference on Automated Deduction, Lecture Notes in Computer Science 310, Springer-Verlag, 1988, pp. 714–729.

  136. Wos, L., Overbeek, R., Lusk, E., and Boyle, J.: Automated Reasoning – Introduction and Applications, 2nd edition, McGraw-Hill, 1992.

  137. Wos, L., Winker, S., McCune, W. W., Overbeek, R., Lusk, E., and Stevens, R.: Automated reasoning contributes to mathematics and logic, in M. Stickel (ed.), Proceedings of the 10th International Conference on Automated Deduction, Lecture Notes in Artificial Intelligence 449, Springer-Verlag, 1990, pp. 485–499.

  138. Zhang, Z.: Automated proofs of equality problems in Overbeek's competition, Journal of Automated Reasoning 11(3) (1993), 333–351.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, James Cook University, Townsville, Australia

    Geoff Sutcliffe

  2. Institut für Informatik, TU München, Munich, Germany

    Christian Suttner

Authors
  1. Geoff Sutcliffe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Suttner
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sutcliffe, G., Suttner, C. The TPTP Problem Library. Journal of Automated Reasoning 21, 177–203 (1998). https://doi.org/10.1023/A:1005806324129

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1005806324129

Search

Navigation