Skip to main content
SpringerLink
Log in

The SAT2002 competition

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

SAT Competition 2002 held in March–May 2002 in conjunction with SAT 2002 (the Fifth International Symposium on the Theory and Applications of Satisfiability Testing). About 30 solvers and 2300 benchmarks took part in the competition, which required more than 2 CPU years to complete the evaluation. In this report, we give the results of the competition, try to interpret them, and give suggestions for future competitions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P.A. Abdulla, P. Bjesse and N. Eén, Symbolic reachability analysis based on SAT-solvers, in: Proceedings of the 6th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’2000) (2000).

  2. F. Bacchus, Enhancing Davis Putnam with extended binary clause reasoning, in: Proceedings of National Conference on Artificial Intelligence (AAAI-2002) (2002).

  3. F. Bacchus, Exploring the computational tradeoff of more reasoning and less searching, in: [49, pp. 7–16] (2002).

  4. L. Baptista and J.P. Marques-Silva, Using randomization and learning to solve hard real-world instances of satisfiability, in: Proceedings of the 6th International Conference on Principles and Practice of Constraint Programming (CP) (2000).

  5. R.J.J. Bayardo and R.C. Schrag, Using CSP look-back techniques to solve real-world SAT instances, in: Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI’97) (AMS, Providence, RI, 1997) pp. 203–208.

  6. A. Biere, A. Cimatti, E.M. Clarke, M. Fujita and Y. Zhu, Symbolic model checking using SAT procedures instead of BDDs, in: Proceedings of Design Automation Conference (DAC’99) (1999).

  7. M. Buro and H.K. Büning, Report on a SAT competition, Bulletin of the European Association for Theoretical Computer Science 49 (1993) 143–151.

    MATH  Google Scholar 

  8. C.-M. Li, B. Jurkowiak and P.W. Purdom Jr, Integrating symmetry breaking into a DLL procedure, in: [49, pp. 149–155] (2002).

  9. A.E. Caldwell, A.B. Kahng and I.L. Markov, Toward CAD-IP reuse: The MARCO GSRC bookshelf of fundamental CAD algorithms, IEEE Design and Test (May 2002) 72-81.

  10. P. Chatalic and L. Simon, Multi-resolution on compressed sets of clauses, in: Twelth International Conference on Tools with Artificial Intelligence (ICTAI’00) (2000) pp. 2–10.

  11. S.A. Cook, The complexity of theorem-proving procedures, in: Proceedings of the Third IEEE Symposium on the Foundations of Computer Science (1971) pp. 151–158.

  12. F. Copty, L. Fix, E. Giunchiglia, G. Kamhi, A. Tacchella and M. Vardi, Benefits of bounded model checking at an industrial setting, in: Proc. of CAV (2001).

  13. E. Dantsin, A. Goerdt, E.A. Hirsch, R. Kannan, J. Kleinberg, C. Papadimitriou, P. Raghavan and U. Schöning, Deterministic (2 − 2 (k + 1 )n algorithm for k-SAT based on local search, Theoretical Computer Science 189(1) (2002) 69–83.

    Article  Google Scholar 

  14. M. Davis, G. Logemann and D. Loveland, A machine program for theorem proving, Communications of the ACM 5(7) (1962) 394–397.

    Article  MATH  MathSciNet  Google Scholar 

  15. M. Davis and H. Putnam, A computing procedure for quantification theory, Journal of the ACM 7(3) (1960) 201–215.

    Article  MATH  MathSciNet  Google Scholar 

  16. O. Dubois, P. André, Y. Boufkhad and J. Carlier, SAT versus UNSAT, in: [29, pp. 415–436] (1996).

  17. O. Dubois and G. Dequen, A backbone-search heuristic for efficient solving of hard 3-SAT formulae, in: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI’01), Seattle, WA (2001).

  18. M.D. Ernst, T.D. Millstein and D.S. Weld, Automatic SAT-compilation of planning problems, in: [28, pp. 1169–1176] (1997).

  19. F. Aloul, A. Ramani, I. Markov and K. Sakallah, Solving difficult SAT instances in the presence of symmetry, in: Design Automation Conference (DAC), New Orleans, LO (2002) pp. 731–736.

  20. J.W. Freeman, Improvements to propositional satisfiability search algorithms, Ph.D. thesis, Departement of Computer and Information Science, University of Pennsylvania, Philadelphia, PA (1995).

  21. E. Goldberg and Y. Novikov, BerkMin: A fast and robust SAT-solver, in: Design, Automation, and Test in Europe (DATE ‘02) (2002) pp. 142–149.

  22. C.P. Gomes, B. Selman and H. Kautz, Boosting combinatorial search through randomization, in: Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI’98), Madison, WI (1998) pp. 431–437.

  23. E.A. Hirsch, SAT local search algorithms: Worst-case study, Journal of Automated Reasoning 24(1/2) (2000) 127–143. Also reprinted in Highlights of Satisfiability Research in the Year 2000, Frontiers in Artificial Intelligence and Applications, Vol. 63 (IOS Press, 2000).

    Article  MATH  MathSciNet  Google Scholar 

  24. E.A. Hirsch, New worst-case upper bounds for SAT, Journal of Automated Reasoning 24(4) (2000) 397–420. Also reprinted in Highlights of Satisfiability Research in the Year 2000, Frontiers in Artificial Intelligence and Applications, Vol. 63 (IOS Press, 2000).

    Article  MATH  MathSciNet  Google Scholar 

  25. E.A. Hirsch and A. Kojevnikov, UnitWalk: A new SAT solver that uses local search guided by unit clause elimination, Annals of Mathematics and Artificial Intelligence 43 (2005) 91–111.

    Article  MATH  MathSciNet  Google Scholar 

  26. J.N. Hooker, Needed: An empirical science of algorithms, Operations Research 42(2) (1994) 201–212.

    Article  MATH  Google Scholar 

  27. J.N. Hooker, Testing heuristics: We have it all wrong, Journal of Heuristics (1996) 32–42.

  28. IJCAI97, Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI’97), Nagoya, Japan (1997).

  29. D. Johnson and M. Trick (eds.), Second DIMACS Implementation Challenge: Cliques, Coloring and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26 (American Mathematical Society, 1996).

  30. H. Kautz and B. Selman (eds.), Proceedings of the Workshop on Theory and Applications of Satisfiability Testing (SAT2001), LICS 2001 Workshop on Theory and Applications of Satisfiability Testing (SAT 2001) (Elsevier Science, 2001).

  31. H.A. Kautz and B. Selman, Planning as satisfiability, in: Proceedings of the 10th European Conference on Artificial Intelligence (ECAI’92) (1992) pp. 359–363.

  32. H.A. Kautz and B. Selman, Pushing the envelope: Planning, propositional logic, and stochastic search, in: Proceedings of the 12th National Conference on Artificial Intelligence (AAAI’96) (1996) pp. 1194–1201.

  33. E. Koutsoupias and C.H. Papadimitriou, On the greedy algorithm for satisfiability, Information Processing Letters 43(1) (1992) 53–55.

    Article  MATH  MathSciNet  Google Scholar 

  34. O. Kullmann, First report on an adaptive density based branching rule for DLL-like SAT solvers, using a database for mixed random conjunctive normal forms created using the Advanced Encryption Standard (AES), Technical Report CSR 19-2002, University of Wales Swansea, Computer Science Report Series (2002). (Extended version of [36].)

  35. O. Kullmann, Investigating the behaviour of a SAT solver on random formulas, Annals of Mathematics and Artificial Intelligence (2002).

  36. O. Kullmann, Towards an adaptive density based branching rule for SAT solvers, using a database for mixed random conjunctive normal forms built upon the Advanced Encryption Standard (AES), in: [49] (2002).

  37. C.-M. Li, A constrained based approach to narrow search trees for satisfiability, Information Processing Letters 71 (1999) 75–80.

    Article  MATH  MathSciNet  Google Scholar 

  38. C.-M. Li, Integrating equivalency reasoning into Davis-Putnam procedure, in: Proceedings of the 17th National Conference in Artificial Intelligence (AAAI’00), Austin, TX (2000) pp. 291–296.

  39. C.-M. Li and Anbulagan, Heuristics based on unit propagation for satisfiability problems, in: [28, pp. 366–371] (1997).

  40. I. Lynce and J.P. Marques Silva, Efficient data structures for backtrack search SAT solvers, in: [49] (2002).

  41. I. Lynce, L. Baptista and J.P. Marques Silva, Stochastic systematic search algorithms for satisfiability, in: [30] (2001).

  42. J.P. Marques-Silva and K.A. Sakallah, GRASP — A new search algorithm for satisfiability, in: Proceedings of IEEE/ACM International Conference on Computer-Aided Design (1996) pp. 220–227.

  43. M.W. Moskewicz, C.F. Madigan, Y. Zhao, L. Zhang and S. Malik, Chaff: Engineering an efficient SAT solver, in: Proceedings of the 38th Design Automation Conference (DAC’01) (2001) pp. 530–535.

  44. F. Okushi and A. Van Gelder, Persistent and quasi-persistent lemmas in propositional model elimination, in: (Electronic) Proc. 6th Int’l Symposium on Artificial Intelligence and Mathematics (2000).; Annals of Mathematics and Artificial Intelligence 40(3–4) (2004) 373–402.

    Article  MATH  MathSciNet  Google Scholar 

  45. R. Ostrowski, E. Grégoire, B. Mazure and L. Sais, Recovering and exploiting structural knowledge from CNF formulas, in: Proc. of the Eighth International Conference on Principles and Practice of Constraint Programming (CP’2002), Ithaca, NY (2002).

  46. R. Paturi, P. Pudlàk and F. Zane, Satisfiability coding lemma, in: Proceedings of the 38th Annual IEEE Symposium on Foundations of Computer Science, FOCS’97 (1997) pp. 566–574.

  47. S. Prestwich, A SAT approach to query optimization in mediator systems, in: [49, pp. 252-259] (2002).

  48. S.D. Prestwich, Randomised backtracking for linear pseudo-Boolean constraint problems, in: Proceedings of Fourth International Workshop on Integration of AI and OR techniques in Constraint Programming for Combinatorial Optimisation Problems (2002).

  49. SAT2002, Fifth International Symposium on the Theory and Applications of Satisfiability Testing, Cincinnati, OH (2002).

  50. R. Schuler, U. Schöning, O. Watanabe and T. Hofmeister, A probabilistic 3-SAT algorithm further improved, in: Proceedings of 19th International Symposium on Theoretical Aspects of Computer Science, STACS 2002 (2002).

  51. B. Selman, H.A. Kautz and B. Cohen, Noise strategies for improving local search, in: Proceedings of the 12th National Conference on Artificial Intelligence (AAAI’94), Seattle (1994) pp. 337–343.

  52. B. Selman, H. Levesque and D. Mitchell, A new method for solving hard satisfiability problems, in: Proceedings of the 10th National Conference on Artificial Intelligence (AAAI’92) (1992) pp. 440–446.

  53. Y. Shang and B.W. Wah, A discrete Lagrangian-based global-search method for solving satisfiability problems, Journal of Global Optimization 12(1) (1998) 61–99.

    Article  MATH  MathSciNet  Google Scholar 

  54. L. Simon and P. Chatalic, SATEx: a Web-based framework for SAT experimentation, in: [30] (2001); http://www.lri.fr/~simon/satex.

  55. G. Sutcliff and C. Suttner, Evaluating general purpose automated theorem proving systems, Artificial Intelligence 131 (2001) 39–54.

    Article  MathSciNet  Google Scholar 

  56. G.S. Tseitin, On the complexity of derivation in the propositional calculus, in: Structures in Constructive Mathematics and Mathematical Logic, Part II, ed. A.O. Slisenko (Consultants Bureau, New York, 1970) 115–125. Translated from Russian.

    Chapter  Google Scholar 

  57. A. Urquhart, Hard examples for resolution, Journal of the Association for Computing Machinery 34(1) (1987) 209–219.

    Article  MATH  MathSciNet  Google Scholar 

  58. A. Van Gelder, Autarky pruning in propositional model elimination reduces failure redundancy, Journal of Automated Reasoning 23(2) (1999) 137–193.

    Article  MATH  MathSciNet  Google Scholar 

  59. A. Van Gelder, Extracting (easily) checkable proofs from a satisfiability solver that employs both pre-order and postorder resolution, in: Seventh Int’l Symposium on AI and Mathematics, Fort Lauderdale, FL (2002).

  60. A. Van Gelder, Generalizations of watched literals for backtracking search, in: Seventh Int’l Symposium on AI and Mathematics, Fort Lauderdale, FL (2002).

  61. A. Van Gelder and F. Okushi, Lemma and Cut strategies for propositional model elimination, Annals of Mathematics and Artificial Intelligence 26(1–4) (1999) 113–132.

    Article  MATH  MathSciNet  Google Scholar 

  62. A. Van Gelder and Y.K. Tsuji, Satisfiability testing with more reasoning and less guessing, in: [29, pp. 559–586] (1996).

  63. M. Velev and R. Bryant, Effective use of Boolean satisfiability procedures in the formal verification of superscalar and VLIW microprocessors, in: Proceedings of the 38th Design Automation Conference (DAC ‘01) (2001) pp. 226–231.

  64. J. Warners and H. van Maaren, Solving satisfiability problems using elliptic approximations: Effective branching rules, Discrete Applied Mathematics 107 (2000) 241–259.

    Article  MATH  MathSciNet  Google Scholar 

  65. H. Zhang, SATO: An efficient propositional prover, in: Proceedings of the International Conference on Automated Deduction (CADE’97), Lecture Notes in Artificial Intelligence, Vol. 1249 (1997) pp. 272-275.

  66. H. Zhang and M.E. Stickel, An efficient algorithm for unit propagation, in: Proceedings of the Fourth International Symposium on Artificial Intelligence and Mathematics (AI-MATH’96), Fort Lauderdale, FL (1996).

  67. L. Zhang, C.F. Madigan, M.W. Moskewicz and S. Malik, Efficient conflict driven learning in a Boolean satisfiability solver, in: International Conference on Computer-Aided Design (ICCAD’01) (2001) pp. 279–285.

  68. L. Zheng and P.J. Stuckey, Improving SAT using 2SAT, in: Proceedings of the Twenty-Fifth Australasian Computer Science Conference (ACSC2002), ed. M.J. Oudshoorn, Melbourne, Australia (2002).

Download references

Author information

Authors and Affiliations

  1. LRI, U.M.R. CNRS 8623, Université Paris-Sud, 91405, Orsay, Cedex, France

    Laurent Simon

  2. CRIL, F.R.E. CNRS 2499, Faculté Jean Perrin, Université d’ Artois, Rue Jean Souvraz SP 18, 62300, Lens, Cedex, France

    Daniel Le Berre

  3. Steklov Institute of Mathematics at St. Petersburg, 27 Fontanka, 191023, St. Petersburg, Russia

    Edward A. Hirsch

Authors
  1. Laurent Simon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Le Berre
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward A. Hirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Simon, L., Le Berre, D. & Hirsch, E.A. The SAT2002 competition. Ann Math Artif Intell 43, 307–342 (2005). https://doi.org/10.1007/s10472-005-0424-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-005-0424-6

Keywords

Search

Navigation