Skip to main content
Springer Nature Link
Log in

Assessing Progress in SAT Solvers Through the Lens of Incremental SAT

  • Conference paper
  • First Online:
Theory and Applications of Satisfiability Testing – SAT 2021 (SAT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12831))

Abstract

There is a wide consensus, which is supported by the hard experimental evidence of the SAT competitions, that clear progress in SAT solver performance has been observed in recent years. However, in the vast majority of practical applications of SAT, one is expected to use SAT solvers as oracles deciding a possibly large number of propositional formulas. In practice, this is often achieved through the use of incremental SAT. Given this fundamental use of SAT solvers, this paper investigates whether recent improvements in solver performance have an observable positive impact on the overall problem-solving efficiency in settings where incremental SAT is mandatory or at least expected. Our results, obtained on a number of well-known practically significant applications, suggest that most improvements made to SAT solvers in recent years have no positive impact on the overall performance when solvers are used incrementally.

Stepan Kochemazov is supported by the Ministry of Science and Higher Education of Russian Federation, research project no. 075-03-2020-139/2 (goszadanie no. 2019-1339). Joao Marques-Silva is supported by the AI Interdisciplinary Institute ANITI, funded by the French program “Investing for the Future – PIA3” under Grant agreement no. ANR-19-PI3A-0004, and by the H2020-ICT38 project COALA “Cognitive Assisted agile manufacturing for a Labor force supported by trustworthy Artificial intelligence”.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    It is generally accepted that the term CDCL was coined by L. Ryan [40].

  2. 2.

    Similarly, in the area of Satisfiability Modulo Theories (SMT) reasoning [3], it is generally accepted that not all optimizations made to SAT and SMT solvers find widespread use.

  3. 3.

    http://fmv.jku.at/chasing-target-phases/.

  4. 4.

    http://fmv.jku.at/kissat/.

  5. 5.

    https://github.com/veinamond/RLNT.

  6. 6.

    https://github.com/biotomas/ipasir.

  7. 7.

    https://satcompetition.github.io/2020/results.html.

  8. 8.

    https://github.com/niklasso/minisat.

  9. 9.

    https://pysathq.github.io/docs/html/api/examples/rc2.html.

  10. 10.

    https://pysathq.github.io/docs/html/api/examples/lsu.html.

  11. 11.

    https://pysathq.github.io/docs/html/api/examples/musx.html.

  12. 12.

    https://www.starexec.org/.

References

  1. Audemard, G., Lagniez, J.-M., Simon, L.: Improving glucose for incremental SAT solving with assumptions: application to MUS extraction. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 309–317. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39071-5_23

    Chapter  MATH  Google Scholar 

  2. Audemard, G., Simon, L.: Predicting learnt clauses quality in modern SAT solvers. In: IJCAI, pp. 399–404 (2009)

    Google Scholar 

  3. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability, 2nd edn. IOS Press, Amsterdam (2021)

    MATH  Google Scholar 

  4. Cook, S.A.: The complexity of theorem-proving procedures. In: STOC, pp. 151–158 (1971)

    Google Scholar 

  5. Davis, M., Logemann, G., Loveland, D.W.: A machine program for theorem-proving. Commun. ACM 5(7), 394–397 (1962)

    Article  MathSciNet  Google Scholar 

  6. Davis, M., Putnam, H.: A computing procedure for quantification theory. J. ACM 7(3), 201–215 (1960)

    Article  MathSciNet  Google Scholar 

  7. Eén, N., Sörensson, N.: An extensible SAT-solver. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 502–518. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24605-3_37

    Chapter  Google Scholar 

  8. Fichte, J.K., Hecher, M., Szeider, S.: A time leap challenge for SAT-solving. In: Simonis, H. (ed.) CP 2020. LNCS, vol. 12333, pp. 267–285. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58475-7_16

    Chapter  Google Scholar 

  9. Gebser, M., Kaminski, R., Kaufmann, B., Schaub, T.: Answer set solving in practice. Synth. Lect. Artif. Intell. Mach. Learn. 6(3), 1–238 (2012). Morgan & Claypool Publishers

    Google Scholar 

  10. Gomes, C.P., Selman, B., Kautz, H.A.: Boosting combinatorial search through randomization. In: AAAI, pp. 431–437 (1998)

    Google Scholar 

  11. Hickey, R., Bacchus, F.: Speeding up assumption-based SAT. In: Janota, M., Lynce, I. (eds.) SAT 2019. LNCS, vol. 11628, pp. 164–182. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-24258-9_11

    Chapter  MATH  Google Scholar 

  12. Ignatiev, A., Morgado, A., Marques-Silva, J.: PySAT: a python toolkit for prototyping with SAT oracles. In: Beyersdorff, O., Wintersteiger, C.M. (eds.) SAT 2018. LNCS, vol. 10929, pp. 428–437. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94144-8_26

    Chapter  MATH  Google Scholar 

  13. Ignatiev, A., Morgado, A., Marques-Silva, J.: RC2: an efficient MaxSAT solver. J. Satisf. Boolean Model. Comput. 11(1), 53–64 (2019)

    MathSciNet  Google Scholar 

  14. Järvisalo, M., Heule, M.J.H., Biere, A.: Inprocessing rules. In: Gramlich, B., Miller, D., Sattler, U. (eds.) IJCAR 2012. LNCS (LNAI), vol. 7364, pp. 355–370. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31365-3_28

    Chapter  Google Scholar 

  15. Katebi, H., Sakallah, K.A., Marques-Silva, J.P.: Empirical study of the anatomy of modern sat solvers. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS, vol. 6695, pp. 343–356. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21581-0_27

    Chapter  MATH  Google Scholar 

  16. Kochemazov, S., Zaikin, O., Semenov, A.A., Kondratiev, V.: Speeding up CDCL inference with duplicate learnt clauses. In: ECAI, pp. 339–346 (2020)

    Google Scholar 

  17. Lagniez, J.-M., Biere, A.: Factoring out assumptions to speed up MUS extraction. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 276–292. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39071-5_21

    Chapter  Google Scholar 

  18. Li, C., Xiao, F., Luo, M., Manyà, F., Lü, Z., Li, Y.: Clause vivification by unit propagation in CDCL sat solvers. Artif. Intell. 279, 103197 (2020)

    Article  MathSciNet  Google Scholar 

  19. Liang, J.H., Ganesh, V., Poupart, P., Czarnecki, K.: Learning rate based branching heuristic for SAT solvers. In: Creignou, N., Le Berre, D. (eds.) SAT 2016. LNCS, vol. 9710, pp. 123–140. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40970-2_9

    Chapter  MATH  Google Scholar 

  20. Liang, J.H., Oh, C., Ganesh, V., Czarnecki, K., Poupart, P.: MapleCOMSPS, MapleCOMSPS\_LRB, MapleCOMSPS\_CHB. In: Procceedings of SAT Competition 2016, vol. B-2016-1, pp. 52–53 (2016)

    Google Scholar 

  21. Luby, M., Sinclair, A., Zuckerman, D.: Optimal speedup of Las Vegas algorithms. Inf. Process. Lett. 47(4), 173–180 (1993)

    Article  MathSciNet  Google Scholar 

  22. Luo, M., Li, C., Xiao, F., Manyà, F., Lü, Z.: An effective learnt clause minimization approach for CDCL SAT solvers. In: IJCAI, pp. 703–711 (2017)

    Google Scholar 

  23. Marques-Silva, J.: Search algorithms for satisfiability problems in combinational switching circuits. Ph.D. thesis, University of Michigan (1995)

    Google Scholar 

  24. Marques-Silva, J., Lynce, I.: On improving MUS extraction algorithms. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS, vol. 6695, pp. 159–173. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21581-0_14

    Chapter  Google Scholar 

  25. Marques-Silva, J., Sakallah, K.A.: GRASP - a new search algorithm for satisfiability. In: ICCAD, pp. 220–227 (1996)

    Google Scholar 

  26. Marques-Silva, J., Sakallah, K.A.: GRASP: a search algorithm for propositional satisfiability. IEEE Trans. Comput. 48(5), 506–521 (1999). https://doi.org/10.1109/12.769433

    Article  MathSciNet  MATH  Google Scholar 

  27. Irkutsk Supercomputer Center of SB RAS. http://hpc.icc.ru

  28. MaxSAT Evaluation 2018. https://maxsat-evaluations.github.io/2018/

  29. MaxSAT Evaluation 2019. https://maxsat-evaluations.github.io/2019/

  30. MaxSAT Evaluation 2020. https://maxsat-evaluations.github.io/2020/

  31. Möhle, S., Biere, A.: Backing backtracking. In: Janota, M., Lynce, I. (eds.) SAT 2019. LNCS, vol. 11628, pp. 250–266. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-24258-9_18

    Chapter  Google Scholar 

  32. Morgado, A., Dodaro, C., Marques-Silva, J.: Core-guided MaxSAT with soft cardinality constraints. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 564–573. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10428-7_41

    Chapter  Google Scholar 

  33. Morgado, A., Heras, F., Liffiton, M.H., Planes, J., Marques-Silva, J.: Iterative and core-guided MaxSAT solving: a survey and assessment. Constraints Int. J. 18(4), 478–534 (2013). https://doi.org/10.1007/s10601-013-9146-2

    Article  MathSciNet  MATH  Google Scholar 

  34. Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S.: Chaff: engineering an efficient SAT solver. In: DAC, pp. 530–535 (2001)

    Google Scholar 

  35. Nadel, A., Ryvchin, V.: Chronological backtracking. In: Beyersdorff, O., Wintersteiger, C.M. (eds.) SAT 2018. LNCS, vol. 10929, pp. 111–121. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94144-8_7

    Chapter  Google Scholar 

  36. Oh, C.: Between SAT and UNSAT: the fundamental difference in CDCL SAT. In: Heule, M., Weaver, S. (eds.) SAT 2015. LNCS, vol. 9340, pp. 307–323. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24318-4_23

    Chapter  Google Scholar 

  37. Ohrimenko, O., Stuckey, P.J., Codish, M.: Propagation via lazy clause generation. Constraints Int. J. 14(3), 357–391 (2009). https://doi.org/10.1007/s10601-008-9064-x

    Article  MathSciNet  MATH  Google Scholar 

  38. Piette, C., Hamadi, Y., Saïs, L.: Vivifying propositional clausal formulae. In: ECAI, pp. 525–529 (2008)

    Google Scholar 

  39. Pipatsrisawat, K., Darwiche, A.: A lightweight component caching scheme for satisfiability solvers. In: Marques-Silva, J., Sakallah, K.A. (eds.) SAT 2007. LNCS, vol. 4501, pp. 294–299. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72788-0_28

    Chapter  Google Scholar 

  40. Ryan, L.: Efficient algorithms for clause-learning SAT solvers. Master’s thesis, School of Computing Science, Simon Fraser University (2004)

    Google Scholar 

  41. Zhang, X., Cai, S.: Relaxed backtracking with rephasing. In: Proceedings of SAT Competition 2020, vol. B-2020-1, pp. 15–16 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ITMO University, St. Petersburg, Russia

    Stepan Kochemazov

  2. Monash University, Melbourne, Australia

    Alexey Ignatiev

  3. IRIT, CNRS, Toulouse, France

    Joao Marques-Silva

Authors
  1. Stepan Kochemazov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexey Ignatiev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joao Marques-Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stepan Kochemazov .

Editor information

Editors and Affiliations

  1. Laboratoire MIS, University of Picardie Jules Verne, Amiens, France

    Chu-Min Li

  2. IIIA-CSIC, Spanish National Research Council (CSIC), Bellaterra, Barcelona, Spain

    Felip Manyà

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kochemazov, S., Ignatiev, A., Marques-Silva, J. (2021). Assessing Progress in SAT Solvers Through the Lens of Incremental SAT. In: Li, CM., Manyà, F. (eds) Theory and Applications of Satisfiability Testing – SAT 2021. SAT 2021. Lecture Notes in Computer Science(), vol 12831. Springer, Cham. https://doi.org/10.1007/978-3-030-80223-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80223-3_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80222-6

  • Online ISBN: 978-3-030-80223-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation