Abstract
This paper reports about the 2004 comparative evaluation of solvers for quantified Boolean formulas (QBFs), the second in a series of non-competitive events established with the aim of assessing the advancements in the field of QBF reasoning and related research. We evaluated sixteen solvers on a test set of about one thousand benchmarks selected from instances submitted to the evaluation and from those available at www.qbflib.org. In the paper we present the evaluation infrastructure, from the criteria used to select the benchmarks to the hardware set up, and we show different views about the results obtained, highlighting the strength of different solvers and the relative hardness of the benchmarks included in the test set.
The work of the first author was partially supported by Université d’Artois, the IUT-Lens and FEDER. The work of the second and the last author was partially supported by ASI, MIUR and a grant from the Intel Corporation. The authors would like to thank all the participants to the QBF evaluation for submitting their work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Le Berre, D., Simon, L., Tacchella, A.: Challenges in the QBF arena: the SAT 2003 evaluation of QBF solvers. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 468–485. Springer, Heidelberg (2004)
Davis, M., Putnam, H.: A computing procedure for quantification theory. Journal of the ACM 7(3), 201–215 (1960)
Davis, M., Logemann, G., Loveland, D.: A machine program for theorem proving. Communications of the ACM 5(7), 394–397 (1962)
Rowley, A.G.D., Gent, I.P., Hoos, H.H., Smyth, K.: Using stochastic local search to solve quantified boolean formulae. In: Rossi, F. (ed.) CP 2003. LNCS, vol. 2833, pp. 348–362. Springer, Heidelberg (2003)
Biere, A.: Resolve and Expand. In: Seventh Intl. Conference on Theory and Applications of Satisfiability Testing (2004), Extended Abstract
Pan, G., Vardi, M.Y.: Symbolic decision procedures for QBF. In: Wallace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 453–467. Springer, Heidelberg (2004)
Gent, I., Walsh, T.: Beyond NP: the QSAT phase transition. In: Proc. of AAAI, pp. 648–653 (1999)
Giunchiglia, E., Narizzano, M., Tacchella, A.: Quantified Boolean Formulas satisfiability library, QBFLIB (2001), www.qbflib.org
Rowley, A.G., Gent, I.P.: Solution learning and solution directed backjumping revisited. Technical Report APES-80-2004, APES Research Group (February 2004)
Le Berre, D., Narizzano, M., Simon, L., Tacchella, A. (eds.): Second QBF solvers evaluation. Pacific Institute of Mathematics (2004), Available on-line at, www.qbflib.org
Rintanen, J.: Partial implicit unfolding in the Davis-Putnam procedure for quantified Boolean formulae. In: Nieuwenhuis, R., Voronkov, A. (eds.) LPAR 2001. LNCS (LNAI), vol. 2250, pp. 362–376. Springer, Heidelberg (2001)
Feldmann, R., Monien, B., Schamberger, S.: A distributed algorithm to evaluate quantified boolean formula. In: Proceedings of the Seventeenth National Conference in Artificial Intelligence (AAAI 2000), pp. 285–290 (2000)
Letz, R.: Lemma and model caching in decision procedures for quantified boolean formulas. In: Egly, U., Fermüller, C. (eds.) TABLEAUX 2002. LNCS (LNAI), vol. 2381, pp. 160–175. Springer, Heidelberg (2002)
Ayari, A., Basin, D.: Bounded model construction for monadic second-order logics. In: Emerson, E.A., Sistla, A.P. (eds.) CAV 2000. LNCS, vol. 1855, pp. 99–113. Springer, Heidelberg (2000)
Castellini, C., Giunchiglia, E., Tacchella, A.: Sat-based planning in complex domains: Concurrency, constraints and nondeterminism. Artificial Intelligence 147(1), 85–117 (2003)
Rowley, A.G., Gent, I.P.: Encoding connect 4 using quantified boolean formulae. Technical Report APES-68-2003, APES Research Group (July 2003)
Mneimneh, M., Sakallah, K.: Computing Vertex Eccentricity in Exponentially Large Graphs: QBF Formulation and Solution. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 411–425. Springer, Heidelberg (2004)
Pan, G., Vardi, M.Y.: Optimizing a BDD-based modal solver. In: Proceedings of the 19th International Conference on Automated Deduction (2003)
Balsiger, P., Heuerding, A., Schwendimann, S.: A benchmark method for the propositional modal logics k, kt, s4. Journal of Automated Reasoning 24(3), 297–317 (2000)
Rintanen, J.: Improvements to the evaluation of quantified boolean formulae. In: Proceedings of the Sixteenth International Joint Conferences on Artificial Intelligence (IJCAI 1999), Stockholm, Sweden, pp. 1192–1197. Morgan Kaufmann, San Francisco (1999)
Scholl, C., Becker, B.: Checking equivalence for partial implementations. In: 38th Design Automation Conference, DAC 2001 (2001)
Giunchiglia, E., Narizzano, M., Tacchella, A.: An Analysis of Backjumping and Trivial Truth in Quantified Boolean Formulas Satisfiability. In: Esposito, F. (ed.) AI*IA 2001. LNCS (LNAI), vol. 2175, p. 111. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Le Berre, D., Narizzano, M., Simon, L., Tacchella, A. (2005). The Second QBF Solvers Comparative Evaluation. In: Hoos, H.H., Mitchell, D.G. (eds) Theory and Applications of Satisfiability Testing. SAT 2004. Lecture Notes in Computer Science, vol 3542. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11527695_28
Download citation
DOI: https://doi.org/10.1007/11527695_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27829-0
Online ISBN: 978-3-540-31580-3
eBook Packages: Computer ScienceComputer Science (R0)