Skip to main content
Springer Nature Link
Log in

SMTQuery: Analysing SMT-LIB String Benchmarks

  • Conference paper
  • First Online:
Formal Methods: Foundations and Applications (SBMF 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15403))

Included in the following conference series:

  • 132 Accesses

Abstract

Constraint satisfaction problems involving strings have been a subject of theoretical study for decades, but it is only in recent years that practical solving methods have begun to emerge. This increasing interest in solving string constraints led to the development of various techniques and solvers, often accompanied by specific benchmark sets. As a result, there is now a substantial corpus of publicly available, yet largely unclassified, such benchmarks. In this context, we present SMTQuery, a framework for maintaining and analyzing benchmarks for SMT string problems. SMTQuery enables the execution of user-defined queries to extract domain-specific information from these benchmarks, facilitating a deeper analysis of the underlying problems. We demonstrate its utility by analyzing over 100,000 benchmarks and training an algorithm selection model to match benchmarks with suitable solvers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://smtquery.github.io/.

References

  1. Abdulla, P.A., et al.: Norn: An SMT solver for string constraints. In: International conference on Computer Aided Verification, pp. 462–469. Springer (2015)

    Google Scholar 

  2. Amadini, R.: A survey on string constraint solving. ACM Comput. Surv. (CSUR) 55(1), 1–38 (2021)

    Article  Google Scholar 

  3. Aydin, A., Bang, L., Bultan, T.: Automata-based model counting for string constraints. In: Computer Aided Verification: 27th International Conference, CAV 2015, San Francisco, CA, USA, July 18-24, 2015, Proceedings, Part I, pp. 255–272. Springer (2015)

    Google Scholar 

  4. Backes, J., et al.: Semantic-based automated reasoning for AWS access policies using SMT. In: 2018 Formal Methods in Computer Aided Design (FMCAD), pp. 1–9 (2018)

    Google Scholar 

  5. Barbosa, H., et al.: cvc5: a versatile and industrial-strength SMT solver. In: International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 415–442. Springer (2022)

    Google Scholar 

  6. Berzish, M., et al.: String theories involving regular membership predicates: from practice to theory and back. In: Lecroq, T., Puzynina, S. (eds.) Combinatorics on Words, pp. 50–64. Springer International Publishing, Cham (2021)

    Chapter  Google Scholar 

  7. Berzish, M., et al.: Towards more efficient methods for solving regular-expression heavy string constraints. Theor. Comput. Sci. 943, 50–72 (2023)

    Article  MathSciNet  Google Scholar 

  8. Berzish, M., Ganesh, V., Zheng, Y.: Z3str3: a string solver with theory-aware heuristics. In: 2017 Formal Methods in Computer Aided Design (FMCAD), pp. 55–59. IEEE (2017)

    Google Scholar 

  9. Blahoudek, F., et al.: Word equations in synergy with regular constraints. In: Chechik, M., Katoen, J.P., Leucker, M. (eds.) Formal Methods, pp. 403–423. Springer International Publishing, Cham (2023)

    Chapter  Google Scholar 

  10. Chen, T., Hague, M., Lin, A.W., Rümmer, P., Wu, Z.: Decision procedures for path feasibility of string-manipulating programs with complex operations. Proc. ACM Program. Lang. 3(POPL), 49:1-49:30 (2019)

    Google Scholar 

  11. Day, J.D., Ehlers, T., Kulczynski, M., Manea, F., Nowotka, D., Poulsen, D.B.: On solving word equations using SAT. In: International Conference on Reachability Problems, pp. 93–106. Springer (2019)

    Google Scholar 

  12. Day, J.D., Ganesh, V., Grewal, N., Manea, F.: On the expressive power of string constraints. Proc. ACM Program. Lang. 7(POPL), 278–308 (2023). https://doi.org/10.1145/3571203

  13. Day, J.D., Kulczynski, M., Manea, F., Nowotka, D., Poulsen, D.B.: Rule-based word equation solving. In: Proceedings of the 8th International Conference on Formal Methods in Software Engineering, pp. 87–97 (2020)

    Google Scholar 

  14. De Moura, L., Bjørner, N.: Z3: An efficient SMT solver. In: TACAS 2008, pp. 337–340. Springer (2008)

    Google Scholar 

  15. Draghici, A., Haase, C., Manea, F.: Semënov arithmetic, affine VASS, and string constraints. In: 41st International Symposium on Theoretical Aspects of Computer Science, STACS 2024. LIPIcs, vol. 289, pp. 29:1–29:19. Schloss Dagstuhl - Leibniz-Zentrum für Informatik (2024). https://doi.org/10.4230/LIPIcs.STACS.2024.29

  16. Hojjat, H., Rümmer, P., Shamakhi, A.: On strings in software model checking. In: Lin, A.W. (ed.) Programming Languages and Systems, pp. 19–30. Springer International Publishing, Cham (2019)

    Chapter  Google Scholar 

  17. Jez, A.: Word Equations in Nondeterministic Linear Space. In: Chatzigiannakis, I., Indyk, P., Kuhn, F., Muscholl, A. (eds.) 44th International Colloquium on Automata, Languages, and Programming (ICALP 2017). Leibniz International Proceedings in Informatics (LIPIcs), vol. 80, pp. 95:1–95:13. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany (2017). http://drops.dagstuhl.de/opus/volltexte/2017/7408

    Google Scholar 

  18. Kan, S., Lin, A.W., Rümmer, P., Schrader, M.: Certistr: A certified string solver (technical report) (2021, to appear in CPP 2022). CoRR arxiv.org/abs/2208.08806

  19. Kausler, S., Sherman, E.: Evaluation of string constraint solvers in the context of symbolic execution. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering. p. 259-270. ASE ’14, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2642937.2643003

  20. Kiezun, A., Ganesh, V., Artzi, S., Guo, P.J., Hooimeijer, P., Ernst, M.D.: Hampi: a solver for word equations over strings, regular expressions, and context-free grammars. ACM Trans. Softw. Eng. Methodol. 21(4), 1 (2013). https://doi.org/10.1145/2377656.2377662

    Article  Google Scholar 

  21. Kulczynski, M., Lotz, K., Nowotka, D., Poulsen, D.B.: Solving string theories involving regular membership predicates using sat. In: Legunsen, O., Rosu, G. (eds.) Model Checking Software, pp. 134–151. Springer International Publishing, Cham (2022)

    Chapter  Google Scholar 

  22. Kulczynski, M., Manea, F., Nowotka, D., Poulsen, D.B.: ZaligVinder: a generic test framework for string solvers. J. Softw. Evol. Process, e2400 (2021)

    Google Scholar 

  23. Lin, A.W., Barceló, P.: String solving with word equations and transducers: towards a logic for analysing mutation xss. In: Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 123–136 (2016)

    Google Scholar 

  24. Lotz, K., et al.: Solving string constraints using sat. In: Enea, C., Lal, A. (eds.) Computer Aided Verification, pp. 187–208. Springer Nature Switzerland, Cham (2023)

    Chapter  Google Scholar 

  25. Mora, F., Berzish, M., Kulczynski, M., Nowotka, D., Ganesh, V.: Z3str4: a multi-armed string solver. In: FM 2021, pp. 389–406. Springer (2021)

    Google Scholar 

  26. Plandowski, W., Rytter, W.: Application of Lempel-Ziv Encodings to the Solution of Word Equations, pp. 731–742. Springer Berlin Heidelberg (1998). https://doi.org/10.1007/BFb0055097

  27. Rice, J.R.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976). https://doi.org/10.1016/S0065-2458(08)60520-3

    Article  Google Scholar 

  28. Saxena, P., Akhawe, D., Hanna, S., Mao, F., McCamant, S., Song, D.: A symbolic execution framework for javascript. In: 2010 IEEE Symposium on Security and Privacy, pp. 513–528 (2010)

    Google Scholar 

  29. Scott, J., Niemetz, A., Preiner, M., Nejati, S., Ganesh, V.: Publisher correction: algorithm selection for SMT. Int. J. Softw. Tools Technol. Transf. 25(5), 799–800 (2023). https://doi.org/10.1007/s10009-023-00714-1

    Article  Google Scholar 

  30. Stanford, C., Veanes, M., Bjørner, N.: Symbolic boolean derivatives for efficiently solving extended regular expression constraints. In: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation. pp. 620–635. PLDI 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3453483.3454066

  31. Yu, F., Alkhalaf, M., Bultan, T.: STRANGER: an automata-based string analysis tool for PHP. In: Proceedings of the 16th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 154–157. TACAS’10, Springer-Verlag, Berlin, Heidelberg (2010)

    Google Scholar 

  32. Zheng, Y., Zhang, X., Ganesh, V.: Z3-str: a z3-based string solver for web application analysis. In: ESEC/SIGSOFT FSE 2013, pp. 114–124 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Kiel University, Kiel, Germany

    Mitja Kulczynski & Kevin Lotz

  2. Department of Computer Science, University of Göttingen, Göttingen, Germany

    Florin Manea &  Paul Sarnighausen-Cahn

  3. Department of Computer Science, Aalborg University, Aalborg, Denmark

    Danny Bøgsted Poulsen

Authors
  1. Mitja Kulczynski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Lotz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florin Manea
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danny Bøgsted Poulsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul Sarnighausen-Cahn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Sarnighausen-Cahn .

Editor information

Editors and Affiliations

  1. Federal Rural University of Pernambuco, Recife, Pernambuco, Brazil

    Sidney C. Nogueira

  2. ENSTA Bretagne, Brest, France

    Ciprian Teodorov

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kulczynski, M., Lotz, K., Manea, F., Poulsen, D.B., Sarnighausen-Cahn, . (2025). SMTQuery: Analysing SMT-LIB String Benchmarks. In: C. Nogueira, S., Teodorov, C. (eds) Formal Methods: Foundations and Applications. SBMF 2024. Lecture Notes in Computer Science, vol 15403. Springer, Cham. https://doi.org/10.1007/978-3-031-78116-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-78116-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-78115-5

  • Online ISBN: 978-3-031-78116-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics