Abstract
Formally verified parsers are powerful tools for preventing the kinds of errors that result from ad hoc parsing and validation of program input. However, verified parsers are often based on formalisms that are not expressive enough to capture the full definition of valid input for a given application. Specifications of many real-world data formats include both a syntactic component and one or more non-context-free semantic properties that a well-formed instance of the format must exhibit. A parser for context-free grammars (CFGs) cannot determine on its own whether an input is valid according to such a specification; it must be supplemented with additional validation checks.
In this work, we present CoStar++, a verified parser interpreter with semantic features that make it highly expressive in terms of both the language specifications it accepts and its output type. CoStar++ provides support for semantic predicates, enabling the user to write semantically rich grammars that include non-context-free properties. The interpreter also supports semantic actions that convert sequential inputs to structured outputs in a principled way. CoStar++ is implemented and verified with the Coq Proof Assistant, and it is based on the ALL(*) parsing algorithm. For all CFGs without left recursion, the interpreter is provably sound, complete, and terminating with respect to a semantic specification that takes predicates and actions into account. CoStar++ runs in linear time on benchmarks for four real-world data formats, three of which have non-context-free specifications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use the term “parser interpreter” instead of “parser generator” because CoStar++ does not generate source code from a grammar; it converts a grammar to an in-memory data structure that a generic driver interprets at parse time.
- 2.
Throughout this paper, nonterminals begin with capital letters and terminals appear in single quotes. When it is necessary to distinguish between terminals and the literal values that they match, we write terminal names in angle brackets (e.g.,
for a terminal that matches an integer). - 3.
Internally, a CoStar++ grammar is a finite map in which each base production \(X \,{:}{:}\!= \gamma \) maps to an annotated production \(X' \,{:}{:}\!= \gamma ' \ \llbracket p \rrbracket ? \ \llbracket f \rrbracket !\). The well-formedness property says that \(X = X'\) and \(\gamma = \gamma '\) for each key/value pair in the map. This property enables the interpreter to retrieve the predicate and action for key \(X := \gamma \).
for a terminal that matches an integer).References
Congress-legislators database (2022). https://github.com/unitedstates/congress-legislators
Arnold, C., Matthews, L.J., Nunn, C.L.: The 10kTrees website: a new online resource for primate phylogeny. Evol. Anthropol. Issues News Rev. 19(3), 114–118 (2010)
Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. J. Am. Stat. Assoc. 74(368), 829–836 (1979)
Danielsson, N.A.: Total parser combinators. In: International Conference on Functional Programming (2010). https://doi.org/10.1145/1863543.1863585
Edelmann, R., Hamza, J., Kunčak, V.: Zippy LL(1) parsing with derivatives. In: Programming Language Design and Implementation (2020). https://doi.org/10.1145/3385412.3385992
Egolf, D., Lasser, S., Fisher, K.: Verbatim: a verified lexer generator. In: LangSec Workshop (2021). https://langsec.org/spw21/papers.html#verbatim
Egolf, D., Lasser, S., Fisher, K.: Verbatim++: verified, optimized, and semantically rich lexing with derivatives. In: Certified Programs and Proofs (2022). https://doi.org/10.1145/3497775.3503694
Jourdan, J.-H., Pottier, F., Leroy, X.: Validating LR(1) parsers. In: Seidl, H. (ed.) ESOP 2012. LNCS, vol. 7211, pp. 397–416. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28869-2_20
Lasser, S., Casinghino, C., Egolf, D., Fisher, K., Roux, C.: GitHub repository for the CoStar++ development and performance evaluation framework (2022). https://github.com/slasser/CoStar
Lasser, S., Casinghino, C., Fisher, K., Roux, C.: A verified LL(1) parser generator. In: Interactive Theorem Proving (2019). https://doi.org/10.4230/LIPIcs.ITP.2019.24
Lasser, S., Casinghino, C., Fisher, K., Roux, C.: CoStar: a verified ALL(*) parser. In: Programming Language Design and Implementation (2021). https://doi.org/10.1145/3453483.3454053
Momot, F., Bratus, S., Hallberg, S.M., Patterson, M.L.: The seven turrets of babel: a taxonomy of LangSec errors and how to expunge them. In: IEEE Cybersecurity Development (2016). https://doi.org/10.1109/SecDev.2016.019
Open American National Corpus (2010). https://www.anc.org/data/oanc/download/
Parr, T., Harwell, S., Fisher, K.: Adaptive LL(*) parsing: the power of dynamic analysis. In: Object-Oriented Programming, Systems, Languages, and Applications (2014). https://doi.org/10.1145/2660193.2660202
Ridge, T.: Simple, functional, sound and complete parsing for all context-free grammars. In: Jouannaud, J.-P., Shao, Z. (eds.) CPP 2011. LNCS, vol. 7086, pp. 103–118. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25379-9_10
Scott, E., Johnstone, A.: GLL parsing. Elect. Notes Theor. Comput. Sci. 253(7), 177–189 (2010). https://doi.org/10.1016/j.entcs.2010.08.041
Acknowledgments
Sam Lasser’s research was supported by a Draper Scholarship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lasser, S., Casinghino, C., Egolf, D., Fisher, K., Roux, C. (2023). Verified ALL(*) Parsing with Semantic Actions and Dynamic Input Validation. In: Rozier, K.Y., Chaudhuri, S. (eds) NASA Formal Methods. NFM 2023. Lecture Notes in Computer Science, vol 13903. Springer, Cham. https://doi.org/10.1007/978-3-031-33170-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-33170-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33169-5
Online ISBN: 978-3-031-33170-1
eBook Packages: Computer ScienceComputer Science (R0)