skip to main content
10.1145/1529282.1529710acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

TWEAST: a simple and effective technique to implement concrete-syntax AST rewriting using partial parsing

Published: 08 March 2009 Publication History

Abstract

Abstract Syntax Trees (ASTs) are commonly used to represent an input/output program in compilers and language processing tools. Many of the tasks of these tools consist in generating and rewriting ASTs. Such an approach can become tedious and hard to maintain for complex operations, namely program transformation, optimization, instrumentation, etc. On the other hand, concrete syntax provides a natural and simpler representation of programs, but it is not usually available as a direct feature of the aforementioned tools. We propose a simple technique to implement AST generation and rewriting in general purpose languages using concrete syntax. Our approach relies on extensions made in the scanner and the parser and the use of objects supporting partial parsing called Texts With Embedded Abstract Syntax Trees (TWEASTs). A compiler for a simple language (Tiger) written in C++ serves as an example, featuring transformations in concrete syntax: syntactic desugaring, optimization, code instrumentation such as bounds-checking, etc. Extensions of this technique to provide a full-fledged concrete-syntax rewriting framework are presented as well.

References

[1]
A. W. Appel. Modern Compiler Implementation in C, Java, ML. Cambridge University Press, 1998.
[2]
E. Balland, P. Brauner, R. Kopetz, P.-E. Moreau, and A. Reilles. Tom: Piggybacking rewriting on Java. In Proceedings of the 18th Conference on Rewriting Techniques and Applications, Lecture Notes in Computer Science. Springer-Verlag, 2007.
[3]
M. Bravenboer, K. T. Kalleberg, R. Vermaas, and E. Visser. Stratego/XT 0.16. Components for transformation systems. In ACM SIGPLAN 2006 Workshop on Partial Evaluation and Program Manipulation (PEPM'06), Charleston, South Carolina, January 2006. ACM SIGPLAN.
[4]
R. Corbett, R. Stallman, and P. Hilfinger. Bison: GNU LALR(1) and GLR parser generator, 2003. http://www.gnu.org/software/bison/bison.html.
[5]
A. Demaille. Making compiler construction projects relevant to core curriculums. In Proceedings of the Tenth Annual Conference on Innovation and Technology in Computer Science Education (ITICSE'05), pages 266--270, Universidade Nova de Lisboa, Monte da Pacarita, Portugal, June 2005.
[6]
A. Demaille. The Tiger Compiler Reference Manual. EPITA Research and Development Laboratory (LRDE), 14--16 rue Voltaire, FR-94270 Le Kremlin-Bicêtre, France, 2007. http://www.lrde.epita.fr/~akim/ccmp/tiger.pdf.
[7]
E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional Computing Series. Addison-Wesley Publishing Company, New York, NY, 1995.
[8]
T. Kuipers and J. Visser. Object-oriented tree traversal with JJForester. Science of Computer Programming, 47(1):59--87, 2003.
[9]
A. Leung. Prop: A C++-based pattern matching language. Technical report, Courant Institute of Mathematical Sciences, 1996.
[10]
T. J. Parr and R. W. Quong. ANTLR: A predicated-LL(k) parser generator. Software, Practice and Experience, 25(7):789--810, 1995.
[11]
V. Paxson, W. Estes, and J. Millaway. The Flex Manual. The Flex Project, September 2007. http://flex.sourceforge.net/manual/index.html.
[12]
E. Visser. A family of syntax definition formalisms. In M. G. J. van den Brand et al., editors, ASF+SDF'95. A Workshop on Generating Tools from Algebraic Specifications, pages 89--126. Technical Report P9504, Programming Research Group, University of Amsterdam, May 1995.
[13]
E. Visser. Scannerless generalized-LR parsing. Technical Report P9707, Programming Research Group, University of Amsterdam, July 1997.
[14]
E. Visser. Meta-programming with concrete object syntax. In D. Batory, C. Consel, and W. Taha, editors, Generative Programming and Component Engineering (GPCE'02), volume 2487 of Lecture Notes in Computer Science, pages 299--315, Pittsburgh, PA, USA, October 2002. Springer-Verlag.
[15]
E. Visser. Program transformation with Stratego/XT: Rules, strategies, tools, and systems in StrategoXT-0.9. In C. Lengauer et al., editors, Domain-Specific Program Generation, volume 3016 of Lecture Notes in Computer Science, pages 216--238. Springer-Verlag, June 2004.
[16]
R. Weatherley. Treecc, the Tree Compiler-Compiler. http://www.southern-storm.com.au/treecc.html, 2002.

Cited By

View all
  • (2016)Synthesizing framework models for symbolic executionProceedings of the 38th International Conference on Software Engineering10.1145/2884781.2884856(156-167)Online publication date: 14-May-2016
  • (2009)The ruby intermediate languageACM SIGPLAN Notices10.1145/1837513.164014844:12(89-98)Online publication date: 26-Oct-2009
  • (2009)The ruby intermediate languageProceedings of the 5th symposium on Dynamic languages10.1145/1640134.1640148(89-98)Online publication date: 26-Oct-2009

Index Terms

  1. TWEAST: a simple and effective technique to implement concrete-syntax AST rewriting using partial parsing

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SAC '09: Proceedings of the 2009 ACM symposium on Applied Computing
      March 2009
      2347 pages
      ISBN:9781605581668
      DOI:10.1145/1529282
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 08 March 2009

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. C++
      2. compiler design
      3. concrete syntax
      4. parsing
      5. program transformation
      6. rewrite rules

      Qualifiers

      • Research-article

      Conference

      SAC09
      Sponsor:
      SAC09: The 2009 ACM Symposium on Applied Computing
      March 8, 2009 - March 12, 2008
      Hawaii, Honolulu

      Acceptance Rates

      Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

      Upcoming Conference

      SAC '25
      The 40th ACM/SIGAPP Symposium on Applied Computing
      March 31 - April 4, 2025
      Catania , Italy

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)1
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 16 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2016)Synthesizing framework models for symbolic executionProceedings of the 38th International Conference on Software Engineering10.1145/2884781.2884856(156-167)Online publication date: 14-May-2016
      • (2009)The ruby intermediate languageACM SIGPLAN Notices10.1145/1837513.164014844:12(89-98)Online publication date: 26-Oct-2009
      • (2009)The ruby intermediate languageProceedings of the 5th symposium on Dynamic languages10.1145/1640134.1640148(89-98)Online publication date: 26-Oct-2009

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media