Staged compilation

Traditional compilers compile and optimize files separately, making worst-case assumptions about the program context in which a file is to be linked. More aggressive compilation architectures perform cross-file interprocedural or whole-program analyses, potentially producing much faster programs but substantially increasing the cost of compilation. Even more radical are systems that perform all compilation and optimization at run-time: such systems can optimize programs based on run-time program and system properties as well as static whole-program properties. However, run-time compilers (also called dynamic compilers or just-in-time compilers) suffer under severe constraints on allowable compilation time, since any time spent compiling steals from time spent running the program. None of these compilation models dominates the others: each has unique strengths and weaknesses not present in the other models.We are developing a new, staged compilation model which strives to combine high run-time code quality with low compilation overhead. Compilation is organized as a series of stages, with stages corresponding to, for example, separate compilation, library linking, program linking, and run-time execution. Any given optimization can be performed at any of these stages; to reduce compilation time while maintaining high effectiveness, an optimization should be performed at the earliest stage that provides the necessary program context information to carry out the optimization effectively. Moreover, a single optimization can itself be spread across multiple stages, with earlier stages performing preplanning work that enables the final stage to complete the optimization quickly. In this way, we hope to produce highly optimized programs, nearly as good as what could be done with a purely run-time compiler that had an unconstrained compilation time budget, but at a much more practical compile time cost.We are building the Whirlwind optimizing compiler as the concrete embodiment of this staged compilation model, initially targeting object-oriented languages. A key component of Whirlwind is a set of techniques for automatically constructing staged compilers from traditional unstaged compilers, including aggressive applications of specialization and other partial evaluation technology.