skip to main content
10.1145/2451512.2451541acmconferencesArticle/Chapter ViewAbstractPublication PagesveeConference Proceedingsconference-collections
research-article

A modular approach to on-stack replacement in LLVM

Published: 16 March 2013 Publication History

Abstract

On-stack replacement (OSR) is a technique that allows a virtual machine to interrupt running code during the execution of a function/method, to re-optimize the function on-the-fly using an optimizing JIT compiler, and then to resume the interrupted function at the point and state at which it was interrupted. OSR is particularly useful for programs with potentially long-running loops, as it allows dynamic optimization of those loops as soon as they become hot.
This paper presents a modular approach to implementing OSR for the LLVM compiler infrastructure. This is an important step forward because LLVM is gaining popular support, and adding the OSR capability allows compiler developers to develop new dynamic techniques. In particular, it will enable more sophisticated LLVM-based JIT compiler approaches. Indeed, other compiler/VM developers can use our approach because it is a clean modular addition to the standard LLVM distribution. Further, our approach is defined completely at the LLVM-IR level and thus does not require any modifications to the target code generation.
The OSR implementation can be used by different compilers to support a variety of dynamic optimizations. As a demonstration of our OSR approach, we have used it to support dynamic inlining in McVM. McVM is a virtual machine for MATLAB which uses a LLVM-based JIT compiler. MATLAB is a popular dynamic language for scientific and engineering applications that typically manipulate large matrices and often contain long-running loops, and is thus an ideal target for dynamic JIT compilation and OSRs. Using our McVM example, we demonstrate reasonable overheads for our benchmark set, and performance improvements when using it to perform dynamic inlining.

References

[1]
LLVM. http://www.llvm.org/.
[2]
A. Adl-Tabatabai, J. Bharadwaj, D. Chen, A. Ghuloum, V. Menon, B. Murphy, M. Serrano, and T. Shpeisman. StarJIT: A Dynamic Compiler for Managed Runtime Environments. Intel Technology Journal, 7(1):19--31, Feb 2003.
[3]
B. Alpern, S. Augart, S. M. Blackburn, M. Butrico, A. Cocchi, P. Cheng, J. Dolby, S. Fink, D. Grove, M. Hind, K. S. McKinley, M. Mergen, J. E. B. Moss, T. Ngo, and V. Sarkar. The Jikes Research Virtual Machine Project: Building an Open-Source Research Community. IBM Syst. J., 44(2):399--417, 2005.
[4]
M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive Optimization in the Jalapeno JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '00, pages 47--65, New York, USA, 2000. ACM.
[5]
M. Arnold, M. Hind, and B. G. Ryder. Online Feedback-Directed Optimization of Java. In Proceedings of the 17th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '02, pages 111--129, New York, USA, 2002. ACM.
[6]
C. Chambers and D. Ungar. Making Pure Object-Oriented Languages Practical. In Conference Proceedings on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA '91, pages 1--15, New York, USA, 1991. ACM.
[7]
M. Chevalier-Boisvert, L. Hendren, and C. Verbrugge. Optimizing MATLAB through Just-In-Time Specialization. In International Conference on Compiler Construction, pages 46--65, March 2010.
[8]
M. Cierniak, G.-Y. Lueh, and J. M. Stichnoth. Practicing JUDO: Java Under Dynamic Optimizations. In Proceedings of the ACMSIGPLAN 2000 Conference on Programming Language Design and Implementation, PLDI '00, pages 13--26, New York, USA, 2000. ACM.
[9]
Cleve Moler. Numerical Computing with MATLAB. SIAM, 2004.
[10]
S. J. Fink and F. Qian. Design, Implementation and Evaluation of Adaptive Recompilation with On-stack Replacement. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO '03, pages 241--252, Washington, DC, USA, 2003. IEEE Computer Society.
[11]
K. Hazelwood and D. Grove. Adaptive Online Context-Sensitive Inlining. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO '03, pages 253--264, Washington, DC, USA, 2003. IEEE Computer Society.
[12]
U. Hölzle, C. Chambers, and D. Ungar. Debugging Optimized Code with Dynamic Deoptimization. In Proceedings of the ACM SIGPLAN 1992 Conference on Programming Language Design and Implementation, PLDI '92, pages 32--43, New York, NY, USA, 1992. ACM.
[13]
U. Hölzle and D. Ungar. A Third-Generation SELF Implementation: Reconciling Responsiveness with Performance. In Proceedings of the Ninth Annual Conference on Object-Oriented Programming Systems, Language, and Applications, OOPSLA '94, pages 229--243, New York, NY, USA, 1994. ACM.
[14]
C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In CGO '04: Proceedings of the International Symposium on Code Generation and Optimization, pages 75--86, Washington, DC, USA, 2004. IEEE Computer Society.
[15]
MathWorks. MATLAB Programming Fundamentals. The Math-Works, Inc., 2009.
[16]
McLAB. The McVM Virtual Machine and its JIT Compiler, 2012. http://www.sable.mcgill.ca/mclab/mcvm_mcjit.html.
[17]
C. Moler. The Growth of MATLAB"and TheMathWorks over Two Decades, 2006. http://www.mathworks.com/company/newsletters/news_notes/clevescorner/jan06.pdf.
[18]
M. Paleczny, C. Vick, and C. Click. The Java HotSpot Server Compiler. In Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium - Volume 1, JVM'01, pages 1--12, Berkeley, CA, USA, 2001. USENIX Association.
[19]
Press, H. William and Teukolsky, A. Saul and Vetterling, T. William and Flannery, P. Brian. Numerical Recipes : the Art of Scientific Computing. Cambridge University Press, 1986.
[20]
L. D. Rose, K. Gallivan, E. Gallopoulos, B. A. Marsolf, and D. A. Padua. FALCON: A MATLAB Interactive Restructuring Compiler. In LCPC '95: Proceedings of the 8th International Workshop on Languages and Compilers for Parallel Computing, pages 269--288, London, UK, 1996. Springer-Verlag.
[21]
S. Soman and C. Krintz. Efficient and General On-Stack Replacement for Aggressive Program Specialization. In Software Engineering Research and Practice, pages 925--932, 2006.
[22]
T. Suganuma, T. Yasue, and T. Nakatani. An Empirical Study of Method In-lining for a Java Just-In-Time Compiler. In Proceedings of the 2nd Java Virtual Machine Research and Technology Symposium, pages 91--104, Berkeley, CA, USA, 2002. USENIX Association.
[23]
M. Süsskraut, T. Knauth, S. Weigert, U. Schiffel, M. Meinhold, and C. Fetzer. Prospect: A Compiler Framework for Speculative Parallelization. In Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization, pages 131--140, New York, USA, 2010. ACM.

Cited By

View all
  • (2022)Principled Composition of Function Variants for Dynamic Software Diversity and Program ProtectionProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3559553(1-5)Online publication date: 10-Oct-2022
  • (2022)Deoptless: speculation with dispatched on-stack replacement and specialized continuationsProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523729(749-761)Online publication date: 9-Jun-2022
  • (2021)Formally verified speculation and deoptimization in a JIT compilerProceedings of the ACM on Programming Languages10.1145/34343275:POPL(1-26)Online publication date: 4-Jan-2021
  • Show More Cited By

Index Terms

  1. A modular approach to on-stack replacement in LLVM

    Recommendations

    Reviews

    William M. Waite

    On-stack replacement is the process of interrupting a running program, optimizing some portion of that program, and then continuing execution from the point of interruption. It dynamically implements a mixed-code approach [1], in which part of the program is compact and slow, and part is expanded and fast. This approach takes advantage of the observation that programs tend to spend most of their time in a tiny fraction of their code. The authors of this paper demonstrate an implementation of on-stack replacement in the context of LLVM, a virtual machine whose instruction set is close to that of conventional hardware. Their explanation of the implementation is clear, with well-chosen flow graphs and code snippets. Given their goals, however, the implementation is complex and therefore the description demands careful attention. A reader will need to be familiar with the basic concepts of code generation and optimization to understand this paper. The implementation was evaluated on 16 programs, by measuring the differences in average running time with and without replacements. Six of the programs never actually performed a replacement, so these cases measured the overhead imposed by the technique (one percent or less). Of the remaining ten programs, three showed improvements of one to four percent and one was sped up by a factor of two. This indicates that the implementation is not particularly costly, and can sometimes provide a significant gain. Although the authors' description follows their implementation, their technique is quite general. It's easy to imagine using the same approach in a wide variety of compilation or execution contexts. Online Computing Reviews Service

    Access critical reviews of Computing literature here

    Become a reviewer for Computing Reviews.

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
    March 2013
    210 pages
    ISBN:9781450312660
    DOI:10.1145/2451512
    • cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 48, Issue 7
      VEE '13
      July 2013
      194 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2517326
      Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 March 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. dynamic optimization
    2. jit compilation
    3. llvm
    4. matlab
    5. mcjit
    6. on-stack replacement

    Qualifiers

    • Research-article

    Conference

    VEE '13

    Acceptance Rates

    Overall Acceptance Rate 80 of 235 submissions, 34%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)8
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Principled Composition of Function Variants for Dynamic Software Diversity and Program ProtectionProceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3559553(1-5)Online publication date: 10-Oct-2022
    • (2022)Deoptless: speculation with dispatched on-stack replacement and specialized continuationsProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523729(749-761)Online publication date: 9-Jun-2022
    • (2021)Formally verified speculation and deoptimization in a JIT compilerProceedings of the ACM on Programming Languages10.1145/34343275:POPL(1-26)Online publication date: 4-Jan-2021
    • (2019)Hack for hireCommunications of the ACM10.1145/335938662:12(32-37)Online publication date: 21-Nov-2019
    • (2019)KRISM—Krylov Subspace-based Optical Computing of Hyperspectral ImagesACM Transactions on Graphics10.1145/334555338:5(1-14)Online publication date: 17-Oct-2019
    • (2018)On-stack replacement, distilledACM SIGPLAN Notices10.1145/3296979.319239653:4(166-180)Online publication date: 11-Jun-2018
    • (2018)Hop, Skip, & JumpACM SIGPLAN Notices10.1145/3296975.318641253:3(1-16)Online publication date: 25-Mar-2018
    • (2018)Revenue Maximization and Ex-Post Budget ConstraintsACM Transactions on Economics and Computation10.1145/32746476:3-4(1-19)Online publication date: 1-Oct-2018
    • (2018)Upward mobility for underrepresented studentsACM Inroads10.1145/32105559:2(72-78)Online publication date: 27-Apr-2018
    • (2018)On-stack replacement, distilledProceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3192366.3192396(166-180)Online publication date: 11-Jun-2018
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media