skip to main content
10.1145/2491411.2491442acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

Will you still compile me tomorrow? static cross-version compiler validation

Published: 18 August 2013 Publication History

Abstract

This paper describes a cross-version compiler validator and measures its effectiveness on the CLR JIT compiler. The validator checks for semantically equivalent assembly language output from various versions of the compiler, including versions across a seven-month time period, across two architectures (x86 and ARM), across two compilation scenarios (JIT and MDIL), and across optimizations levels. For month-to-month comparisons, the validator achieves a false alarm rate of just 2.2%. To help understand reported semantic differences, the validator performs a root-cause analysis on the counterexample traces generated by the underlying automated theorem proving tools. This root-cause analysis groups most of the counterexamples into a small number of buckets, reducing the number of counterexamples analyzed by hand by anywhere from 53% to 96%. The validator ran on over 500,000 methods across a large suite of test programs, finding 12 previously unknown correctness and performance bugs in the CLR compiler.

References

[1]
M. Barnett, B.-Y. E. Chang, R. DeLine, B. Jacobs, and K. R. M. Leino. Boogie: A modular reusable verifier for object-oriented programs. In Formal Methods for Components and Objects (FMCO), volume 4111, 2006.
[2]
A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler. A few billion lines of code later: using static analysis to find bugs in the real world. In Communications of the ACM, Feb. 2010.
[3]
J. Chen, C. Hawblitzel, F. Perry, M. Emmi, J. Condit, D. Coetzee, and P. Pratikakis. Type-preserving compilation for large-scale optimizing object-oriented compilers. SIGPLAN Not., 43(6):183–192, 2008.
[4]
L. M. de Moura and N. Bjørner. Z3: An efficient SMT solver. In TACAS, pages 337–340, 2008.
[5]
C. Hawblitzel and E. Petrank. Automated verification of practical garbage collectors. In POPL, pages 441–453, 2009.
[6]
M. Jose and R. Majumdar. Cause clue clauses: Error localization using maximum satisfiability. In PLDI, 2011.
[7]
S. K. Lahiri, C. Hawblitzel, M. Kawaguchi, and H. Rebˆ elo. Symdiff: A language-agnostic semantic diff tool for imperative programs. In Computer Aided Verification (CAV ˇ S12), 2012.
[8]
X. Leroy. Formal certification of a compiler back-end or: programming a compiler with a proof assistant. In Principles of Programming Languages (POPL ’06), pages 42–54, 2006.
[9]
G. C. Necula. Translation validation for an optimizing compiler. In Programming Language Design and Implementation (PLDI ’00), pages 83–94, 2000.
[10]
A. Pardoe. Clr inside out: Program silverlight with the coreclr. In MSDN Magazine, Aug. 2008.
[11]
A. Pnueli, M. Siegel, and E. Singerman. Translation validation. In Tools and Algorithms for Construction and Analysis of Systems (TACAS ’98), pages 151–166, 1998.
[12]
S. Ramaswamy. Deep dive into the kernel of .NET on Windows Phone 8. In Build Conference, Nov. 2012.
[13]
D. Ramos and D. Engler. Practical, low-effort equivalence verification of real code. In CAV, 2011.
[14]
M. Stepp, R. Tate, and S. Lerner. Equality-based translation validator for llvm. In CAV, 2011.
[15]
R. Tate, M. Stepp, Z. Tatlock, and S. Lerner. Equality saturation: a new approach to optimization. In Principles of Programming Languages (POPL ’09), pages 264–276, 2009.
[16]
J. Tristan, P. Govereau, and G. Morrisett. Evaluating value-graph translation validation for llvm. In Programming Language Design and Implementation (PLDI ’11), pages 295–305, 2011.
[17]
X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in c compilers. In PLDI. ACM Press, 2011.
[18]
J. Zhao, S. Zdancewic, S. Nagarakatte, and M. M. K. Martin. Formalizing the llvm intermediate representation for verified program transformation. In POPL, 2012.

Cited By

View all
  • (2024)Assembly Function Recognition in Embedded Systems as an Optimization ProblemMathematics10.3390/math1205065812:5(658)Online publication date: 23-Feb-2024
  • (2024)AsFuzzer: Differential Testing of Assemblers with Error-Driven Grammar InferenceProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680345(1099-1111)Online publication date: 11-Sep-2024
  • (2024)Automating the correctness assessment of AI-generated code for security contextsJournal of Systems and Software10.1016/j.jss.2024.112113216(112113)Online publication date: Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEC/FSE 2013: Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering
August 2013
738 pages
ISBN:9781450322379
DOI:10.1145/2491411
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 August 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Compilers
  2. Translation Validation
  3. Verification

Qualifiers

  • Research-article

Conference

ESEC/FSE'13
Sponsor:

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)2
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Assembly Function Recognition in Embedded Systems as an Optimization ProblemMathematics10.3390/math1205065812:5(658)Online publication date: 23-Feb-2024
  • (2024)AsFuzzer: Differential Testing of Assemblers with Error-Driven Grammar InferenceProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680345(1099-1111)Online publication date: 11-Sep-2024
  • (2024)Automating the correctness assessment of AI-generated code for security contextsJournal of Systems and Software10.1016/j.jss.2024.112113216(112113)Online publication date: Oct-2024
  • (2023)Automatic Target Description File GenerationJournal of Computer Science and Technology10.1007/s11390-022-1919-x38:6(1339-1355)Online publication date: 30-Nov-2023
  • (2021)Language-parametric compiler validation with application to LLVMProceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3445814.3446751(1004-1019)Online publication date: 19-Apr-2021
  • (2021)Automatically Checking Semantic Equivalence between Versions of Large-Scale C Projects2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST49551.2021.00045(329-339)Online publication date: Apr-2021
  • (2020)GauntletProceedings of the 14th USENIX Conference on Operating Systems Design and Implementation10.5555/3488766.3488805(683-699)Online publication date: 4-Nov-2020
  • (2020)A Survey of Compiler TestingACM Computing Surveys10.1145/336356253:1(1-36)Online publication date: 6-Feb-2020
  • (2019)Computing summaries of string loops in C for better testing and refactoringProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3314221.3314610(874-888)Online publication date: 8-Jun-2019
  • (2019)Semantic program alignment for equivalence checkingProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3314221.3314596(1027-1040)Online publication date: 8-Jun-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media