research-article

CompCertTSO: A Verified Compiler for Relaxed-Memory Concurrency

Authors:

Jaroslav Ševčík,

Viktor Vafeiadis,

Francesco Zappa Nardelli,

Suresh Jagannathan,

Peter SewellAuthors Info & Claims

Journal of the ACM (JACM), Volume 60, Issue 3

Article No.: 22, Pages 1 - 50

https://doi.org/10.1145/2487241.2487248

Published: 01 June 2013 Publication History

Abstract

In this article, we consider the semantic design and verified compilation of a C-like programming language for concurrent shared-memory computation on x86 multiprocessors. The design of such a language is made surprisingly subtle by several factors: the relaxed-memory behavior of the hardware, the effects of compiler optimization on concurrent code, the need to support high-performance concurrent algorithms, and the desire for a reasonably simple programming model. In turn, this complexity makes verified compilation both essential and challenging.

We describe ClightTSO, a concurrent extension of CompCert’s Clight in which the TSO-based memory model of x86 multiprocessors is exposed for high-performance code, and CompCertTSO, a formally verified compiler from ClightTSO to x86 assembly language, building on CompCert. CompCertTSO is verified in Coq: for any well-behaved and successfully compiled ClightTSO source program, any permitted observable behavior of the generated assembly code (if it does not run out of memory) is also possible in the source semantics. We also describe some verified fence-elimination optimizations, integrated into CompCertTSO.

References

[1]

Martn Abadi and Leslie Lamport. 1991. The existence of refinement mappings. Theoret. Comput. Sci. 253--284.

Digital Library

[2]

Sarita V. Adve and Mark D. Hill. 1990. Weak ordering---a new definition. In Proceedings of the 17th International Symposium on Computer Architecture (ISCA). ACM, New York, 2--14.

Digital Library

[3]

Jade Alglave. 2010. A shared memory poetics. Ph.D. dissertation. Université Paris 7.

[4]

Jade Alglave, Luc Maranget, Susmit Sarkar, and Peter Sewell. 2010. Fences in weak memory models. In Proceedings of the 22nd International Conference on Computer Aided Verification (CAV). Springer-Verlag, Berlin, Heidelberg, 258--272.

Digital Library

[5]

Mark Batty, Scott Owens, Susmit Sarkar, Peter Sewell, and Tjark Weber. 2011. Mathematizing C++ concurrency. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 55--66.

Digital Library

[6]

Mark Batty, Kayvan Memarian, Scott Owens, Susmit Sarkar, and Peter Sewell. 2012. Clarifying and compiling C/C++ concurrency: From C++11 to POWER. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 509--520.

Digital Library

[7]

Pete Becker Ed. 2011. Programming Languages --- C++, ISO/IEC 14882:2011 (N3242). (last accessed 5/13). (N3242 is a near-final draft standard which is available online).

[8]

Nick Benton and Chung-Kil Hur. 2009. Biorthogonality, step-indexing and compiler correctness. In Proceedings of the 14th ACM SIGPLAN International Conference on Functional Programming (ICFP). ACM, New York, 97--108.

Digital Library

[9]

Sandrine Blazy and Xavier Leroy. 2009. Mechanized semantics for the Clight subset of the C language. J. Automat. Reason. 43, 3, 263--288.

[10]

Hans-J. Boehm. 2005. Threads cannot be implemented as a library. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 261--268.

Digital Library

[11]

Hans-J. Boehm and Sarita V. Adve. 2008. Foundations of the C++ concurrency memory model. In Proceedings of the SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 68--78.

Digital Library

[12]

Sebastian Burckhardt, Rajeev Alur, and Milo M. K. Martin. 2007. CheckFence: Checking consistency of concurrent data types on relaxed memory models. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 12--21.

Digital Library

[13]

Sebastian Burckhardt, Madanlal Musuvathi, and Vasu Singh. 2010. Verifying local transformations on relaxed memory models. In Proceedings of the International Conference on Compiler Construction (CC). Springer-Verlag, Berlin, 104--123.

Digital Library

[14]

Chi Cao Minh, JaeWoong Chung, Christos Kozyrakis, and Kunle Olukotun. 2008. STAMP: Stanford transactional applications for multi-processing. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC). 35--46.

[15]

Pietro Cenciarelli, Alexander Knapp, and Eleonora Sibilio. 2007. The Java memory model: Operationally, denotationally, axiomatically. In Proceedings of the 16th European Symposium on Programming (ESOP). Lecture Notes in Computer Science, vol. 4421, Springer-Verlag, Berlin, 331--346.

Digital Library

[16]

Adam Chlipala. 2010. A verified compiler for an impure functional language. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 93--106.

Digital Library

[17]

Coq. 2011. The Coq proof assistant. http://coq.inria.fr/, version 8.3pl1.

[18]

Dave Dice, Ori Shalev, and Nir Shavit. 2006. Transactional locking II. In Proceedings of the 20th International Conference on Distributed Computing (DISC). Springer-Verlag, Berlin, 194--208.

Digital Library

[19]

Keir Fraser. 2003. Practical lock freedom. Ph.D. dissertation. University of Cambridge.

[20]

Georges Gonthier and Assia Mahboubi. 2007. A small scale reflection extension for the Coq system. Tech. rep. 6455. INRIA.

[21]

Aquinas Hobor, Andrew W. Appel, and Francesco Zappa Nardelli. 2008. Oracle semantics for concurrent separation logic. In Proceedings of the European Symposium on Programming (ESOP). Lecture Notes in Computer Science, vol. 4960. Springer-Verlag, Berlin, 353--367.

Digital Library

[22]

Thuan Quang Huynh and Abhik Roychoudhury. 2007. Memory model sensitive bytecode verification. Formal Methods in System Design 31, 3, 281--305.

Digital Library

[23]

ISO. 2011. Programming languages --- C, ISO/IEC 9899:2011 (N1547). http://www.open-std.org/jtc1/sc22/wg14/www/docs/PostBatavia.tar.bz2 (last accessed 5/13). (N1547 is a near-final draft standard which is available online).

[24]

Michael Kuperstein, Martin Vechev, and Eran Yahav. 2010. Automatic inference of memory fences. In Proceedings of the Formal Methods in Computer-Aided Design (FMCAD). FMCAD Inc., Austin, TX, 111--120.

Digital Library

[25]

Leslie Lamport. 1979. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Trans. Comput. C-28, 9, 690--691.

Digital Library

[26]

Doug Lea. 1999. Concurrent Programming in Java 2nd Ed. Design Principles and Patterns. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.

Digital Library

[27]

Jaejin Lee and David A. Padua. 2001. Hiding relaxed memory consistency with a compiler. IEEE Trans. Comput. 50, 824--833. Issue 8.

Digital Library

[28]

Xavier Leroy. 2009a. The compcert verified compiler, v. 1.5. http://compcert.inria.fr/release/compcert-1.5.tgz (last accessed 5/13).

[29]

Xavier Leroy. 2009b. Formal verification of a realistic compiler. Comm. ACM 52, 7, 107--115.

Digital Library

[30]

Xavier Leroy. 2009c. A formally verified compiler back-end. J. Automat. Reason. 43, 4, 363--446.

Digital Library

[31]

Xavier Leroy. 2013. The Compcert verified compiler, v. 1.12.1. (2013). http://compcert.inria.fr/release/compcert-1.12.1.tgz (last accessed 5/13).

[32]

Xavier Leroy and Sandrine Blazy. 2008. Formal verification of a C-like memory model and its uses for verifying program transformations. J. Automat. Reason. 41, 1, 1--31.

Digital Library

[33]

Linux. 1999. Linux Kernel mailing list, thread “spin_unlock optimization(i386)”, 119 messages, Nov. 20--Dec. 7th. http://www.gossamer-threads.com/lists/engine?post=105365;list=linux (last accessed 5/13).

[34]

Andreas Lochbihler. 2010. Verifying a compiler for Java threads. In Proceedings of the European Symposium on Programming (ESOP). A. D. Gordon Ed., Springer, 427--447.

Digital Library

[35]

Andreas Lochbihler. 2012. Java and the Java memory model -- a unified, machine-checked formalisation. In Proceedings of the European Symposium on Programming (ESOP). Springer-Verlag, Berlin, 497--517.

Digital Library

[36]

Nancy Lynch and Frits Vaandrager. 1995. Forward and backward simulations I: Untimed systems. Inf. Comput. 121, 2, 214--233.

Digital Library

[37]

Jeremy Manson, William Pugh, and Sarita V. Adve. 2005. The Java memory model. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 378--391.

Digital Library

[38]

Daniel Marino, Abhayendra Singh, Todd Millstein, Madanlal Musuvathi, and Satish Narayanasamy. 2011. A case for an SC-preserving compiler. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 199--210.

Digital Library

[39]

John McCarthy and James Painter. 1967. Correctness of a compiler for arithmetic expressions. In Proceedings of a Symposium in Applied Mathematics. Vol. 19, Mathematical Aspects of Computer Science, J. T. Schwartz Ed., American Mathematical Society, Providence, R.I., 33--41.

[40]

Robin Milner. 1989. Communication and Concurrency. Prentice Hall International.

Digital Library

[41]

Robin Milner and R. Weyrauch. 1972. Proving compiler correctness in a mechanized logic. In Machine Intelligence 7, B. Meltzer and D. Michie Eds., American Elsevier, New York, 51--70.

[42]

Etienne Morel and Claude Renvoise. 1979. Global optimization by suppression of partial redundancies. Comm. ACM 22, 2, 96--103.

Digital Library

[43]

Magnus O. Myreen. 2010. Verified just-in-time compiler on x86. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 107--118.

Digital Library

[44]

Scott Owens. 2010. Reasoning about the implementation of concurrency abstractions on x86-TSO. In Proceedings of the 24th European Conference on Object-Oriented Programming (ECOOP). Lecture Notes in Computer Science, vol. 6183. Springer-Verlag, Berlin, 478--503.

Digital Library

[45]

Scott Owens, Susmit Sarkar, and Peter Sewell. 2009. A better x86 memory model: x86-TSO. In Proceedings of the 22nd International Conference on Theorem Proving in Higher Order Logics (TPHOLs). Lecture Notes in Computer Science, vol. 5674. Springer-Verlag, Berlin, 391--407.

Digital Library

[46]

William Pugh. 2000. The Java memory model is fatally flawed. Concurrency Pract. Exper. 12, 6, 445--455.

[47]

Susmit Sarkar, Peter Sewell, Francesco Zappa Nardelli, Scott Owens, Tom Ridge, Thomas Braibant, Magnus O. Myreen, and Jade Alglave. 2009. The semantics of x86-CC multiprocessor machine code. In Proceedings of the 36th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 379--391.

Digital Library

[48]

Susmit Sarkar, Peter Sewell, Jade Alglave, Luc Maranget, and Derek Williams. 2011. Understanding POWER multiprocessors. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 175--186.

Digital Library

[49]

Susmit Sarkar, Kayvan Memarian, Scott Owens, Mark Batty, Peter Sewell, Luc Maranget, Jade Alglave, and Derek Williams. 2012. Synchronising C/C++ and POWER. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 311--322.

Digital Library

[50]

Jaroslav Ševčík and David Aspinall. 2008. On validity of program transformations in the Java memory model. In Proceedings of the 22nd European Conference on Object-Oriented Programming (ECOOP). Springer-Verlag, Berlin, 27--51.

Digital Library

[51]

Jaroslav Ševčík, Viktor Vafeiadis, Francesco Zappa Nardelli, Suresh Jagannathan, and Peter Sewell. 2011. Relaxed-memory concurrency and verified compilation. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). ACM, New York, 43--54.

Digital Library

[52]

Peter Sewell. 1997. On implementations and semantics of a concurrent programming language. In Proceedings of the 8th International Conference on Concurrency Theory (CONCUR). Lecture Notes in Computer Science, Springer-Verlag, Berlin, 391--405.

Digital Library

[53]

Peter Sewell, Susmit Sarkar, Scott Owens, Francesco Zappa Nardelli, and Magnus O. Myreen. 2010a. x86-TSO: A rigorous and usable programmer’s model for x86 multiprocessors. Comm. ACM 53, 7, 89--97.

Digital Library

[54]

Peter Sewell, Francesco Zappa Nardelli, Scott Owens, Gilles Peskine, Thomas Ridge, Susmit Sarkar, and Rok Strniša. 2010b. Ott: Effective tool support for the working semanticist. J. Function. Prog. 20, 1, 71--122.

Digital Library

[55]

Dennis Shasha and Marc Snir. 1988. Efficient and correct execution of parallel programs that share memory. ACM Trans. Prog. Lang. Syst. 10, 2, 282--312.

Digital Library

[56]

SPARC International, Inc. 1992. The SPARC Architecture Manual, V. 8, Revision SAV080SI9308. http://www.sparc.org/standards/V8.pdf (last accessed 5/13).

Digital Library

[57]

SPARC International, Inc. 1994. The SPARC Architecture Manual, V. 9. http://www.sparc.com/standards/SPARCV9.pdf (last accessed 5/13).

Digital Library

[58]

Gordon Stewart and Andrew W. Appel. 2011. Local actions for a curry-style operational semantics. In Proceedings of the 5th ACM Workshop on Programming Languages Meets Program Verification (PLPV). ACM, New York, 31--42.

Digital Library

[59]

Zehra Sura, Xing Fang, Chi-Leung Wong, Samuel P. Midkiff, Jaejin Lee, and David Padua. 2005. Compiler techniques for high performance sequentially consistent Java programs. In Proceedings of the 10th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP). ACM, New York, 2--13.

Digital Library

[60]

Emina Torlak, Mandana Vaziri, and Julian Dolby. 2010. MemSAT: Checking axiomatic specifications of memory models. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 341--350.

Digital Library

[61]

R. Kent Treiber. 1986. Systems programming: Coping with parallelism. Tech. rep. TR RJ 5118. IBM.

[62]

Viktor Vafeiadis and Francesco Zappa Nardelli. 2011. Verifying fence elimination optimisations. In Proceedings of the 18th International Conference on Static Analysis (SAS). Springer-Verlag, Berlin, 146--162.

Digital Library

Cited By

Chappe NHenrio LZakowski YStark KTimany ABlazy STabareau N(2025)Monadic Interpreters for Concurrent Memory ModelsProceedings of the 14th ACM SIGPLAN International Conference on Certified Programs and Proofs10.1145/3703595.3705890(283-298)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3703595.3705890
Bender JNaik MPereira FTitzer B(2024)Correct Compilation of Concurrent C CodeProceedings of the Workshop Dedicated to Jens Palsberg on the Occasion of His 60th Birthday10.1145/3694848.3694856(39-42)Online publication date: 22-Oct-2024
https://dl.acm.org/doi/10.1145/3694848.3694856
Nagar KSahoo AChowdhury RJagannathan S(2024)Automated Robustness Verification of Concurrent Data Structure Libraries against Relaxed Memory ModelsProceedings of the ACM on Programming Languages10.1145/36898028:OOPSLA2(2578-2605)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689802
Show More Cited By

Index Terms

CompCertTSO: A Verified Compiler for Relaxed-Memory Concurrency

Recommendations

Relaxed-memory concurrency and verified compilation
POPL '11: Proceedings of the 38th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages

In this paper, we consider the semantic design and verified compilation of a C-like programming language for concurrent shared-memory computation above x86 multiprocessors. The design of such a language is made surprisingly subtle by several factors: ...
Relaxed-memory concurrency and verified compilation
POPL '11

In this paper, we consider the semantic design and verified compilation of a C-like programming language for concurrent shared-memory computation above x86 multiprocessors. The design of such a language is made surprisingly subtle by several factors: ...
Formally Verified Native Code Generation in an Effectful JIT: Turning the CompCert Backend into a Formally Verified JIT Compiler

Modern Just-in-Time compilers (or JITs) typically interleave several mechanisms to execute a program. For faster startup times and to observe the initial behavior of an execution, interpretation can be initially used. But after a while, JITs dynamically ...

Comments

Information & Contributors

Information

Published In

cover image Journal of the ACM

Journal of the ACM Volume 60, Issue 3

June 2013

308 pages

ISSN:0004-5411

EISSN:1557-735X

DOI:10.1145/2487241

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2013

Accepted: 01 April 2013

Revised: 01 February 2013

Received: 01 November 2011

Published in JACM Volume 60, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

120
Total Citations
View Citations
660
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chappe NHenrio LZakowski YStark KTimany ABlazy STabareau N(2025)Monadic Interpreters for Concurrent Memory ModelsProceedings of the 14th ACM SIGPLAN International Conference on Certified Programs and Proofs10.1145/3703595.3705890(283-298)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3703595.3705890
Bender JNaik MPereira FTitzer B(2024)Correct Compilation of Concurrent C CodeProceedings of the Workshop Dedicated to Jens Palsberg on the Occasion of His 60th Birthday10.1145/3694848.3694856(39-42)Online publication date: 22-Oct-2024
https://dl.acm.org/doi/10.1145/3694848.3694856
Nagar KSahoo AChowdhury RJagannathan S(2024)Automated Robustness Verification of Concurrent Data Structure Libraries against Relaxed Memory ModelsProceedings of the ACM on Programming Languages10.1145/36898028:OOPSLA2(2578-2605)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689802
Yang ZShirako JSarkar V(2024)Fully Verified Instruction SchedulingProceedings of the ACM on Programming Languages10.1145/36897398:OOPSLA2(791-816)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689739
Geeson LBrotherston JDijkstra WDonaldson ASmith LSorensen TWickerson J(2024)Mix Testing: Specifying and Testing ABI Compatibility of C/C++ Atomics ImplementationsProceedings of the ACM on Programming Languages10.1145/36897278:OOPSLA2(442-467)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689727
Craaijo JVerbeek FRavindran B(2024)libLISA: Instruction Discovery and Analysis on x86-64Proceedings of the ACM on Programming Languages10.1145/36897238:OOPSLA2(333-361)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689723
Beck CYoon IChen HZakowski YZdancewic S(2024)A Two-Phase Infinite/Finite Low-Level Memory Model: Reconciling Integer–Pointer Casts, Finite Space, and undef at the LLVM IR Level of AbstractionProceedings of the ACM on Programming Languages10.1145/36746528:ICFP(789-817)Online publication date: 15-Aug-2024
https://dl.acm.org/doi/10.1145/3674652
Mansky WDu K(2024)An Iris Instance for Verifying CompCert C ProgramsProceedings of the ACM on Programming Languages10.1145/36328488:POPL(148-174)Online publication date: 5-Jan-2024
https://dl.acm.org/doi/10.1145/3632848
Geeson LSmith L(2024)Compiler Testing with Relaxed Memory Models2024 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)10.1109/CGO57630.2024.10444836(334-348)Online publication date: 2-Mar-2024
https://doi.org/10.1109/CGO57630.2024.10444836
Kim JGu RShao Z(2024) : A simplified and abstract multicore hardware model for large scale system software formal verification Journal of Systems Architecture10.1016/j.sysarc.2023.103049147(103049)Online publication date: Feb-2024
https://doi.org/10.1016/j.sysarc.2023.103049
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents