research-article

RTHMS: a tool for data placement on hybrid memory system

Authors:

Roberto Gioiosa,

Pietro Cicotti,

Stefano MarkidisAuthors Info & Claims

ISMM 2017: Proceedings of the 2017 ACM SIGPLAN International Symposium on Memory Management

Pages 82 - 91

https://doi.org/10.1145/3092255.3092273

Published: 18 June 2017 Publication History

Abstract

Traditional scientific and emerging data analytics applications require fast, power-efficient, large, and persistent memories. Combining all these characteristics within a single memory technology is expensive and hence future supercomputers will feature different memory technologies side-by-side. However, it is a complex task to program hybrid-memory systems and to identify the best object-to-memory mapping. We envision that programmers will probably resort to use default configurations that only require minimal interventions on the application code or system settings. In this work, we argue that intelligent, fine-grained data placement can achieve higher performance than default setups.

We present an algorithm for data placement on hybrid-memory systems. Our algorithm is based on a set of single-object allocation rules and global data placement decisions. We also present RTHMS, a tool that implements our algorithm and provides recommendations about the object-to-memory mapping. Our experiments on a hybrid memory system, an Intel Knights Landing processor with DRAM and HBM, show that RTHMS is able to achieve higher performance than the default configuration. We believe that RTHMS will be a valuable tool for programmers working on complex hybrid-memory systems.

References

[1]

The CORAL Benchmarks. https://asc.llnl.gov/CORALbenchmarks/, 2017. {Online; accessed 15-Janurary-2017}. Rodinia:Accelerating Compute-Intensive Applications with Accelerators. https://www.cs.virginia.edu/~skadron/wiki/rodinia/ index.php/, 2017. {Online; accessed 15-Janurary-2017}. The DGEMM Benchmark. http://www.nersc.gov/research-anddevelopment/apex/apex-benchmarks/dgemm/, 2017. {Online; accessed 15-Janurary-2017}. Knights Landing (KNL) Testing & Development Platform. http://www. archer.ac.uk/documentation/knl-guide/, 2017.

[2]

{Online; accessed 15-Janurary-2017}. The Graph500 Benchmark. http://www.graph500.org/, 2017. {Online; accessed 15-Janurary-2017}. Multithreaded Transposition of Square Matrices with Common Code for Intel Xeon Processors and Intel Xeon Phi Coprocessors. https: //colfaxresearch.com/multithreaded-transpositionof-square-matrices-with-common-code-for-intel-xeonprocessors-and-intel-xeon-phi-coprocessors/, 2017.

[3]

{Online; accessed 15-Janurary-2017}. Xsbench: The Monte Carlo macroscopic cross section lookup benchmark. https://github.com/ANL-CESAR/XSBench, 2017. {Online; accessed 01-January-2017}. J. Absar and F. Catthoor. Analysis of scratch-pad and data-cache performance using statistical methods. In Asia and South Pacific Conference on Design Automation, 2006., pages 6 pp.–, 2006.

Digital Library

[4]

N. Chatterjee, M. Shevgoor, R. Balasubramonian, A. Davis, Z. Fang, R. Illikkal, and R. Iyer. Leveraging heterogeneity in dram main memories to accelerate critical word access. In Proceedings of the 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO-45, pages 13–24, 2012.

Digital Library

[5]

S. R. Dulloor, A. Roy, Z. Zhao, N. Sundaram, N. Satish, R. Sankaran, J. Jackson, and K. Schwan. Data tiering in heterogeneous memory systems. In Proceedings of the Eleventh European Conference on Computer Systems, page 15. ACM, 2016.

Digital Library

[6]

A. Hassan, H. Vandierendonck, and D. S. Nikolopoulos. Software-managed energy-efficient hybrid DRAM/NVM main memory. In Proceedings of the 12th ACM International Conference on Computing Frontiers, CF ’15, pages 23:1–23:8, 2015.

Digital Library

[7]

I. Karlin, A. Bhatele, J. Keasler, B. L. Chamberlain, J. Cohen, Z. Devito, R. Haque, D. Laney, E. Luke, F. Wang, et al. Exploring traditional and emerging parallel programming models using a proxy application. In Parallel & Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on, pages 919–932. IEEE, 2013.

Digital Library

[8]

G. Kestor, R. Gioiosa, D. J. Kerbyson, and A. Hoisie. Quantifying the energy cost of data movement in scientific applications. In 2013 IEEE International Symposium on Workload Characterization (IISWC), pages 56–65, 2013.

[9]

D. Li, J. S. Vetter, G. Marin, C. McCurdy, C. Cira, Z. Liu, and W. Yu. Identifying opportunities for byte-addressable non-volatile memory in extreme-scale scientific applications. In Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS ’12, pages 945–956, 2012.

Digital Library

[10]

C.-K. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallace, V. J. Reddi, and K. Hazelwood. Pin: building customized program analysis tools with dynamic instrumentation. In Acm sigplan notices, volume 40, pages 190–200. ACM, 2005.

Digital Library

[11]

J. A. Mandelman, R. H. Dennard, G. B. Bronner, J. K. DeBrosse, R. Divakaruni, Y. Li, and C. J. Radens. Challenges and future directions for the scaling of dynamic random-access memory (dram). IBM J. Res. Dev., 46(2-3):187–212, 2002.

Digital Library

[12]

J. D. McCalpin. A survey of memory bandwidth and machine balance in current high performance computers. IEEE TCCA Newsletter, pages 19– 25, 1995.

[13]

I. B. Peng, S. Markidis, E. Laure, G. Kestor, and R. Gioiosa. Exploring application performance on emerging hybrid-memory supercomputers. In High Performance Computing and Communications; 2016 IEEE 18th International Conference on, pages 473–480. IEEE, 2016.

[14]

I. B. Peng, R. Gioiosa, G. Kestor, P. Cicotti, E. Laure, and S. Markidis. Exploring the performance benefit of hybrid memory system on HPC environments. In Parallel and Distributed Processing Symposium Workshops, 2017 IEEE International. IEEE, 2017.

[15]

L. E. Ramos, E. Gorbatov, and R. Bianchini. Page placement in hybrid memory systems. In Proceedings of the international conference on Supercomputing, pages 85–95. ACM, 2011.

Digital Library

[16]

D. Shen, X. Liu, and F. X. Lin. Characterizing emerging heterogeneous memory. In Proceedings of the 2016 ACM SIGPLAN International Symposium on Memory Management, pages 13–23. ACM, 2016.

Digital Library

[17]

B. Wang, B. Wu, D. Li, X. Shen, W. Yu, Y. Jiao, and J. S. Vetter. Exploring hybrid memory for gpu energy efficiency through software-hardware codesign. In Proceedings of the 22nd international conference on Parallel architectures and compilation techniques, pages 93–102. IEEE Press, 2013.

Digital Library

[18]

W. Wei, D. Jiang, S. A. McKee, J. Xiong, and M. Chen. Exploiting program semantics to place data in hybrid memory. In Proceedings of the 2015 International Conference on Parallel Architecture and Compilation (PACT), PACT ’15, pages 163–173, 2015.

Digital Library

Cited By

Heroux MBujack RGokhale MIonkov LIwabuchi KJantz MJones TLakshmiranganatha SLang MLee JOlson MPakin SPearce RPietarila Graham JTang LTurton TWilliams S(2025)The ECP SICM projectInternational Journal of High Performance Computing Applications10.1177/1094342024128824339:1(193-207)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1177/10943420241288243
Kammerdiener BMcMichael JJantz MDoshi KJones TBlackburn SPetrank E(2023)Flexible and Effective Object Tiering for Heterogeneous Memory SystemsProceedings of the 2023 ACM SIGPLAN International Symposium on Memory Management10.1145/3591195.3595277(163-175)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591195.3595277
Wang YFu YLiu YChen ZXiao N(2022)Characterizing and Optimizing Hybrid DRAM-PM Main Memory System with Application Awareness2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE54114.2022.9774718(879-884)Online publication date: 14-Mar-2022
https://doi.org/10.23919/DATE54114.2022.9774718
Show More Cited By

Index Terms

RTHMS: a tool for data placement on hybrid memory system
1. General and reference
  1. Cross-computing tools and techniques

Recommendations

RTHMS: a tool for data placement on hybrid memory system
ISMM '17

Traditional scientific and emerging data analytics applications require fast, power-efficient, large, and persistent memories. Combining all these characteristics within a single memory technology is expensive and hence future supercomputers will ...
Design and implementation of bandwidth-aware memory placement and migration policies for heterogeneous memory systems
ICS '17: Proceedings of the International Conference on Supercomputing

Heterogeneous memory systems that comprise memory nodes based on widely-different device technologies (e.g., DRAM and nonvolatile memory (NVM)) are emerging in various computing domains ranging from high-performance to embedded computing. Despite the ...
Enabling Hybrid PCM Memory System with Inherent Memory Management
RACS '16: Proceedings of the International Conference on Research in Adaptive and Convergent Systems

Replacing the traditional volatile main memory, e.g., DRAM, with a non-volatile phase change memory (PCM) has become a possible solution to reduce the energy consumption of computing systems. To further reduce the bit cost of PCM, the development trend ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISMM 2017: Proceedings of the 2017 ACM SIGPLAN International Symposium on Memory Management

June 2017

127 pages

ISBN:9781450350440

DOI:10.1145/3092255

General Chair:
Christoph M. Kirsch
University of Salzburg, Austria
,
Program Chair:
Ben L. Titzer
Google, Germany

ACM SIGPLAN Notices Volume 52, Issue 9
ISMM '17
September 2017
127 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3156685
Editor:
Matthew Fluet
Issue’s Table of Contents

Copyright © 2017 ACM.

© 2017 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the United States Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the DOE Office of Science Advanced Scientific Computing Research through the ARGO project
the DOE Office of Science Advanced Scientific Computing Research through the CENATE project
the European Commission through the SAGE project

Conference

ISMM '17

Sponsor:

SIGPLAN

ISMM '17: International Symposium on Memory Management

June 18, 2017

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 72 of 156 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

48
Total Citations
View Citations
638
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Heroux MBujack RGokhale MIonkov LIwabuchi KJantz MJones TLakshmiranganatha SLang MLee JOlson MPakin SPearce RPietarila Graham JTang LTurton TWilliams S(2025)The ECP SICM projectInternational Journal of High Performance Computing Applications10.1177/1094342024128824339:1(193-207)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1177/10943420241288243
Kammerdiener BMcMichael JJantz MDoshi KJones TBlackburn SPetrank E(2023)Flexible and Effective Object Tiering for Heterogeneous Memory SystemsProceedings of the 2023 ACM SIGPLAN International Symposium on Memory Management10.1145/3591195.3595277(163-175)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591195.3595277
Wang YFu YLiu YChen ZXiao N(2022)Characterizing and Optimizing Hybrid DRAM-PM Main Memory System with Application Awareness2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE54114.2022.9774718(879-884)Online publication date: 14-Mar-2022
https://doi.org/10.23919/DATE54114.2022.9774718
Olson MKammerdiener BJantz MDoshi KJones T(2022)Online Application Guidance for Heterogeneous Memory SystemsACM Transactions on Architecture and Code Optimization10.1145/353385519:3(1-27)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3533855
Zhao XZhou JGuan HWang WLiu XLiu TZhou HMoreira JMueller FEtsion Y(2021)NumaPerfProceedings of the 35th ACM International Conference on Supercomputing10.1145/3447818.3460361(52-62)Online publication date: 3-Jun-2021
https://dl.acm.org/doi/10.1145/3447818.3460361
Ren JLuo JPeng IWu KLi DZhou HMoreira JMueller FEtsion Y(2021)Optimizing large-scale plasma simulations on persistent memory-based heterogeneous memory with effective data placement across memory hierarchyProceedings of the 35th ACM International Conference on Supercomputing10.1145/3447818.3460356(203-214)Online publication date: 3-Jun-2021
https://dl.acm.org/doi/10.1145/3447818.3460356
Chen TLiu HLiao XJin H(2021)Resource abstraction and data placement for distributed hybrid memory poolFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-020-9448-715:3Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1007/s11704-020-9448-7
Khan AGoens AHameed FCastrillon JDi Natale GFummi F(2020)Generalized data placement strategies for racetrack memoriesProceedings of the 23rd Conference on Design, Automation and Test in Europe10.5555/3408352.3408693(1502-1507)Online publication date: 9-Mar-2020
https://dl.acm.org/doi/10.5555/3408352.3408693
Khan AGoens AHameed FCastrillon J(2020)Generalized Data Placement Strategies for Racetrack Memories2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE48585.2020.9116245(1502-1507)Online publication date: Mar-2020
https://doi.org/10.23919/DATE48585.2020.9116245
Effler TJantz MJones T(2020)Performance Potential of Mixed Data Management Modes for Heterogeneous Memory Systems2020 IEEE/ACM Workshop on Memory Centric High Performance Computing (MCHPC)10.1109/MCHPC51950.2020.00007(10-16)Online publication date: Nov-2020
https://doi.org/10.1109/MCHPC51950.2020.00007
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten