Mochi: Composing Data Services for High-Performance Computing Environments

Ross, Robert B.; Amvrosiadis, George; Carns, Philip; Cranor, Charles D.; Dorier, Matthieu; Harms, Kevin; Ganger, Greg; Gibson, Garth; Gutierrez, Samuel K.; Latham, Robert; Robey, Bob; Robinson, Dana; Settlemyer, Bradley; Shipman, Galen; Snyder, Shane; Soumagne, Jerome; Zheng, Qing

doi:10.1007/s11390-020-9802-0

Title: Mochi: Composing Data Services for High-Performance Computing Environments

Journal Article · Fri Jan 17 00:00:00 EST 2020 · Journal of Computer Science and Technology

DOI:https://doi.org/10.1007/s11390-020-9802-0· OSTI ID:1596688

Ross, Robert B. ^[1]; Amvrosiadis, George ^[2]; Carns, Philip ^[1]; Cranor, Charles D. ^[2]; Dorier, Matthieu ^[1]; Harms, Kevin ^[1]; Ganger, Greg ^[2]; Gibson, Garth ^[3]; Gutierrez, Samuel K. ^[4]; Latham, Robert ^[1]; Robey, Bob ^[4]; Robinson, Dana ^[5]; Settlemyer, Bradley ^[4]; Shipman, Galen ^[4]; Snyder, Shane ^[1]; Soumagne, Jerome ^[5]; Zheng, Qing ^[2]

Argonne National Lab. (ANL), Lemont, IL (United States)
Carnegie Mellon Univ., Pittsburgh, PA (United States)
Vector Inst. for Artificial Intelligence, Toronto (Canada)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
HDF Group Champaign, IL (United States)

Technology enhancements and the growing breadth of application workflows running on high-performance computing (HPC) platforms drive the development of new data services that provide high performance on these new platforms, provide capable and productive interfaces and abstractions for a variety of applications, and are readily adapted when new technologies are deployed. The Mochi framework enables composition of specialized distributed data services from a collection of connectable modules and subservices. Rather than forcing all applications to use a one-size-fits-all data staging and I/O software configuration, Mochi allows each application to use a data service specialized to its needs and access patterns. This paper introduces the Mochi framework and methodology. The Mochi core components and microservices are described. Examples of the application of the Mochi methodology to the development of four specialized services are detailed. Finally, a performance evaluation of a Mochi core component, a Mochi microservice, and a composed service providing an object model is performed. The paper concludes by positioning Mochi relative to related work in the HPC space and indicating directions for future work.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Lab. (ANL), Argonne, IL (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC02-06CH11357; AC02-05CH11231

OSTI ID:: 1596688

Journal Information:: Journal of Computer Science and Technology, Vol. 35, Issue 1; ISSN 1000-9000

Publisher:: Springer NatureCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 17 works

Citation information provided by
Web of Science

References (26)

Scaling Embedded In-Situ Indexing with DeltaFS Zheng, Qing; Cranor, Charles D.; Guo, Danhao SC18: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2018.00006	conference	November 2018
Evaluation of HPC Application I/O on Object Storage Systems Liu, Jialin; Koziol, Quincey; Butler, Gregory F. 2018 IEEE/ACM 3rd International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS) https://doi.org/10.1109/PDSW-DISCS.2018.00005	conference	November 2018
BESPOKV: Application Tailored Scale-Out Key-Value Stores Anwar, Ali; Cheng, Yue; Huang, Hai SC18: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2018.00005	conference	November 2018
Toward Scalable and Asynchronous Object-Centric Data Management for HPC Tang, Houjun; Byna, Suren; Tessier, Francois 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) https://doi.org/10.1109/CCGRID.2018.00026	conference	May 2018
Dragonfly+: Low Cost Topology for Scaling Datacenters Shpiner, Alexander; Haramaty, Zachy; Eliad, Saar 2017 IEEE 3rd International Workshop on High-Performance Interconnection Networks in the Exascale and Big-Data Era (HiPINEB) https://doi.org/10.1109/HiPINEB.2017.11	conference	February 2017
Mercury: Enabling remote procedure call for high-performance computing Soumagne, Jerome; Kimpe, Dries; Zounmevo, Judicael 2013 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2013.6702617	conference	September 2013
LABIOS: A Distributed Label-Based I/O System Kougkas, Anthony; Devarajan, Hariharan; Lofstead, Jay Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing - HPDC '19 https://doi.org/10.1145/3307681.3325405	conference	January 2019
Overview of 3D NAND Technologies and Outlook Invited Paper Venkatesan, Srinath; Aoulaiche, Marc 2018 Non-Volatile Memory Technology Symposium (NVMTS) https://doi.org/10.1109/NVMTS.2018.8603104	conference	October 2018
Slim Fly: A Cost Effective Low-Diameter Network Topology Besta, Maciej; Hoefler, Torsten SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.34	conference	November 2014
FusionFS: Toward supporting data-intensive scientific applications on extreme-scale high-performance computing systems Zhao, Dongfang; Zhang, Zhao; Zhou, Xiaobing 2014 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData.2014.7004214	conference	October 2014
ROOT — An object oriented data analysis framework Brun, Rene; Rademakers, Fons Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 389, Issue 1-2 https://doi.org/10.1016/S0168-9002(97)00048-X	journal	April 1997
Malacology: A Programmable Storage System Sevilla, Michael A.; Watkins, Noah; Jimenez, Ivo Proceedings of the Twelfth European Conference on Computer Systems - EuroSys '17 https://doi.org/10.1145/3064176.3064208	conference	January 2017
Long-Time Dynamics through Parallel Trajectory Splicing Perez, Danny; Cubuk, Ekin D.; Waterland, Amos Journal of Chemical Theory and Computation, Vol. 12, Issue 1 https://doi.org/10.1021/acs.jctc.5b00916	journal	December 2015
DataSpaces: an interaction and coordination framework for coupled simulation workflows Docan, Ciprian; Parashar, Manish; Klasky, Scott Cluster Computing, Vol. 15, Issue 2 https://doi.org/10.1007/s10586-011-0162-y	journal	February 2011
Technology-Driven, Highly-Scalable Dragonfly Topology Kim, John; Dally, Wiliam J.; Scott, Steve ACM SIGARCH Computer Architecture News, Vol. 36, Issue 3 https://doi.org/10.1145/1394608.1382129	journal	June 2008
PapyrusKV: a high-performance parallel key-value store for distributed NVM architectures Kim, Jungwon; Lee, Seyong; Vetter, Jeffrey S. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '17 https://doi.org/10.1145/3126908.3126943	conference	January 2017
The design and implementation of a log-structured file system Rosenblum, Mendel; Ousterhout, John K. ACM Transactions on Computer Systems, Vol. 10, Issue 1, p. 26-52 https://doi.org/10.1145/146941.146943	journal	February 1992
Argobots: A Lightweight Low-Level Threading and Tasking Framework Seo, Sangmin; Amer, Abdelhalim; Balaji, Pavan IEEE Transactions on Parallel and Distributed Systems, Vol. 29, Issue 3 https://doi.org/10.1109/TPDS.2017.2766062	journal	March 2018
Methodology for the Rapid Development of Scalable HPC Data Services Dorier, Matthieu; Settlemyer, Brad; Shipman, Galen 2018 IEEE/ACM 3rd International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS) https://doi.org/10.1109/PDSW-DISCS.2018.00013	conference	November 2018
Programmable Caches with a Data Management Language and Policy Engine Sevilla, Michael A.; Maltzahn, Carlos; Alvaro, Peter 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) https://doi.org/10.1109/CCGRID.2018.00035	conference	May 2018
RADOS: a scalable, reliable storage service for petabyte-scale storage clusters Weil, Sage A.; Leung, Andrew W.; Brandt, Scott A. Proceedings of the 2nd international workshop on Petascale data storage held in conjunction with Supercomputing '07 - PDSW '07 https://doi.org/10.1145/1374596.1374606	conference	January 2007
Platform Storage Performance With 3D XPoint Technology Hady, Frank T.; Foong, Annie; Veal, Bryan Proceedings of the IEEE, Vol. 105, Issue 9 https://doi.org/10.1109/JPROC.2017.2731776	journal	September 2017
GekkoFS - A Temporary Distributed File System for HPC Applications Vef, Marc-Andre; Moti, Nafiseh; SuB, Tim 2018 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2018.00049	conference	September 2018
Massively parallel loading Frings, Wolfgang; Ahn, Dong H.; LeGendre, Matthew Proceedings of the 27th international ACM conference on International conference on supercomputing - ICS '13 https://doi.org/10.1145/2464996.2465020	conference	January 2013
The NumPy Array: A Structure for Efficient Numerical Computation van der Walt, Stéfan; Colbert, S. Chris; Varoquaux, Gaël Computing in Science & Engineering, Vol. 13, Issue 2 https://doi.org/10.1109/MCSE.2011.37	journal	March 2011
An Ephemeral Burst-Buffer File System for Scientific Applications Wang, Teng; Mohror, Kathryn; Moody, Adam SC16: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2016.68	conference	November 2016

Similar Records

SYMBIOSYS: A Methodology for Performance Analysis of Composable HPC Data Services

Conference · Fri Jan 01 00:00:00 EST 2021 · OSTI ID:1596688

Ramesh, Srinivasan; Malony, Allen D.; Carns, Philip; +4 more

A Software Defined Storage Approach to Exascale Storage Services (Final Technical Report)

Technical Report · Wed Sep 11 00:00:00 EDT 2019 · OSTI ID:1596688

Soumagne, Jerome; Ross, Robert B.; Shipman, Galen; +12 more

Enabling NVM for Data-Intensive Scientific Services

Conference · Fri Jan 01 00:00:00 EST 2016 · OSTI ID:1596688

Carns, Philip; Jenkins, John; Seo, Sangmin; +5 more

Related Subjects

97 MATHEMATICS AND COMPUTING
data-intensive computing
distributed services
high-performance computing
storage and I/O

Title: Mochi: Composing Data Services for High-Performance Computing Environments

Citation Formats

References (26)

Similar Records

Related Subjects