skip to main content
10.1145/2488551.2488598acmotherconferencesArticle/Chapter ViewAbstractPublication PageseurompiConference Proceedingsconference-collections
research-article

A hierarchical parallel storage system based on distributed memory for large scale systems

Published: 15 September 2013 Publication History

Abstract

This paper presents the design and implementation of a storage system for high performance systems based on a multiple level I/O caching architecture. The solution relies on Memcached as a parallel storage system, preserving its powerful capacities such as transparency, quick deployment, and scalability. The designed parallel storage system targets to reduce the I/O latency in data-intensive high performance applications. The proposed solution consists of a user-level library and extended Memcached servers. The solution aims to be hierarchical by deploying Memcached-based I/O servers across all the infrastructure data path. Our experiments demonstrate that our solution is up to 40% faster than PVFS2.

References

[1]
G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. Disk-locality in datacenter computing considered irrelevant. In HotOS'13, pages 12--12, Berkeley, CA, USA, 2011. USENIX Association.
[2]
G. Bell, J. Gray, and A. Szalay. Petascale computational systems. Computer, 39(1):110--112, jan. 2006.
[3]
J. Dongarra, P. Beckman, T. Moore, and Aerts. The international exascale software project roadmap. Int. J. High Perform. Comput. Appl., 25(1):3--60, Feb. 2011.
[4]
B. Fitzpatrick. Distributed caching with memcached. Linux J., 2004(124):5--, aug 2004.
[5]
F. Isaila, J. G. Blas, J. Carretero, R. Latham, and R. Ross. Design and evaluation of multiple-level data staging for blue gene systems. IEEE Transactions on Parallel and Distributed Systems, 22(6):946--959, 2011.

Cited By

View all
  • (2023)LAMP: Improving Compression Ratio for AMR Applications via Level Associated Mapping-Based PreconditioningIEEE Transactions on Computers10.1109/TC.2023.329744272:12(3370-3382)Online publication date: Dec-2023
  • (2023)Reducing energy footprint in cloud computing: a study on the impact of clustering techniques and scheduling algorithms for scientific workflowsComputing10.1007/s00607-023-01182-w105:10(2231-2261)Online publication date: 13-May-2023
  • (2023)Hercules: Scalable and Network Portable In-Memory Ad-Hoc File System for Data-Centric and High-Performance ApplicationsEuro-Par 2023: Parallel Processing10.1007/978-3-031-39698-4_46(679-693)Online publication date: 24-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
EuroMPI '13: Proceedings of the 20th European MPI Users' Group Meeting
September 2013
289 pages
ISBN:9781450319034
DOI:10.1145/2488551
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

  • ARCOS: Computer Architecture and Technology Area, Universidad Carlos III de Madrid

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 September 2013

Check for updates

Author Tags

  1. distributed cache
  2. memcached
  3. parallel storage system

Qualifiers

  • Research-article

Funding Sources

Conference

EuroMPI '13
Sponsor:
  • ARCOS
EuroMPI '13: 20th European MPI Users's Group Meeting
September 15 - 18, 2013
Madrid, Spain

Acceptance Rates

EuroMPI '13 Paper Acceptance Rate 22 of 47 submissions, 47%;
Overall Acceptance Rate 66 of 139 submissions, 47%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 11 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)LAMP: Improving Compression Ratio for AMR Applications via Level Associated Mapping-Based PreconditioningIEEE Transactions on Computers10.1109/TC.2023.329744272:12(3370-3382)Online publication date: Dec-2023
  • (2023)Reducing energy footprint in cloud computing: a study on the impact of clustering techniques and scheduling algorithms for scientific workflowsComputing10.1007/s00607-023-01182-w105:10(2231-2261)Online publication date: 13-May-2023
  • (2023)Hercules: Scalable and Network Portable In-Memory Ad-Hoc File System for Data-Centric and High-Performance ApplicationsEuro-Par 2023: Parallel Processing10.1007/978-3-031-39698-4_46(679-693)Online publication date: 24-Aug-2023
  • (2023)Malleability Techniques for HPC SystemsParallel Processing and Applied Mathematics10.1007/978-3-031-30445-3_7(77-88)Online publication date: 27-Apr-2023
  • (2022)IMSS: In-Memory Storage System for Data Intensive ApplicationsHigh Performance Computing. ISC High Performance 2022 International Workshops10.1007/978-3-031-23220-6_13(190-205)Online publication date: 29-May-2022
  • (2021)A Data-Aware Scheduling Strategy for Executing Large-Scale Distributed WorkflowsIEEE Access10.1109/ACCESS.2021.30678159(47354-47364)Online publication date: 2021
  • (2021)Cloud Computing for Enabling Big Data AnalysisCloud Computing and Services Science10.1007/978-3-030-72369-9_4(84-109)Online publication date: 23-Mar-2021
  • (2020)Improving Performance and Energy Consumption in Embedded Systems via Binary Acceleration: A SurveyACM Computing Surveys10.1145/336976453:1(1-36)Online publication date: 6-Feb-2020
  • (2020)Autonomous Visual Navigation for Mobile RobotsACM Computing Surveys10.1145/336896153:1(1-34)Online publication date: 6-Feb-2020
  • (2020)Multiple Workflows Scheduling in Multi-tenant Distributed SystemsACM Computing Surveys10.1145/336803653:1(1-39)Online publication date: 6-Feb-2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media