Skip to main content
SpringerLink
Log in

Enabling dynamic file I/O path selection at runtime for parallel file system

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Parallel file systems are experiencing more and more applications from various fields. Various applications have different I/O workload characteristics, which have diverse requirements on accessing storage resources. However, parallel file systems often adopt the “one-size-fits-all” solution, which fails to meet specific application needs and hinders the full exploitation of potential performance. This paper presents a framework to enable dynamic file I/O path selection with fine granularity at runtime. The framework adopts a file handle-rich scheme to allow file systems choose corresponding optimizations to serve I/O requests. Consistency control algorithms are proposed to ensure data consistency while changing optimizations at runtime. One case study on our prototype shows that choosing proper optimizations can improve the I/O performance for small files and large files by up to 40 and 64.4 %, respectively. Another case study shows that the data prefetch performance for real-world application traces can be improved by up to 193 % by selecting correct prefetch patterns. Simulations in large-scale environment also show that our method is scalable and both the memory consumption and the consistency control overhead can be negligible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. BerkeleyDB (2012). http://db.cs.berkeley.edu/

  2. HECIOS: the high end computing I/O simulator (2012). http://www.parl.clemson.edu/hecios/

  3. Lustre File System (2012). http://www.lustre.org

  4. Mdtest HPC Benchmark (2012). http://sourceforge.net/projects/mdtest/

  5. The Parallel Virtual File System (2012). http://www.pvfs.org

  6. Zlib compression library (2012). http://www.zlib.net/

  7. Abd-El-Malek M, Courtright II, WV, Cranor C et al (2005) Ursa minor: versatile cluster-based storage. In: USENIX FAST. San Francisco, pp 59–72

  8. Abe Y, Gibson G (2010) pWalrus: towards better integration of parallel file systems into cloud storage. IASDS 2010, Cluster 2010IEEE Computer Society, Heraklion, pp 1–7

  9. Al-Kiswany S (2013) Embracing diversity: optimizing distributed storage systems for diverse deployment environments. Ph.D. thesis, University of British Columbia

  10. Al-Kiswany S, Gharaibeh A, Ripeanu M (2010) The case for a versatile storage system. SIGOPS Oper Syst Rev 44(1):10–14

    Article  Google Scholar 

  11. Bell G, Gray J, Szalay A (2006) Petascale computational systems. Computer 39(1):110–112

    Article  Google Scholar 

  12. Byna S, Chen Y, Sun XH, Thakur R, Gropp W (2008) Parallel I/O prefetching using MPI file caching and I/O signatures. In: SC 2008. ACM/IEEE, Piscataway, pp 1–12

  13. Calderón A, García-Carballeira F, Sánchez LM, García JD, Fernandez J (2009) Fault tolerant file models for parallel file systems: introducing distribution patterns for every file. J Supercomput 47(3):312–334

    Article  Google Scholar 

  14. Carns P, Lang S, Ross R, Vilayannur M, Kunkel J, Ludwig T (2009) Small-file access in parallel file systems. In: IPDPS 2009. IEEE Computer Society, Rome, pp 1–11

  15. Chen J, Roth PC, Chen Y (2013) Using pattern-models to guide SSD deployment for big data in HPC systems. In: BigData 2013

  16. Chen Y, Sun XH, Thakur R, Song H, Jin H (2010) Improving parallel I/O performance with data layout awareness. Cluster 2010, CLUSTER ’10IEEE Computer Society, Washington, DC, pp 302–311

  17. Devulapalli A, Wyckoff P (2007) File creation strategies in a distributed metadata file system. In: IEEE IPDPS, Long Beach, pp 1–10

  18. Dong B, Li X, Wu Q, Xiao L, Ruan L (2012) A dynamic and adaptive load balancing strategy for parallel file system with large-scale I/O servers. J Parallel Distrib Comput 72(10):1254–1268

    Article  Google Scholar 

  19. Gharaibeh A, Al-Kiswany S, Ripeanu M (2008) Configurable security for scavenged storage systems. In: ACM StorageSS, Alexandria, pp 55–62

  20. He J, Bent J, Torres A, Grider G, Gibson G, Maltzahn C, Sun XH (2013) I/O acceleration with pattern detection. In: HPDC 2013. ACM, New York, pp 25–36

  21. Hendricks J, Sambasivan RR, Sinnamohideen S, Ganger GR (2006) Improving small file performance in object-based storage. Tech. Rep. CMU-PDL-06-104, Parallel Data Lab, Carnegie Mellon University

  22. Kuhn M, Kunkel JM, Ludwig T (2009) Dynamic file system semantics to enable metadata optimizations in PVFS. Concurr Comput Pract Exper 21(14):1775–1788

    Article  Google Scholar 

  23. Li J, Qiu M, Ming Z, Quan G, Qin X, Gu Z (2012) Online optimization for scheduling preemptable tasks on IaaS cloud systems. J Parallel Distrib Comput 72(5):666–677

    Article  Google Scholar 

  24. Li X, Dong B, Xiao L, Ruan L, Liu D (2012) CEFLS: a cost-effective file lookup service in a distributed metadata file system. In: CCGrid 2012. IEEE Computer Society, Washington, DC, pp 25–32

  25. Li X, Dong B, Xiao L, Ruan L, Liu D (2012) HCCache: a hybrid client-side cache management scheme for I/O-intensive workloads in network-based file systems. In: PDCAT 2012. IEEE Computer Society, Washington, DC, pp 467–473

  26. Li Z, Chen Z, Srinivasan SM, Zhou Y (2004) C-Miner: mining block correlations in storage systems. In: USENIX FAST, Berkeley, pp 173–186

  27. Madhyastha TM, Reed DA (2002) Learning to classify parallel input/output access patterns. IEEE Trans Parallel Distrib Syst 13(8):802–813

    Article  Google Scholar 

  28. Molina-Estolano E, Gokhale M, Maltzahn C, May J, Bent J, Brandt S (2009) Mixing Hadoop and HPC workloads on parallel filesystems. In: PDSW 2009. ACM, Portland, pp 1–5

  29. Narayan S, Chandy JA (2010) ATTEST: ATTributes-based Extendable STorage. J Syst Softw 83(4):548–556

    Article  Google Scholar 

  30. Pan A, Walters JP, Pai VS, Kang DID, Crago SP (2012) Integrating high performance file systems in a cloud computing environment. In: Proceedings of the 2012 SC companion: high performance computing, networking storage and analysis, SCC ’12IEEE Computer Society, Washington, DC, pp 753–759

  31. Patrick CM, Kandemir M, Karaköy M, Son SW, Choudhary A (2010) Cashing in on hints for better prefetching and caching in PVFS and MPI-IO. In: HPDC 2010. ACM, Chicago, pp 191–202

  32. Pérez MS, Carretero J, García F, Peña JM, Robles V (2006) MAPFS: a flexible multiagent parallel file system for clusters. Future Gener Comput Syst 22(5):620–632

    Article  Google Scholar 

  33. Prost JP, Treumann R, Hedges R, Jia B, Koniges A (2001) MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS. In: SC 2001. ACM, New York, pp 1–17

  34. Qiu M, Sha EHM (2009) Cost minimization while satisfying hard/soft timing constraints for heterogeneous embedded systems. ACM Trans Des Autom Electron Syst 14(2):25:1–25:30

    Google Scholar 

  35. Schmuck F, Haskin R (2002) GPFS: a shared-disk file system for large computing clusters. In: USENIX FAST. USENIX Association, Berkeley, pp 231–244

  36. Shamsi J, Khojaye M, Qasmi M (2013) Data-intensive cloud computing: requirements, expectations, challenges, and solutions. J Grid Comput 11(2):281–310

    Article  Google Scholar 

  37. Tantisiriroj W, Patil S, Gibson G (2008) Data-intensive file systems for Internet services: a rose by any other name. Tech. Rep. CMU-PDL-08-114, Parallel Data Lab, Carnegie Mellon University

  38. Tantisiriroj W, Son SW, Patil S, Lang SJ, Gibson G, Ross RB (2011) On the duality of data-intensive file system design: reconciling HDFS and PVFS. In: SC 2011. ACM, New York, pp 67:1–67:12

  39. Uysal M, Acharya A, Saltz J (1997) Requirements of I/O systems for parallel machines: an application-driven study. Tech. Rep. UMIACS-TR-97-49, University of Maryland, College Park

  40. Vairavanathan E, Al-Kiswany S, Costa LBa, Zhang Z, Katz DS, Wilde M, Ripeanu M (2012) A workflow-aware storage system: an opportunity study. In: CCGRID 2012. IEEE Computer Society, Washington, DC, pp 326–334

  41. Vilayannur M, Nath P, Sivasubramaniam A (2005) Providing tunable consistency for a parallel file store. In: USENIX FAST, San Francisco, pp 17–30

  42. Wei Q, Xie C, Li X, Cao Q (2007) Research and design of an attribute-managed storage-cluster based on TCP/IP network. In: IEEE NPC, Dalian, pp 332–336

  43. Xia P, Feng D, Jiang H, Tian L, Wang F (2008) FARMER: a novel approach to file access correlation mining and evaluation reference model for optimizing peta-scale file system performance. In: HPDC 2008. ACM, Boston, pp 185–196

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grant No. 61232009 and 61370059, the Doctoral Fund of Ministry of Education of China under Grant No. 20101102110018, the fund of the State Key Laboratory of Software Development Environment under Grant No. SKLSDE-2012ZX-23, the Hi-tech Research and Development Program of China (863 Program) under Grant No. 2011AA01A205 and the Beijing Natural Science Foundation under Grant No. 4122042. Prof. Qiu is supported by NSF CNS-1359557 and NSFC 61071061.

Author information

Authors and Affiliations

  1. State Key Laboratory of Software Development Environment, Beijing, 100191, China

    Xiuqiao Li, Limin Xiao, Bin Dong & Li Ruan

  2. School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Xiuqiao Li, Limin Xiao, Bin Dong & Li Ruan

  3. Department of Computer Engineering, San Jose State University, One Washington Square, San Jose, CA, 95192-0180, USA

    Meikang Qiu

Authors
  1. Xiuqiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Limin Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meikang Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Ruan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuqiao Li.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, X., Xiao, L., Qiu, M. et al. Enabling dynamic file I/O path selection at runtime for parallel file system. J Supercomput 68, 996–1021 (2014). https://doi.org/10.1007/s11227-013-1077-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-013-1077-6

Keywords

Search

Navigation