skip to main content
10.1145/2834976.2834982acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article

Experiences in using os-level virtualization for block I/O

Published: 15 November 2015 Publication History

Abstract

Today, HPC clusters commonly use Resource Management Systems such as PBS and TORQUE to share physical resources. These systems enable resources to be shared by assigning nodes to users exclusively in non-overlapping time slots. With virtualization technology, users can run their applications on the same node with low mutual interference. However, the overhead introduced by the virtual machine monitor or hypervisor is too high to be accepted, because efficiency is key to many HPC applications. OS-level virtualization (such as Linux Containers) offers a lightweight virtualization layer, which promises a near-native performance and is adopted by some BigData resource sharing platforms such as Mesos. Nevertheless, OS-level virtualization's overhead and isolation on block devices have not been completely evaluated, especially when applied to a shared distributed/parallel file system (D/PFS) such as HDFS or Lustre. In this paper, we thoroughly evaluate the overhead and isolation involved in sharing block I/O via OS-level virtualization on the local disk and D/PFSs. Meanwhile, to assign D/PFS storage resources to users, a middleware system is proposed and implemented to bridge the configuration gap between virtual clusters and remote D/PFSs.

References

[1]
Docker, https://www.docker.com/.
[2]
KVM. http://www.linux-kvm.org/page/Main_Page.
[3]
Linux container, https://linuxcontainers.org/.
[4]
Lustre filesystem. http://www.lustre.org/.
[5]
MPI-IO Test. http://institute.lanl.gov/data/software/.
[6]
Openvz, http://www.openvz.org.
[7]
Paraview, http://www.paraview.org/.
[8]
TORQUE Resource Manager http://www.adaptivecomputing.com/products/open-source/torque/.
[9]
Vmware, http://www.vmware.com/.
[10]
Xen, http://www.xenproject.org.
[11]
Dhruba Borthaku. The Hadoop Distributed File System: Architecture and Design.
[12]
J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2004.
[13]
Garth Gibson, Gary Grider, Andree Jacobson, and Wyatt Lloyd. Probe: A thousand-node experimental cluster for computer systems research. USENIX; login, 38(3), 2013.
[14]
Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. Mesos: a platform for fine-grained resource sharing in the data center. In Proceedings of the 8th USENIX conference on Networked systems design and implementation, NSDI'11, pages 22--22, Berkeley, CA, USA, 2011. USENIX Association.
[15]
Nikolaus Huber, Marcel von Quast, Michael Hauck, and Samuel Kounev. Evaluating and modeling virtualization performance overhead for cloud environments. In CLOSER, pages 563--573, 2011.
[16]
Junbin Kang, Benlong Zhang, Tianyu Wo, Chunming Hu, and Jinpeng Huai. Multilanes: providing virtualized storage for os-level virtualization on many cores. In FAST, pages 317--329, 2014.
[17]
Mingliang Liu, Ye Jin, Jidong Zhai, Yan Zhai, Qianqian Shi, Xiaosong Ma, and Wenguang Chen. Acic: automatic cloud i/o configurator for hpc applications. In High Performance Computing, Networking, Storage and Analysis (SC), 2013 International Conference for, pages 1--12. IEEE, 2013.
[18]
Shuangcheng Niu, Jidong Zhai, Xiaosong Ma, Xiongchao Tang, and Wenguang Chen. Cost-effective cloud hpc resource provisioning by building semi-elastic virtual clusters. In High Performance Computing, Networking, Storage and Analysis (SC), 2013 International Conference for, pages 1--12. IEEE, 2013.
[19]
Jongse Park, Daewoo Lee, Bokyeong Kim, Jaehyuk Huh, and Seungryoul Maeng. Locality-aware dynamic vm reconfiguration on mapreduce clouds. In Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing, pages 27--36. ACM, 2012.
[20]
Stephen Soltesz, Herbert Pötzl, Marc E Fiuczynski, Andy Bavier, and Larry Peterson. Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In ACM SIGOPS Operating Systems Review, volume 41, pages 275--287. ACM, 2007.
[21]
Tom White. Hadoop: The Definitive Guide. O'Reilly Media, original edition, June 2009, ISBN: 0596521979.
[22]
Miguel G Xavier, Marcelo Veiga Neves, Fabio D Rossi, Tiago C Ferreto, Timoteo Lange, and Cesar AF De Rose. Performance evaluation of container-based virtualization for high performance computing environments. In Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference on, pages 233--240. IEEE, 2013.

Cited By

View all
  • (2018)Achieving Load Balance for Parallel Data Access on Distributed File SystemsIEEE Transactions on Computers10.1109/TC.2017.274922967:3(388-402)Online publication date: 1-Mar-2018

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PDSW '15: Proceedings of the 10th Parallel Data Storage Workshop
November 2015
59 pages
ISBN:9781450340083
DOI:10.1145/2834976
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2015

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

  • US National Science Foundation

Conference

SC15
Sponsor:

Acceptance Rates

PDSW '15 Paper Acceptance Rate 9 of 25 submissions, 36%;
Overall Acceptance Rate 17 of 41 submissions, 41%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 11 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2018)Achieving Load Balance for Parallel Data Access on Distributed File SystemsIEEE Transactions on Computers10.1109/TC.2017.274922967:3(388-402)Online publication date: 1-Mar-2018

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media