Abstract
Recently, a number of cloud platforms and services have been developed for data intensive computing, including Hadoop, Sector, CloudStore (formerly KFS), HBase, and Thrift. In order to benchmark the performance of these systems, to investigate their interoperability, and to experiment with new services based on flexible compute node and network provisioning capabilities, we have designed and implemented a large scale testbed called the Open Cloud Testbed (OCT). Currently OCT has 120 nodes in 4 data centers: Baltimore, Chicago (two locations), and San Diego. In contrast to other cloud testbeds, which are in small geographic areas and which are based on commodity Internet services, the OCT is a wide area testbed and the 4 data centers are connected with a high performance 10Gb/s network, based on a foundation of dedicated lightpaths. This testbed can address the requirements of extremely large data streams that challenge other types of distributed infrastructure. We have also developed several utilities to support the development of cloud computing systems and services, including novel node and network provisioning services, a monitoring system, and an RPC system. In this paper, we describe the OCT concepts, architecture, infrastructure, a few benchmarks that were developed for this platform, interoperability studies, and results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gu, Y., Grossman, R.: Sector and Sphere: The Design and Implementation of a High Performance Data Cloud. Theme Issue of the Philosophical Transactions of the Royal Society A: Crossing Boundaries: Computational Science, E-Science and Global E-Infrastructure 367(1897), 2429–2445 (2009)
Hadoop, http://hadoop.apache.org/core/
CloudStore, http://kosmosfs.sourceforge.net/
Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: The Eucalyptus Open-source Cloud-computing System. In: Proceedings of Cloud Computing and Its Applications, Chicago, Illinois (October 2008)
Amazon EC2 and S3, http://aws.amazon.com/
Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google File System, pub. In: 19th ACM Symposium on Operating Systems Principles, Lake George, NY (October 2003)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: OSDI 2004: Sixth Symposium on Operating System Design and Implementation, San Francisco, CA (December 2004)
HP Technical Report, HP-2009-122: Open Cirrus Cloud Computing Testbed: Federated Data Centers for Open Source Systems and Services Research
Provos, N., McNamee, D., Mavrommatis, P., Wang, K., Modadugu, N.: The Ghost In The Browser - Analysis of Web-based Malware. In: Proceedings of the First Workshop on Hot Topics in Understanding Botnets, HotBots (2007)
OCT Monitor, http://angle.ncdm.uic.edu/simnetup/
Gu, Y., Grossman, R.L.: UDT: UDP-based Data Transfer for High-Speed Wide Area Networks. Computer Networks 51(7) (May 2007)
Travestino, F., Mambretti, J., Karmous-Edwards, G.: GridNetworks: Enabling Grids With Advanced Communication Services. Wiley, Chichester (2006)
Bennett, C., Grossman, R., Seidman, J.: Open Cloud Consortium Technical Report TR-09-01, MalStone: A Benchmark for Data Intensive Computing (April 2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Grossman, R., Gu, Y., Sabala, M., Bennet, C., Seidman, J., Mambratti, J. (2010). The Open Cloud Testbed: Supporting Open Source Cloud Computing Systems Based on Large Scale High Performance, Dynamic Network Services. In: Doulamis, A., Mambretti, J., Tomkos, I., Varvarigou, T. (eds) Networks for Grid Applications. GridNets 2009. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 25. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11733-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-11733-6_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-11732-9
Online ISBN: 978-3-642-11733-6
eBook Packages: Computer ScienceComputer Science (R0)