Two-mode data distribution scheme for heterogeneous storage in data centers | IEEE Conference Publication | IEEE Xplore

Two-mode data distribution scheme for heterogeneous storage in data centers


Abstract:

Fast growing "Big Data" demands present new challenges to the traditional distributed storage system solutions. In order to support cloud-scale data centers, new types of...Show More

Abstract:

Fast growing "Big Data" demands present new challenges to the traditional distributed storage system solutions. In order to support cloud-scale data centers, new types of distributed storage systems are emerging. They are designed to scale to thousands of nodes, maintain petabytes of data and be highly reliable. The support for virtual machines is also becoming essential as it is one of the most important technology that supports cloud computing. To meet these needs, these distributed storage systems are implemented with advanced data distribution schemes. Data are striped and distributed across the storage cluster based on distribution algorithms instead of mapping tables. The existing algorithms usually balance the data distribution across nodes proportional to their capacity. However, they overlook distinct performance characteristics across different nodes and devices in the emerging heterogeneous storage environment. We propose a two-mode data distribution scheme in this study to maximize the overall performance and keep data balanced across the storage cluster at the same time. The working principle of the two-mode data distribution scheme is provided. We also present a new data read and write strategy to work with the two-mode scheme. We evaluate the computation time for data distribution using two-mode scheme and analyze its implication on the overall IO performance. We expect significant performance improvement while it still needs more analytical and experimental evaluation to further examine the details.
Date of Conference: 29 October 2015 - 01 November 2015
Date Added to IEEE Xplore: 28 December 2015
ISBN Information:
Conference Location: Santa Clara, CA, USA

Contact IEEE to Subscribe

References

References is not available for this document.