Welcome to the InfoSci Platform
Reference Hub1
A Context-Based Performance Enhancement Algorithm for Columnar Storage in MapReduce with Hive

A Context-Based Performance Enhancement Algorithm for Columnar Storage in MapReduce with Hive

Yashvardhan Sharma, Saurabh Verma, Sumit Kumar, Shivam U.
Copyright: © 2013 |Volume: 3 |Issue: 4 |Pages: 13
ISSN: 2156-1834|EISSN: 2156-1826|EISBN13: 9781466635012|DOI: 10.4018/ijcac.2013100104
Cite Article Cite Article

MLA

Sharma, Yashvardhan, et al. "A Context-Based Performance Enhancement Algorithm for Columnar Storage in MapReduce with Hive." IJCAC vol.3, no.4 2013: pp.38-50. http://doi.org/10.4018/ijcac.2013100104

APA

Sharma, Y., Verma, S., Kumar, S., & U., S. (2013). A Context-Based Performance Enhancement Algorithm for Columnar Storage in MapReduce with Hive. International Journal of Cloud Applications and Computing (IJCAC), 3(4), 38-50. http://doi.org/10.4018/ijcac.2013100104

Chicago

Sharma, Yashvardhan, et al. "A Context-Based Performance Enhancement Algorithm for Columnar Storage in MapReduce with Hive," International Journal of Cloud Applications and Computing (IJCAC) 3, no.4: 38-50. http://doi.org/10.4018/ijcac.2013100104

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

To achieve high reliability and scalability, most large-scale data warehouse systems have adopted the cluster-based architecture. In this context, MapReduce has emerged as a promising architecture for large scale data warehousing and data analytics on commodity clusters. The MapReduce framework offers several lucrative features such as high fault-tolerance, scalability and use of a variety of hardware from low to high range. But these benefits have resulted in substantial performance compromise. In this paper, we propose the design of a novel cluster-based data warehouse system, Daenyrys for data processing on Hadoop – an open source implementation of the MapReduce framework under the umbrella of Apache. Daenyrys is a data management system which has the capability to take decision about the optimum partitioning scheme for the Hadoop's distributed file system (DFS). The optimum partitioning scheme improves the performance of the complete framework. The choice of the optimum partitioning is query-context dependent. In Daenyrys, the columns are formed into optimized groups to provide the basis for the partitioning of tables vertically. Daenyrys has an algorithm that monitors the context of current queries and based on the observations, it re-partitions the DFS for better performance and resource utilization. In the proposed system, Hive, a MapReduce-based SQL-like query engine is supported above the DFS.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.