Welcome to the InfoSci Platform
Reference Hub3
An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE

An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE

Geetha J., Uday Bhaskar N, Chenna Reddy P.
Copyright: © 2018 |Volume: 10 |Issue: 2 |Pages: 14
ISSN: 1935-5661|EISSN: 1935-567X|EISBN13: 9781522543251|DOI: 10.4018/IJICTHD.2018040101
Cite Article Cite Article

MLA

Geetha J., et al. "An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE." IJICTHD vol.10, no.2 2018: pp.1-14. http://doi.org/10.4018/IJICTHD.2018040101

APA

Geetha J., Uday Bhaskar N, & Chenna Reddy P. (2018). An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE. International Journal of Information Communication Technologies and Human Development (IJICTHD), 10(2), 1-14. http://doi.org/10.4018/IJICTHD.2018040101

Chicago

Geetha J., Uday Bhaskar N, and Chenna Reddy P. "An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE," International Journal of Information Communication Technologies and Human Development (IJICTHD) 10, no.2: 1-14. http://doi.org/10.4018/IJICTHD.2018040101

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

Data intensive systems aim to efficiently process “big” data. Several data processing engines have evolved over past decade. These data processing engines are modeled around the MapReduce paradigm. This article explores Hadoop's MapReduce engine and propose techniques to obtain a higher level of optimization by borrowing concepts from the world of High Performance Computing. Consequently, power consumed and heat generated is lowered. This article designs a system with a pipelined dataflow in contrast to the existing unregulated “bursty” flow of network traffic, the ability to carry out both Map and Reduce tasks in parallel, and a system which incorporates modern high-performance computing concepts using Remote Direct Memory Access (RDMA). To establish the claim of an increased performance measure of the proposed system, the authors provide an algorithm for RoCE enabled MapReduce and a mathematical derivation contrasting the runtime of vanilla Hadoop. This article proves mathematically, that the proposed system functions 1.67 times faster than the vanilla version of Hadoop.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.