Welcome to the InfoSci Platform
Reference Hub7
Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data

Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data

Padmashree Ravindra, Kemafor Anyanwu
Copyright: © 2014 |Volume: 10 |Issue: 1 |Pages: 26
ISSN: 1552-6283|EISSN: 1552-6291|EISBN13: 9781466657007|DOI: 10.4018/ijswis.2014010101
Cite Article Cite Article

MLA

Ravindra, Padmashree, and Kemafor Anyanwu. "Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data." IJSWIS vol.10, no.1 2014: pp.1-26. http://doi.org/10.4018/ijswis.2014010101

APA

Ravindra, P. & Anyanwu, K. (2014). Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data. International Journal on Semantic Web and Information Systems (IJSWIS), 10(1), 1-26. http://doi.org/10.4018/ijswis.2014010101

Chicago

Ravindra, Padmashree, and Kemafor Anyanwu. "Nesting Strategies for Enabling Nimble MapReduce Dataflows for Large RDF Data," International Journal on Semantic Web and Information Systems (IJSWIS) 10, no.1: 1-26. http://doi.org/10.4018/ijswis.2014010101

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

Graph and semi-structured data are usually modeled in relational processing frameworks as “thin” relations (node, edge, node) and processing such data involves a lot of join operations. Intermediate results of joins with multi-valued attributes or relationships, contain redundant subtuples due to repetition of single-valued attributes. The amount of redundant content is high for real-world multi-valued relationships in social network (millions of Twitter followers of popular celebrities) or biological (multiple references to related proteins) datasets. In MapReduce-based platforms such as Apache Hive and Pig, redundancy in intermediate results contributes avoidable costs to the overall I/O, sorting, and network transfer overhead of join-intensive workloads due to longer workflows. Consequently, providing techniques for dealing with such redundancy will enable more nimble execution of such workflows. This paper argues for the use of a nested data model for representing intermediate data concisely using nesting-aware dataflow operators that allow for lazy and partial unnesting strategies. This approach reduces the overall I/O and network footprint of a workflow by concisely representing intermediate results during most of a workflow's execution, until complete unnesting is absolutely necessary. The proposed strategies are integrated into Apache Pig and experimental evaluation over real-world and synthetic benchmark datasets confirms their superiority over relational-style MapReduce systems such as Apache Pig and Hive.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.