Abstract
Skewness in input data can degrade the performance of MapReduce. There are many approaches to address this problem. This paper compares and contrasts these approaches to observe their performance on syntactic and real-world data. The results show that all of the algorithms studied in this paper can improve the execution time of MapReduce with skewed data. However, there are some limitations to improvement, especially when data is not heavily skewed; the overhead of the algorithms might overcome their benefits.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Irandoost, M.A., Rahmani, A.M., Setayeshi, S.: MapReduce data skewness handling: a systematic literature review. Int. J. Parallel Prog. 47, 907–950 (2019). https://doi.org/10.1007/s10766-019-00627-0
Kwon, Y.C., Ren, K., Balazinska, M., Howe, B., Rolia, J.: Managing skew in Hadoop. IEEE Data Eng. Bull. 36 (2013)
Xie, J., et al.: Improving MapReduce performance through data placement in heterogeneous Hadoop clusters. In: Proceedings of the 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Ph.D. Forum. IEEE Computer Society (2010)
Gua, Z., Pierce, M., Fox, G., Zhou, M.: Automatic task reorganization in MapReduce. In: Proceedings of the 2011 IEEE International Conference on Cluster Computing. IEEE Computer Society (2011)
Vernica, R., Balman, A., Beyer, K.S., Ercegovac, V.: Adaptive MapReduce using situation-aware mappers. In: Proceedings of the 15th International Conference on Extending Database Technology. ACM (2012)
Guo, Y., Rao, J., Cheng, D., Zhou, X.: iShuffle: improving Hadoop performance with shuffle-on-write. IEEE Trans. Parallel Distrib. Syst. 28 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kanteewong, N., Boonma, P. (2023). A Comparative Study on Improvement of MapReduce Performance with Skewed Data. In: Barolli, L. (eds) Advances in Intelligent Networking and Collaborative Systems. INCoS 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 182. Springer, Cham. https://doi.org/10.1007/978-3-031-40971-4_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-40971-4_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40970-7
Online ISBN: 978-3-031-40971-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)