Skip to main content
SpringerLink
Log in

A Survey of Speculative Execution Strategy in MapReduce

  • Conference paper
  • First Online:
Cloud Computing and Security (ICCCS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10039))

Included in the following conference series:

Abstract

MapReduce is a parallel computing programming model designed to process large-scale data. Therefore, the accuracy and efficiency for computing are needed to be assured and speculative execution is an efficient method for calculation of fault tolerance. It reaches the goals of shortening the execution time and increasing the cluster throughput through selecting slow tasks and speculative copy these tasks on a fast machine to be executed. Hadoop naïve speculative execution strategy assumes that the cluster is homogeneous, and this assumption leads to the poor performance in heterogeneous environment. Several speculative execution strategies which aim to improve the MapReduce Performance in the heterogeneous environments are reviewed in this paper like LATE, MCP, ex-MCP and ERUL, then the comparison between these methods are listed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Armbrust, M., Fox, A., Griffith, R., Joseph, A., Katz, R., Konwinski, A., Zaharia, M.: A view of cloud computing. Commun. ACM 53(4), 50–58 (2010)

    Article  Google Scholar 

  2. Ghemawat, S., Gobioff, H., Leung, S.T.: The Google file system. ACM SIGOPS Oper. Syst. Rev. 37(5), 29–43 (2003)

    Article  Google Scholar 

  3. Dean, J., Ghemawa, S.: MapReduce: simplified data processing on large clusters. Proc. Oper. Syst. Des. Implement. 51(1), 107–113 (2004)

    Google Scholar 

  4. Chang, F., Dean, J., Ghemawa, S.: A distributed storage system for structured data. ACM Trans. Comput. Syst. 26(2), 1–26 (2008)

    Article  Google Scholar 

  5. Apache Hadoop (2013). http://Hadoop.Apache.Org/

  6. Vijayalakshmi, B., Ravi, P.R.: The down of big Data-Hbase. In: IEEE 2014 Conference on IT in Business, Industry and Government (2014)

    Google Scholar 

  7. Apache Pig (2014). http://pig.apache.org/

  8. Apache Hive (2014). https://hive.apache.org/

  9. Xia, Z.H., Wang, X.H., Sun, X.H., Wang, Q.: A secure and dynamic multi-keyword ranked search scheme over encrypted cloud data. IEEE Trans. Parallel Distrib. Syst. 27(2), 340–352 (2015)

    Article  MathSciNet  Google Scholar 

  10. Fu, Z.J., Ren, K., Shu, J.G., Sun, X.M.: Enabling personalized search over encrypted outsourced data with efficiency improvement. IEEE Trans. Parallel Distrib. Syst. (in press)

    Google Scholar 

  11. Fu, Z.J., Sun, X.M., Li, Q., Zhou, L., Shu, J.G.: Achieving efficient cloud search services: multi-keyword ranked search over encrypted cloud data supporting parallel computing. IEICE Trans. Commun. E98-B(1), 190–200 (2015)

    Article  Google Scholar 

  12. Yoo, D.G., Sim, K.M.: A comparative review of job scheduling for MapReduce. In: IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS), pp. 353–358. IEEE (2011)

    Google Scholar 

  13. Isard, M., Budiu, M., Yu, Y., Birrel, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, pp. 59–72. ACM (2007)

    Google Scholar 

  14. Nenavath, S.N., Atul, N.: A review of adaptive approaches to MapReduce scheduling in heterogeneous environments. In: International Conference on Advances in Computing, Communications and Informatics, pp. 677–683. IEEE (2014)

    Google Scholar 

  15. Zaharia, M., Konwinski, A., Joseph, A., Katz, R., Stoica, I.: Improving MapReduce performance in heterogeneous environments. Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI), pp. 29–42 (2008)

    Google Scholar 

  16. Chen, Q., Liu, C., Xiao, Z.: Improving MapReduce performance using smart speculative execution strategy. IEEE Trans. Comput. 63(4), 954–967 (2014)

    Article  MathSciNet  Google Scholar 

  17. Huang, X., Zhang, L.X., Li, R.F., Wan, L.J., Li, K.Q.: Novel heuristic speculative execution strategies in heterogeneous distributed environments. Comput. Electr. Eng. 50, 166–179 (2015)

    Article  Google Scholar 

  18. Wu, H.C., Li, K., Tang, Z., Zhang, L.: A heuristic speculative execution strategy in heterogeneous distributed environments. In: 2014 Sixth International Symposium on Parallel Architectures, Algorithms and Programming (PAAP), pp. 268–273 (2014)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the NSFC (61300238, 61300237, 61232016, 1405254, 61373133), Marie Curie Fellowship (701697-CAR-MSCA-IFEF-ST), Basic Research Programs (Natural Science Foundation) of Jiangsu Province (BK20131004) and the PAPD fund.

Author information

Authors and Affiliations

  1. Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Qi Liu

  2. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, China

    Qi Liu & Dandan Jin

  3. Jiangsu Engineering Centre of Network Monitoring, Nanjing University of Information Science and Technology, Nanjing, China

    Dandan Jin

  4. School of Computing, Edinburgh Napier University, 10 Colinton Road, Edinburgh, EH10 5DT, UK

    Xiaodong Liu

  5. The University of Salford, Salford, Greater Manchester, M5 4WT, UK

    Nigel Linge

Authors
  1. Qi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dandan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaodong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nigel Linge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qi Liu .

Editor information

Editors and Affiliations

  1. University of Information Science and Technology, Nanjing, China

    Xingming Sun

  2. Michigan State University , EAST LANSING, Michigan, USA

    Alex Liu

  3. National Dong Hwa University , Shoufeng, Taiwan

    Han-Chieh Chao

  4. Purdue University , West Lafayette, Indiana, USA

    Elisa Bertino

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Liu, Q., Jin, D., Liu, X., Linge, N. (2016). A Survey of Speculative Execution Strategy in MapReduce. In: Sun, X., Liu, A., Chao, HC., Bertino, E. (eds) Cloud Computing and Security. ICCCS 2016. Lecture Notes in Computer Science(), vol 10039. Springer, Cham. https://doi.org/10.1007/978-3-319-48671-0_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48671-0_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48670-3

  • Online ISBN: 978-3-319-48671-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation