Skip to main content

Makespan Minimization for Batch Tasks in Data Centers

  • Conference paper
  • First Online:
Security, Privacy and Anonymity in Computation, Communication and Storage (SpaCCS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 10067))

Abstract

Makespan is one for crucial factors to determine the performance of job scheduling in cloud data center, short makespan could lead to more job throughput and less energy consumption. In this paper, we study the joint task and data assignment problem to realized makespan minimization. We propose the data migration method to overcome the memory space limitation of servers, and realize better data locality for task execution. We conduct extensive simulations, and the simulation results show that our algorithm has significant improvement on makespan reduction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Apach Hadoop. http://hadoop.apache.org

  2. Ahmad, F., Chakradhar, S., Raghunathan, A., Vijaykumar, T.: Shufflewatcher: shuffle-aware scheduling in multi-tenant mapreduce clusters. In: USENIX ATC (2014)

    Google Scholar 

  3. Ananthanarayanan, G., Agarwal, S., Kandula, S., Greenberg, A., Stoica, I., Harlan, D., Harris, E.: Scarlett: coping with skewed content popularity in mapreduce clusters. In: EuroSys (2011)

    Google Scholar 

  4. Ananthanarayanan, G., Ghodsi, G., Wang, A., Borthakur, D., Kandula, S., Shenker, S., Stoica, I.: Pacman: coordinated memory caching for parallel jobs. In: USENIX NSDI (2012)

    Google Scholar 

  5. Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  6. Eltabakh, M., Tian, Y., Ozcan, F., Gemulla, R., Krettek, A., McPherson, J.: Cohadoop: flexible data placement and its exploitation in hadoop. In: VLDB Endow (2011)

    Google Scholar 

  7. Grandl, R., Ananthanarayanan, G., Kandula, S., Rao, S., Akella, A.: Multi-resource packing for cluster schedulers. In: ACM SIGCOMM (2014)

    Google Scholar 

  8. Jalaparti, V., Bodik, P., Menache, I., Rao, S., Makarychev, K., Caesar, M.: Network-aware scheduling for data-parallel jobs: plan when you can. In: ACM SIGCOMM (2015)

    Google Scholar 

  9. Leung, J., Kelly, L., Anderson, J.H.: Handbook of Scheduling: Algorithms, Models, and Performance Analysis. CRC Press, Boca Raton (2004)

    Google Scholar 

  10. Maguluri, S.T., Srikant, R.: Scheduling jobs with unknown duration in clouds. In: IEEE INFOCOM (2013)

    Google Scholar 

  11. Tan, J., Meng, X., Zhang, L.: Coupling task progress for mapreduce resource-aware scheduling. In: IEEE INFOCOM (2013)

    Google Scholar 

  12. Verma, A., Cherkasova, L., Campbell, R.H.: Two sides of a coin: Optimizing the schedule of mapreduce jobs to minimize their makespan and improve cluster performance. In: IEEE MASCOTS (2012)

    Google Scholar 

  13. Wang, W., Zhu, K., Ying, L., Tan, J., Zhang, L.: Map task scheduling in mapreduce with data locality: throughput and heavy-traffic optimality. In: IEEE INFOCOM (2013)

    Google Scholar 

  14. Wolf, J., Rajan, D., Hildrum, K., Khandekar, R., Kumar, V., Parekh, S., Wu, K.L., Balmin, A.: Flex: a slot allocation scheduling optimizer for mapredcue workloads. In: ACM/IFIP/USENIX Middleware (2010)

    Google Scholar 

  15. Zaharia, M., Borthakur, D., Sarma, J.S., Elmeleegy, K., Shenker, S., Stoica, I.: Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In: EuroSys (2010)

    Google Scholar 

Download references

Acknowledgments

This work is supported in part by the Jiangsu Natural Science Foundation under Grant No. BK20160813, National High Technology Research and Development Program of China under Grant No. 2015AA015303, Project Funded by China Postdoctoral Science Foundation, Fundamental Research Funds for the Central Universities under Grant NO. NS2016097.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xin Li .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Li, X., Tang, C. (2016). Makespan Minimization for Batch Tasks in Data Centers. In: Wang, G., Ray, I., Alcaraz Calero, J., Thampi, S. (eds) Security, Privacy and Anonymity in Computation, Communication and Storage. SpaCCS 2016. Lecture Notes in Computer Science(), vol 10067. Springer, Cham. https://doi.org/10.1007/978-3-319-49145-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49145-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49144-8

  • Online ISBN: 978-3-319-49145-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics