Abstract
We study online scheduling on two uniform machines in the MapReduce system. Each job consists of two sets of tasks, namely the map tasks and reduce tasks. A job’s reduce tasks can only be processed after all its map tasks are finished. The map tasks are fractional, i.e., they can be arbitrarily split and processed on different machines in parallel. Our goal is to find a schedule that minimizes the makespan. We consider two variants of the problem, namely the cases involving preemptive reduce tasks and non-preemptive reduce tasks. We provide lower bounds for both variants. For preemptive reduce tasks, we present an optimal online algorithm with a competitive ratio of \(\frac{\sqrt{s^{2}+2s+5}+1-s}{2}\), where \(s\ge 1\) is the ratio between the speeds of the two machines. For non-preemptive reduce tasks, we show that the \({ LS}\)-like algorithm is optimal and its competitive ratio is \(\frac{2s+1}{s+1}\) if \(s<\frac{1+\sqrt{5}}{2}\) and \(\frac{s+1}{s}\) if \(s\ge \frac{1+\sqrt{5}}{2}\).
Similar content being viewed by others
References
Chang, H., Kodialam, M., Kompella, R.R., Lakshman, T.V.: Scheduling in MapReduce-like systems for fast completion time. In: Proceedings of INFOCOM’14, pp. 3074–3082 (2015)
Chen, C., Xu, Y., Zhu, Y., Sun, C.: Online MapReduce scheduling problem of minimizing the makespan. J. Comb. Optim. 33(2), 590–608 (2017)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Proc. Oper. Syst. Des. Implement. 51(1), 107–113 (2004)
Epstein, L., Noga, J., Seiden, S., Sgall, J., Woeginger, G.: Randomized on-line scheduling on two uniform machines. J. Sched. 4, 71–92 (2001)
Huang, J., Zheng, F., Xu, Y., Liu, M.: Online MapReduce processing on two identical parallel machines. J. Combin. Optim. 35(1), 216–223 (2018)
Jiang, Y., Zhu, Y., Wu, W., Li, D.: Makespan minimization for MapReduce systems with different servers. Future Gener. Comput. Syst. 67, 13–21 (2017)
Jiang, Y., Zhou, W., Zhou, P.: An optimal preemptive algorithm for online MapReduce scheduling on two parallel machines. Asia-Pac. J. Oper. Res. 35(3), 1850013 (2018)
Luo, T., Zhu, Y., Wu, W., Xu, Y., Du, D.: Online makespan minimization in MapReduce-like systems with complex reduce tasks. Optim. Lett. 11, 271–277 (2017)
Le, Y., Liu, J., Ergun, F., Wang, D.: Online load balancing for MapReduce with skewed data input. In: Proceeding of INFOCOM’14, pp. 2004–2012 (2014)
Moseley, B., Dasgupta, A., Kumar, R., Sarlós, T.: On scheduling in Map-Reduce and flowshops. In: Proceedings of the Twenty-Third Annual ACM Symposium on Parallelism in Algorithms and Architectures, pp. 289–298 (2011)
Wen, J., Du, D.: Preemptive on-line scheduling for two uniform processors. Oper. Res. Lett. 23, 113–116 (1998)
Zheng, Y., Shroff, N.B., Sinha, P.: A new analytical technique for designing provably efficient MapReduce schedulers. In: Proceeding of INFOCOM’13, pp. 1600–1608 (2013)
Zhu, Y., Jiang, Y., Wu, W., Ding, L., Teredesai, A., Li, D., Lee, W.: Minimizing makespan and total completion time in MapReduce-like systems. In: Proceeding of INFOCOM’14, pp. 2166–2174 (2014)
Acknowledgements
Jiang was supported in part by the National Natural Science Foundation of China (Grant No. 11571013). Cheng was supported in part by The Hong Kong Polytechnic University under the Fung Yiu King—Wing Hang Bank Endowed Professorship in Business Administration. Ji was supported in part by Zhejiang Provincial Natural Science Foundation of China (Grant No. LR15G010001).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jiang, Y., Zhou, P., Cheng, T.C.E. et al. Optimal online algorithms for MapReduce scheduling on two uniform machines. Optim Lett 13, 1663–1676 (2019). https://doi.org/10.1007/s11590-018-01384-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-018-01384-8