Abstract
Data-intensive applications in data center networks generate a large number of parallel data streams, where the strategy of flow scheduling and the allocation of network bandwidth have become research hotspot issues in this field. Compared with the scheduling of a single data stream, coflow that aims to improve the overall performance of parallel applications can transmit the application layer semantics to the network layer, which is conducive to scheduling decisions by taking full advantage of the application layer semantics. This paper focuses on coflow scheduling with the goal of optimizing completion time, where we review the existing scheduling frameworks and discuss the ideal characteristics in future work. The existing schemes fall into two categories of centralized scheduling and distributed scheduling. Centralized scheduling makes scheduling decisions through the global view of the central scheduler, and distributed scheduling makes scheduling decisions through the local view of adjacent switches. The existing scheduling schemes have made great progress in time optimization, while in-depth research is still needed in the future in terms of fault tolerance, scalability, and starvation avoidance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Awan, N., Khan, S., Khalid, M., Rahmani, I., Tahir, M.: Machine learning-enabled power scheduling in IoT-based smart cities 10 (2021)
Boutin, E., et al.: Apollo: scalable and coordinated scheduling for cloud-scale computing. In: 11th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 14), pp. 285–300 (2014)
Cao, J., Kerr, G., Arya, K., Cooperman, G.: Transparent checkpoint-restart over infiniband. In: Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed, pp. 13–24 (2014)
Cho, I., Jang, K., Han, D.: Credit-scheduled delay-bounded congestion control for datacenters. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 239–252 (2017)
Chowdhury, M., Stoica, I.: Coflow: a networking abstraction for cluster applications. In: Proceedings of the 11th ACM Workshop on Hot Topics in Networks, pp. 31–36 (2012)
Chowdhury, M., Stoica, I.: Efficient coflow scheduling without prior knowledge. ACM SIGCOMM Comput. Commun. Rev. 45(4), 393–406 (2015)
Chowdhury, M., Zaharia, M., Ma, J., Jordan, M.I., Stoica, I.: Managing data transfers in computer clusters with orchestra. ACM SIGCOMM Comput. Commun. Rev. 41(4), 98–109 (2011)
Chowdhury, M., Zhong, Y., Stoica, I.: Efficient coflow scheduling with varys. In: Proceedings of the 2014 ACM Conference on SIGCOMM, pp. 443–454 (2014)
Dogar, F.R., Karagiannis, T., Ballani, H., Rowstron, A.: Decentralized task-aware scheduling for data center networks. ACM SIGCOMM Comput. Commun. Rev. 44(4), 431–442 (2014)
Ghorbani, S., Yang, Z., Godfrey, P.B., Ganjali, Y., Firoozshahian, A.: Drill: micro load balancing for low-latency data center networks. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 225–238 (2017)
Handley, M., et al.: Re-architecting datacenter networks and stacks for low latency and high performance. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 29–42 (2017)
Li, Z., Zhang, Y., Li, D., Chen, K., Peng, Y.: Optas: decentralized flow monitoring and scheduling for tiny tasks. In: IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9. IEEE (2016)
Luo, S., Yu, H., Zhao, Y., Wang, S., Yu, S., Li, L.: Towards practical and near-optimal coflow scheduling for data center networks. IEEE Trans. Parallel Distrib. Syst. 27(11), 3366–3380 (2016)
Qiu, Z., Stein, C., Zhong, Y.: Minimizing the total weighted completion time of coflows in datacenter networks. In: Proceedings of the 27th ACM Symposium on Parallelism in Algorithms and Architectures, pp. 294–303 (2015)
Roy, A., Zeng, H., Bagga, J., Porter, G., Snoeren, A.C.: Inside the social network’s (datacenter) network. In: Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, pp. 123–137 (2015)
Tan, H.: Joint online coflow routing and scheduling in data center networks. IEEE/ACM Trans. Netw. 27(5), 1771–1786 (2019)
Wang, Z., et al.: Efficient scheduling of weighted coflows in data centers. IEEE Trans. Parallel Distrib. Syst. 30(9), 2003–2017 (2019)
Yan, Y., Kong, Y., Fu, Z.: Dynamic resource scheduling in emergency environment. J. Inf. Hiding Priv. Prot. 1(3), 143 (2019)
Yang, G., Jiang, Y., Li, Q., Jia, X., Xu, M.: Cross-layer self-similar coflow scheduling for machine learning clusters. In: 2018 27th International Conference on Computer Communication and Networks (ICCCN), pp. 1–9. IEEE (2018)
Zhang, H., et al.: Da&fd-deadline-aware and flow duration-based rate control for mixed flows in dcns. IEEE/ACM Trans. Netw. 27(6), 2458–2471 (2019)
Zhang, H., Shi, X., Guo, Y., Wang, Z., Yin, X.: More load, more differentiation-let more flows finish before deadline in data center networks. Comput. Netw. 127, 352–367 (2017)
Zhang, H., et al.: Guaranteeing deadlines for inter-data center transfers. IEEE/ACM Trans. Netw. 25(1), 579–595 (2016)
Zhang, H., Chen, L., Yi, B., Chen, K., Chowdhury, M., Geng, Y.: Coda: toward automatically identifying and scheduling coflows in the dark. In: Proceedings of the 2016 ACM SIGCOMM Conference, pp. 160–173 (2016)
Zhao, Y., et al.: Rapier: Integrating routing and scheduling for coflow-aware data center networks. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 424–432. IEEE (2015)
Zheng, J.: Django: bilateral coflow scheduling with predictive concurrent connections. J. Parallel Distrib. Comput. 152, 45–56 (2021)
Zhou, Q.: Fast coflow scheduling via traffic compression and stage pipelining in datacenter networks. IEEE Trans. Comput. 68(12), 1755–1771 (2019)
Acknowledgement
This work was supported by the Hainan Provincial Natural Science Foundation of China (620RC560, 2019RC096, 620RC562), the Scientific Research Setup Fund of Hainan University (KYQD(ZR)1877), the Program of Hainan Association for Science and Technology Plans to Youth R&D Innovation (QCXM201910), and the National Natural Science Foundation of China (61802092, 62162021).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, D., Cao, G., Xiao, L., Yao, J., Cao, X. (2022). Research Progress and Trend of Coflow Time-Optimal Scheduling in Data Center Network. In: Sun, X., Zhang, X., Xia, Z., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2022. Lecture Notes in Computer Science, vol 13339. Springer, Cham. https://doi.org/10.1007/978-3-031-06788-4_47
Download citation
DOI: https://doi.org/10.1007/978-3-031-06788-4_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06787-7
Online ISBN: 978-3-031-06788-4
eBook Packages: Computer ScienceComputer Science (R0)