Skip to main content
SpringerLink
Log in

SDN-based bandwidth scheduling for prioritized data transfer between data centers

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The widespread deployment of scientific applications and business services of various types on clouds requires the transfer of big data with different priorities between geographically distributed cloud-based data centers. As a result, Cloud Service Providers (CSP) face a significant challenge to fully utilize the expensive bandwidth resources of the links connecting data centers while guaranteeing Quality of Experience (QoE) for users. Modern data centers are increasingly adopting Software-Defined Networking (SDN) technology, which provides the capability of advance bandwidth reservation. This paper focuses on the collaborative scheduling of multiple prioritized user requests, namely, advance bandwidth reservation with a lower priority and immediate bandwidth reservation with a higher priority, to maximize the total user satisfaction. We formulate this co-scheduling problem with preemption as a generic optimization problem, which is shown to be NP-complete. We design a heuristic algorithm to maximize the number of successfully scheduled requests and minimize the number of preempted advance reservation requests, while minimizing the completion time of each request. Extensive results from simulations with randomly generated networks and emulation-based experiments on an SDN testbed show that our scheduling scheme significantly outperforms greedy approaches in terms of user satisfaction degree, a normalized quantification parameter we define to measure users’ QoE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The simulation and experimental data will be made available upon request.

References

  1. Hou, A., Wu, C., Zuo, L., Quan, D., Li, Y., Zhu, M., Duan, Q., Fang, D.: Co-scheduling of advance and immediate bandwidth reservations for inter-data center transfer. In: 2019 IEEE/ACM Workshop on Innovating the Network for Data-Intensive Science (INDIS), in Conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’19), Denver, Co, USA, 2019

  2. Kathiravelu, P., Veiga, L.: SDN middlebox architecture for resilient transfers. In: 2017 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), 2017, pp. 560–563. https://doi.org/10.23919/INM.2017.7987329

  3. Grosvenor, M.P., Schwarzkopf, M., Gog, I., Watson, R.N.M., Moore, A.W., Hand, S., Crowcroft, J.: Queues don’t matter when you can jump them! In: 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI ’15), Oakland, CA, USA, 4–6, 2015

  4. Zuo, L.: Intelligent and flexible bandwidth scheduling for data transfers in dedicated high-performance networks. IEEE Trans. Netw. Serv. Manag. 17(4), 2364–2379 (2020). https://doi.org/10.1109/TNSM.2020.3012888

    Article  Google Scholar 

  5. Zuo, L., Zhu, M.M., Wu, C.Q.: Bandwidth reservation strategies for scheduling maximization in dedicated networks. IEEE Trans. Netw. Serv. Manag. (2018). https://doi.org/10.1109/TNSM.2018.2794300

    Article  Google Scholar 

  6. Lin, Y., Wu, Q.: Complexity analysis and algorithm design for advance bandwidth scheduling in dedicated networks. IEEE/ACM Trans. Netw. 21(1), 14–27 (2013)

    Article  Google Scholar 

  7. Zuo, L., Wu, C., Rao, N., Hou, A., Chang, C.: Bandwidth preemption for high-priority data transfer on dedicated channels. In: IEEE International Conference ICCCN, Hangzhou, China, 2018

  8. Xie, C., Ghani, N.: Admission control in networks with prioritized advance reservation. In: Proceedings of International Conference on Computer Communications and Networks, 2009. ICCCN 2009, pp. 1–6 (2009)

  9. Dharam, P.: QoS routing for big data transfer in software-defined networks. PhD Thesis, The University of Memphis, Memphis (2014)

  10. Hou, A., Wu, C., Fang, D., Zuo, L., Zhu, M., Zhang, X., Qiao, R., Yin, X.: Bandwidth scheduling for big data transfer with deadline constraint between data centers. In: Workshop on Innovating the Network for Data-Intensive Science (INDIS), in Conjunction with the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC’18), Dallas, Texas, USA, 2018

  11. Lin, Y., Wu, Q.: On design of bandwidth scheduling algorithms for multiple data transfers in dedicated networks. In: ACM/IEEE Symposium on Architecture for NETWORKING and Communications Systems, ANCS 2008, San Jose, California, USA, 2008, pp. 151–160

  12. Mininet: an instant virtual network on your laptop (or other PC). http://mininet.org/. Accessed 1 Jan 2019

  13. Jin, X., Li, Y., Wei, D., Li, S., Gao, J., Xu, L., Li, W.X. G., Rexford, J.: Optimizing bulk transfers with software-defined optical WAN. In: Proceedings of SIGCOMM, 2016, pp. 87–100. https://doi.org/10.1145/2934872.2934904

  14. Paasch, C., Ferlin, S., Bonaventure, O.: Experimental evaluation of multipath TCP schedulers. In: ACM SIGCOMM Workshop on Capacity Sharing Workshop, 2014. https://doi.org/10.1145/2630088.2631977

  15. Aktas, M.F., Haldeman, G., Parashar, M.: Scheduling and flexible control of bandwidth and in-transit services for end-to-end application workflows. Future Gener. Comput. Syst. 56, 284–294 (2015). https://doi.org/10.1016/j.future.2015.09.011

    Article  Google Scholar 

  16. Open vSwitch: an open virtual switch. http://openvswitch.org/. Accessed 1 Jan 2019

  17. SDN controller. http://www.opendaylight.org/. Accessed 1 Jan 2019

  18. OSCARS. http://www.es.net/oscars. Accessed 1 May 2018

Download references

Acknowledgements

This research is sponsored by the Key Research and Development Plan of Shaanxi Province, China under Grant No. 2018GY-011, and Xi’an Science and Technology Plan Project under Grant No. 2019218214GXRC018CG019-GXYD18.2 with Northwest University, China.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Northwest University, Shaanxi International Joint Research Centre for the Battery-Free Internet of Things, Xi’an, 710127, Shaanxi, China

    Aiqin Hou, Dawei Quan, Yangyang Li & Dingyi Fang

  2. Department of Computer Science, New Jersey Institute of Technology, Newark, NJ, 07102, USA

    Chase Q. Wu

  3. Information Science and Technology Department, Penn State University, Abington, PA, 19001, USA

    Qiang Duan

  4. Computer Science Department, California State University, Dominguez Hills, Carson, CA, 90747, USA

    Liudong Zuo

  5. Department of Computer Science, Montclair State University, Montclair, NJ, 07043, USA

    Michelle M. Zhu

Authors
  1. Aiqin Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chase Q. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiang Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dawei Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liudong Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yangyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michelle M. Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dingyi Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors of this paper made the following contributions to the work in our submission. AH: Conceptualization, Methodology, Writing-Original Draft, Formal analysis, Investigation, Funding acquisition. CW: Conceptualization, Formal analysis, Methodology, Supervision, Investigation, Writing-Review and Editing. QD: Resources, Investigation, Writing-Review and Editing. DQ and YL: Visualization, Validation, Investigation, Data curation. LZ and MZ: Methodology, Data curation, Writing-Review and Editing. DF: Data curation, Writing-Review and Editing

Corresponding author

Correspondence to Chase Q. Wu.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethical approval

This research does not involve any human participants and/or animals.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Some preliminary results in this manuscript have been published in INDIS’19 in conjunction with SC’19 [1].

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, A., Wu, C.Q., Duan, Q. et al. SDN-based bandwidth scheduling for prioritized data transfer between data centers. Cluster Comput 25, 3019–3034 (2022). https://doi.org/10.1007/s10586-021-03364-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03364-7

Keywords

Search

Navigation