Abstract
Dataflow computing has been regarded one of the most promising computing paradigms in the big data era. With the vast distribution of data sources, it is significant to deploy the dataflow based applications in distributed environment to digest these data. In dataflow computing, the data flows shall be transferred between different processing units to the accomplish the predefined semantics. Software-defined networking (SDN) has emerged as an effective network management technology to orchestrate the data flows among these processing units. For each data flow, a forwarding rule shall be inserted into the forwarding table of each switch on the routing path. However, the number of rules that can be inserted in one forwarding table is limited. We are motivated to take such constraints into the consideration of dataflow applications deployment in distributed computing environment managed by SDN. An efficient deployment algrotithm is proposed and evaluated in this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. In: Proceedings of OSDI, vol. 16, pp. 265–283 (2016)
Botero, J.F., Hesselbach, X., Duelli, M., Schlosser, D., Fischer, A., De Meer, H.: Energy efficient virtual network embedding. IEEE Commun. Lett. 16(5), 756–759 (2012)
Chowdhury, N.M.K., Rahman, M.R., Boutaba, R.: Virtual network embedding with coordinated node and link mapping. In: INFOCOM 2009, pp. 783–791. IEEE (2009)
Fischer, A., Botero, J.F., Beck, M.T., de Meer, H., Hesselbach, X.: Virtual network embedding: a survey. IEEE Commun. Surv. Tutor. 15(4), 1888–1906 (2013)
Giroire, F., Moulierac, J., Phan, T.K.: Optimizing rule placement in software-defined networks for energy-aware routing. In: 2014 IEEE Global Communications Conference (GLOBECOM), pp. 2523–2529. IEEE (2014)
Gonzalez, J.E., Xin, R.S., Dave, A., Crankshaw, D., Franklin, M.J., Stoica, I.: Graphx: graph processing in a distributed dataflow framework. In: Proceedings of OSDI, pp. 599–613. USENIX, Broomfield (2014)
Gu, L., Zeng, D., Guo, S., Xiang, Y., Hu, J.: A general communication cost optimization framework for big data stream processing in geo-distributed data centers. IEEE Trans. Comput. 65(1), 19–29 (2016)
Kang, N., Liu, Z., Rexford, J., Walker, D.: Optimizing the one big switch abstraction in software-defined networks. In: Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies, pp. 13–24. ACM (2013)
Kreutz, D., Ramos, F.M., Verissimo, P.E., Rothenberg, C.E., Azodolmolky, S., Uhlig, S.: Software-defined networking: a comprehensive survey. Proc. IEEE 103(1), 14–76 (2015)
Murray, D.G., Schwarzkopf, M., Smowton, C., Smith, S., Madhavapeddy, A., Hand, S.: CIEL: a universal execution engine for distributed data-flow computing. In: Proceedings of ACM/USENIX NSDI, pp. 113–126 (2011)
Sun, G., Yu, H., Anand, V., Li, L.: A cost efficient framework and algorithm for embedding dynamic virtual network requests. Futur. Gener. Comput. Syst. 29(5), 1265–1277 (2013)
Acknowledgment
This work was partially sponsored by the National Basic Research 973 Program of China under grant 2015CB352403, the National Natural Science Foundation of China (NSFC) (61602301, 61402425).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Zhang, Y., Li, Y., Zheng, L., Zeng, D. (2018). On Cost Efficient Dataflow Computing Program Deployment in SDN Managed Distributed Computing Environment. In: Romdhani, I., Shu, L., Takahiro, H., Zhou, Z., Gordon, T., Zeng, D. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 252. Springer, Cham. https://doi.org/10.1007/978-3-030-00916-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-00916-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00915-1
Online ISBN: 978-3-030-00916-8
eBook Packages: Computer ScienceComputer Science (R0)