Abstract
This article describes network resource control system for HPC based on software defined network (SDN). For job scheduling this system uses BackFill algorithm modifications with job assignment algorithms Summed Distance Minimization and Maximum Distance Minimization we have developed. We also offer data flow control methods for SDN in high-performance systems such as reactive and proactive algorithms. Our experiment has shown that BackFill scheduling algorithm in combination with Summed Distance Minimization and the proactive routing algorithm demonstrates a significant decrease in execution time for the reference communication-intensive job flow on a cluster.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Feitelson, D., Weil, A.: Utilization and predictability in scheduling the IBM SP2 with backfilling. In: Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, pp. 542–546 (1998)
Kovalenko, V.N., Semyachkin, D.A.: Using BackFill in grid-systems. In: Proceedings of the International Conference on Distributing Computing and Grid-technologies in Science and Education, pp. 139–144 (2004)
Polezhaev, P.N.: Simulator of cluster and its management system used for research of job scheduling algorithms. Bulletin of SUSU – Mathematical Modeling and Programming 35(6,211), 79–90 (2010)
McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., Turner, J.: Openflow: enabling innovation in campus networks. ACM SIGCOMM Computer Communication Review 38, 69–74 (2008)
Intro to OpenFlow, https://www.opennetworking.org/standards/intro-to-openflow
Chowdhury, M.: Managing data transfers in computer clusters with Orchestra. In: Proceedings of the ACM SIGCOMM 2011, pp. 98–109 (2011)
Hong, C.-Y., Caesar, M., Godfrey, P.B.: Finishing flows quickly with preemptive scheduling. ACM SIGCOMM Computer Communication Review, Special October Issue SIGCOMM 2012 42(4), 127–138 (2012)
Narayan, S., Bailey, S., Daga, A.: Hadoop acceleration in an OpenFlow-based cluster. In: Proceedings of 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, pp. 535–538 (2012)
Narayan, S., Bailey, S., Daga, A., et al.: Openflow enabled Hadoop over local and wide area clusters. In: Proceedings of High Performance Computing, Networking, Storage and Analysis, pp. 1625–1628 (2012)
Chowdhury, M., Stoica, I.: Coflow: a networking abstraction for cluster applications. In: Proceedings of the 11th ACM Workshop on Hot Topics in Networks, pp. 31–36 (2012)
Chowdhury, M., Stoica, I.: Coflow: an application layer abstraction for cluster networking. Technical report EECS-2012-184 (2012)
Bender, M.A., Bunde, D.P., Fekete, S.P., Leung, V.J., Meijer, H., Phillips, C.A.: Communication-aware processor allocation for supercomputers: finding point sets of small average distance. Algorithmica 50(2), 279–298 (2008)
Mache, J., Lo, V., Windisch, K.: Minimizing message-passing contention in fragmentation-free processor allocation (2012), http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.55.390&rep=rep1&type=pdf
NOX-Classic wiki, https://github.com/noxrepo/nox-classic/wiki
Cbench - Scalable Cluster Benchmarking and Testing, http://sourceforge.net/apps/trac/cbench/
Aida, K., Kasahara, H., Narita, S.: Job Scheduling Scheme for Pure Space Sharing among Rigid Jobs. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1998. LNCS, vol. 1459, pp. 98–121. Springer, Heidelberg (1998)
Feitelson, G.: Workload Modeling for Computer Systems Performance Evaluation. Workload modeling book draft, Ver. 0.37, 508 p. (2012)
Lublin, U., Feitelson, G.: The workload on parallel supercomputers: modeling the characteristics of rigid job. Journal of Parallel and Distributed Computing 63(11), 542–546 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Polezhaev, P., Shukhman, A., Ushakov, Y. (2014). Network Resource Control System for HPC Based on SDN. In: Balandin, S., Andreev, S., Koucheryavy, Y. (eds) Internet of Things, Smart Spaces, and Next Generation Networks and Systems. NEW2AN 2014. Lecture Notes in Computer Science, vol 8638. Springer, Cham. https://doi.org/10.1007/978-3-319-10353-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-10353-2_19
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10352-5
Online ISBN: 978-3-319-10353-2
eBook Packages: Computer ScienceComputer Science (R0)