Fei Lei
ABSTRACT
Interconnection network plays an essential role in the architecture of large-scale high performance computing (HPC) systems. In the paper, we construct a novel family of low-diameter topologies, Galaxyfly, using techniques of algebraic graphs over finite fields. Galaxyfly is guaranteed to retain a small constant diameter while achieving a flexible tradeoff between network scale and bisection bandwidth. Galaxyfly lowers the demands for high radix of network routers and is able to utilize routers with merely moderate radix to build exascale interconnection networks. We present effective congestion-aware routing algorithms for Galaxyfly by exploring its algebraic property. We conduct extensive simulations and analysis to evaluate the performance, cost and power consumption of Galaxyfly against state-of-the-art topologies. The results show that our design achieves better performance than most existing topologies under various routing algorithms and traffic patterns, and is cost-effective to deploy for exascale HPC systems.
- D. Abts, M. R. Marty, P. M. Wells, P. Klausler, and H. Liu. Energy proportional datacenter networks. In Proceedings of International Symposium on Computer Architecture (ISCA), pages 338--347, 2010. Google Scholar
Digital Library
- J. H. Ahn, N. Binkert, A. Davis, M. McLaren, and R. S. Schreiber. Hyperx: Topology, routing, and packaging of efficient large-scale networks. In Proceedings of International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), pages 41:1--41:11, 2009. Google ScholarDigital Library
- S. Ankit, H. Chi-Yao, P. Lucian, and G. P. Brighten. Jellyfish: Networking data centers randomly. In Proceedings of Symposium on Network System Design and Implementation (NSDI), pages 225--238, 2012. Google ScholarDigital Library
- M. Besta and T. Hoefler. Slim fly: A cost effective low-diameter network topology. In Proceedings of International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), pages 348--359, 2014. Google ScholarDigital Library
- N. Binkert, A. Davis, N. P. Jouppi, M. McLaren, N. Muralimanohar, R. Schreiber, and J. H. Ahn. The role of optics in future high radix switch design. ACM Sigarch Computer Architecture News, 39(3):437--448, June 2011. Google ScholarDigital Library
- N. Chrysos, C. Minkenberg, M. Rudquist, C. Basso, and B. Vanderpool. Scoc: High-radix switches made of bufferless clos networks. In Proceedings of High-Performance Computer Architecture (HPCA), pages 402--414, 2015.Google ScholarCross Ref
- G. Faanes, A. Bataineh, D. Roweth, T. Court, E. Froese, B. Alverson, T. Johnson, J. Kopnick, M. Higgins, and J. Reinhard. Cray cascade: A scalable hpc system based on a dragonfly network. In Proceedings of International Conference for High Performance Computing, Networking, Storage, and Analysis (SC), pages 103:1--103:9, 2012. Google ScholarDigital Library
- I. Fujiwara, M. Koibuchi, H. Matsutani, and H. Casanova. Skywalk: a topology for hpc networks with low-delay switc. In Proceedings of International Parallel and Distributed Processing Symposium (IPDPS), pages 263--272, 2014. Google ScholarDigital Library
- C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, and et al. Bcube: A high performance, server-centric network architecture for modular data centers. In Proceedings of International Conference on the applications, technologies, architectures, and protocols for computer communication (SIGCOMM), pages 63--74, 2009. Google ScholarDigital Library
- C. Guo, H. Wu, K. Tan, L. Shi, Y. Zhang, and S. Lu. Dcell: A scalable and fault-tolerant network structure for data centers. ACM SIGCOMM Computer Communication Review, 38(4):75--86, Aug. 2008. Google ScholarDigital Library
- P. R. Hafner. Geometric realisation of the graphs of mckay- miller- siran. Journal of Combinatorial Theory, 90(2):223--232, 2004. Google ScholarDigital Library
- N. Hamedazimi, Z. Qazi, H. Gupta, V. Sekar, S. R. Das, J. P. Longtin, H. Shah, and A. Tanwer. Firefly: A reconfigurable wireless data center fabric using free-space optics. In Proceedings of International Conference on the applications, technologies, architectures, and protocols for computer communication (SIGCOMM), pages 319--330, 2014. Google ScholarDigital Library
- https://en.wikipedia.org/wiki/Isomorphism.Google Scholar
- http://www.avagotech.com/optical_fpga#.Google Scholar
- http://www.colfaxdirect.com.Google Scholar
- http://www.mellanoxstore.com.Google Scholar
- F. Ikki, K. Michihiro, O. Tomoya, M. Hiroki, and C. Henri. Augmenting low-latency hpc network with free-space optical links. In Proceedings of High-Performance Computer Architecture (HPCA), pages 390--401, 2015.Google Scholar
- S. Jeloka, R. Das, R. G. Dreslinski, T. Mudge, and D. Blaauw. Hi-rise: A high-radix switch for 3d integration with single-cycle arbitration. In Proceedings of IEEE Computer Society Technical Committee on Microprogramming and Microarchitecture (MICRO), pages 471--483, 2014. Google ScholarDigital Library
- J. Kim, W. J. Dally, and D. Abts. Flattened butterfly: A cost-efficient topology for high-radix networks. In Proceedings of International Symposium on Computer Architecture (ISCA), pages 126--137, 2007. Google ScholarDigital Library
- J. Kim, W. J. Dally, S. Scott, and D. Abts. Technology-driven, highly-scalable dragonfly topology. In Proceedings of International Symposium on Computer Architecture (ISCA), pages 77--88, 2008. Google ScholarDigital Library
- M. Koibuchi, I. Fujiwara, H. Matsutani, and H. Casanova. Layout-conscious random topologies for hpc off-chip interconnects. In Proceedings of High-Performance Computer Architecture (HPCA), pages 484--495, 2013. Google ScholarDigital Library
- M. Koibuchi, H. Matsutani, H. Amano, D. F. Hsu, and H. Casanova. A case for random shortcut topologies for hpc interconnects. ACM Sigarch Computer Architecture News, 40(3):177--188, 2012. Google ScholarDigital Library
- S. B. Mark, D. Mark, H. Ram, K. James, L. Tom, R. Todd, D. U. Keith, and R. C. Zak. Intel omni-path architecture: Enabling scalable, high performance fabrics. In Proceedings of Symposium on High-Performance Interconnects (HOTI), pages 402--414, 2015.Google Scholar
- B. D. McKay, M. Miller, and J. Siran. A note on large graphs of diameter two and given maximum degree. Journal of Combinatorial Theory, 74(1):110--118, 1998. Google ScholarDigital Library
- T. P. Morgan. The road to 200g networks starts with the transceiver. http://www.nextplatform.com.Google Scholar
- J. Nan, U. B. Daniel, M. George, B. James, T. Brian, K. John, and J. D. William. A detailed and flexible cycle-accurate network-on-chip simulator. In Proceedings of International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 86--96, 2013.Google Scholar
- A. Putnam, A. Caulfield, E. Chung, D. Chiou, K. Constantinides, and et al. A reconfigurable fabric for accelerating large-scale datacenter services. In Proceedings of International Symposium on Computer Architecture (ISCA), pages 13--24, 2014. Google ScholarDigital Library
- S. Satpathy, K. Sewell, T. Manville, Y.-P. Chen, R. Dreslinski, D. Sylvester, T. Mudge, and D. Blaauw. A 4.5Tb/s 3.4Tb/s/W 64x64 switch fabric with self-updating least-recently-granted priority and quality-of-service arbitration in 45nm CMOS. In Proceedings of International Solid-State Circuits Conference (ISSCC), pages 478--480, 2012.Google Scholar
- S. Scott, D. Abts, J. Kim, and W. J. Dally. The blackwidow high-radix clos network. In Proceedings of International Symposium on Computer Architecture (ISCA), pages 16--28, 2006. Google ScholarDigital Library
- L. Xiangke, P. Zhengbin, W. Kefei, L. Yutong, X. Min, X. Jun, D. Dezun, and S. Guang. High performance interconnect network for tianhe system. Journal of Computer Science & Technology, 30(2):259--272, 2015.Google ScholarCross Ref
- Galaxyfly: A Novel Family of Flexible-Radix Low-Diameter Topologies for Large-Scales Interconnection Networks
Recommendations
A case for random shortcut topologies for HPC interconnects
ISCA '12: Proceedings of the 39th Annual International Symposium on Computer ArchitectureAs the scales of parallel applications and platforms increase the negative impact of communication latencies on performance becomes large. Fortunately, modern High Performance Computing (HPC) systems can exploit low-latency topologies of high-radix ...
Flattened Butterfly Topology for On-Chip Networks
With the trend towards increasing number of cores in a multicore processors, the on-chip network that connects the cores needs to scale efficiently. In this work, we propose the use of high-radix networks in on-chip networks and describe how the ...
A case for random shortcut topologies for HPC interconnects
ISCA '12As the scales of parallel applications and platforms increase the negative impact of communication latencies on performance becomes large. Fortunately, modern High Performance Computing (HPC) systems can exploit low-latency topologies of high-radix ...
Comments