skip to main content
10.1145/1555754.1555783acmconferencesArticle/Chapter ViewAbstractPublication PagesiscaConference Proceedingsconference-collections
research-article

Indirect adaptive routing on large scale interconnection networks

Published: 20 June 2009 Publication History

Abstract

Recently proposed high-radix interconnection networks [10] require global adaptive routing to achieve optimum performance. Existing direct adaptive routing methods are slow to sense congestion remote from the source router and hence misroute many packets before such congestion is detected. This paper introduces indirect global adaptive routing (IAR) in which the adaptive routing decision uses information that is not directly available at the source router. We describe four IAR routing methods: credit round trip (CRT) [10], progressive adaptive routing (PAR), piggyback routing (PB), and reservation routing (RES). We evaluate each of these methods on the dragonfly topology under both steady-state and transient loads. Our results show that PB, PAR, and CRT all achieve good performance. PB provides the best absolute performance, with 2-7% lower latency on steady-state uniform random traffic at 70% load, while PAR provides the fastest response on transient loads. We also evaluate the implementation costs of the indirect adaptive routing methods and show that PB has the lowest implementation cost requiring <1% increase in the total storage of a typical high-radix router.

References

[1]
R. V. Boppana and S. Chalasani. A comparison of adaptive wormhole routing algorithms. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 351--360, San Diego, CA, 1993. ACM.
[2]
W. Choi and S. K. Das. Design and performance analysis of a proxy-based indirect routing scheme in ad hoc wireless networks. Mob. Netw. Appl., 8(5):499--515, 2003.
[3]
C. Clos. A study of non-blocking switching networks. Bell System Technical Journal, 32:406--424, 1953.
[4]
W. Dally and B. Towles. Principles and Practices of Interconnection Networks. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2003.
[5]
W. J. Dally. Virtual-channel flow control. In ISCA'90: Proceedings of the 17th annual international symposium on Computer Architecture, pages 60--68, New York, NY, USA, 1990. ACM.
[6]
P. T. Gaughan and S. Yalamanchili. Adaptive routing protocols for hypercube interconnection networks. Computer, 26(5):12--23, 1993.
[7]
P. Gratz, B. Grot, and S. W. Keckler. Regional congestion awareness for load balance in networks-on-chip. High Performance Computer Architecture, 2008. HPCA 2008. IEEE 14th International Symposium on, pages 203--214, Feb 2008.
[8]
L. Gravano, G. D. Pifarre, P. E. Berman, and J. L. C. Sanz. Adaptive deadlock- and livelock-free routing with all minimal paths in torus networks. IEEE Trans. Parallel Distrib. Syst., 5(12):1233--1251, 1994.
[9]
J. Kim, W. J. Dally, and D. Abts. Flattened butterfly: a cost-efficient topology for high-radix networks. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 126--137, San Diego, CA, 2007.
[10]
J. Kim, W. J. Dally, S. Scott, and D. Abts. Technology-driven, highly-scalable dragonfly network. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 77--88, Beijing, China, 2008.
[11]
J. Kim, W. J. Dally, S. Scott, and D. Abts. Cost-efficient dragonfly topology for large-scale system. IEEE Micro Top Picks, 29(1):33--40, Jan/Feb 2009.
[12]
J. Kim, W. J. Dally, B. Towles, and A. Gupta. Microarchitecture of a high-radix router. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 420--431, Madison, WI, Jun. 2005.
[13]
A. Kumar, L.-S. Peh, and N. Jha. Token flow control. In IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 342--353, Nov. 2008.
[14]
D. H. Linder and J. C. Harden. An adaptive and fault tolerant wormhole routing strategy for k-ary n-cubes. IEEE Trans. Comput., 40(1):2--12, 1991.
[15]
O. Lysne, S.-A. Reinemo, T. Skeie, A. G. Solheim, T. Sødring, L. P. Huse, and B. D. Johnsen. The interconnection network - architectural challenges for utility computing data centres. IEEE Computer, 41(9):62--69, 2008.
[16]
S. Scott, D. Abts, J. Kim, and W. J. Dally. The blackwidow high-radix clos network. In Proc. of the International Symposium on Computer Architecture (ISCA), pages 16--28, Boston, MA, 2006.
[17]
S. L. Scott and G. M. Thorson. The cray T3E network: Adaptive routing in a high performance 3D torus. In Hot Interconnects, pages 147--156, August 1996.
[18]
A. Singh. Load-Balanced Routing in Interconnection Networks. PhD thesis, Stanford University, 2005.
[19]
A. Singh, W. J. Dally, A. K. Gupta, and B. Towles. Adaptive channel queue routing on k-ary n-cubes. In SPAA '04: Proceedings of the sixteenth annual ACM symposium on Parallelism in algorithms and architectures, pages 11--19, New York, NY, USA, 2004. ACM.
[20]
L. G. Valiant. A scheme for fast parallel communication. SIAM Journal on Computing, 11(2):350--361, 1982.

Cited By

View all
  • (2024)Opportunistic Packet Forwarding for Proactive Transport in Datacenters2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619903(1-9)Online publication date: 3-Jun-2024
  • (2024)Enhanced UGAL Routing Schemes for Dragonfly NetworksProceedings of the 38th ACM International Conference on Supercomputing10.1145/3650200.3656602(449-459)Online publication date: 30-May-2024
  • (2024)MUSE: A Runtime Incrementally Reconfigurable Network Adapting to HPC Real-Time Traffic2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00073(765-779)Online publication date: 27-May-2024
  • Show More Cited By

Index Terms

  1. Indirect adaptive routing on large scale interconnection networks

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ISCA '09: Proceedings of the 36th annual international symposium on Computer architecture
      June 2009
      510 pages
      ISBN:9781605585260
      DOI:10.1145/1555754
      • cover image ACM SIGARCH Computer Architecture News
        ACM SIGARCH Computer Architecture News  Volume 37, Issue 3
        June 2009
        495 pages
        ISSN:0163-5964
        DOI:10.1145/1555815
        Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 20 June 2009

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. dragonfly
      2. interconnection networks
      3. routing

      Qualifiers

      • Research-article

      Conference

      ISCA '09
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 543 of 3,203 submissions, 17%

      Upcoming Conference

      ISCA '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)96
      • Downloads (Last 6 weeks)7
      Reflects downloads up to 05 Mar 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Opportunistic Packet Forwarding for Proactive Transport in Datacenters2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619903(1-9)Online publication date: 3-Jun-2024
      • (2024)Enhanced UGAL Routing Schemes for Dragonfly NetworksProceedings of the 38th ACM International Conference on Supercomputing10.1145/3650200.3656602(449-459)Online publication date: 30-May-2024
      • (2024)MUSE: A Runtime Incrementally Reconfigurable Network Adapting to HPC Real-Time Traffic2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00073(765-779)Online publication date: 27-May-2024
      • (2023)GRAP: Group-level Resource Allocation Policy for Reconfigurable Dragonfly Network in HPCProceedings of the 37th International Conference on Supercomputing10.1145/3577193.3593732(437-449)Online publication date: 21-Jun-2023
      • (2023)Workload Interference Prevention with Intelligent Routing and Flexible Job Placement on DragonflyProceedings of the 2023 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation10.1145/3573900.3591119(23-33)Online publication date: 21-Jun-2023
      • (2023)Adaptive Routing with Hierarchical Reinforcement Learning on Dragonfly NetworksICC 2023 - IEEE International Conference on Communications10.1109/ICC45041.2023.10278794(403-409)Online publication date: 28-May-2023
      • (2023)Efficient Intra-Rack Resource Disaggregation for HPC Using Co-Packaged DWDM Photonics2023 IEEE International Conference on Cluster Computing (CLUSTER)10.1109/CLUSTER52292.2023.00021(158-172)Online publication date: 31-Oct-2023
      • (2022)Study of Workload Interference with Intelligent Routing on DragonflySC22: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41404.2022.00025(1-14)Online publication date: Nov-2022
      • (2022)Congestion management in high-performance interconnection networks using adaptive routing notificationsThe Journal of Supercomputing10.1007/s11227-022-04926-179:7(7804-7834)Online publication date: 7-Dec-2022
      • (2021)Q-adaptiveProceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing10.1145/3431379.3460650(189-200)Online publication date: 21-Jun-2021
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media