Elsevier

Computer Networks

Volume 50, Issue 12, 24 August 2006, Pages 2013-2027
Computer Networks

Efficient large-scale BGP simulations

https://doi.org/10.1016/j.comnet.2005.09.033Get rights and content

Abstract

Simulation has been the method of choice for analyzing large, complex, and highly volatile systems. One of these systems is the inter-domain routing infrastructure of the Internet. Despite the need for high quality Border Gateway Protocol (BGP) simulation tools, traditional BGP simulators have limitations either in their modeling fidelity or in their scalability. In this work we introduce BGP++, a scalable BGP simulator that employs state-of-the-art techniques to address the abstraction-scalability trade-off.

BGP++ builds on high quality software in network simulation, routing and parallel-distributed simulation to deliver a detailed yet scalable implementation of BGP. Moreover, with respect to the needs of researchers and operators, BGP++ has a CISCO-like configuration language, a seamless partitioning engine for parallel-distributed simulations and a configuration toolset that expedites common simulation configuration tasks.

Introduction

Modeling and simulation analysis has played a key role in the field of computer networks. Typically, vendors and researchers evaluate prospective architectures and perform comprehensive “what if” analysis using simulation. Simulation is also used for parameter-tuning, problem diagnosis and performance optimizations. It is immensely important in research on large, complex and heterogeneous systems, like the Internet, where analytical models and laboratory testbeds do not capture the detail or the sheer volume of the system. One of these systems is the BGP infrastructure of the Internet.

BGP is the de-facto inter-domain routing protocol in the Internet. It is the “glue” that interconnects more than 16,000 Autonomous Systems (AS) of diverse geopolitical nature. In contrast to Interior Gateway Protocols (IGP), BGP is a policy-based protocol. Business relationships and agreements between ASs determine how packets are routed.

During the last few years the inter-domain routing infrastructure has attracted substantial research interest. Some of the widely researched problems are its slow convergence [1], [2], [3], [4], [5], policy conflicts [6], [7], instability [8], [9], misconfigurations [10], lack of security [11], table growth [12], [13] and path inflation [14], [15], [16]. These problems necessitate fixing or even replacing the current routing architecture. Several examples of proposed new routing architectures can be found in the following references [17], [18], [19], [20], [21], [22], [23]. Yet, it is not clear how BGP should be modified.

The main reason the future of the routing architecture is dubious, is the lack of necessary tools to comprehensively understand the current infrastructure and to evaluate new alternatives. Measuring the performance of BGP is strongly prohibited by ownership and lack of measurement infrastructure. Analytical models of BGP are simplistic and do not capture the complexity, configurability and heterogeneity of the protocol. BGP simulators can only perform moderate-scale simulations of a few hundreds of routers, from which we cannot draw reliable conclusions.

In this work we are concerned with building a simulation tool that will help researchers shed light on the design flaws of the current BGP infrastructure and evaluate the performance of new architectures. BGP++ is designed along the scalability-realism diptych, enabling for the first time large-scale as well as detailed BGP simulations. BGP++ is not a new bottom-up implementation of BGP, but it capitalizes on and benefits from existing high quality software on network simulation, routing and parallel-distributed simulation. The following software are the basic components on which we build BGP++:

  • 1.

    ns-2 [24] is a discrete-event network simulator that serves as a common platform on which researchers can test and compare their proposals. ns-2 has evolved into the most widely used simulator in networking literature. It includes numerous implementations of protocols with a special emphasis on TCP variants.

  • 2.

    GNU Zebra [25] and its ancestor, Quagga [26],1 is a detailed open-source implementation of BGP. It is used by a large community of providers, vendors and researchers for testing and experimenting with the protocol. It is also used for routing by small ASs that cannot afford buying expensive routers.

  • 3.

    PDNS [27] is the parallel-distributed version of the ns-2 simulator. It enables large-scale simulations by distributing the simulation model on multiple workstations, thereby granting more physical resources. In contrast to its counterpart, SSFnet [28], PDNS supports both shared-memory multiprocessors and distributed-memory clusters of workstations. Support of distributed architectures offers more physical resources, overwhelming the limits inherent in parallel architectures. Remarkably, PDNS was recently used by Fujimoto et al. [29] to realize the largest network simulations ever, of more than 5 million network nodes.

Our contributions can be summarized as follows:

  • 1.

    We develop and make publicly available [30] a packet-level BGP simulator on the widely used ns-2 simulation platform.

  • 2.

    We integrate the BGP implementation of Zebra into ns-2, making the minimum possible changes to the original software. We realize an accurate BGP simulator that supports most of the details of Zebra’s BGP implementation, including a CISCO-like configuration language.

  • 3.

    We use and extend PDNS to support parallel-distributed BGP simulations. We also evaluate the performance of alternative model partitioning algorithms for parallel-distributed simulations.

  • 4.

    We identify the representation of BGP routing tables as the main source of memory consumption in BGP simulations. We introduce a compact routing table data structure that exploits the redundancy of information in BGP routing tables and realizes significant memory savings.

  • 5.

    We propose and develop a simple generic technique to speedup simulation trials using process checkpointing.

  • 6.

    We survey recent advancements in measuring the Internet. We highlight measurement data and related models that should be explored to enhance the fidelity of BGP simulations.

  • 7.

    We develop a seamless partitioning and configuration engine for parallel-distributed BGP simulations that hides the complexity of PDNS configuration and brings parallel-distributed simulation closer to the general ns-2 user. We also develop an automatic generator of CISCO-like configuration. We combine these tools in a toolset that expedites common BGP simulation and configuration practices.

The remainder of this paper is organized as follows. Section 2 elaborates on previous efforts on BGP simulation. Section 3 describes in detail the development of BGP++. Section 4 introduces three techniques that we use to materialize large-scale BGP simulations. Section 5 surveys the developments in the field of Internet measurements with respect to BGP. Section 6 introduces a toolset to expedite configuration, partitioning and scheduling of BGP simulations. Finally, Section 7 talks about future directions of our research.

Section snippets

Related work

The last few years there has been a considerable work on BGP simulation research. The BGP model by Premore in the SSFnet [28] simulator was the first detailed simulation model of BGP and is currently the most widely used BGP simulator. It supports most important BGP features and extensions. The main limitation of SSFnet is that it exhibits considerable memory demand, thereby preventing simulations larger than a few hundred of BGP routers.

Independently and in parallel with BGP++ a number of

BGP++ development

The process of modeling a system is subject to the abstraction-scalability trade-off. Higher abstraction results in more efficient models, which therefore are more scalable. On the other hand, detail is required to thoroughly capture the characteristics of the system. To create detailed simulation models we chose to incorporate the Zebra open-source BGP implementation into the ns-2 simulation environment. The main advantage of our approach is that the simulator inherits the detail,

Scaling BGP++

The need for large-scale BGP simulations stems from the scale of the BGP infrastructure. More than 16,000 ASs use BGP, including a number of global providers that administrate thousands of internal routers. Also, the advent of the Internet measurements research over the last few years has enhanced the information known about BGP, making it possible to construct more detailed and realistic simulations. However, this comes at the cost of more simulation resources, underlining the need for

How to simulate BGP

Development of realistic simulation models is a challenging endeavor. The reliability of conclusions drawn from simulation analysis depends highly on the accuracy and the correctness of the simulation models. The accuracy of these models can be enhanced by exploring Internet measurements. In this section we survey the data and models available from Internet measurement studies that can be used to construct realistic BGP simulations.

A BGP simulation may include one or more of the following:

BGP++ configuration toolset

We develop a toolset to expedite typical simulation configuration tasks. The simulation toolset can read simple user input files, generate CISCO-like configuration files, partition parallel-distributed simulations, generate configuration for PDNS and schedule multiple simulation runs. Fig. 6 illustrates a block diagram of the software architecture of the toolset. There are three main components: a configuration generator, a partitioner and a scheduler.

The configuration component takes as input

Conclusion and future work

In this work we develop BGP++, a scalable and detailed BGP simulator that is built on top of high quality software in network simulation (ns-2), routing (Zebra) and parallel-distributed simulation (PDNS).

BGP research has ample fertile ground for simulation-based analysis. BGP++ opens new directions in using simulation to improve our understanding of the Internet inter-domain infrastructure.

Evaluating the future performance of the inter-domain infrastructure is a problem in which simulation is

Acknowledgement

This work is supported in part by NSF under contract numbers ANI-9977544 and ANI-0136969; and in part by DARPA under contract number N66002-00-1-8934.

Xenofontas A. Dimitropoulos is a Fulbright scholar, working towards his Ph.D. in the Department of Electrical and Computer Engineering at the Georgia Institute of Technology. He received his Bachelor’s in Physics from the Aristotle University of Thessaloniki. His research interests include Internet measurements, modeling of communication systems and large-scale simulation techniques.

References (72)

  • K. Varadhan et al.

    Persistent route oscillations in inter-domain routing

    Computer Networks

    (2000)
  • H. Chang et al.

    Towards capturing representative AS-level Internet topologies

    Computer Networks Journal

    (2004)
  • C. Labovitz, A. Ahuja, A. Bose, F. Jahanian, Delayed Internet routing convergence, in: Proceedings of ACM SIGCOMM,...
  • C. Labovitz, A. Ahuja, R. Wattenhofer, S. Venkatachary, The impact of Internet policy and topology on delayed routing...
  • D. Pei et al.

    Improving BGP convergence through consistency assertions

  • A. Bremler-Barr, Y. Afek, S. Schwarz, Improved BGP convergence via Ghost Flushing (2003). Available from:...
  • D. Pei, M. Azuma, N. Nguyen, J. Chen, D. Massey, L. Zhang, BGP-RCN: Improving BGP Convergence Through Root Cause...
  • T. Griffin, G. Willfong, A safe path vector protocol, in: Proceedings of IEEE INFOCOMM,...
  • T.G. Griffin, A.D. Jaggard, V. Ramachandran, Design principles of policy languages for path vector protocols, in:...
  • R. Govindan, A. Reddy, An analysis of inter-domain topology and route stability, in: Proceedings of INFOCOM,...
  • R. Mahajan, D. Wetherall, T. Anderson, Understanding BGP misconfiguration, in: ACM SIGCOMM,...
  • S. Kent et al.

    Secure border gateway protocol (Secure-BGP)

    IEEE Journal on Selected Areas in Communications

    (2000)
  • G. Huston, BGP table...
  • T. Bates, Routing table history. Available from:...
  • N. Spring, R. Mahajan, T. Anderson, Quantifying the causes of path inflation, in: ACM SIGCOMM,...
  • L. Gao, F. Wang, The extent of AS path inflation by routing policies, in: Proceeding of Global Internet 2002, 2002, pp....
  • H. Tangmunarunkit, R. Govindan, S. Shenker, Internet path inflation due to policy routing, in: Proceeding of SPIE ITCom...
  • I. Castineyra, N. Chiappa, M. Steenstrup, The Nimrod Routing Architecture, IETF, RFC 1992,...
  • F. Kastenholz, ISLAY: A New Routing and Addressing Architecture, IRTF, Internet Draft,...
  • X. Yang, NIRA: a new Internet routing architecture, Ph.D. Thesis, Massachusetts Institute of Technology, September...
  • P. Verkaik, A. Broido, kc claffy, R. Gao, Y. Hyun, R. van der Pol, Beyond CIDR aggregation, cAIDA Technical Report...
  • L. Subramanian, M. Caesar, C.T. Ee, M. Handley, M. Mao, S. Shenker, I. Stoica, Towards a next generation inter-domain...
  • S. Agarwal, C.-N. Chuah, R. Katz, OPCA: Robust interdomain policy routing and traffic control, in: IEEE OPENARCH,...
  • A. Snoeren, B. Raghavan, Decoupling policy from mechanism in internet routing, in: Proceedings of ACM SIGCOMM Workshop...
  • S. McCanne, S. Floyd, The LBNL network simulator. Available from: <http://www.isi.edu/nsnam>, Lawrence Berkeley...
  • K. Ishiguro, GNU Zebra. Available from:...
  • P. Jakma, V. Jardin, A. Schorr, H. Tepper, G. Troxel, Quagga Routing Suite. Available from:...
  • G.F. Riley, R.M. Fujimoto, M.H. Ammar, Parallel/distributed ns. Available from:...
  • J.H. Cowie, D.M. Nicol, A.T. Ogielski, Modeling the global Internet, Computing in Science and...
  • R.M. Fujimoto, K.S. Perumalla, A. Park, M.A.H. Wu, G.F. Riley, Large-scale network simulation. How big? How fast? in:...
  • X. Dimitropoulos, G. Riley, BGP++ webpage. Available from:...
  • B. Quoitin, S. Uhlig, Modeling the routing of an autonomous system with C-BGP, IEEE Network, vol. 19(6), November...
  • F. Hao et al.

    An Internet scale simulation setup for BGP

    Computer Communication Review

    (2003)
  • T.D. Feng, R. Ballantyne, L. Trajkovi, Implementation of BGP in a network simulator, in: Proceedings of Advanced...
  • B. Quoitin, Towards a BGP model in Javasim,...
  • B.K. Szymanski, Y. Liu, R. Gupta, Parallel network simulation under distributed Genesis, in: Proceedings of...
  • Cited by (20)

    • Measurement of large-scale BGP events: Definition, detection, and analysis

      2016, Computer Networks
      Citation Excerpt :

      Simulation is important for studying giant and complex systems such as the BGP system. To integrate our method with BGP simulation tools (e.g., [26]), one needs to set up a special BGP router that peers with other BGP routers for the purpose of gathering updates. The major advantage of working with simulation tools, according to our understanding, is that each BGP router in a simulation tool could work as a monitor.

    • Modeling a policy-capable path-vector routing protocol using Jacobi iteration over a path algebra

      2011, Computer Networks
      Citation Excerpt :

      Regarding distributed simulation, it is possible to simulate larger network sizes more quickly by distributing the network experiment across multiple computing platforms [19]. However, except for some treatment in [20,21], distributed simulation has not been widely used for performing BGP simulations. This is possibly due to the non-existent or unsupported distributed BGP simulation capabilities within popular public-domain discrete-event network simulation tools (e.g., SSFNET [15], ns2 [6]).

    • Expected convergence properties of BGP

      2011, Computer Networks
      Citation Excerpt :

      The work in [22] develops testing techniques for BGP (or other path vector routing protocols) that can scale to large topologies. Simulation is an important tool in understanding BGP performance and the BGP++ simulator presented in [23] is intended to be detailed yet scalable to large topologies. Finally, in [24], the underlying graph theoretical problem solved by BGP is generalized to a fractional version for which solutions are always shown to exist.

    • BGPy: The BGP Python Security Simulator

      2023, ACM International Conference Proceeding Series
    View all citing articles on Scopus

    Xenofontas A. Dimitropoulos is a Fulbright scholar, working towards his Ph.D. in the Department of Electrical and Computer Engineering at the Georgia Institute of Technology. He received his Bachelor’s in Physics from the Aristotle University of Thessaloniki. His research interests include Internet measurements, modeling of communication systems and large-scale simulation techniques.

    George F. Riley is an Assistant Professor of Electrical and Computer Engineering at the Georgia Institute of Technology. He received his Ph.D. in computer science from the Georgia Institute of Technology, College of Computing, in August 2001. His research interests are large-scale simulation using distributed simulation methods. He is the developer of Parallel/Distributed ns2 (PDNS), and the Georgia Tech Network Simulator (GTNetS).

    View full text