Efficient large-scale BGP simulations
Introduction
Modeling and simulation analysis has played a key role in the field of computer networks. Typically, vendors and researchers evaluate prospective architectures and perform comprehensive “what if” analysis using simulation. Simulation is also used for parameter-tuning, problem diagnosis and performance optimizations. It is immensely important in research on large, complex and heterogeneous systems, like the Internet, where analytical models and laboratory testbeds do not capture the detail or the sheer volume of the system. One of these systems is the BGP infrastructure of the Internet.
BGP is the de-facto inter-domain routing protocol in the Internet. It is the “glue” that interconnects more than 16,000 Autonomous Systems (AS) of diverse geopolitical nature. In contrast to Interior Gateway Protocols (IGP), BGP is a policy-based protocol. Business relationships and agreements between ASs determine how packets are routed.
During the last few years the inter-domain routing infrastructure has attracted substantial research interest. Some of the widely researched problems are its slow convergence [1], [2], [3], [4], [5], policy conflicts [6], [7], instability [8], [9], misconfigurations [10], lack of security [11], table growth [12], [13] and path inflation [14], [15], [16]. These problems necessitate fixing or even replacing the current routing architecture. Several examples of proposed new routing architectures can be found in the following references [17], [18], [19], [20], [21], [22], [23]. Yet, it is not clear how BGP should be modified.
The main reason the future of the routing architecture is dubious, is the lack of necessary tools to comprehensively understand the current infrastructure and to evaluate new alternatives. Measuring the performance of BGP is strongly prohibited by ownership and lack of measurement infrastructure. Analytical models of BGP are simplistic and do not capture the complexity, configurability and heterogeneity of the protocol. BGP simulators can only perform moderate-scale simulations of a few hundreds of routers, from which we cannot draw reliable conclusions.
In this work we are concerned with building a simulation tool that will help researchers shed light on the design flaws of the current BGP infrastructure and evaluate the performance of new architectures. BGP++ is designed along the scalability-realism diptych, enabling for the first time large-scale as well as detailed BGP simulations. BGP++ is not a new bottom-up implementation of BGP, but it capitalizes on and benefits from existing high quality software on network simulation, routing and parallel-distributed simulation. The following software are the basic components on which we build BGP++:
- 1.
ns-2 [24] is a discrete-event network simulator that serves as a common platform on which researchers can test and compare their proposals. ns-2 has evolved into the most widely used simulator in networking literature. It includes numerous implementations of protocols with a special emphasis on TCP variants.
- 2.
GNU Zebra [25] and its ancestor, Quagga [26],1 is a detailed open-source implementation of BGP. It is used by a large community of providers, vendors and researchers for testing and experimenting with the protocol. It is also used for routing by small ASs that cannot afford buying expensive routers.
- 3.
PDNS [27] is the parallel-distributed version of the ns-2 simulator. It enables large-scale simulations by distributing the simulation model on multiple workstations, thereby granting more physical resources. In contrast to its counterpart, SSFnet [28], PDNS supports both shared-memory multiprocessors and distributed-memory clusters of workstations. Support of distributed architectures offers more physical resources, overwhelming the limits inherent in parallel architectures. Remarkably, PDNS was recently used by Fujimoto et al. [29] to realize the largest network simulations ever, of more than 5 million network nodes.
Our contributions can be summarized as follows:
- 1.
We develop and make publicly available [30] a packet-level BGP simulator on the widely used ns-2 simulation platform.
- 2.
We integrate the BGP implementation of Zebra into ns-2, making the minimum possible changes to the original software. We realize an accurate BGP simulator that supports most of the details of Zebra’s BGP implementation, including a CISCO-like configuration language.
- 3.
We use and extend PDNS to support parallel-distributed BGP simulations. We also evaluate the performance of alternative model partitioning algorithms for parallel-distributed simulations.
- 4.
We identify the representation of BGP routing tables as the main source of memory consumption in BGP simulations. We introduce a compact routing table data structure that exploits the redundancy of information in BGP routing tables and realizes significant memory savings.
- 5.
We propose and develop a simple generic technique to speedup simulation trials using process checkpointing.
- 6.
We survey recent advancements in measuring the Internet. We highlight measurement data and related models that should be explored to enhance the fidelity of BGP simulations.
- 7.
We develop a seamless partitioning and configuration engine for parallel-distributed BGP simulations that hides the complexity of PDNS configuration and brings parallel-distributed simulation closer to the general ns-2 user. We also develop an automatic generator of CISCO-like configuration. We combine these tools in a toolset that expedites common BGP simulation and configuration practices.
The remainder of this paper is organized as follows. Section 2 elaborates on previous efforts on BGP simulation. Section 3 describes in detail the development of BGP++. Section 4 introduces three techniques that we use to materialize large-scale BGP simulations. Section 5 surveys the developments in the field of Internet measurements with respect to BGP. Section 6 introduces a toolset to expedite configuration, partitioning and scheduling of BGP simulations. Finally, Section 7 talks about future directions of our research.
Section snippets
Related work
The last few years there has been a considerable work on BGP simulation research. The BGP model by Premore in the SSFnet [28] simulator was the first detailed simulation model of BGP and is currently the most widely used BGP simulator. It supports most important BGP features and extensions. The main limitation of SSFnet is that it exhibits considerable memory demand, thereby preventing simulations larger than a few hundred of BGP routers.
Independently and in parallel with BGP++ a number of
BGP++ development
The process of modeling a system is subject to the abstraction-scalability trade-off. Higher abstraction results in more efficient models, which therefore are more scalable. On the other hand, detail is required to thoroughly capture the characteristics of the system. To create detailed simulation models we chose to incorporate the Zebra open-source BGP implementation into the ns-2 simulation environment. The main advantage of our approach is that the simulator inherits the detail,
Scaling BGP++
The need for large-scale BGP simulations stems from the scale of the BGP infrastructure. More than 16,000 ASs use BGP, including a number of global providers that administrate thousands of internal routers. Also, the advent of the Internet measurements research over the last few years has enhanced the information known about BGP, making it possible to construct more detailed and realistic simulations. However, this comes at the cost of more simulation resources, underlining the need for
How to simulate BGP
Development of realistic simulation models is a challenging endeavor. The reliability of conclusions drawn from simulation analysis depends highly on the accuracy and the correctness of the simulation models. The accuracy of these models can be enhanced by exploring Internet measurements. In this section we survey the data and models available from Internet measurement studies that can be used to construct realistic BGP simulations.
A BGP simulation may include one or more of the following:
- •
BGP++ configuration toolset
We develop a toolset to expedite typical simulation configuration tasks. The simulation toolset can read simple user input files, generate CISCO-like configuration files, partition parallel-distributed simulations, generate configuration for PDNS and schedule multiple simulation runs. Fig. 6 illustrates a block diagram of the software architecture of the toolset. There are three main components: a configuration generator, a partitioner and a scheduler.
The configuration component takes as input
Conclusion and future work
In this work we develop BGP++, a scalable and detailed BGP simulator that is built on top of high quality software in network simulation (ns-2), routing (Zebra) and parallel-distributed simulation (PDNS).
BGP research has ample fertile ground for simulation-based analysis. BGP++ opens new directions in using simulation to improve our understanding of the Internet inter-domain infrastructure.
Evaluating the future performance of the inter-domain infrastructure is a problem in which simulation is
Acknowledgement
This work is supported in part by NSF under contract numbers ANI-9977544 and ANI-0136969; and in part by DARPA under contract number N66002-00-1-8934.
Xenofontas A. Dimitropoulos is a Fulbright scholar, working towards his Ph.D. in the Department of Electrical and Computer Engineering at the Georgia Institute of Technology. He received his Bachelor’s in Physics from the Aristotle University of Thessaloniki. His research interests include Internet measurements, modeling of communication systems and large-scale simulation techniques.
References (72)
- et al.
Persistent route oscillations in inter-domain routing
Computer Networks
(2000) - et al.
Towards capturing representative AS-level Internet topologies
Computer Networks Journal
(2004) - C. Labovitz, A. Ahuja, A. Bose, F. Jahanian, Delayed Internet routing convergence, in: Proceedings of ACM SIGCOMM,...
- C. Labovitz, A. Ahuja, R. Wattenhofer, S. Venkatachary, The impact of Internet policy and topology on delayed routing...
- et al.
Improving BGP convergence through consistency assertions
- A. Bremler-Barr, Y. Afek, S. Schwarz, Improved BGP convergence via Ghost Flushing (2003). Available from:...
- D. Pei, M. Azuma, N. Nguyen, J. Chen, D. Massey, L. Zhang, BGP-RCN: Improving BGP Convergence Through Root Cause...
- T. Griffin, G. Willfong, A safe path vector protocol, in: Proceedings of IEEE INFOCOMM,...
- T.G. Griffin, A.D. Jaggard, V. Ramachandran, Design principles of policy languages for path vector protocols, in:...
- R. Govindan, A. Reddy, An analysis of inter-domain topology and route stability, in: Proceedings of INFOCOM,...
Secure border gateway protocol (Secure-BGP)
IEEE Journal on Selected Areas in Communications
An Internet scale simulation setup for BGP
Computer Communication Review
Cited by (20)
Measurement of large-scale BGP events: Definition, detection, and analysis
2016, Computer NetworksCitation Excerpt :Simulation is important for studying giant and complex systems such as the BGP system. To integrate our method with BGP simulation tools (e.g., [26]), one needs to set up a special BGP router that peers with other BGP routers for the purpose of gathering updates. The major advantage of working with simulation tools, according to our understanding, is that each BGP router in a simulation tool could work as a monitor.
Modeling a policy-capable path-vector routing protocol using Jacobi iteration over a path algebra
2011, Computer NetworksCitation Excerpt :Regarding distributed simulation, it is possible to simulate larger network sizes more quickly by distributing the network experiment across multiple computing platforms [19]. However, except for some treatment in [20,21], distributed simulation has not been widely used for performing BGP simulations. This is possibly due to the non-existent or unsupported distributed BGP simulation capabilities within popular public-domain discrete-event network simulation tools (e.g., SSFNET [15], ns2 [6]).
Expected convergence properties of BGP
2011, Computer NetworksCitation Excerpt :The work in [22] develops testing techniques for BGP (or other path vector routing protocols) that can scale to large topologies. Simulation is an important tool in understanding BGP performance and the BGP++ simulator presented in [23] is intended to be detailed yet scalable to large topologies. Finally, in [24], the underlying graph theoretical problem solved by BGP is generalized to a fractional version for which solutions are always shown to exist.
BGPy: The BGP Python Security Simulator
2023, ACM International Conference Proceeding SeriesCutting Through the Noise to Infer Autonomous System Topology
2022, Proceedings - IEEE INFOCOM
Xenofontas A. Dimitropoulos is a Fulbright scholar, working towards his Ph.D. in the Department of Electrical and Computer Engineering at the Georgia Institute of Technology. He received his Bachelor’s in Physics from the Aristotle University of Thessaloniki. His research interests include Internet measurements, modeling of communication systems and large-scale simulation techniques.
George F. Riley is an Assistant Professor of Electrical and Computer Engineering at the Georgia Institute of Technology. He received his Ph.D. in computer science from the Georgia Institute of Technology, College of Computing, in August 2001. His research interests are large-scale simulation using distributed simulation methods. He is the developer of Parallel/Distributed ns2 (PDNS), and the Georgia Tech Network Simulator (GTNetS).