Genesis: A scalable distributed system for large-scale parallel network simulation

Abstract

The complexity and dynamics of the Internet is driving the demand for scalable and efficient network simulation. In this paper, we describe a novel approach to scalability and efficiency of parallel network simulations. This approach is based on partitioning of a network into domains and of the simulation time into intervals. Each domain is simulated independently of and concurrently with the others over the same simulation time interval. At the end of each interval, traffic statistics data, including per flow average packet delays and packet drop rates, are exchanged between domain simulators. The simulators iterate over the same time interval until the exchanged information converges, that is until the certain metric of a difference between the information exchanged in two subsequent iterations is smaller than a prescribed precision. After convergence, all simulators progress to the next time interval. This approach allows the parallelization with infrequent synchronization, and achieves significant simulation speedups.

Large memory size required by simulation software hinders the simulation of large-scale networks. To overcome this problem, our system supports distribution of network information by assigning to each participating simulator only data related to the part of the network that it simulates. Such a solution supports simulations of large-scale networks on machines with modest memory size.

Introduction

In simulating large-scale networks at the packet-level, a major difficulty is the enormous computational power needed to execute all events that packets undergo in a network [5]. Conventional simulation techniques require tight synchronization for each individual event that crosses the processor boundary [1]. The inherent characteristics of network simulations are fine granularity of events (individual packet transitions in a network) and high ratio of events that transfer a packet from a network node assigned to one processor to the network node assigned to the other. These two factors severely limit parallel efficiency of the network simulation executed under the traditional protocols [1].

Another difficulty is the size of memory required by large-scale network simulations. With the current trend of simulating ever larger and more complicated networks, the memory size becomes a bottleneck. Typically, the entire network configuration and routing information needs to be accessible to each processor and such a solution requires large total memory during construction of the simulated network. Additionally, the memory needed by each processor increases also with the intensity of traffic flows that dictate the size of the future event list. Although memory requirements can be tampered by the good design and implementation of the simulation software [7], we believe that to simulate truly large networks, the comprehensive, distributed memory approach is needed.

This paper describes our long-term research on developing an architecture that can efficiently simulate very large heterogeneous networks in near real time [17]. Our approach combines simulation and modeling in a single execution. The network parts, called domains, are simulated at the packet-level concurrently with each other. Each domain maintains the model of the network external to itself which is built by using periodically output from other domain simulations. If this model is faithfully representing the network, the domain simulation will exactly represent the behavior of its domain, so its output will ensure the correctness of models maintained by the other simulations. Each domain simulation repeats its execution, each time adjusting its model of the rest of the network and feeding the other simulations with increasingly precise model of its own domain. As a result, all domain simulations collectively converge to the consistent state of all domains and all models.

Thanks to the coarse granularity synchronization mechanism employed, our system is able to use different simulators in a single coherent network simulation. Hence we called it General Network Simulation Integration System, or Genesis in short.

Genesis addresses also the large memory requirement problem in large-scale network simulations. Many parallel simulation systems achieved speed-up in simulation time, however, they also required that every machine involved had big enough memory to hold the full network information. This requirement is most easily achieved through a system with shared memory. In Genesis, in contrast, memory usage is fully distributed. Each Genesis domain simulator stores only a part of the network, together with some additional information which is required to cooperate with other simulators. In such a way, large networks can be simulated by clusters of machines with smaller dedicated memory on each of them.

Our approach underlying Genesis can also be seen as a variant of a general scheme for optimistic simulation referred to as Time-Space Mappings proposed by Chandy and Sherman in [2]. Although all optimistic simulations can be viewed as variants of this scheme, very few apply, as we do, iterations over the same time interval to find a solution.