Large-scale urban traffic simulation with Scala and high-performance computing system

Highlights

• A Scala/Akka-based large scale simulation system is presented.

• The system is applied to the problem of urban traffic simulation.

• The details of implementation are presented.

• Very promising results of scalability up to 1000 cores are shown.

Abstract

High-performance computing systems make it possible to implement large-scale simulations of natural phenomena. However, in order to develop efficacious and efficient solutions, easy-to-use software platforms and applications are required. Up until now, the most popular solutions in this area were based on the message passing interface. Next, the authors successfully tackled this problem using Erlang; now, we focus on Scala/Akka — a popular and widely used standard in parallel and distributed programming. The paper focuses on the scalable implementation of a traffic simulation system in an asynchronous and notably desynchronized way. In addition to describing the concept of the system, a series of experiments on a cluster of up to 1000 nodes (24,000 cores) is presented and discussed, showing the efficiency and scalability. The main aim of the paper is to show that the implementation of high-performance computing-grade software solutions lies at the hand of any programmer proficient in Java Virtual Machine-related technologies.

Introduction

Computer simulation of natural phenomena is a widely used method that helps us understand and predict changes in a system's state over time. With the growing computational power of modern hardware, one can expect a corresponding growth in the performance and accuracy of such simulations. However, the astonishing computational power promised by contemporary high-performance computing platforms (HPC) is not straightforward due to their hardware architecture. The power is provided by the multiplication of relatively slow nodes, and the platforms are composed of thousands of independent processors. This fact creates the need for massively parallel versions of algorithms. In the case of data-intensive computations (like the simulation of natural phenomena), creating an efficient parallel implementation poses a great challenge.

The phenomenon considered in this paper is an urban traffic simulation, which has been the subject of active research since the 1950s [11]. Over the years, three different levels of details of the traffic model have been proposed: macro-, meso-, and microscopic [35]. The microscopic models, which represent the individual features of different cars and drivers, are currently dynamically researched and used in road engineering. However, the growing complexity of models, the need for simulating large environments (like whole cities) and the desire to get results immediately, make the simulation problem a complex computational challenge. One of the possible solution is to use a cluster of computers.

Despite the many attempts to create parallel traffic simulation algorithms, the problem of executing a microscopic traffic simulation on HPC hardware is still open. A parallel version of a well-recognized Nagel–Schreckenberg theoretical traffic model (actually based on a specifically configured cellular automaton [22]) was proposed back in 1994 [21], which proved this possibility. More recent research [26], [19], [25], [18] has shown various scalability problems caused by synchronization overheads when the common model is updated by several computing nodes simultaneously.

Most commonly used traffic simulators, both open-source and commercial, rarely offer the possibility of accelerating the simulation by using multiple processors. The well-known SUMO simulator [20] is a single-process application. A distributed version of SUMO, called dSUMO has been presented in [5], however the authors utilize only 4 cores on 2 PCs. Another mature agent-based traffic simulation, MATSim [16], supports the parallel computing inside a single computing node. TRANSIMS open-source project, supported by Kai Nagel, has been used to create a distributed simulation [27], which improved performance, but also did not scale well. Commercial simulators, like EMME/2 [2] or PTV VISSIM [3] do not advertise the ability of using HPC infrastructures. AnyLogic, a very mature, commercial general-purpose simulation framework [1], supports execution in Cloud according to the claims by the supporting company, but no technical details are provided. There are also prototypes of simulation frameworks to be found in the literature (see, e.g. [31], [29]), however to our best knowledge they do not reach HPC-grade scalability, although the latter cited paper claims to offer a very efficient hybrid (CPU/GPGPU) simulation framework.

Having the HPC infrastructure at hand cannot be treated as attaining the Holy Grail of computing, as HPC-grade appliances are not so straightforward to implement, debug, nor use. This motivates research on new and alternative programming models, languages, and runtime systems. Starting from universal MPI-based systems [10], the technology evolved; recently, very efficient ways of leveraging HPC based on the utilization of functional languages have arisen. Indeed, we have already approached a similar simulation problem using Erlang [34], successfully obtaining and testing a scalable and efficient system. However, the steep learning curve of this technology does not make it a tool of choice for all HPC-oriented developers. Therefore, we also wanted to try to use very popular JVM-based tools (JVM - Java Virtual Machine); particularly, Scala technology, which has become greatly successful in joining the ubiquitous nature of Java with the ease and elegance of the functional paradigm.

In this paper, we present a highly scalable urban traffic simulation system implemented in Scala and Akka technology. Nowadays, Scala is a popular general-purpose programming language with support for functional programming and strong typing [24]. Akka is a toolkit that simplifies the construction of concurrent and distributed applications on the Java Virtual Machine (particularly well-known from the support of actor-based concurrency) with inspiration drawn from Erlang [12], [32].

These technologies have been designed to create massively parallel systems. They provide several mechanisms and tools dedicated to handling HPC hardware. The implemented simulation algorithm can be distributed between hundreds of computing nodes, providing almost linear scalability. The main contributions of our work are as follows:

• Implementation of a new distributed car traffic simulation created entirely in Scala and Akka, making it easier to get to know with HPC computing for programmers who are familiar with the JVM environment.

• Presentation of a series of experiments on state-of-the-art HPC infrastructure, showing the good scalability and efficiency results on up to 1000 nodes with 24,000 cores in total, proving the usability of the proposed framework.

The paper is organized as follows. In the next section, the parallel processing in Scala and Akka is discussed; the actual model of traffic in the parallel context is then described. The following section shows the implementation details of the simulation system using the previously mentioned Scala and Akka technologies. The scalability of the presented solution is tested and discussed in detail in Section 5. Finally, the paper is concluded, suggesting further research directions.