Eric Mikida
ABSTRACT
Parallel Discrete Event Simulations (PDES) running at large scales involve the coordination of billions of very fine grain events distributed across a large number of processes. At such large scales optimistic synchronization protocols, such as TimeWarp, allow for a high degree of parallelism between processes, but with the additional complexity of managing event rollback and cancellation. This can become especially problematic in models that exhibit imbalance resulting in low event efficiency, which increases the total amount of work required to run a simulation to completion. Managing this complexity becomes key to achieving a high degree of performance across a wide range of models. In this paper, we address this issue by analyzing the relationship between synchronization cost and event efficiency. We first look at how these two characteristics are coupled via the computation of Global Virtual Time (GVT). We then introduce dynamic load balancing, and show how, when combined with low overhead GVT computation, we can achieve higher efficiency with less synchronization cost. In doing so, we achieve up to 2x better performance on a variety of benchmarks and models of practical importance.
- The charm+ parallel programming system manual. http://charm.cs.illinois.edu/manuals/html/charm+/manual.html.Google Scholar
- Ross source code on github. https://github.com/carothersc/ROSS, visited 2016-03--20.Google Scholar
- B. Acun, A. Gupta, N. Jain, A. Langer, H. Menon, E. Mikida, X. Ni, M. Robson, Y. Sun, E. Totoni, L. Wesolowski, and L. Kale. Parallel Programming with Migratable Objects: Charm+ in Practice. SC, 2014. Google ScholarDigital Library
- P. D. Barnes, Jr., C. D. Carothers, and D. R. e. a. Jefferson. Warp speed: Executing time warp on 1,966,080 cores. In Conference on Principles of Advanced Discrete Simulation, SIGSIM-PADS, pages 327--336, New York, NY, USA, 2013. Google ScholarDigital Library
- D. Bauer, G. Yaun, C. D. Carothers, M. Yuksel, and S. Kalyanaraman. Seven-o'clock: A new distributed gvt algorithm using network atomic operations. In Proceedings of the 19th Workshop on Principles of Advanced and Distributed Simulation, PADS '05, pages 39--48, Washington, DC, USA, 2005. IEEE Computer Society. Google ScholarDigital Library
- D. W. Bauer Jr., C. D. Carothers, and A. Holder. Scalable time warp on blue gene supercomputers. In Proceedings of the 2009 ACM/IEEE/SCS 23rd Workshop on Principles of Advanced and Distributed Simulation, PADS '09, pages 35--44, Washington, DC, USA, 2009. IEEE Computer Society. Google ScholarDigital Library
- C. D. Carothers, D. Bauer, and S. Pearce. ROSS: A high-performance, low-memory, modular Time Warp system. Journal of Parallel and Distributed Computing, 62(11):1648--1669, 2002.Google ScholarDigital Library
- C. D. Carothers and R. M. Fujimoto. Efficient execution of time warp programs on heterogeneous, now platforms. IEEE Trans. Parallel Distrib. Syst., 11(3):299--317, Mar. 2000. Google ScholarDigital Library
- G. G. Chen, Boleslaw, and K. Szymanski. Time quantum gvt: A scalable computation of the global virtual time in parallel discrete event simulations.Google Scholar
- N. Choudhury, Y. Mehta, T. L. Wilmarth, E. J. Bohm, and L. V. . Kalé. Scaling an optimistic parallel simulation of large-scale interconnection networks. In Proceedings of the Winter Simulation Conference, 2005. Google ScholarDigital Library
- E. Deelman and B. K. Szymanski. Dynamic load balancing in parallel discrete event simulation for spatially explicit problems. In Parallel and Distributed Simulation, 1998. PADS 98. Proceedings. Twelfth Workshop on, pages 46--53, May 1998. Google ScholarDigital Library
- R. M. Fujimoto. Performance of time warp under synthetic workloads. Distributed Simulation Conference, 1990.Google Scholar
- R. M. Fujimoto and M. Hybinette. Computing global virtual time in shared-memory multiprocessors. ACM Trans. Model. Comput. Simul., 7(4):425--446, Oct. 1997. Google ScholarDigital Library
- D. W. Glazer and C. Tropper. On process migration and load balancing in time warp. IEEE Transactions on Parallel and Distributed Systems, 4(3):318--327, Mar 1993. Google ScholarDigital Library
- Z. X. F. Gomes, B. Unger, and J. Cleary. A fast asynchronous gvt algorithm for shared memory multiprocessor architectures. SIGSIM Simul. Dig., 25(1):203--208, July 1995. Google ScholarDigital Library
- E. J. Gonsiorowski, J. M. LaPre, and C. D. Carothers. Improving accuracy and performance through automatic model generation for gate-level circuit pdes with reverse computation. In Proceedings of the 3rd ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, SIGSIM PADS '15, pages 87--96, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- D. Jefferson and H. Sowizral. Fast Concurrent Simulation Using the Time Warp Mechanism. In Proceedings of the Conference on Distributed Simulation, pages 63--69, July 1985.Google Scholar
- F. Mattern. Efficient algorithms for distributed snapshopts and global virtual time approximation. Journal of Parallel and Distributed Computing, 18:423--434, 1993. Google ScholarDigital Library
- H. Menon and L. Kalé. A distributed dynamic load balancer for iterative applications. In Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis, SC '13, pages 15:1--15:11, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- S. Meraji, W. Zhang, and C. Tropper. On the scalability and dynamic load-balancing of optimistic gate level simulation. Trans. Comp.-Aided Des. Integ. Cir. Sys., 29(9):1368--1380, Sept. 2010. Google ScholarDigital Library
- E. Mikida, N. Jain, E. Gonsiorowski, P. D. Barnes, Jr., D. Jefferson, C. Carothers, and L. V. Kale. Towards pdes in a message-driven paradigm: A preliminary case study using charm+. In Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, SIGSIM PADS '16. ACM, May 2016. Google ScholarDigital Library
- M. Mubarak, C. D. Carothers, R. Ross, and P. Carns. Modeling a million-node dragonfly network using massively parallel discrete-event simulation. In High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:, pages 366--376, Nov 2012. Google ScholarDigital Library
- M. Mubarak, C. D. Carothers, R. B. Ross, and P. Carns. Using massively parallel simulation for mpi collective communication modeling in extreme-scale networks. In Proceedings of the 2014 Winter Simulation Conference, WSC '14, pages 3107--3118, Piscataway, NJ, USA, 2014. IEEE Press. Google ScholarDigital Library
- K. S. Perumalla, A. J. Park, and V. Tipparaju. Gvt algorithms and discrete event dynamics on 129k+ processor cores. In High Performance Computing (HiPC), 2011 18th International Conference on, pages 1--11, Dec 2011. Google ScholarDigital Library
- K. S. Perumalla, A. J. Park, and V. Tipparaju. Discrete event execution with one-sided and two-sided gvt algorithms on 216,000 processor cores. ACM Trans. Model. Comput. Simul., 24(3):16:1--16:25, June 2014. Google ScholarDigital Library
- A. B. Sinha, L. V. Kale, and B. Ramkumar. A dynamic and adaptive quiescence detection algorithm. Technical Report 93--11, Parallel Programming Laboratory, Department of Computer Science, University of Illinois, Urbana-Champaign, 1993.Google Scholar
- S. Srinivasan and P. F. Reynolds, Jr. Non-interfering gvt computation via asynchronous global reductions. In Proceedings of the 25th Conference on Winter Simulation, WSC '93, pages 740--749, New York, NY, USA, 1993. ACM. Google ScholarDigital Library
- J. S. Steinman, C. A. Lee, L. F. Wilson, and D. M. Nicol. Global virtual time and distributed synchronization. In Proceedings of the Ninth Workshop on Parallel and Distributed Simulation, PADS '95, pages 139--148, Washington, DC, USA, 1995. IEEE Computer Society. Google ScholarDigital Library
- N. Wolfe, C. D. Carothers, M. Mubarak, R. Ross, and P. Carns. Modeling a million-node slim fly network using parallel discrete-event simulation. In Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, SIGSIM-PADS '16, pages 189--199, New York, NY, USA, 2016. ACM. Google ScholarDigital Library
- Y. Xu, W. Cai, H. Aydt, M. Lees, and D. Zehe. An asynchronous synchronization strategy for parallel large-scale agent-based traffic simulations. In Proceedings of the 3rd ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, SIGSIM PADS '15, pages 259--269, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- S. B. Yoginath and K. S. Perumalla. Optimized hypervisor scheduler for parallel discrete event simulations on virtual machine platforms. In Proceedings of the 6th International ICST Conference on Simulation Tools and Techniques, SimuTools '13, pages 1--9, ICST, Brussels, Belgium, Belgium, 2013. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering). Google ScholarDigital Library
Index Terms
- Adaptive Methods for Irregular Parallel Discrete Event Simulation Workloads
Recommendations
Towards PDES in a Message-Driven Paradigm: A Preliminary Case Study Using Charm++
SIGSIM-PADS '16: Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete SimulationDiscrete event simulations (DES) are central to exploration of "what-if" scenarios in many domains including networks, storage devices, and chip design. Accurate simulation of dynamically varying behavior of large components in these domains requires ...
An Adaptive Non-Blocking GVT Algorithm
SIGSIM-PADS '19: Proceedings of the 2019 ACM SIGSIM Conference on Principles of Advanced Discrete SimulationIn optimistic Parallel Discrete Event Simulations (PDES), the Global Virtual Time (GVT) computation is an important aspect of performance. It must be performed frequently enough to ensure simulation progress and free memory, while still incurring ...
Parallel Discrete-Event Simulation (PDES): a case study in design, development, and performance using SPEEDES
Can parallel simulations efficiently exploit a network of workstations? Why haven't PDES models followed standard modeling methodologies? Will the field of PDES survive, and if so, in what form? Researchers in the PDES field have addressed these ...
Comments