Synchronization in federation community networks
Introduction
Distributed simulation technology facilitates the construction of a large scale simulation with simulation components of various types, which can be developed independently and distributed geographically. The High Level Architecture (HLA) defines the rules, interface specification and object model template to support reusability and interoperability amongst the simulation components, known as federates [12]. While the HLA serves as the de facto standard for distributed simulations, the Runtime Infrastructure (RTI) software provides services1to support and synchronize the interactions amongst different federates conforming to the HLA standard to sustain an overall simulation application, known as a federation as shown in Fig. 1(A).
In the case where the problem domain is particularly complex or involves multiple collaborative parties, the analysts often need to construct large scale HLA-based simulations which may involve a large number of federates and vast computing resources over a network or the Internet. Some typical examples are: military commission rehearsal, Internet gaming, biology simulation, and supply chain simulation. Sometimes such large scale simulations need to be constructed upon multiple simulation federations. Despite the tremendous advantages brought by the HLA technologies, the HLA standard does not explicitly sustain interoperation between federates across the boundaries of federations. To address this issue, a method has been proposed to harness a network of federations to achieve a common goal in the form of a “federation community” [24]. Fig. 1(B) illustrates an example of federation community network. In addition to the advantages of using flat federations,2 simulation developers and users can benefit from the federation community method in (1) improving the scalability of large and complex applications by reducing the bandwidth requirements through the partitioning of network load and the filtering out of irrelevant data amongst federations, (2) enhancing composability of simulation by enabling simulation development using legacy federations, (3) supporting interoperation between heterogeneous federations and RTIs, and (4) facilitating information security [2], [3], [4], [6], [23], [24]. The applications of a federation community have been extensively discussed in the existing literature, including the architecture for information hiding adopted in secured supply chain simulations developed by our team [4].
The emergence of Grid computing technologies meets the requirement of large amounts of distributed computational and data resources by the increasing size and complexity of simulation applications. The Grid provides a flexible, secure and coordinated resource sharing environment which can facilitate distributed simulation execution [27]. In the last few years, there has been an increasing interest in taking advantage of the Grid technologies to execute HLA simulations over the Internet. A few research groups have successfully enabled the Grid technologies to RTI, which either use middleware approaches to encapsulate vendor-provided RTIs [8], [27], [30], [31], [32] or implement the RTI directly using Grid services [14], [26]. While the existing Grid-enabled simulation techniques have proved highly advantageous, users can even benefit more if a whole federation is accessible through Grid services. Once a Grid-enabled federation community is constructed, the computational resources within administrative domains can be further exploited with the reusability3 of legacy federates maximized when enabling the Grid technology.
Simulation federates in a federation community should be provided with RTI services as in a flat federation [18] so that they can interact with each other and the RTIs involved, irrespective of whether these federates operate in the same federation or not. Support to the RTI services related to Federation Management, Declaration Management, Object Management (OM) and Data Distribution Management (DDM) has been well addressed to facilitate the execution of federation communities [2], [3], [21], [22]. Nevertheless, a more important and challenging issue, namely Time Management (TM) over federation community networks, has received little attention. Time management is concerned with the mechanisms for controlling the advancement of each federate along the federation time axis and synchronizing event (the terms “event” and “message” are used interchangeably in the rest of the paper) delivery amongst federates. Without a properly designed synchronization mechanism, the overall simulation execution upon a federation community is susceptible to state inconsistencies, e.g., the federates in different federations receiving messages for the same set of events in different orders. However, the current IEEE 1516 HLA standard is not intended to foster time management across the boundaries of federations [18]. There only exist a few preliminary or non-standard methods to address this issue. Hence, there exists a pressing need for a generic synchronization mechanism for HLA-based federation communities.
This paper proposes a generic approach to synchronizing federates and events within federation community networks. The proposed synchronization algorithms have been developed based on the gateway federate approach for constructing federation communities. The algorithms operate inside the gateway federates, and the time constraints from any federation are propagated through the whole network to coordinate the progresses of the whole federation community along the simulation time axis. The approach has also been successfully applied onto Grid-enabled federation communities.
A shorter version of this paper was presented in the 1st IEEE International Workshop on Advances of CyberInfrastructure [5] in the context of a Grid-enabled infrastructure for simulation of large and complex scenarios. The rest of this paper is organized as follows: Section 2 gives an overview of federation communities and related issues. Section 3 discusses existing work and analyses the challenges to be addressed. Section 4 details the algorithms for synchronizing Timestamp Order events crossing federation boundaries and proves the algorithms’ correctness. Section 5 presents the benchmarking experiments, which examine the correctness of the synchronization mechanism and evaluate its performance. In Section 6, we conclude with a summary and proposals on future work.
Section snippets
HLA federation communities
The technology for constructing federation communities has been well studied. Federation communities may be constructed for different objectives (see Section 1) and various architectures. Grid computing technologies can also be combined with federation community approaches to further benefit simulation users.
Time management in federation communities
In the context of HLA-based simulations, a federate can be both time regulating and time constrained, either of them, or neither. Time regulating federates may send time stamp ordered (TSO) events while time constrained federates are able to receive TSO events in time order. Each regulating federate must specify a “lookahead” value, and ensure that it will not generate any TSO event earlier than its current time plus lookahead [18]. After requesting a time advance from the RTI and getting
Synchronization algorithms for federation community networks
This section proposes a generic synchronization mechanism that takes advantage of the Gateway approach to constructing federation communities. Two novel synchronization algorithms are introduced which are employed by gateway federates, one for layered federations and another for star link federations (see Fig. 3). The basic idea of the proposed mechanism is to let a gateway throttle the time advance of the federations linked together by the gateway. The two proposed algorithms acquire the time
Experiments and results
In order to verify the correctness and investigate the overhead incurred in the proposed synchronization mechanism and its scalability, we carried out a number of experiments to benchmark the efficiency of synchronization in federation communities (multiple-layer federation community, peer-to-peer federation community and Grid-enabled federation community) in both LAN environments and a WAN environment between the UK (Birmingham) and Singapore.
The DMSO RTI NG 1.3 with Java bindings and its
Conclusions and future work
This paper is concerned with issues related to synchronization in federation community networks with alternative approaches for constructing federation communities and various architectures of federation community networks are introduced. The paper presents a mechanism for supporting synchronization crossing the boundaries of interlinked federations. We have investigated the issues and design of two generic synchronization algorithms. Based upon the gateway federate approach, the
Acknowledgment
This research was supported by the National Natural Science Foundation of China under the project grants No. 60804036 and No. 60805045. It is a collaborative work among the University of Birmingham, Nanyang Technological University, and Yanshan University under the Birmingham-Warwick Science City Research Interdisciplinary Alliance.
Dan Chen is a joint HEFCE Fellow with the University of Birmingham and the University of Warwick (United Kingdom). He is also a Professor with the Institute of Electrical Engineering, Yanshan University (China). He was a Postdoctoral Research Fellow with School of Computer Engineering at Nanayang Technological University (NTU, Singapore). He received a B.Sc. in applied physics from Wuhan University (China) and a M.Eng. in computer science from Huazhong University of Science and Technology
References (32)
- et al.
A decoupled federate architecture for high level architecture-based distributed simulation
Journal of Parallel and Distributed Computing
(2008) - et al.
Large scale agent-based simulation on the grid
Future Generation Computer Systems
(2008) - C. Bouwens, D. Hurrell, D. Shen, Implementing ownership management services with a bridge federate, in: Proceedings of...
- W. Cai, G. Li, S.J. Turner, B.S. Lee, L. Liu, Automatic construction of hierarchical federation architectures, in:...
- W. Cai, G. Li, S.J. Turner, B.S. Lee, L. Liu, Implementation of federation management services over federation...
- W. Cai, S.J. Turner, B.P. Gan, Hierarchical federations: An architecture for information hiding, in: Proceedings of the...
- D. Chen, C. Bian, Towards hybrid grid infrastructure for large simulations, in: Proceedings of the 15th IEEE...
- D. Chen, B.S. Lee, W. Cai, S.J. Turner, Design and development of a cluster gateway for cluster-based hla distributed...
- et al.
Algorithms for HLA-based distributed simulation cloning
ACM Transactions on Modeling and Computer Simulation
(2005) - et al.
Towards fault-tolerant HLA-based distributed simulations
Simulation: Transactions of the Society for Modeling and Simulation International
(2008)
Standards for simulation: As simple as possible but not simpler, the high level architecture for simulation
Simulation: Transactions of the Society for Modeling and Simulation International
Time management in the DoD high level architecture
ACM SIGSIM Simulation Digest Archive
Time management in the high level architecture
Simulation: Transactions of the Society for Modeling and Simulation International
Cited by (27)
Adapting HLA-based co-simulation for interdependent infrastructure resilience management
2023, Automation in ConstructionHybrid modelling and simulation of huge crowd over a hierarchical Grid architecture
2013, Future Generation Computer SystemsCitation Excerpt :The synchronization module is responsible for delivering these events in a correct order and coordinating time advances of all simulation models. More details about the synchronization algorithm are available at [27]. We have developed a large simulation system of a huge crowd’s evacuation and its interaction with vehicles in an urban area.
Towards enabling Cyberinfrastructure as a Service in Clouds
2013, Computers and Electrical EngineeringCitation Excerpt :Computing Clouds render users with services to access hardware, software and data resources, thereafter an integrated computing platform as a service. The Cloud computing paradigm distinguishes itself from other computing paradigms, such as Grid computing [9–11], Global computing, Internet Computing in the following aspects [12,13]: Utility computing model: Users obtain and employ computing platforms in computing Clouds as easily as they access a traditional public utility (such as electricity, water, natural gas, or telephone network).
Granular cloning: Intra-object parallelism in ensemble studies
2018, SIGSIM-PADS 2018 - Proceedings of the 2018 ACM SIGSIM Conference on Principles of Advanced Discrete SimulationCloning agent-based simulation
2017, ACM Transactions on Modeling and Computer SimulationLarge-Scale Simulation: Models, Algorithms, and Applications
2017, Large-Scale Simulation: Models, Algorithms, and Applications
Dan Chen is a joint HEFCE Fellow with the University of Birmingham and the University of Warwick (United Kingdom). He is also a Professor with the Institute of Electrical Engineering, Yanshan University (China). He was a Postdoctoral Research Fellow with School of Computer Engineering at Nanayang Technological University (NTU, Singapore). He received a B.Sc. in applied physics from Wuhan University (China) and a M.Eng. in computer science from Huazhong University of Science and Technology (China). After that, he received another M.Eng. and a Ph.D. from NTU in year 2002 and 2006 respectively. His research interests include: computer-based modelling and simulation, high performance computing, and neuroinformatics. Recently, he has been working in dynamics analysis of large crowd and high performance computing in neuroinformatics.
Stephen J. Turner is a Professor and the head of the Computer Science Division, School of Computer Engineering at Nanyang Technological University (NTU, Singapore). He joined NTU in 1999. Previously, he was a Senior Lecturer in Computer Science at Exeter University (UK). He received his MA in Mathematics and Computer Science from Cambridge University (UK) and his M.Sc. and Ph.D. in Computer Science from Manchester University (UK). His current research interests include: parallel and distributed simulation, distributed virtual environments, grid computing and multi-agent systems. He is steering committee chair of the Principles of Advanced and Distributed Simulation conference and advisory committee member of the Distributed Simulation and Real Time Applications symposium. He is also an area editor of ACM Transactions on Modeling and Computer Simulation (TOMACS).
Wentong Cai is a Professor in the Division of Computer Science, School of Computer Engineering at Nanyang Technological University, Singapore. He is also the Director of the Parallel and Distributed Computing Centre. He received his Ph.D. in Computer Science from University of Exeter (UK) in 1991. His current research interests include: Parallel & Distributed Simulation, Multi-agent Systems, and Grid and Cluster Computing. He is an associate editor of the ACM Transactions on Modeling and Computer Simulation (TOMACS), and editorial board member of the Multiagents and Grid Systems—An International Journal.
Georgios K. Theodoropoulos is a Reader in the School of Computer Science at the University of Birmingham (UK) where he has set up and leads the Distributed Systems Lab. He is also a founding Director of the Midlands e-Science Centre. He received his Diploma in Computer Engineering from the University of Patras (Greece) and his M.Sc. and Ph.D. in Computer Science from the University of Manchester (UK). His current research is in parallel and distributed simulation, distributed virtual environments, Grid computing and Peer-to-Peer systems. He is currently Area Editor of the Simulation: Transactions of the Society for Computer Simulation International (SCS) and editorial board member of the International Journal of Simulation and Process Modelling.
Muzhou Xiong is a Ph.D. candidate in Huazhong University of Science and technology, P.R. China. Also he currently is a Research Associate in Nanyang Technological University, Singapore. He obtained his B.S. degree in computer science from Huazhong University of Science and Technology in 2002. His research interests include grid computing, network storage, automatic storage management, and parallel and distributed simulation. His recent research has focused on modeling of human behavior model for high dense crowds.
Michael Lees is currently an Assistant Professor at Nanyang Technological University (Singapore). He completed his Ph.D. at Nottingham University (UK) in 2006 and went on to post-doctoral positions at Nottingham University and Birmingham University. He received his BSc from Edinburgh University (UK) in Artificial Intelligence and Computer Science. His research interests include: Agent-based modeling and simulation, large-scale distributed simulation and high-performance computing.