Elsevier

Computer Networks

Volume 56, Issue 6, 19 April 2012, Pages 1731-1744
Computer Networks

P2P consistency support for large-scale interactive applications

https://doi.org/10.1016/j.comnet.2012.01.012Get rights and content

Abstract

Peer-to-Peer (P2P) systems have been widely used by networked interactive applications to relieve the drawback and reduce the reliance on well-provisioned servers. A core challenge is to provide consistency maintenance for a massive number of users in a P2P manner. This requires propagating updates on time by only using the uplink bandwidth from individual users instead of relying on dedicated servers. In this paper, we present a P2P system called PPAct to provide consistency maintenance for large-scale fast-interactive applications. We use massive multi-player online games as example applications to illustrate PPAct. The design can be directly applied to other interactive applications. We adopt the Area-of-Interest (AOI) filtering method, which is proposed in prior works [1], [2], to reduce bandwidth consumption of update delivery. We solve the AOI’s critical problem of bandwidth shortage in hot regions by dynamically balancing the workload of each region in a distributed way. We separate the roles of view discovery from consistency maintenance by assigning players as “region hosts” and “object holders.” A region host is responsible for tracking objects and players within a region, and an object holder is responsible for sending updates about an object to interested players. Lookup queries for view discovery are processed by region hosts, while consistency maintenance of objects is taken by object holders. Separating the roles not only alleviates the workload overflow in hot regions, but also speeds up view discovery and update delivery. Another key idea is that peers contribute spare bandwidth in a fully distributed way to forwarding updates about objects of interest. Thus popular, high-demand objects will have more peers forward updates. We also present how to select capable and reliable players for region hosts and object holders.

A P2P network simulator is developed to evaluate PPAct on two major types of online games: role-playing games (RPGs) and first-person shooter (FPS) games. The results demonstrate that PPAct successfully supports 10,000 players in RPGs and 1500 players in FPS games. PPAct outperforms SimMud [2] in RPGs and Donnybrook [3] in FPS games by 40% and 30% higher successful update rates respectively.

Introduction

Modern networked interactive applications, such as online games [3], performance monitoring [4], and collaborative editing [5], [6], have moved from dedicated server designs to P2P paradigms for better scalability and robustness. A core challenge for P2P systems is to provide consistency maintenance for all users by only using resources from these users.

P2P managed massive multi-player online games (MMOGs) are representative applications that have been extensively studied [1], [2], [3]. Some online game platforms, e.g., Microsoft XBox Live [7], have deployed their P2P services for commercial use. P2P systems for managing MMOGs have the following goals: (1) scalability to serve a large number of players; (2) consistency support for high-frequency interactions; (3) compatibility to different types of games; (4) easy configurability by heterogeneous players. Unfortunately, there is no existing P2P system that achieves all these goals. In this paper, we design a P2P system, PPAct, that accomplishes all these goals supporting large-scale fast-interactive applications. MMOG is used as an example to illustrate how these goals are satisfied.

Insufficient uplink bandwidth. Online game applications require that each player should have a correct view of game status. The views of all players should be consistent. Thus, a player needs to get timely updates. In the worst case, a player needs every update from all other players to maintain consistency. The total number of updates sent by players is quadratic to the total number of players. However, the total supply of uplink bandwidth used for update delivery increases only linearly with the number of players. The asymmetric bandwidth allocation is dominated by downlink and thus uplink bandwidth is quite limited for normal players. The mismatch between the supply and demand of uplink bandwidth is a major obstacle to good scalability.

We adopt Area-of-Interest (AOI) filtering [1], [2] to reduce the number of updates sent to each player. The game map is partitioned into regions, and each player receives updates about surrounding regions. These regions are called view regions. Each player subscribes to appropriate view regions in real-time. However, the AOI filtering only alleviates bandwidth shortage but does not solve it fundamentally.

We make additional bandwidth savings by reducing the overhead of view region discovery. We organize all regions into a two-dimensional DHT (2-D DHT) based on their horizontal and vertical positions. As game mobility studies [8] suggest, player movements are mostly continuous. A 2-D DHT provides low-cost region discovery for continuous movements. In addition, the frequency of receiving updates from subscribed regions is tuned temporarily to offset the gap between the supply and demand of bandwidth due to update bursts. Since new updates are sent out periodically, players may ignore obsolete updates in favor of newer ones.

Workload imbalance. The interests of players are usually clustered to a few regions according to the power law distribution of player population density [9]. The players who are responsible for sending updates about the hot regions may not have enough bandwidth to deliver them on time, which is called the AOI “hot spot” problem. A straightforward solution is to adjust region size related to popularity as in the previous works [10], [1]. A popular region is divided into several small regions while an unpopular region is kept large. However, varying region size complicates region partitioning in P2P systems. Players change interests as the game evolves, consequently, locations and sizes of hot areas keep changing. It is hard to correctly partition regions because all players should be synchronized and provided with such frequently changing information. Another solution is migrating peers from unpopular regions to popular regions to share the workload as in the previous work [11]. However, it is difficult to correctly perform migration in dynamic P2P systems. In PPAct, we use a uniform region size so that region partitioning information is easily maintained by every player. In this paper, we use peers and players interchangeably, both mean the users of a P2P system.

We solve the AOI “hot spot” problem by breaking down the workload of each region. We separate the object discovery from the object consistency maintenance by assigning each region to a region-host and each object to an object-holder. A region-host is a peer who tracks objects inside a region and players who want to receive updates about this region. An object-holder is a peer who holds the primary copy of an object and sends out updates about the object. All objects inside a region are registered to the region-host by their object-holders. Thus, players only query the region-host to subscribe to all objects within that region.

We further break down the workload of update delivery by having each player contribute spare bandwidth to subscribed objects. This avoids overloading object holders of popular objects. Since demand for receiving updates of an object is determined by the number of object subscribers, our scheme ensures that the objects with higher demands have more players to contribute. Moreover, bandwidth is contributed in a fully distributed manner as players piggyback bandwidth information onto their region discovery queries without adding extra overhead.

From our observation, the workload of a region host is more stable than that of an object holder. The number of regions assigned to a region host varies by the region host’s capacity. When assigning multiple regions to a peer, we avoid assigning neighboring regions to the same peer so the regions are less likely to have player crowds simultaneously. A region host should also be reliable since it is the contact point for players and object holders. We develop an analytical model for region host selection with both capacity and reliability threshold requirements.

Incompatibility to various games. There are two types of popular online games: action games and role-playing games (RPGs). Each type has its distinctive features and thus different application requirements. An action game requires players to use quick reflexes, accuracy, and timing to overcome obstacles. A popular sub-genre is called first person shooter (FPS). Thus, an action game involves frequent updates and imposes stringent latency bounds for update delivery. Most RPGs cast the player in the role of one or more “adventurers” who specialize in specific skill sets and progress through a predetermined storyline. Compared to FPS games, RPGs have lower update rates and tolerate to longer update latency, but support a larger number of players. RPGs have updates from a great number of Non-Player Characters (NPC) and player avatars, unlike FPS games, which only have updates from player avatars [12]. Existing techniques may address either game type but overlook adaptability. For instance, Donnybrook [3] works well for FPS games, but only scales up to 900 players and does not handle NPCs for RPGs. Another work, SimMud [2], manages NPCs for RPGs, but can rarely meet the latency bound of FPS games.

PPAct is the first P2P system designed to be compatible for different types of games. The separation of region-hosts from object holders not only speeds up update delivery but also provides flexible consistency maintenance for both avatars and NPCs. With all these fully distributed techniques, PPAct satisfies the stringent update latency bound of FPS games and provides good scalability for RPGs. Our extensive simulations show that PPAct successfully supports 10,000 players in RPGs with successful update rate 40% higher than that of SimMud [2], and 1500 players in FPS games with successful update rate 30% higher than that of Donnybrook [3]. The paper makes the following contributions while designing the new PPAct system for maintaining consistency in a large interactive application.

  • (1)

    We decouple the roles of view discovery and consistency maintenance to speed up lookup and update delivery for the stringent latency requirements of FPS games, and to flexibly support object consistency maintenance for RPGs.

  • (2)

    We propose a scheme that balances the workload of update delivery dynamically by having each player contribute its spare bandwidth to its subscribed objects. This solves the AOI’s problems of bandwidth shortage in hot regions.

  • (3)

    We present schemes for selecting region hosts and object holders with capability and reliability requirements.

  • (4)

    We develop a simulator to extensively evaluate the performance of PPAct on two major types of online games. The performance of our work is compared with two recent P2P game systems: Donnybrook [3] on FPS games and SimMud [2] on RPGs. The major design factors are also verified through simulation results.

The rest of the paper is organized as follows. Section 2 gives the background on consistency maintenance of online games. Section 3 and Section 4 present PPAct techniques for fulfilling our design goals. Section 5 shows simulation results. Section 6 reviews prior work, and Section 7 concludes the paper.

Section snippets

Background

In most MMOGs, each player plays a role in a virtual world. Each player is represented in the game by an avatar. A game includes both immutable and mutable objects. Immutable objects (e.g., landscape) are downloaded by game client software. Mutable objects are updated by either players (e.g., avatars, food, tools) or automated algorithms (e.g., NPCs). Every mutable object is represented by its state. For example, an avatar’s state may include its position, health, possessions; a tool’s state

Rendezvous enabled range query processing and subscription

In this section, we first give an overview of how a player obtains its view in PPAct. Then, we present the region partitioning scheme, and the update forwarding scheme. Finally, we present how to map regions to region hosts and objects to object holders.

Region-host organization and selection

In this section, we introduce how to organize regions into a 2-D DHT and how to route lookup queries. We then analytically model the region host selection with both reliability and capability threshold requirements.

Experimental methodology

We developed a simulator to evaluate the efficiency of PPAct in FPS games and RPGs.

Game settings. Two game workload generators are developed: one for FPS games and the other for RPGs. Game traffic characteristics are based on trace data of Counter Strike [24] for FPS games and ShenZhou Online games [12] for RPGs, as summarized in Table 2. The actions of the players and their movements in FPS games are based on the networked game mobility model [8], which simulates the real FPS player behaviors.

P2P managed online game systems

In the literature, two major techniques are proposed to reduce bandwidth consumption of update delivery. One is Area-of-Interest (AOI) filtering [1], [2], where updates about an object is only sent to the players within the same region as the object. This method works well when players have limited proximity to each other, but does not help when players are clustered in a few regions, which is unavoidable due to the power law distribution of the player population density [9]. As a result,

Conclusion

In this paper, we proposed a P2P system called PPAct to provide consistency maintenance for large-scale fast-interactive applications. We presented the design and evaluation of PPAct by taking massive multi-player online games as example applications. Our work focused on three major problems of consistency maintenance: insufficient uplink bandwidth for the update delivery, workload imbalance among players, and incompatibility for various types of games.

We adopted the AOI filtering method to

Yi Hu received her B.Eng. degree in Computer Science from City University of Hong Kong in 2006. Currently, she is a PhD student, advised by Dr. Laxmi N. Bhuyan, in Computer Science and Engineering Department at University of California, Riverside.

References (39)

  • A. Bharambe, J. Pang, S. Seshan, Colyseus: a distributed architecture for online multiplayer games, in: NSDI,...
  • B. Knutsson, W.X.H. Lu, B. Hopkins, Peer-to-peer support for massively multiplayer games, in: IEEE INFOCOM,...
  • A. Bharambe, J.R. Douceur, J.R. Lorch, T. Moscibroda, J. Pang, S. Seshan, X. Zhuang, Donnybrook: enabling large-scale,...
  • S. Agarwal, J.R. Lorch, Matchmaking for online games and other latency-sensitive P2P systems, in: ACM SIGCOMM,...
  • G. Oster, P. Urso, P. Molli, A. Imine, Data consistency for P2P collaborative editing, in: CSCW,...
  • D. Li, R. Li, Ensuring content and intention consistency in real-time group editors, in: IEEE ICDCS,...
  • XboxLIVE, 2008....
  • S.A. Tan, W. Lau, A. Loh, Networked game mobility model for fist-person-shooter games, in: NetGames,...
  • D. Pittman, C.G. Dickey, A measurement study of virtual populations in massively multiplayer online games, in:...
  • M. Varvello, C. Diot, E. Biersack, P2P second life: experimental validation using Kad, in: IEEE INFOCOM,...
  • A. Bharambe, M. Agrawal, S. Seshan, Mercury: supporting scalable multi-attribute range queries, in: ACM SIGCOMM,...
  • K.T. Chen et al.

    Game traffic analysis: an mmorpg perspective

    Computer Networks

    (2006)
  • T. Beigbeder, R. Coughlan, C. Lusher, J. Plunkett, E. Agu, M. Claypool, The effects of loss and latency on user...
  • D. Stutzbach, R. Rejaie, Understanding churn in peer-to-peer networks, in: ACM IMC,...
  • J. Falkner, M. Piatek, J.P. John, A. Krishnamurthy, T. Anderson, Profiling a million user DHT, in: ACM IMC,...
  • A. Rowstron, P. Druschel, Pastry: scalable, distributed object location and routing for large-scale peer-to-peer...
  • A. Rowstron, P. Druschel, Storage management and caching in past, a large-scale, persistent peer-to-peer storage...
  • M. Castro et al.

    Scribe: a large scale and decentralized application level multicast infrastructure

    IEEE J-SAC

    (2002)
  • J. Mickens, B. Noble, Exploiting availability prediction in distributed systems, in: NSDI,...
  • Cited by (9)

    • BMMI-tree: A Peer-to-Peer m-ary tree using 1-m node splitting for an efficient multidimensional complex query search

      2019, Journal of Parallel and Distributed Computing
      Citation Excerpt :

      Higher scalability, dynamicity, self-organizing capabilities, fault-tolerance, symmetric communication i.e., each peer/node being identical and independent, multi-purpose resource sharing i.e., each peer acting as a server, client and router and locality of data are the factors that make P2P systems essential for numerous applications [1,3,23,28,47]. The P2P is now extended to the applications like intelligent resource discovery [31], streaming media and telephony [29], mobile P2P networks [32], multi player interactive gaming [13], video content managing [24], P2P in cloud computing for service sharing [26,49] etc. The researchers recently have focused on P2P overlay networks (logical networks built on the top of the existing network, generally work at application layer) supporting Multidimensional Indexing (MI) to deal with the multidimensional queries and applications efficiently and effectively [19,47].

    • A heuristic data update mechanism in unstructured mobile P2P systems

      2017, Ad Hoc Networks
      Citation Excerpt :

      In the past decades, traditional peer-to-peer (P2P) networks, in which each peer can directly communicate and share resources with other peers have rapidly evolved and received substantial research and commerce attention [1–4].

    • Next level peer-to-peer overlay networks under high churns: a survey

      2020, Peer-to-Peer Networking and Applications
    • Improving multidimensional point query search using multi-way peer-to-peer tree network

      2019, International Journal of Information and Communication Technology
    View all citing articles on Scopus

    Yi Hu received her B.Eng. degree in Computer Science from City University of Hong Kong in 2006. Currently, she is a PhD student, advised by Dr. Laxmi N. Bhuyan, in Computer Science and Engineering Department at University of California, Riverside.

    Laxmi N. Bhuyan is Distinguished Professor and Chairman of Computer Science and Engineering Department at the University of California, Riverside (UCR). Prior to joining UCR in January 2001, he was a professor of Computer Science at Texas A&M University (1989–2000) and Program Director of the Computer System Architecture Program at the National Science Foundation (1998–2000). He has also worked as a consultant to Intel and HP Labs.

    Min Feng received his B.Eng. degree in Computer Science and Technology from Tongji University, Shanghai in 2005, and his M.Phil. degree in Computer Science from City University of Hong Kong in 2007. Currently, he is a PhD student, advised by Dr. Rajiv Gupta, in Computer Science and Engineering Department at University of California, Riverside.

    View full text