ABSTRACT
We propose, OMen, a distributed system for dynamically maintaining overlays for topic-based publish/subscribe (pub/sub) systems. In particular, OMen supports churn-resistant construction of topic-connected overlays (TCO), which organizes all nodes interested in the same topic in a directly connected dissemination sub-overlay. While aiming at pub/sub deployments in data centers, OMen internally leverages selected peer-to-peer technologies, such as T-Man as the underlying topology maintenance protocol.
Existing approaches for constructing pub/sub TCOs are (i) centralized algorithms that guarantee low node degrees at the cost of prohibitive running time and (ii) decentralized protocols that are time efficient while lacking bounds on node degrees.
We show both analytically and experimentally that OMen combines the best from both worlds. Namely, OMen achieves (i) low node degrees, close to centralized algorithms, and (ii) high efficiency, scalability, and load balance, comparable to decentralized protocols. Our evaluation uses both synthetic pub/sub workloads and real-world ones extracted from Facebook and Twitter. We generate churn traces with Google cluster data.
- Google Cluster Data. http://code.google.com/p/googleclusterdata.Google Scholar
- IBM Watson IoT Platform. http://internetofthings.ibmcloud.com/.Google Scholar
- R. Baldoni, R. Beraldi, V. Quema, L. Querzoni, and S. Tucci-Piergiovanni. Tera: Topic-based event routing for peer-to-peer architectures. In DEBS '07. Google ScholarDigital Library
- M. Castro, P. Druschel, A.-M. Kermarrec, and A. Rowstron. SCRIBE: A large-scale and decentralized application-level multicast infrastructure. JSAC, 2002. Google ScholarDigital Library
- C. Chen and Y. Tock. Design of routing protocols and overlay topologies for topic-based publish/subscribe on small-world networks. In Middleware Industry '15. Google ScholarDigital Library
- C. Chen, Y. Tock, and H.-A. Jacobsen. Overlay design for topic-based publish/subscribe under node degree constraints. In ICDCS '16.Google Scholar
- C. Chen, R. Vitenberg, and H.-A. Jacobsen. Scaling construction of low fan-out overlays for topic-based publish/subscribe systems. In ICDCS'11. Google ScholarDigital Library
- G. Chockler, R. Melamed, Y. Tock, and R. Vitenberg. Constructing scalable overlays for pub-sub with many topics: Problems, algorithms, and evaluation. Podc'07. Google ScholarDigital Library
- G. Chockler, R. Melamed, Y. Tock, and R. Vitenberg. Spidercast: A scalable interest-aware overlay for topic-based pub/sub communication. In DEBS'07. Google ScholarDigital Library
- V. Chvatal. A greedy heuristic for the set-covering problem. Mathematics of Operations Research, 1979. Google ScholarDigital Library
- B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H.-A. Jacobsen, N. Puz, D. Weaver, and R. Yerneni. Pnuts: Yahoo!'s hosted data serving platform. Proc. VLDB Endow., 2008. Google ScholarDigital Library
- S. El-Ansary, L. Alima, P. Brand, and S. Haridi. Efficient broadcast in structured p2p networks. In Peer-to-Peer Systems II. 2003.Google Scholar
- S. Girdzijauskas, G. Chockler, Y. Vigfusson, Y. Tock, and R. Melamed. Magnet: practical subscription clustering for internet-scale publish/subscribe. Debs'10. Google ScholarDigital Library
- D. S. Hochbaum. Approximation algorithms for the set covering and vertex cover problems. SIAM Journal on Computing, 1982.Google ScholarCross Ref
- M. Jelasity, R. Guerraoui, A.-M. Kermarrec, and M. van Steen. The peer sampling service: experimental evaluation of unstructured gossip-based implementations. In Middleware '04. Google ScholarDigital Library
- M. Jelasity, A. Montresor, and O. Babaoglu. T-man: Gossip-based fast overlay topology construction. Computer Networks, 2009. Google ScholarDigital Library
- J. Kleinberg. The small-world phenomenon: An algorithmic perspective. In STOC'00. Google ScholarDigital Library
- H. Kwak, C. Lee, H. Park, and S. Moon. What is Twitter, a social network or a news media? WWW'10. Google ScholarDigital Library
- A. Lakshman and P. Malik. Cassandra: a decentralized structured storage system. SIGOPS Operating System Review, 2010. Google ScholarDigital Library
- L. Lamport. The part-time parliament. ACM Trans. Comput. Syst., pages 133--169, 1998. Google ScholarDigital Library
- J. Leitao, J. Pereira, and L. Rodrigues. HyParView: A membership protocol for reliable gossip-based broadcast. In DSN '07. Google ScholarDigital Library
- H. Liu, V. Ramasubramanian, and E. G. Sirer. Client behavior and feed characteristics of RSS, a publish-subscribe system for web micronews. IMC'05. Google ScholarDigital Library
- M. Onus and A. W. Richa. Parameterized maximum and average degree approximation in topic-based publish-subscribe overlay network design. ICDCS'10. Google ScholarDigital Library
- M. Matos, P. Felber, R. Oliveira, J. O. Pereira, and E. Riviere. Scaling up publish/subscribe overlays using interest correlation for link sharing. TPDS, 2013. Google ScholarDigital Library
- A. Montresor and M. Jelasity. PeerSim: A scalable P2P simulator. In P2P'09.Google Scholar
- D. Ongaro and J. Ousterhout. In search of an understandable consensus algorithm. In USENIX ATC'14. Google ScholarDigital Library
- J. A. Patel, E. Rivière, I. Gupta, and A.-M. Kermarrec. Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems. Computer Networks, 53(13):2304--2320, 2009. Google ScholarDigital Library
- F. Rahimian, S. Girdzijauskas, A. H. Payberah, and S. Haridi. Vitis: A gossip-based hybrid overlay for internet-scale publish/subscribe enabling rendezvous routing in unstructured overlay networks. IPDPS'11. Google ScholarDigital Library
- J. Reumann. Pub/Sub at Google. Lecture & Personal Communications at EuroSys & CANOE Summer School, Oslo, Norway, Aug 2009.Google Scholar
- S. Rhea, D. Geels, T. Roscoe, and J. Kubiatowicz. Handling churn in a DHT. In ATEC '04. Google ScholarDigital Library
- V. Setty, M. van Steen, R. Vitenberg, and S. Voulgaris. Poldercast: Fast, robus, and scalable architecture for p2p topic-based pub/sub. Middleware'12. Google ScholarDigital Library
- Y. Tock, N. Naaman, A. Harpaz, and G. Gershinsky. Hierarchical clustering of message flows in a multicast data dissemination system. In IASTED PDCS, 2005.Google Scholar
- G. Urdaneta, G. Pierre, and M. V. Steen. Towards a fully decentralized and collaborative hosting infrastructure for Wikipedia. In WikiSym'08.Google Scholar
- C. Wilson, B. Boe, A. Sala, K. P. Puttaswamy, and B. Y. Zhao. User interactions in social networks and their implications. In EuroSys '09. Google ScholarDigital Library
Index Terms
- OMen: overlay mending for topic-based publish/subscribe systems under churn
Recommendations
BeaConvey: Co-Design of Overlay and Routing for Topic-based Publish/Subscribe on Small-World Networks
DEBS '18: Proceedings of the 12th ACM International Conference on Distributed and Event-based SystemsDistributed pub/sub must make principal design choices with regards to overlay topologies and routing protocols. It is challenging to tackle both aspects together, and most existing work merely considers one. We argue the necessity to address both ...
mDHT: multicast-augmented DHT architecture for high availability and immunity to churn
CCNC'09: Proceedings of the 6th IEEE Conference on Consumer Communications and Networking ConferenceThis paper presents mDHT, a novel architectural enhancement to DHT using multicast service discovery. In mDHT, a group of host computers in a subnet participate in a DHT overlay as a single node. A query is routed from subnet to subnet until it reaches ...
Performance analysis of structured peer-to-peer overlays for mobile networks
Distributed Hash Table DHT based Peer-to-Peer P2P overlays have been widely researched and deployed in many applications such as file sharing, IP telephony, content distribution and media streaming applications. However, their deployment has largely ...
Comments