Skip to main content
SpringerLink
Log in

An efficient and scalable framework for content-based publish/subscribe systems

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

Challenges for content-based publish/sub scribe systems include efficient subscription management and event matching, load balancing, and efficient and scalable event delivery. This paper presents an efficient and scalable framework for content-based publish/subscribe systems. We propose using K-D trees to dynamically partition and organize subscriptions, thereby preserving subscription locality, minimizing event matching load and ensuring load balance across nodes. We propose an efficient event delivery mechanism that cleverly exploits embedded trees in distributed hash tables to disseminate events. We show that the latency of event publication and delivery is low. The event delivery mechanism can deliver events to a large number of subscribers at low latency and overhead, consuming modest bandwidth.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In DHTs, node ID can be either obtained when a node joins the overlay or determined by hashing its IP address or public key.

  2. We need to prefix “0” to the string of 0s and 1s extracted from the split history.

  3. This is two-fold. First, the subscriptions in the two partitions are probably less similar if we choose the most distinguishing attribute as the split attribute. Second, splitting along the median of the attribute values is to balance the load between two descendant nodes.

  4. Subscription installation may not be so performance critical as event publication.

  5. Search for the longest matching-prefix for a subscription s in the cache is a greedy algorithm which starts checking from the entry with the longest label. Each cache entry uses its split history to find the longest prefix of its label for s and we choose the longest matching-prefix among all the entries.

  6. A summary filter covers the subscriptions pushed away by exploiting covering relationships between subscriptions [31].

References

  1. Fabret F, Jacobsen HA, Llirbat F, Pereira J, Ross KA, Shasha D (2001) Filtering algorithms and implementation for very fast publish/subscribe systems. In: Proceedings of the 2001 ACM SIGMOD, vol. 30. Santa Barbara, CA, pp 115–126

  2. Segall B, Arnold D (1997) Elvin has left the building: a publish/subscribe notification service with quenching. In: Proceedings of AUUG. Brisbane, Australia, pp 243–255, September

  3. Triantafillou P, Economides A (2004) Subscription summarization: a new paradigm for efficient publish/subscribe systems. In: Proceedings of the 24th IEEE ICDCS. Tokyo, Japan, pp 562–571, March

  4. Carzaniga A, Wolf AL (2003) Forwarding in a content-based network. In: Proceedings of ACM SIGCOMM. Karlsruhe, Germany, pp 163–174, August

  5. Carzaniga A, Rutherford MJ, Wolf AL (2004) A routing scheme for content-based networking. In: Proceedings of IEEE INFOCOM. Hongkong, China, pp 918–928, March

  6. Cao F, Singh JP (2005) MEDYM: match-early and dynamic multicast for content-based publish-subscribe service networks. In: Proceedings of the 4th international workshop on distributed event-based systems. Washington, DC, pp 370–376

  7. Gupta A, Sahin OD, Agrawal D, Abbadi AE (2004) Meghdoot: content-based publish/subscribe over P2P networks. In: ACM/IFIP/USENIX 5th international middleware conference. Toronto, Ontario, Canada, October

  8. Stoica I, Morris R, Karger D, Kaashoek M, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of ACM SIGCOMM. San Diego, CA, pp 149–160, August

  9. Rowstron A, Druschel P (2001) Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Proceedings of the 18th IFIP/ACM international conference on distributed system platforms (Middleware). Heidelberg, Germany, pp 329–350, November

  10. Zhao BY, Kubiatowicz JD, Joseph AD (2001) Tapestry: an infrastructure for fault-tolerance wide-area location and routing. Tech. Rep. UCB/CSD-01-1141, Computer Science Division, University of California, Berkeley, April

  11. Ratnasamy S, Francis P, Handley M, Karp R, Shenker S (2001) A scalable content-addressable network. In: Proceedings of ACM SIGCOMM. San Diego, CA, pp 161–172, August

  12. Aguilera MK, Strom RE, Sturman DC, Astley M, Chandra TD (1999) Matching events in a content-based subscription system. In: Proceedings of the 8th ACM symposium on principles of distributed computing (PODC). Atlanta, GA, pp 53–61, May

  13. Li G, Hou S, Jacobsen H-A (2005) A unified approach to routing, covering and merging in publish/subscribe systems based on modified binary decision diagrams. In: Proceedings of international onference on distributed computing systems (ICDCS), Columbus, OH, June

  14. Banavar G, Chandra T, Mukherjee B, Nagarajarao J, Strom RE, Sturman DC (1999) An efficient multicast protocol for content-based publish-subscribe systems. In: Proceedings of the 19th IEEE ICDCS. Washington, DC, pp 262–272, June

  15. Carzaniga A, Rosenblum DS, Wolf AL (2001) Design and evaluation of a wide-area event notification service. ACM Trans Comput Syst 19(3):332–383

    Article  Google Scholar 

  16. Banavar G, Chandra T, Mukherjee B, Nagarajarao J, Strom RE, Sturman DC (1999) An efficient multicast protocol for content-based publish-subscribe systems. In: Proceedings of the 19th IEEE international conference on distributed computing systems(ICDCS). Austin, TX, pp 262–272, May

  17. Cao F, Singh JP (2004) Efficient event routing in content-basedpublish/subscribe service networks. In: Proceedings of INFOCOM, vol. 2. Hong Kong, China, pp 929–940, March

  18. Zhang R, Hu YC (2005) HYPER: a hybrid approach to efficient content-based publish/subscribe. In: Proceedings of international conference on distributed computing systems (ICDCS). Columbus, OH, June

  19. Rowstron AIT, Kermarrec A-M, Castro M, Druschel P (2001) SCRIBE: the design of a large-scale event notification infrastructure. In: Proceedings of the 3rd international networked group communication, pp 30–43

  20. Zhuang SQ, Zhao BY, Joseph AD, Katz RH, Kubiatowicz J (2001) Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination. In: Proceedings of the eleventh international workshop on network and operating system support for digital audio and video (NOSSDAV). Port Jefferson, New York, pp 11–20, June

  21. Castro M, Druschel P, Kermarrec A-M, Nandi A, Rowstron A, Singh A (2003) Splitstream: high-bandwidth multicast in cooperative environments. In: Proceedings of the 19th ACM symposium on operating systems principles (SOSP). Bolton Landing, NY, October

  22. Tam D, Azimi R, Jacobsen H-A (2003) Building content-based publish/subscribe systems with distributed hash tables. In: Proceedings of the international workshop on databases, information systems and peer-to-peer computing. Berlin, Germany, September

  23. Terpstra WW, Behnel S, Fiege L, Zeidler A, Buchmann AP (2003) A peer-to-peer approach to content-based publish/subscribe. In: Proceedings of the second international workshop on distributed event-based systems (DEBS). San Diego, CA, June

  24. Triantafillou P, Aekaterinidis I (2004) Content-based publish-subscribe over structured P2P networks. In: Proceedings of the third international workshop on distributed event-based systems (DEBS). Edinburgh, Scotland, UK, pp 104–109, May

  25. Perng G, Wang C, Reiter MK (2004) Providing content-based services in a peer-to-peer environment. In: Proceedings of the third international workshop on distributed event-based systems (DEBS). Edinburgh, Scotland, UK, pp 74–79, May

  26. Choi Y, Park K, Park D (2004) HOMED: a peer-to-peer overlay architecture for large-scale content-based publish/subscribe systems. In: Proceedings of the third international workshop on distributed event-based systems (DEBS). Edinburgh, Scotland, UK, pp 20–25, May

  27. Zhu Y, Hu Y (2007) Ferry: an P2P-based architecture for content-based publish/subscribe services. IEEE Trans Parallel Distrib Syst 18(5):672–685

    Article  Google Scholar 

  28. Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517

    Article  MATH  MathSciNet  Google Scholar 

  29. Zhang C, Krishnamurthy A, Wang RY (2005) Brushwood: distributed trees in peer-to-peer systems. In: Proceedings of 4th international workshop on peer-to-peer systems (IPTPS), Ithaca, NY, February

  30. Chawathe Y, Ramabhadran S, Ratnasamy S, LaMarca A, Shenker S, Hellerstein J (2005) A case study in building layered DHT applications. In: Proceedings of SIGCOMM. Philadelphia, PA, pp 97–108, August

  31. Wang Y-M, Qiu L, Achlioptas D, Das G, Larson P, Wang HJ (2002) Subscription partitioning and routing in content-based publish/subscribe systems. In: Proceedings of the 16th international symposium on distributed computing (DISC). Toulouse, France, October

Download references

Acknowledgements

We thank Adair Dingle for her valuable comments and inspiring discussions on the first draft of this paper. We also thank anonymous reviewers for their constructive feedbacks which helped improve the paper.

Author information

Authors and Affiliations

  1. Department of CSSE, Seattle University, Seattle, WA, 98122, USA

    Yingwu Zhu

  2. Computer Science and Computer Engineering Department, University of Arkansas, Fayetteville, AR, 72701, USA

    Haiying Shen

Authors
  1. Yingwu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiying Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingwu Zhu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhu, Y., Shen, H. An efficient and scalable framework for content-based publish/subscribe systems. Peer-to-Peer Netw. Appl. 1, 3–17 (2008). https://doi.org/10.1007/s12083-007-0001-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-007-0001-3

Keywords

Search

Navigation