Pooya Salehi
Kaiwen Zhang
ABSTRACT
Distributed overlay-based publish/subscribe systems provide a selective and scalable communication paradigm for connecting components of a distributed application. Existing overlay-based systems only guarantee delivery of notifications to clients that are already known by all brokers in the overlay. Nonetheless, due to the propagation delay, it takes time for a client's interests to be received by all brokers comprising the overlay. The message propagation delay and unclear delivery guarantees during this time increase the complexity of developing distributed applications based on the pub/sub paradigm. In this paper, we propose a collection of message processing and delivery guarantees that allows clients to clearly define the set of publications they receive. Based on our evaluation, these delivery guarantees can reduce buffering requirements on clients by up to 10 times, prevent missing notifications due to the propagation delay, and provide clients with primitive building blocks that simplify application development. We evaluate our proposed routing algorithms and show that a pub/sub system can provide the proposed delivery guarantees without increasing its resource requirements or hindering its throughput.
- R. Baldoni, M. Contenti, S. T. Piergiovanni, and A. Virgillito. 2003. Modeling publish/subscribe communication systems: towards a formal approach. In WORDS'03.Google Scholar
- Mike Burrows. 2006. The Chubby Lock Service for Loosely-coupled Distributed Systems. In OSDI '06.Google Scholar
- Antonio Carzaniga, David S. Rosenblum, and others. 2001. Design and Evaluation of a Wide-area Event Notification Service. ACM Trans. Comput. Syst. (2001).Google Scholar
- Jacques Chester. 2020. Introducing Knative Serving. https://freecontent.manning.com/introducing-knative-serving/. (2020). Accessed: 2020-09-11.Google Scholar
- Gregory V. Chockler, Idit Keidar, and Roman Vitenberg. 2001. Group Communication Specifications: A Comprehensive Study. ACM Comput. Surv. (2001).Google Scholar
- Gianpaolo Cugola, Matteo Migliavacca, and Alessandro Monguzzi. 2007. On Adding Replies to Publish-subscribe. In DEBS '07.Google ScholarDigital Library
- Knative Documentation. 2020. Knative Serving. https://knative.dev/docs/serving/. (2020). Accessed: 2020-09-11.Google Scholar
- Eli Fidler, Hans-Arno Jacobsen, Guoli Li, and Serge Mankovskii. 2005. The PADRES Distributed Publish/Subscribe System. International Conference on Feature Interactions in Telecommunications and Software Systems (ICFI'05) (July 2005), 12-30.Google Scholar
- Apache Software Foundation. 2018. Apache Pulsar pub/sub messaging platform. https://pulsar.incubator.apache.org/. (2018). Accessed: 2018-04-01.Google Scholar
- Sebastian Frischbier, Alessandro Margara, Tobias Freudenreich, and others. 2013. Aggregation for Implicit Invocations. In AOSD '13.Google Scholar
- Google. 2018. Cloud Pub/Sub: A Google-Scale Messaging Service. https://cloud.google.com/pubsub/architecture. (2018). Accessed: 2018-05-01.Google Scholar
- Google. 2018. Google Public NTP. https://developers.google.com/time/. (2018). Accessed: 2018-03-05.Google Scholar
- Jonathan C. Hill, John C. Knight, Aaron M. Crickenberger, and Richard Honhard. 2002. Publish and Subscribe with Reply. Technical Report. University of Virginia.Google Scholar
- Songlin Hu, Vinod Muthusamy, Guoli Li, and Hans-Arno Jacobsen. 2009. Transactional Mobility in Distributed Content-based Publish/Subscribe Systems. In 29th IEEE International Conference on Distributed Computing Systems (ICDCS). 101--110.Google Scholar
- Patrick Hunt, Mahadev Konar, Flavio P. Junqueira, and others. 2010. ZooKeeper: Wait-free Coordination for Internet-scale Systems. In USENIXATC'10.Google Scholar
- Randall Hunt. 2017. Keeping Time With Amazon Time Sync Service. https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-time-sync-service. (2017). Accessed: 2018-03-05.Google Scholar
- Hans-Arno Jacobsen, Alex Cheung, Guoli Li, Balasubramaneyam Maniymaran, Vinod Muthusamy, and Reza Sherafat Kazemzadeh. 2010. The PADRES Publish/Subscribe System. In Principles and Applications of Distributed Event-Based Systems. IGI Global, 164--205.Google Scholar
- Martin Jergler, Kaiwen Zhang, and Hans-Arno Jacobsen. 2018. Multi-client Transactions in Distributed Publish/Subscribe Systems. In ICDCS.Google Scholar
- Reza Sherafat Kazemzadeh. 2012. Overlay neighborhoods for distributed publish/subscribe systems. Ph.D. Dissertation. University of Toronto.Google Scholar
- Reza Sherafat Kazemzadeh and Hans-Arno Jacobsen. 2009. Reliable and Highly Available Distributed Publish/Subscribe Service. In SRDS '09.Google Scholar
- M. Kim, A. Mohindra, V. Muthusamy, R. Ranchal, V. Salapura, A. Slominski, and R. Khalaf. 2016. Building scalable, secure, multi-tenant cloud services on IBM Bluemix. IBM Journal of Research and Development 60 (2016).Google Scholar
- Jay Kreps, Neha Narkhede, Jun Rao, and others. 2011. Kafka: A distributed messaging system for log processing. In Proceedings of the NetDB. 1--7.Google Scholar
- Kfir Lev-Ari, Edward Bortnikov, Idit Keidar, and Alexander Shraer. 2016. Modular Composition of Coordination Services. In USENIX ATC '16.Google Scholar
- Guoli Li, Shuang Hou, and H. Jacobsen. 2005. A Unified Approach to Routing, Covering and Merging in Publish/Subscribe Systems Based on Modified Binary Decision Diagrams. In ICDCS '05.Google Scholar
- Guoli Li, Vinod Muthusamy, and Hans-Arno Jacobsen. 2008. Adaptive Content-based Routing in General Overlay Topologies. In Middleware '08.Google Scholar
- M. Li, F. Ye, M. Kim, H. Chen, and H. Lei. 2011. A Scalable and Elastic Publish/Subscribe Service. In IPDPS '11.Google Scholar
- Microsoft. 2018. Azure regions. https://azure.microsoft.com/en-us/regions/. (2018). Accessed: 2018-01-15.Google Scholar
- Neha Narkhede, Gwen Shapira, and Todd Palino. 2016. Kafka: The Definitive Guide. O'Reilly Media, Inc.Google Scholar
- Navneet Kumar Pandey, Kaiwen Zhang, Stephane Weiss, Hans-Arno Jacobsen, and Roman Vitenberg. 2015. Minimizing the Communication Cost of Aggregation in Publish/Subscribe Systems. In 35th IEEE International Conference on Distributed Computing Systems (ICDCS).Google Scholar
- G. Pardo-Castellote. 2003. OMG Data-Distribution Service: architectural overview. In ICDCS Workshops '03.Google ScholarCross Ref
- Pooya Salehi, Christoph Doblander, and Hans-Arno Jacobsen. 2016. Highly-available Content-based Publish/Subscribe via Gossiping. In DEBS '16.Google ScholarDigital Library
- Yogeshwer Sharma, Philippe Ajoux, Petchean Ang, David Callies, Abhishek Choudhary, Laurent Demailly, Thomas Fersch, Liat Atsmon Guz, Andrzej Kotulski, Sachin Kulkarni, Sanjeev Kumar, Harry Li, Jun Li, Evgeniy Makeev, Kowshik Prakasam, Robbert Van Renesse, Sabyasachi Roy, Pratyush Seth, Yee Jiun Song, Benjamin Wester, Kaushik Veeraraghavan, and Peter Xie. 2015. Wormhole: Reliable Pub-Sub to Support Geo-replicated Internet Services. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). USENIX Association, Oakland, CA, 351--366. https://www.usenix.org/conference/nsdi15/technical-sessions/presentation/sharmaGoogle ScholarDigital Library
- Colby Tresness. 2018. Understanding serverless cold start. https://azure.microsoft.com/en-us/blog/understanding-serverless-cold-start/. (2018). Accessed: 2020-09-11.Google Scholar
- L. Vargas, L. I. W. Pesonen, E. Gudes, and J. Bacon. 2007. Transactions in Content-Based Publish/Subscribe Middleware. In ICDCS Workshops '07.Google Scholar
- WonderNetwork. 2018. Global ping statistics. https://wondernetwork.com/pings. (2018). Accessed: 2018-01-15.Google Scholar
Index Terms
- On Delivery Guarantees in Distributed Content-Based Publish/Subscribe Systems
Recommendations
Dependable distributed content-based publish/subscribe systems: doctoral symposium
DEBS '16: Proceedings of the 10th ACM International Conference on Distributed and Event-based SystemsContent-based publish/subscribe systems provide an efficient communication paradigm that allows decoupling of information producers and consumers across location and time. Distributed overlay-based publish/subscribe systems, while scalable, face many ...
Load Balancing Content-Based Publish/Subscribe Systems
Distributed content-based publish/subscribe systems suffer from performance degradation and poor scalability caused by uneven load distributions typical in real-world applications. The reason for this shortcoming is the lack of a load balancing scheme. ...
Dynamic load balancing in distributed content-based publish/subscribe
Middleware '06: Proceedings of the ACM/IFIP/USENIX 2006 International Conference on MiddlewareDistributed content-based publish/subscribe systems to date suffer from performance degradation and poor scalability caused by uneven load distributions typical in real-world applications. The reason for this shortcoming is due to the lack of a load ...
Comments