Jianxia Chen
ABSTRACT
Recently, decentralized publish-subscribe (pub-sub) systems have gained popularity as a scalable asynchronous messaging paradigm over wide-area networks. Most existing pub-sub systems, however, have been designed with the implicit assumption that published data is clean and accurate. As the pub-sub paradigm is incorporated in real-world applications with human participants, this assumption becomes increasingly invalid due to the inherent noise in the event stream. The noise can take many forms, including redundant, incomplete, inaccurate, and even malicious event messages.
This paper explores the distributed computing issues involved in handling event streams with redundant and incomplete messages. Given a distributed broker overlay-based pub-sub system, we present our initial ideas for (1) aggregating event information scattered across multiple messages generated by different publishers and (2) eliminating redundant event messages. Key to our approach is the concept of an event-gatherer---a designated broker in the routing graph that acts as a proxy sink for all messages of a particular event---located at the graph center of the corresponding routing tree. This paper proposes a novel decentralized algorithm to find this graph center. Early results show that the proposed scheme typically reduces the message load by over 60% with less than 25% time overhead to subscribers.
- Digg (http://digg.com).Google Scholar
- Twitter (http://twitter.com).Google Scholar
- TIB/Rendezvous. White paper, 1999.Google Scholar
- Raman Adaikkalavan and Sharma Chakravarthy. Events must be complete in event processing! In Proceedings of ACM-SAC, 2008. Google ScholarDigital Library
- Mert Akdere, Ugur Çetintemel, and Nesime Tatbul. Plan-based complex event detection across distributed sources. In Proceedings of VLDB, 2008.Google ScholarDigital Library
- Roberto Baldoni, Roberto Beraldi, Vivien Quéma, Leonardo Querzoni, and Sara Tucci Piergiovanni. TERA: topic-based event routing for peer-to-peer architectures. In Proceedings of DEBS, 2007. Google ScholarDigital Library
- Roberto Baldoni, Carlo Marchetti, Antonio Virgillito, and Roman Vitenberg. Content-based Publish-Subscribe over Structured Overlay Networks. In Proceedings ICDCS, 2005. Google ScholarDigital Library
- G. Banavar, T. Chandra, B. Mukherjee, J. Nagarajarao, R. E. Strom, and D. C. Sturman. An Efficient Multicast Protocol for Content-Based Publish-Subscribe Systems. In Proceedings of ICDCS 1999, 1999. Google ScholarDigital Library
- Martin Bauer and Kurt Rothermel. How to Observe Real-World Events through a Distributed World Model. In Proceedings of ICPADS, 2004. Google ScholarDigital Library
- Antonio Carzaniga, David S. Rosenblum, and Alexander L. Wolf. Design and evaluation of a wide-area event notification service. ACM Transactions on Computer Systems, 19(3):332--383, 2001. Google ScholarDigital Library
- Antonio Carzaniga, Matthew. J. Rutherford, and Alexander L. Wolf. A Routing Scheme for Content-Based Networking. In Proceedings of INFOCOM 2004, 2004.Google ScholarCross Ref
- M. Castro, P. Druschel, A-M. Kermarrec, and A. Rowstron. SCRIBE: A Large-Scale and Decentralised Application-level Multicast Infrastructure. IEEE Journal on Selected Areas in Communications (JSAC), 2002.Google Scholar
- Jianxia Chen, Lakshmish Ramaswamy, and David K. Lowenthal. Agele: Dealing with redundant and partial events in a real-world publish-subscribe system. Technical Report UGA-CS-TR-09.001, 2009.Google Scholar
- Gregory Chockler, Roie Melamed, Yoav Tock, and Roman Vitenberg. Constructing scalable overlays for pub-sub with many topics. In Proceedings of PODC, 2007. Google ScholarDigital Library
- Paolo Costa, Matteo Migliavacca, Gian Pietro Picco, and Gianpaolo Cugola. Epidemic Algorithms for Reliable Content-Based Publish-Subscribe: An Evaluation. In Proceedings of ICDCS, 2004. Google ScholarDigital Library
- Patrick Th. Eugster, Rachid Guerraoui, and Christian Heide Damm. On Objects and Events. In Proceedings of OOPSLA, 2001. Google ScholarDigital Library
- Patrick Th Pascal Felber, Rachid Guerraoui, and Anne-Marie Kermarrec. The Many Faces of Publish/Subscribe. ACM Computing Surveys, 35(2), 2003. Google ScholarDigital Library
- W. Fenner, M. Rabinovich, K K. Ramakrishnan, D. Srivastava, and Yin Zhang. XTreeNet: scalable overlay networks for XML content dissemination and querying. In Proceedings WCW, 2005. Google ScholarDigital Library
- Ludger Fiege, Mariano Cilia, Gero Mühl, and Alejandro P. Buchmann. Publish-Subscribe Grows Up: Support for Management, Visibility Control, and Heterogeneity. IEEE Internet Computing, 10(1), 2006. Google ScholarDigital Library
- Abhishek Gupta, Ozgur D. Sahin, Divyakant Agrawal, and Amr El Abbadi. Meghdoot: content-based publish/subscribe over P2P networks. In Middleware 2004, 2004. Google ScholarDigital Library
- Yongqiang Huang and Hector Garcia-Molina. Publish/subscribe in a mobile environment. Wireless Networks, 10(6), 2004. Google ScholarDigital Library
- Navendu Jain, Michael Dahlin, Yin Zhang, Dmitry Kit, Prince Mahajan, and Praveen Yalagandula. STAR: Self-Tuning Aggregation for Scalable Monitoring. In Proceedings of VLDB, 2007. Google ScholarDigital Library
- Zbigniew Jerzak and Christof Fetzer. Bloom Filter Based Routing for Content-based Publish/Subscribe. In Proceedings of DEBS, 2008. Google ScholarDigital Library
- Oana Jurca, Sebastian Michel, Alexandre Herrmann, and Karl Aberer. Query Driven Operator Placement for Complex Event Detection over Data Streams. In Proceedings of EuroSSC, 2008.Google Scholar
- R. Lewis. Advanced Messaging with MSMQ and MQSeries, 1999.Google Scholar
- José Mocito, J. Alfonso Briones-García, Boris Koldehofe, Hugo Miranda, and Luís Rodrigues. Geographical Distribution of Subscriptions for Content-based Publish/Subscribe in MANETs. In Middleware (Companion), 2008. Google ScholarDigital Library
- Peter Pietzuch and Jean Bacon. Hermes: A Distributed Event-Based Middleware Architecture. In Proceedings DEBS, 2002. Google ScholarDigital Library
- Peter R. Pietzuch, Jonathan Ledlie, Jeffrey Shneidman, Mema Roussopoulos, Matt Welsh, and Margo I. Seltzer. Network-Aware Operator Placement for Stream-Processing Systems. In Proceedings of ICDE, 2006. Google ScholarDigital Library
- Thomas Repantis, Xiaohui Gu, and Vana Kalogeraki. Synergy: Sharing-Aware Component Composition for Distributed Stream Processing Systems. In Proceedgins of Middleware, 2006. Google ScholarDigital Library
- Bill Segall, David Arnold, Julian Boot, Michael Henderson, and Ted Phelps. Content Based Routing with Elvin4. In Proceedings of AUUG2k, 2000.Google Scholar
- Linlin Song. A Distributed Algorithm for Graph Center Problem. Master's thesis, 2003.Google Scholar
- Mudhakar Srivatsa and Ling Liu. Securing Publish-Subscribe Overlay Services With EventGuard. In Proceedings of ACM-CCS, 2005. Google ScholarDigital Library
- Sasu Tarkoma. Dynamic content-based channels: meeting in the middle. In Proceedings of DEBS, 2008. Google ScholarDigital Library
- David Thaler and Chinya V. Ravishankar. Distributed Center-Location Algorithms. IEEE Journal on Selected Areas in Communications, 15(3), 1997. Google ScholarDigital Library
- Robert Voigt, Robert Barton, and Shridhar Shukla. A Tool for Configuring Multicast Data Distribution Over Global Networks. In Proceedings of INET, 1995.Google Scholar
- Spyros Voulgaris, Etienne Riviere, Anne-Marie Kermarrec, and Maarten van Steen. Sub-2-Sub: Self-Organizing Content-Based Publish Subscribe for Dynamic Large Scale Collaborative Networks. In Proceedings of the 5th international workshop on peer-to-peer systems, Feb 2006.Google Scholar
- David Wall. Mechanisms for Broadcast and Selective Broadcast. PhD thesis, Stanford University, 1980. Google ScholarDigital Library
Index Terms
- Towards efficient event aggregation in a decentralized publish-subscribe system
Recommendations
Distributed event aggregation for content-based publish/subscribe systems
DEBS '14: Proceedings of the 8th ACM International Conference on Distributed Event-Based SystemsModern data-intensive applications handling massive event streams such as real-time traffic monitoring require support for both rich data filtering and aggregation. While the pub/sub communication paradigm provides an effective solution for the sought ...
Publish-Subscribe Grows Up: Support for Management, Visibility Control, and Heterogeneity
Message-oriented middleware is used to decouple the operation of cooperating applications. Existing approaches have concentrated mainly on scalability issues, but dynamic business processes and the integration of a wide range of data sources and ...
Exploitng event stream interpretation in publish-subscribe systems
PODC '01: Proceedings of the twentieth annual ACM symposium on Principles of distributed computingPublish-subscribe messaging middleware typically offers limited and low-level options for quality of service, such as best-effort delivery versus reliable delivery, or ordered versus unordered. We propose a new, high-level approach to specifying quality ...
Comments