Abstract
The sheer volume of data delivered via the Internet requires a more flexible and powerful communication model. As an expressive loosely-coupled asynchronous messaging model, Publish-Subscribe (Pub/Sub) system has been widely used. Traditional topic-based Pub/Sub system fails to understand the information of messages delivered, all messages must be previously classified into a set of topics. Content-based Pub/Sub system can dynamically choose subscribers for each message by its metadata. Existing distributed Pub/Sub systems are built on the overlay network consists of message brokers, which can adapt to heterogeneous network but inevitably impairs performance. In this paper, we designed a novel centralized tiered content-based Pub/Sub system with a four-layer architecture. In access layer, a customized naming strategy is proposed to achieve high availability. Internal message routing is finished in routing layer and sharding scheme is used to lower routing overhead. In computation layer, a two-step streaming computation model is used to boost the performance. In storage layer we adopt column-oriented database HBase for persistence. A set of comprehensive experiments were conduct to verify that our system achieve excellent performance, linear scalability and high availability.
This work has been financially supported by Shenzhen Key Fundamental Research Projects (Grant No. JCYJ20170412151008290, JCYJ20170306091556329, JCYJ20170412150946024).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Banavar, G., Chandra, T., Mukherjee, B., Nagarajarao, J.: An efficient multicast protocol for content-based publish-subscribe systems. In: Proceedings of IEEE International Conference on Distributed Computing Systems, 1999, pp. 262–272 (1999)
Strom, R., Banavar, G., Chandra, T., Kaplan, M., Miller, K., Mukherjee, B., et al.: Gryphon: an information flow based approach to message brokering. Computer Science arXiv:cs/9810019 (1998)
Cao, F., Singh, J.P.: MEDYM: match-early with dynamic multicast for content-based publish-subscribe networks. In: Alonso, G. (ed.) Middleware 2005. LNCS, vol. 3790, pp. 292–313. Springer, Heidelberg (2005). https://doi.org/10.1007/11587552_15
Carzaniga, A., Rosenblum, D.S., Wolf, A.L.: Design and evaluation of a wide-area event notification service. ACM Trans. Comput. Syst. 19(3), 283–334 (2003)
Apache Storm. http://storm.apache.org. Accessed 08 Oct 2017
MQTT. http://mqtt.org. Accessed 08 Oct 2017
Netty. https://netty.io. Accessed 08 Oct 2017
Lei, K., Hou, C., Li, L., Xu, K.: A RCP-based congestion control protocol in named data networking. In: 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pp. 538–541. IEEE (2015)
Wong, T., Katz, R., Mccanne, S.: An evaluation of preference clustering in large-scale multicast applications. In: Proceedings of IEEE Nineteenth Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2000, vol. 2, pp. 451–460. IEEE (2000)
Wang, Y.M., Qiu, L., Achlioptas, D., Das, G., Larson, P., Wang, H.J.: Subscription partitioning and routing in content-based publish/subscribe systems. In: Disc Proceedings of International Symposium on Distributed Computing (2002)
Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B.: The R*-tree: an efficient and robust access method for points and rectangles. ACM Sigmod Rec. 19(2), 22–331 (1990)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Huang, L., Liu, L., Chen, J., Lei, K. (2018). An Implementation of Content-Based Pub/Sub System via Stream Computation. In: Qiu, M. (eds) Smart Computing and Communication. SmartCom 2017. Lecture Notes in Computer Science(), vol 10699. Springer, Cham. https://doi.org/10.1007/978-3-319-73830-7_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-73830-7_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73829-1
Online ISBN: 978-3-319-73830-7
eBook Packages: Computer ScienceComputer Science (R0)