Abstract
Monitoring the traffic of wide area networks consisting of several autonomous systems connected through a high-speed backbone is a challenge due to the huge amount of traffic. Keeping logs for obtaining measurements is unfeasible. This work describes a distributed real-time strategy for backbone traffic monitoring that does not employ logs and allows arbitrary metrics to be collected about the traffic of the backbone as a whole. Traffic is sampled by monitors that are distributed across the backbone and are accessed by a Stream Processing Engine (SPE). Besides the distributed monitoring architecture, we present an implementation (BackStreamDB) that was deployed on a national backbone. Case studies are described that show the system flexibility. Experiments are reported in which we evaluated the amount of traffic that can be handled.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Abadi, D.J., Ahmad, Y., Balazinska, M., Çentintemel, U., Cherniack, M., Hwang, J.H., Lindner, W., Maskey, A.S., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S.: The Design of the Borealis Stream Processing Engine. In: Proc. of the Conf. on Innovative Data Systems Research, pp. 277–289 (2005)
Amato, G., Chessa, S., Vairo, C.: MaD-WiSe: A Distributed Stream Management System for Wireless Sensor Networks. Software Practice and Experience 40(5), 431–451 (2010)
Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., Widom, J.: STREAM: The Stanford Data Stream Management System. IEEE Data Engineering Bulletin 26(1), 19–26 (2003)
Bin, L., Chuang, L., Jian, Q., Jianping, H., Ungsunan, P.: A NetFlow-based Flow Analisys and Monitoring System in Enterprise Networks. Computer Networks 52(5), 1074–1092 (2008)
Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M.J., Hellerstein, J.M., Hong, W., Krishnamurthy, S., Madden, S., Raman, V., Reiss, F., Shah, M.: TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In: Proc. of the Conf. on Innovative Data Systems Research (2003)
Cottrell, L.: Network Monitoring Tools (2011), http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html
Cranor, C., Johnson, T., Spataschek, O.: Gigascope: A Stream Database for Network Applications. In: Proc. of the ACM SIGMOD Int. Conf. on Management of Data Conference, pp. 647–651 (2003)
Dubendorfer, T., Wagner, A., Plattner, B.: A Framework for Real-Time Worm Attack Detection and Backbone Monitoring. In: Proc. of the IEEE Int. Workshop on Critical Infrastructure Protection, pp. 3–12 (2005)
Hanemann, A., Boote, J.W., Boyd, E.L., Durand, J., Kudarimoti, L., Łapacz, R., Swany, D.M., Trocha, S., Zurawski, J.: PerfSONAR: A Service Oriented Architecture for Multi-domain Network Monitoring. In: Benatallah, B., Casati, F., Traverso, P. (eds.) ICSOC 2005. LNCS, vol. 3826, pp. 241–254. Springer, Heidelberg (2005)
Koudas, N., Srivastava, D.: Data Stream Query Processing: A Tutorial. In: Proc. of the Int. Conf. on Very Large Data Bases, p. 1149 (2003)
Ligocki, N., Gomes, C.L., Hara, C.: A Flexible Network Monitoring Tool based on a Data Stream Management System. In: Proc. of the IEEE Symp. on Computers and Communications, pp. 800–805 (2008)
Munz, G., Carle, G.: Distributed Network Analysis Using TOPAS and Wireshark. In: Network Operations and Management Symp. Workshops, pp. 161–164 (2008)
Phaal, P., Panchen, S., McKee, N.: RFC 3176: InMon Corporation’s sFlow: A Method for Monitoring Traffic in Switched and Routed Networks (2001)
Plagemann, T., Goebel, V., Bergamini, A., Tolu, G., Urvoy-Keller, G., Biersack, E.W.: Using Data Stream Management Systems for Traffic Analysis – A Case Study. In: Barakat, C., Pratt, I. (eds.) PAM 2004. LNCS, vol. 3015, pp. 215–226. Springer, Heidelberg (2004)
Zhou, A., Yan, Y., Gong, X., Chang, J., Dai, D.: SMART: A System for Online Monitoring Large Volumes of Network Traffic. In: Proc. of the IEEE Int. Conf. on Data Engineering, pp. 1576–1579 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lyra, C., Hara, C.S., Duarte, E.P. (2012). BackStreamDB: A Distributed System for Backbone Traffic Monitoring Providing Arbitrary Measurements in Real-Time. In: Taft, N., Ricciato, F. (eds) Passive and Active Measurement. PAM 2012. Lecture Notes in Computer Science, vol 7192. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28537-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-28537-0_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28536-3
Online ISBN: 978-3-642-28537-0
eBook Packages: Computer ScienceComputer Science (R0)