Abstract
For network operators, understanding the types and volumes of traffic carried on the Internet is fundamental to maintaining its stability, reliability, security, and performance. Having efficient and comprehensive network monitoring systems is the key to achieving this understanding. The process of network monitoring varies in complexity from simple long term collection of link utilization statistics to complicated ad-hoc upper-layer protocol analysis for detecting network intrusions, tuning network performance, and debugging protocols. Existing network monitoring tools suffer from critical shortcomings and can no longer fully address network monitoring and debugging needs. To address these problems, we have created Gigascope—a fast and flexible stream database for network monitoring. Gigascope was designed around two key aspects. First, Gigascope has a highly flexible SQL-like query language, GSQL, for its interface. Using a database query language provides us with great flexibility and allows Gigascope to be quickly adapted for new problems—only the high-level query need be changed. Second, Gigascope was designed using the overriding principle of reducing data as early as possible to allow high-speed monitoring. Gigascope queries are automatically broken up into hierarchical components. Low-level components can run on the network interface card itself, reducing data before it reaches the main system bus. High-level query components may run either in kernel or user space and can be used to extract application layer information from the network.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
K.G. Anagnostakis, S. Ioannidis, S. Miltchev, J. Ioannidis, M.B. Greenwald, J.M. Smith, Efficient packet monitoring for network management, in Proceedings of IFIP/IEEE Network Operations and Management Symposium (NOMS) (2002)
A. Arasu, B. Babcock, S. Babu, J. McAlister, J. Widom, Characterizing memory requirements for queries over continuous data streams, in Principles of Database Systems (2002)
B. Babcock, S. Babu, M. Datar, R. Motwani, J. Widom, Models and issues in data stream systems, in Principles of Database Systems (2002), pp. 1–16
S. Babu, J. Widom, Continuous queries over data streams. SIGMOD Rec. 30(3), 109–120 (2001)
P. Bonnet, J. Gehrke, P. Seshadri, Towards sensor database systems, in 2nd Intl. Conf. on Mobile Data Management (2001)
J.D. Case, M. Fedor, M.L. Schoffstall, C. Davin, RFC 1157: Simple Network Management Protocol (SNMP) (1990)
Cisco. Netflow services and application. http://www.cisco.com/
G. Cormode, T. Johnson, F. Korn, S. Muthukrishnan, D. Srivastava, Holistic udafs at streaming speeds, in Proc. ACM SIGMOD (2004)
C. Cortes, K. Fisher, D. Pregibon, A. Rogers, F.S. Hancock, A language for extracting signatures from data streams, in Proc. Sixth Intl. Conf. on Knowledge Discovery and Data Mining (2000), pp. 9–17
DSKI. Dski—the data stream kernel interface. http://www.ittc.ku.edu/datastream
Endace. Endace web page. http://www.endace.com
N.C. Hutchinson, L.L. Peterson, Design of the x-Kernel, in Proceedings of the SIGCOMM’88 Symposium, Stanford, Calif. (1988), pp. 65–75
V. Jacobson, C. Malan, S. McCanne, Libpcap and tcpdump home page. http://www.tcpdump.org/
S. Madden, M. Franklin, Fjording the stream: an architecture for queries over streaming sensor data, in Intl. Conf. on Data Engineering (2002)
G.R. Malan, F. Jahanian, An extensible probe architecture for network protocol performance measurement, in ACM SIGCOMM’98 (1998)
S. McCanne, V. Jacobson, The BSD packet filter: a new architecture for user-level packet capture, in USENIX Winter (1993), pp. 259–270
Narus. Narus platform. http://www.narus.com/w/solutions/platform/
Niksun. Product solutions. http://www.niksun.com/product-list.html
P. Seshadri, M. Livny, R. Ramakrishnan, The design and implementation of a sequence database system, in Proc. of the 22nd VLDB Conf. (1996)
J.W. Stewart III, BGP4: Inter-Domain Routing in the Internet (Addison-Wesley, Reading, 1999)
M. Sullivan, A. Heybey, Tribeca: a system for managing large databases of network traffic, in Proc. USENIX Annual Technical Conf. (1998)
D. Terry, D. Goldberg, D. Nichols, B. Oki, Continuous queries over append-only databases, in Proc. ACM SIGMOD Conf. (1992), pp. 321–330
P. Tucker, D. Maier, T. Sheard, L. Fegaras, Exploiting punctuation semantics in continuous data streams. IEEE Trans. Knowl. Data Eng. 15(3), 555–568 (2003)
S. Waldbusser, RFC 2819: remote monitoring management information base (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Cranor, C.D., Johnson, T., Spatscheck, O. (2016). Stream Processing Techniques for Network Management. In: Garofalakis, M., Gehrke, J., Rastogi, R. (eds) Data Stream Management. Data-Centric Systems and Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28608-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-28608-0_21
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28607-3
Online ISBN: 978-3-540-28608-0
eBook Packages: Computer ScienceComputer Science (R0)