Skip to main content
SpringerLink
Log in

Topology-Aware Correlated Network Anomaly Event Detection and Diagnosis

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

For purposes such as end-to-end monitoring, capacity planning, and performance bottleneck troubleshooting across multi-domain networks, there is an increasing trend to deploy interoperable measurement frameworks such as perfSONAR. These deployments expose vast data archives of current and historic measurements, which can be queried using web services. Analysis of these measurements using effective schemes to detect and diagnose anomaly events is vital since it allows for verifying if network behavior meets expectations. In addition, it allows for proactive notification of bottlenecks that may be affecting a large number of users. In this paper, we describe our novel topology-aware scheme that can be integrated into perfSONAR deployments for detection and diagnosis of network-wide correlated anomaly events. Our scheme involves spatial and temporal analyses on combined topology and uncorrelated anomaly events information for detection of correlated anomaly events. Subsequently, a set of ‘filters’ are applied on the detected events to prioritize them based on potential severity, and to drill-down upon the events “nature” (e.g., event burstiness) and “root-location(s)” (e.g., edge or core location affinity). To validate our scheme, we use traceroute information and one-way delay measurements collected over 3 months between the various U.S. Department of Energy national lab network locations, published via perfSONAR web services. Further, using real-world case studies, we show how our scheme can provide helpful insights for detection, visualization and diagnosis of correlated network anomaly events, and can ultimately save time, effort, and costs spent on network management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Hanemann, A., Boote, J., Boyd, E., Durand, J., Kudarimoti, L., Lapacz, R., Swany, M., Trocha, S., Zurawski, J.: perfSONAR: a service oriented architecture for multi-domain network monitoring. In: Proceedings of Service Oriented Computing, LNCS 3826, pp. 241–254. Springer (http://www.perfSONAR.net) (2005)

  2. Zurawski, J., Swany, M., Gunter, D.: Scalable framework for representation and exchange of network measurements. In: Proceedings of IEEE TRIDENTCOM (2006)

  3. Guok, C., Robertson, D., Thompson, M., Lee, J., Tierney, B., Johnston, W.: Intra and interdomain circuit provisioning using the OSCARS reservation system. In: Proceedings of IEEE/ICST Conference on Broadband Communications, Networks, and Systems (2006)

  4. Allen, J.: Driving by the rear-view mirror: managing a network with cricket. In: Proceedings of USENIX Network Administration Conference (1999)

  5. Matthews, W., Cottrell, L.: The PingER project: active internet performance monitoring for the HENP community. IEEE Commun. Mag. Netw. Traffic Meas. Exp. 38(5), 130–136 (2000)

    Google Scholar 

  6. Zurawski, J., Boote, J. et al.: Hierarchically federated registration and lookup within the perfSONAR framework. In: Proceedings of IFIP/IEEE Integrated Management Symposium (2007)

  7. McGregor, A., Braoun, H-W.: Automated event detection for active measurement systems. In: Proceedings of Passive and Active Measurement Workshop (2001)

  8. Logg, C., Cottrell, L.: Experiences in traceroute and available bandwidth change analysis. In: Proceedings of ACM SIGCOMM Network Troubleshooting Workshop (2004)

  9. Calyam, P., Pu, J., Mandrawa, W., Krishnamurthy, A.: OnTimeDetect: dynamic network anomaly notification in perfSONAR deployments. In: Proceedings of IEEE/ACM MASCOTS (2010)

  10. Mahimkar, A., Yates, J., Zhang, Y., Shaikh, A., Wang, J., Ee, C.: Troubleshooting chronic conditions in large IP networks. ACM SIGCOMM CoNEXT (2008)

  11. Plonka, D., Barford, P.: Network anomaly confirmation, diagnosis and remediation. In: Proceedings of IEEE Allerton Conference on Communication, Control, and Computing (2009)

  12. Palla, G., lszl Barabsi, A., Vicsek, T., Hungary, B.: Quantifying social group evolution. In: Proceedings of Nature (2007)

  13. Collaboration, A., Laurens, P., Severini, H., Wolff, S., Lake, A., Kee, S., Zurawski, J., Wlodek, T.: Monitoring the US ATLAS network infrastructure with perfSONAR-ps. In: Proceedings of Conferences on Computing in High Energy and Nuclear Physics (CHEP) (2012)

  14. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: Proceedings of ACM SIGCOMM (2004)

  15. Soule, A., Salamatian, K., Taft, N.: Combining filtering and statistical methods for anomaly detection. In: Proceedings of Conference on Internet Measurement (2005)

  16. Zonglin, L., Guangmin, H., Xingmiao, Y., Dan, Y.: Detecting distributed network traffic anomaly with network-wide correlation analysis. In: Proceedings of EURASIP J. Adv. Signal Process (2009)

  17. Zhou, Y., Hu, G.: Network-wide anomaly detection based on router connection relationships. IEICE Trans. 94-B(8), 2239–2242 (2011)

    Google Scholar 

  18. Yalagandula, P., Lee, S., Sharma, P., Banerjee, S.: Correlations in end-to-end network metrics: impact on large scale network monitoring. In: Proceedings of IEEE INFOCOM Workshops (2008)

  19. Yalagandula, P., Lee, S., Sharma, P., Banerjee, S.: Leveraging correlations between capacity and available bandwidth to scale network monitoring. In: Proceedings of IEEE GLOBECOM (2010)

  20. Mutt, E., Sharma, M., Soman, J., Kothapalli, K., Mitra, A.: Graph theoretic approach for studying correlated motions in biomolecules. In: Proceedings of IEEE NaBIC (2009)

  21. Fernandes, G., Owezarski, P.: Automated classification of network traffic anomalies. In: Proceedings of SecureComm (2009)

  22. Wang, T., Srivatsa, M., Agrawal, D., Liu, L.: Spatio-temporal patterns in network events. In: Proceedings of ACM Co-NEXT (2010)

  23. Hanemann, A., Jeliazkov, V., Kvittem, O., Marta, L., Metzger, J., Velimirovic, I.: Complementary visualization of perfSONAR network performance measurements. In: Proceedings of IEEE International Conference on Internet Surveillance and Protection (2006)

  24. Bruackhoff, D., Dimitropoulos, X., Wagner, A., Salamatian, K.: Anomaly extraction in backbone networks using associated rules. IEEE/ACM Trans. Netw. 20(6), 1788–1799 (2012)

    Google Scholar 

  25. Chan, E., Luo, X., Fok, W., Li, W., Chang, R.: Non-cooperative diagnosis of submarine cable faults. In: Proceedings of Passive and Active Measurement Conference (PAM) (2011)

  26. Dainotti, A., Pescape, A., Ventre, G.: Worm traffic analysis and characterization. In: Proceedings of IEEE International Conference on Communications (ICC) (2007)

  27. Dainotti, A., King, A., Claffy, K., Papale, F., Pescape, A.: Analysis of a "/0" stealth scan from a Botnet. In: Proceedings of ACM SIGCOMM/SIGMETRICS Internet Measurement Conference (IMC) (2012)

  28. Calyam, P., Krymskiy, D., Sridharan, M., Schopis, P.: Active and passive measurements on campus, regional and national network backbone paths. In: Proceedings of IEEE ICCCN (2005)

  29. Grigoriev, M., Demar, P., Eads, D., Tierney, B., Metzger, J., Lake, A., Frey, M., Calyam, P.: E-Center: Collaborative platform for the wide area network users. In: Proceedings of Conferences on Computing in High Energy and Nuclear Physics (CHEP) (2012)

  30. Dhanapalan, M., Calyam, P., Sridharan, M.: Nagios adaptive plateau anomaly detection (APD) Plugin- v1.4. http://anonsvn.internet2.edu/svn/perfSONAR-PS/branches/osc-APD-Nagios/perfSONAR_PS-Nagios/doc/APD_README.txt (2012)

  31. Marchetta, P., Merindol, P., Donnet, B., Pescape, A., Pansiot, J.-J.: Topology discovery at the router level: a new hybrid tool targeting ISP networks. IEEE J Sel. Areas Commun. (JSAC) 29(9), 1776–1787 (2011)

    Article  Google Scholar 

  32. Donato, W., Marchetta, P., Pescape, A.: Detecting third-party addresses in traceroute IP paths. In: Proceedings of ACM SIGCOMM (2012)

  33. Ellson, J., Gansner, E., Koutsofios, E., North, S., Woodhull, G.: Graphviz and dynagraph—static and dynamic graph drawing tools. In: Proceedings of Graph Drawing Software, pp. 127–148. Springer (2003)

Download references

Author information

Authors and Affiliations

  1. University of Missouri-Columbia, Columbia, MO, USA

    Prasad Calyam

  2. The Ohio State University, Columbus, OH, USA

    Manojprasadh Dhanapalan & Rajiv Ramnath

  3. The Samraksh Company, Dublin, OH, USA

    Mukundan Sridharan

  4. RENCI, Chapel Hill, NC, USA

    Ashok Krishnamurthy

Authors
  1. Prasad Calyam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manojprasadh Dhanapalan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mukundan Sridharan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ashok Krishnamurthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rajiv Ramnath
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prasad Calyam.

Additional information

This material is based upon work supported by the Department of Energy under Award Numbers: DE-SC0001331 and DE-SC0007531. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Calyam, P., Dhanapalan, M., Sridharan, M. et al. Topology-Aware Correlated Network Anomaly Event Detection and Diagnosis. J Netw Syst Manage 22, 208–234 (2014). https://doi.org/10.1007/s10922-013-9286-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10922-013-9286-0

Keywords

Search

Navigation