Skip to main content
Springer Nature Link
Log in

Anomaly Detection Approaches for Communication Networks

  • Chapter
  • First Online:
Algorithms for Next Generation Networks

Part of the book series: Computer Communications and Networks ((CCN))

  • 1740 Accesses

Abstract

In recent years, network anomaly detection has become an important area for both commercial interests as well as academic research. Applications of anomaly detection typically stem from the perspectives of network monitoring and network security. In network monitoring, a service provider is often interested in capturing such network characteristics as heavy flows, flow size distributions, and the number of distinct flows. In network security, the interest lies in characterizing known or unknown anomalous patterns of an attack or a virus.

In this chapter we review two main approaches to network anomaly detection: streaming algorithms, and machine learning approaches with a focus on unsupervised learning. We discuss the main features of the different approaches and discuss their pros and cons. We conclude the chapter by presenting some open problems in the area of network anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ahmed T., Coates M., Lakhina A.: Multivariate Online Anomaly Detection Using Kernel Recursive Least Squares. Proc. of 26th IEEE International Conference on Computer Communications (2007)

    Google Scholar 

  2. Ahmed T., Oreshkin B., Coates M.: Machine Learning Approaches to Network Anomaly Detection. Proc. of International Measurement Conference (2007)

    Google Scholar 

  3. Andersen D., Feamster N., Bauer S., Balaskrishman H.: Topology inference from BGP routing dynamics. Proc. SIGCOM Internet Measurements Workshop, Marseille, France (2002)

    Google Scholar 

  4. Androulidakis G., Papavassiliou S.: Improving Network Anomaly Detection via Selective Flow-Based Sampling. Communications, IET. Vol. 2, no. 3, 399–409 (2008)

    Article  Google Scholar 

  5. Barford P., Kline J., Plonka D., Ron A.: A Signal Analysis of Network Traffic Anomalies. Proc. of the 2nd ACM SIGCOMM Workshop on Internet Measurements, 71–82 (2002)

    Google Scholar 

  6. Cormode G., Korn F., Muthukrishnan S. D., Srivastava D.: Finding Hierarchical Heavy Hitters in Data Streams. Proc. of VLDB, Berlin, Germany (2003)

    Google Scholar 

  7. Cormode G., Muthukrishan S.: Improved Data Stream Summaries: The Count-Min Sketch and Its Applications. Tech. Rep. 03-20, DIMACS (2003)

    Google Scholar 

  8. Cormode G., Johnson T., Korn F., Muthukrishnan S. Spatscheck O., Srivastava D.: Holistic UDAFs at Streaming Speeds. Proc. of ACM SIGMOD, Paris, France (2004)

    Google Scholar 

  9. Cormode G., Korn F, Muthukrishnan S., Srivastava D.: Diamond in the Rough: Finding Hierarchical Heavy Hitters in Multi-Dimensional Data. Proc. of ACM SIGMOD, 155–166 (2004)

    Google Scholar 

  10. Cormode G., Muthukrishnan S.: What’s New: Finding Significant Differences in Network Data Streams. IEEE/ACM Trans. Netw. 13(6):1219–1232 (2005)

    Article  Google Scholar 

  11. Cormode G., Korn. F., Muthukrishnan S., Srivastava D: Finding Hierarchical Heavy Hitters in Streaming Data. ACM Trans. Knowledge Discovery from Data 1(4) (2008)

    Google Scholar 

  12. Deshpande S., Thottan M., Sikdar B.: Early Detection of BGP Instabilities Resulting From Internet Worm Attacks. Proc. of IEEE Globecom, Dallas, TX (2004)

    Google Scholar 

  13. Duda R. O., Hart P., Stork D.: Pattern Classification, 2nd edn. John Willy and Sons (2001)

    Google Scholar 

  14. Duffield N.G., Lund C., Thorup M.: Properties and Prediction of Flow Statistics from Sampled Packet Streams. Proc. of ACM SIGCOMM Internet Measurement Workshop (2002)

    Google Scholar 

  15. Ensafi R., Dehghanzadeh S., Mohammad R., Akbarzadeh T.: Optimizing Fuzzy K-Means for Network Anomaly Detection Using PSO. Computer Systems and Applications, IEEE/ACS International Conference, 686–693 (2008)

    Google Scholar 

  16. Erjongmanee S., Ji C.: Inferring Internet Service Disruptions upon A Natural Disaster. To appear at 2nd International Workshop on Knowledge Discovery from Sensor Data (2008)

    Google Scholar 

  17. Estan C., Varghese G.: New Directions in Traffic Measurement and Accounting. Proc. of ACM SIGCOMM, New York, USA (2002)

    Google Scholar 

  18. Gao Y., Li Z., Chen Y.: A DoS Resilient Flow-level Intrusion Detection Approach for High-speed Networks, Proc. of IEEE International Conference on Distributed Computing Systems (2006)

    Google Scholar 

  19. Gu Y., McCallum A., Towsley D.: Detecting Anomalies in Network Traffic Using Maximum Entropy Estimation. Proc. of IMC (2005)

    Google Scholar 

  20. Haffner P., Sen S., Spatscheck O., Wang D.: ACAS: Automated Construction of Application Signatures. Proc. of ACM SIGCOMM Workshop on Mining Network Data, Philadelphia, (2005)

    Google Scholar 

  21. Hajji H.: Statistical Analysis of Network Traffic for Adaptive Faults Detection. IEEE Trans. Neural Networks. Vol. 16, no. 5, 1053–1063 (2005)

    Article  Google Scholar 

  22. He Q., Shayman M.A.: Using Reinforcement Learning for Pro-Active Network Fault Management. Proc. of Communication Technology. Vol. 1, 515–521 (2000)

    Google Scholar 

  23. Hood C.S., Ji C.: Proactive Network Fault Detection. IEEE Tran. Reliability. Vol. 46 3, 333–341 (1997)

    Article  Google Scholar 

  24. Huang L., Nguyen X., Garofalakis M., Jordan M.I., Joseph A., Taft N.: Communication-Efficient Online Detection of Network-Wide Anomalies. Proc. of 26th Annual IEEE Conference on Computer Communications (2007)

    Google Scholar 

  25. Huang Y., Feamster N., Lakhina A., Xu J.: Diagnosing Network Disruptions with Network-Wide Analysis. Proc. of ACM SIGMETRICS (2007)

    Google Scholar 

  26. Ide T., Kashima H.: Eigenspace-Based Anomaly Detection in Computer Systems. Proc. of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, Seattle, 440–449 (2004)

    Google Scholar 

  27. Kim S.S., Reddy A.: Statistical Techniques for Detecting Traffic Anomalies Through Packet Header Data. Accepted by IEEE/ACM Tran. Networking (2008)

    Google Scholar 

  28. Kline K., Nam S., Barford P., Plonka D., Ron A.: Traffic Anomaly Detection at Fine Time Scales with Bayes Nets. To appear in the International Conference on Internet Monitoring and Protection (2008)

    Google Scholar 

  29. Krishnamurthy B., Sen S., Zhang Y., Chan Y.: Sketch-Based Change Detection: Methods, Evaluation, and Applications. Proc. of ACM SIGCOMM IMC, Florida, USA (2003)

    Google Scholar 

  30. Lall S., Sekar V., Ogihara M., Xu J., Zhang H.: Data Streaming Algorithms for Estimating Entropy of Network Traffic. Proc. of ACM SIGMETRICS (2006)

    Google Scholar 

  31. Lakhina A., Crovella M., Diot C.: Diagnosing Network-Wide Traffic Anomalies. Proc. of ACM SIGCOMM (2004)

    Google Scholar 

  32. Lakhina A., Papagiannaki K., Crovella M., Diot C., Kolaczyk E. N., Taft N.: Structural Analysis of Network Traffic Flows. Proc. of ACM SIGMETRICS (2004)

    Google Scholar 

  33. Lakhina A., Crovella M., Diot C.: Mining Anomalies Using Traffic Feature Distributions. Proc. of ACM SIGCOMM, Philadelphia, PA (2005)

    Google Scholar 

  34. Lee W., Stolfo F., Mok K.W.: A Data Mining Framework for Building Intrusion Detection Models. Proc. of In IEEE Symposium on Security and Privacy (1999)

    Google Scholar 

  35. Lee W., Xiang D.: Information-Theoretic Measures for Anomaly Detection. Proc. of IEEE Symposium on Security and Privacy (2001)

    Google Scholar 

  36. Leland W. E., Taqqu M. S., Willinger W., Wilson D. V.: On the Self-Similar Nature of Ethernet Traffic, Proc. of ACM SIGCOMM (1993)

    Google Scholar 

  37. Mai J., Chuah C., Sridharan A., Ye T., Zang H.: Is Sampled Data Sufficient for Anomaly Detection? Proc. of 6th ACM SIGCOMM conference on Internet measurement, Rio de Janeriro, Brazil. 165–176 (2006)

    Google Scholar 

  38. Mandjes M., Saniee I., Stolyar A. L.: Load Characterization and Anomaly Detection for Voice over IP traffic. IEEE Tran. Neural Networks. Vol.16, no. 5, 1019–1026 (2005)

    Article  Google Scholar 

  39. Manku G. S., Motwani R.: Approximate Frequency Counts over Data Streams. Proc. of IEEE VLDB, Hong Kong, China (2002)

    Google Scholar 

  40. Maxion R. A., Tan K. M. C.: Benchmarking Anomaly-Based Detection Systems. Proc. International Conference on Dependable Systems and Networks (2000)

    Google Scholar 

  41. Miller E. L., Willsky A. S.: Multiscale, Statistical Anomaly Detection Analysis and Algorithms for Linearized Inverse Scattering Problems. Multidimensional Systems and Signal Processing. Vol. 8, 151–184 (1997)

    MATH  Google Scholar 

  42. Ricciato F., Fleischer W.: Bottleneck Detection via Aggregate Rate Analysis: A Real Case in a 3G Network. Proc. IEEE/IFIP NOMS (2004)

    Google Scholar 

  43. Ringberg H., Soule A., Rexford J., Diot C.: Sensitivity of PCA for Traffic Anomaly Detection. Proc. of ACM SIGMETRICS (2007)

    Google Scholar 

  44. Rish I., Brodie M., Sheng M., Odintsova N., Beygelzimer A., Grabarnik G., Hernandez K.: Adaptive Diagnosis in Distributed Systems. IEEE Tran. Neural Networks. Vol. 16, No. 5, 1088–1109 (2005)

    Google Scholar 

  45. Schweller R., Gupta A., Parsons E., Chen Y.: Reversible Sketches for Efficient and Accurate Change Detection over Network Data Streams. Proc. of IMC, Italy (2004)

    Google Scholar 

  46. Schweller R., Li Z., Chen Y., Gao Y., Gupta A., Zhang Y., Dinda P., Kao M., Memik G.: Reverse hashing for High-Speed Network Monitoring: Algorithms, Evaluation, and Applications. Proc. of IEEE INFOCOM (2006)

    Google Scholar 

  47. Soule A., Salamatian K., Taft N.: Combining Filtering and Statistical Methods for Anomaly Detection. Proc. of IMC Workshop (2005)

    Google Scholar 

  48. Steinder M., Sethi A.S.: Probabilistic Fault Localization in Communication Systems Using Belief Networks. IEEE/ACM Trans. Networking. Vol. 12, No. 5, 809–822 (2004)

    Article  Google Scholar 

  49. Tavallaee M., Lu W., Iqbal S. A., Ghorbani A.: A Novel Covariance Matrix Based Approach for Detecting Network Anomalies. Communication Networks and Services Research Conference (2008)

    Google Scholar 

  50. Thottan M., Ji C.: Anomaly Detection in IP Networks. IEEE Trans. Signal Processing, Special Issue of Signal Processing in Networking, Vol. 51, No. 8, 2191–2204 (2003)

    Article  Google Scholar 

  51. Thottan M., Ji C.: Proactive Anomaly Detection Using Distributed Intelligent Agents. IEEE Network. Vol. 12, no. 5, 21–27 (1998)

    Article  Google Scholar 

  52. Venkataraman S., Song D., Gibbons P., Blum A.: New Streaming Algorithms for Fast Detection of Superspreaders. Proc. of Network and Distributed Systems Security Symposium (2005)

    Google Scholar 

  53. Venkataraman S., Caballero J., Song D., Blum A., Yates J.: Black-box Anomaly Detection: Is it Utopian?” Proc. of the Fifth Workshop on Hot Topics in Networking (HotNets-V), Irvine, CA (2006)

    Google Scholar 

  54. Xie Y., Kim H.A., O’Hallaron D. R., Reiter M. K., Zhang H.: Seurat: A Pointillist Approach to Anomaly Detection. Proc. of the International Symposium on Recent Advances in Intrusion Detection (RAID) (2004)

    Google Scholar 

  55. Wang H., Zhang D., Shin K. G.: Detecting SYN flooding attacks. Proc. of IEEE INFOCOM (2002)

    Google Scholar 

  56. Xu J.: Tutorial on Network Data Streaming. SIGMETRICS (2007)

    Google Scholar 

  57. Yang Y., Deng F., Yang H.: An Unsupervised Anomaly Detection Approach using Subtractive Clustering and Hidden Markov Model. Communications and Networking in China. 313–316 (2007)

    Google Scholar 

  58. Yeung D. S., Jin S., Wang X.: Covariance-Matrix Modeling and Detecting Various Flooding Attacks. IEEE Tran. Systems, Man and Cybernetics, Part A, vol. 37, no. 2, 157–169 (2007)

    Article  Google Scholar 

  59. Zhang Y., Singh S., Sen S., Duffield N., Lund C.: Online Identification of Hierarchical Heavy Hitters: Algorithms, Evaluation and Applications. Proc. of ACM SIGCOMM conference on Internet measurement. 101–114 (2004)

    Google Scholar 

  60. Zhang J., Rexford J., Feigenbaum J.: Learning-Based Anomaly Detection in BGP Updates. Proc. of ACM SIGCOMM MineNet workshop (2005)

    Google Scholar 

  61. Zhang Y., Ge Z., Greenberg A., Roughan M.: Network Anomography. Proc. of ACM/USENIX Internet Measurement Conference (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bell Labs, Alcatel-Lucent, 600-700 Mountain Avenue, Murray Hill, NJ, 07974, U.S.A.

    Marina Thottan

  2. Division of Mathematics and Sciences, Roane State Community College, 276 Patton Lane, Harriman, TN, 37748, U.S.A.

    Guanglei Liu

  3. School of Electrical and Computer Engineering, Georgia Institute of Technology, 777 Atlantic Drive, Atlanta, GA, 30332, U.S.A.

    Chuanyi Ji

Authors
  1. Marina Thottan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanglei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuanyi Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marina Thottan .

Editor information

Editors and Affiliations

  1. AT & T Labs Research, Park Ave. 180, Florham Park, 07932, U.S.A.

    Graham Cormode

  2. Networking Research Lab., Bell Labs, Mountain Ave. 600-700, Murray Hill, 07974, U.S.A.

    Marina Thottan

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag London Limited

About this chapter

Cite this chapter

Thottan, M., Liu, G., Ji, C. (2010). Anomaly Detection Approaches for Communication Networks. In: Cormode, G., Thottan, M. (eds) Algorithms for Next Generation Networks. Computer Communications and Networks. Springer, London. https://doi.org/10.1007/978-1-84882-765-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-84882-765-3_11

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-84882-764-6

  • Online ISBN: 978-1-84882-765-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics