Skip to main content
Springer Nature Link
Log in

A New Anomaly Detection Method Based on IGTE and IGFE

  • Conference paper
  • First Online:
International Conference on Security and Privacy in Communication Networks (SecureComm 2014)

  • 826 Accesses

Abstract

Network anomalies have been a serious challenge for the Internet nowadays. In this paper, two new metrics, IGTE (Inter-group Traffic Entropy) and IGFE (Inter-group Flow Entropy), are proposed for network anomaly detection. It is observed that IGTE and IGFE are highly correlated and usually change synchronously when no anomaly occurs. However, once anomalies occur, this highly linear correlation would be destroyed. Based on this observation, we propose a linear regression model built upon IGTE and IGFE, to detect the network anomalies. We use both CERNET2 netflow data and synthetic data to validate the regression model and its corresponding detection method. The results show that the regression-based method works well and outperforms the well known wavelet-based detection method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Andrysiak, T., Saganowski, Ł., Choraś, M.: DDoS attacks detection by means of greedy algorithms. In: Choraś, R.S. (ed.) Image Processing and Communications Challenges 4. AISC, vol. 184, pp. 301–308. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  2. Barford, P., Kline, J., Plonka, D., Ron, A.: A signal analysis of network traffic anomalies. In: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurment, pp. 71–82. ACM (2002)

    Google Scholar 

  3. The r project for statistical computing. http://www.r-project.org/

  4. Cisco systems netflow services export version 9. http://www.rfc-base.org/rfc-3954.html

  5. Brauckhoff, D., Salamatian, K., May, M.: Applying pca for traffic anomaly detection: Problems and solutions. In: INFOCOM 2009, pp. 2866–2870. IEEE (2009)

    Google Scholar 

  6. Casella, G., Berger, R.L.: Statistical Inference. Duxbury Press, Belmont (1990)

    MATH  Google Scholar 

  7. Cong, F., Hautakangas, H., Nieminen, J., Mazhelis, O., Perttunen, M., Riekki, J., Ristaniemi, T.: Applying wavelet packet decomposition and one-class support vector machine on vehicle acceleration traces for road anomaly detection. In: Guo, C., Hou, Z.-G., Zeng, Z. (eds.) ISNN 2013, Part I. LNCS, vol. 7951, pp. 291–299. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  8. Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006)

    Article  MathSciNet  Google Scholar 

  9. Guzman, J., Poblete, B.: On-line relevant anomaly detection in the twitter stream: an efficient bursty keyword detection model. In: Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description, pp. 31–39. ACM (2013)

    Google Scholar 

  10. Jamshed, M.A., Lee, J., Moon, S., Yun, I., Kim, D., Lee, S., Yi, Y., Park, K.: Kargus: a highly-scalable software-based intrusion detection system. In: Proceedings of the 2012 ACM Conference on Computer and Communications Security, pp. 317–328. ACM (2012)

    Google Scholar 

  11. Jiang, D., Zhang, P., Xu, Z., Yao, C., Qin, W.: A wavelet-based detection approach to traffic anomalies. In: 2011 Seventh International Conference on Computational Intelligence and Security (CIS), pp. 993–997. IEEE (2011)

    Google Scholar 

  12. Jiang, H., Zhang, G., Xie, G., Salamatian, K., Mathy, L.: Scalable high-performance parallel design for network intrusion detection systems on many-core processors. In: Proceedings of the Ninth ACM/IEEE Symposium on Architectures for Networking and Communications Systems, pp. 137–146. IEEE Press (2013)

    Google Scholar 

  13. Kuzmanovic, A., Knightly, E.W.: Low-rate tcp-targeted denial of service attacks: the shrew vs. the mice and elephants. In: Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, pp. 75–86 (2003)

    Google Scholar 

  14. Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. ACM SIGCOMM Comput. Commun. Rev. 34, 219–230 (2004). ACM

    Article  Google Scholar 

  15. Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. ACM SIGCOMM Comput. Commun. Rev. 35, 217–228 (2005). ACM

    Article  Google Scholar 

  16. Palmieri, F., Fiore, U.: Network anomaly detection through nonlinear analysis. Comput. Secur. 29(7), 737–755 (2010)

    Article  Google Scholar 

  17. Paxson, V.: Bro: a system for detecting network intruders in real-time. Comput. Netw. 31(23), 2435–2463 (1999)

    Article  Google Scholar 

  18. Ringberg, H., Soule, A., Rexford, J., Diot, C.: Sensitivity of PCA for traffic anomaly detection. ACM SIGMETRICS Perform. Eval. Rev. 35, 109–120 (2007). ACM

    Article  Google Scholar 

  19. Roesch, M., et al.: Snort: Lightweight intrusion detection for networks. In: LISA, pp. 229–238 (1999)

    Google Scholar 

  20. Ross, S.M.: Introductory statistics. Academic Press (2010)

    Google Scholar 

  21. Rubinstein, B.I., Nelson, B., Huang, L., Joseph, A.D., Lau, S.h., Rao, S., Taft, N., Tygar, J.: Antidote: understanding and defending against poisoning of anomaly detectors. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference, pp. 1–14. ACM (2009)

    Google Scholar 

  22. Silveira, F., Diot, C., Taft, N., Govindan, R.: Astute: detecting a different class of traffic anomalies. ACM SIGCOMM Comput. Commun. Rev. 40(4), 267–278 (2010)

    Article  Google Scholar 

  23. Simmross-Wattenberg, F., Asensio-Perez, J.I., Casaseca-de-la Higuera, P., Martin-Fernandez, M., Dimitriadis, I.A., Alberola-López, C.: Anomaly detection in network traffic based on statistical inference and alpha-stable modeling. IEEE Trans. Dependable Secure Comput. 8(4), 494–509 (2011)

    Article  Google Scholar 

  24. Soldo, F., Metwally, A.: Traffic anomly detection based on the IP size distribution. In: 2012 Proceedings IEEE INFOCOM, pp. 2005–2013 (2012)

    Google Scholar 

  25. Vasiliadis, G., Polychronakis, M., Ioannidis, S.: Midea: a multi-parallel intrusion detection architecture. In: Proceedings of the 18th ACM Conference on Computer and Communications Security, pp. 297–308. ACM (2011)

    Google Scholar 

  26. Wang, W., Lu, D., Zhou, X., Zhang, B., Mu, J.: Statistical wavelet-based anomaly detection in big data with compressive sensing. EURASIP J. Wireless Commun. Networking 2013(269), 1–6 (2013)

    Google Scholar 

  27. Winter, P., Lampesberger, H., Zeilinger, M., Hermann, E.: On detecting abrupt changes in network entropy time series. In: De Decker, B., Lapon, J., Naessens, V., Uhl, A. (eds.) CMS 2011. LNCS, vol. 7025, pp. 194–205. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  28. Wu, J., Cui, Z., Shi, Y., Su, D.: Traffic flow anomaly detection based on wavelet denoising and support vector regression. J. Algorithms Comput. Technol. 7(2), 209–226 (2013)

    Article  Google Scholar 

  29. Yaacob, A.H., Tan, I.K., Chien, S.F., Tan, H.K.: Arima based network anomaly detection. In: Second International Conference on Communication Software and Networks. ICCSN 2010, pp. 205–209. IEEE (2010)

    Google Scholar 

  30. Zhang, B., Yang, J., Wu, J., Qin, D., Gao, L.: Mcst: Anomaly detection using feature stability for packet-level traffic. In: 2011 13th Asia-Pacific Network Operations and Management Symposium (APNOMS), pp. 1–8. IEEE (2011)

    Google Scholar 

  31. Zhang, B., Yang, J., Wu, J., Qin, D., Gao, L.: Pca-subspace method is it good enough for network-wide anomaly detection. In: 2012 IEEE Network Operations and Management Symposium (NOMS), pp. 359–367. IEEE (2012)

    Google Scholar 

  32. Zhang, B., Yang, J., Wu, J., Wang, Z.: Mbst: detecting packet-level traffic anomalies by feature stability. Comput. J. 56(10), 1176–1188 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful to Lujing Sun for providing us with the Netflow data from CERNET2, and to Kun Wen and Chenxi Li for many helpful discussions. We also thank anonymous reviewers for their constructive comments. This work is supported by the National Basic Research Program of China under Grant No. 2012CB315806, the National Natural Science Foundation of China under Grant No. 61170211, 61202356, 61161140454, Specialized Research Fund for the Doctoral Program of Higher Education under Grant No. 20110002110056, 20130002110058.

Author information

Authors and Affiliations

  1. Tsinghua National Laboratory for Information Science and Technology (TNList), Institute for Network Sciences and Cyberspace, Tsinghua University, Beijing, 100084, China

    Ziyu Wang, Jiahai Yang & Fuliang Li

Authors
  1. Ziyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiahai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fuliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ziyu Wang .

Editor information

Editors and Affiliations

  1. CAS, Institute of Information Engineering, Beijing, China

    Jin Tian

  2. Institute of Information Engineering, Beijing, China

    Jiwu Jing

  3. IBM Thomas J. Watson Research Center, New York, New York, USA

    Mudhakar Srivatsa

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper

Wang, Z., Yang, J., Li, F. (2015). A New Anomaly Detection Method Based on IGTE and IGFE. In: Tian, J., Jing, J., Srivatsa, M. (eds) International Conference on Security and Privacy in Communication Networks. SecureComm 2014. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 153. Springer, Cham. https://doi.org/10.1007/978-3-319-23802-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23802-9_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23801-2

  • Online ISBN: 978-3-319-23802-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics