Skip to main content
SpringerLink
Log in

Semi-unsupervised Machine Learning for Anomaly Detection in HTTP Traffic

  • Conference paper
  • First Online:
Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 403))

Abstract

Currently, the growing popularity of publicly available web services is one of the driving forces for so-called “web hacking” activities. The main contribution of this paper is the semi-unsupervised anomaly detection method for HTTP traffic anomaly detection. We made the assumption that during the learning phase (for the captured volume of HTTP traffic), only small friction of samples is labelled. Our experiments show that the proposed method allows us to achieve the ratios of true positive and false positive errors below 1 %.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Symantec, Internet Security Threat Report, vol. 19. http://www.symantec.com/security_response/publications/threatreport.jsp (2014)

  2. WhiteHat Website Security Statistics Report. https://www.whitehatsec.com/resource/stats.html

  3. SNORT project homepage. http://www.snort.org/

  4. Sharma, M., Toshniwal, D.: Pre-clustering algorithm for anomaly detection and clustering that uses variable size buckets. In: 1st International Conference on Recent Advances in Information Technology (RAIT), pp. 515–519, 15–17 March 2012

    Google Scholar 

  5. Adaniya, M.H.A.C., Lima, M.F., Rodrigues, J.J.P.C., Abrao, T., Proenca, M.L.: Anomaly detection using DSNS and firefly harmonic clustering algorithm. In: IEEE International Conference on Communications (ICC), pp. 1183–1187, 10–15 June 2012

    Google Scholar 

  6. Mazel, J., Casas, P., Labit, Y., Owezarski, P.: Sub-space clustering, inter-clustering results association and anomaly correlation for unsupervised network anomaly detection. In: 7th International Conference on Network and Service Management (CNSM), pp. 1–8, 24–28 Oct 2011

    Google Scholar 

  7. Yang, C., Deng, F., Yang, H.: An unsupervised anomaly detection approach using subtractive clustering and hidden markov model. In: Second International Conference on Communications and Networking in China. CHINACOM’07, pp. 313–316, 22–24 Aug 2007

    Google Scholar 

  8. Liang, H., Wei-wu, R., Fei, R.: An adaptive anomaly detection based on hierarchical clustering. In: 1st International Conference on Information Science and Engineering (ICISE), pp. 1626–1629, 26–28 Dec 2009

    Google Scholar 

  9. Pons, P., Latapy, M.: Computing communities in large networks using random walks. J. Graph Algorithms Appl. 10(2), 191–218 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  10. Liao, Q., Blaich, A., Van Bruggen, D., Striegel, A.: Managing networks through context: graph visualization and exploration. Comput. Netw. 54, 2809–2824 (2010)

    Article  Google Scholar 

  11. Ricciato, F., Fleischer, W.: Bottleneck detection via aggregate rate analysis: a real casein a 3G network. In: Proceedings of the IEEE/IFIP NOMS (2004)

    Google Scholar 

  12. Thottan, M., Ji, C.: Anomaly detection in IP networks. IEEE Trans. Signal Process. [Special Issue of Signal Processing in Networking], 51(8): 2191–2204 (2003)

    Google Scholar 

  13. Rish, I., Brodie, M., Sheng, M., Odintsova, N., Beygelzimer, A., Grabarnik, G., Hernandez, K.: Adaptive diagnosis in distributed systems. IEEE Trans. Neural Netw. 16(5), 1088–1109 (2005)

    Article  Google Scholar 

  14. Soule, A., Salamatian, K., Taft, N.: Combining filtering and statistical methods for anomaly detection. In: Proceedings of IMC Workshop (2005)

    Google Scholar 

  15. Ma, J., Dai, G., Xu, Z.: Network anomaly detection using dissimilarity-based one-class SVM classifier. In: International Conference on Parallel Processing Workshops. ICPPW’09, pp. 409–414, 22–25 Sept 2009

    Google Scholar 

  16. Ma, R., Liu, Y., Lin, X., Wang, Z.: Network anomaly detection using RBF neural network with hybrid QPSO. In: IEEE International Conference on Networking, Sensing and Control. ICNSC, pp. 1284–1287, 6–8 April 2008

    Google Scholar 

  17. Gaddam, S.R., Phoha, V.V., Balagani, K.S.: K-Means+ID3: a novel method for supervised anomaly detection by cascading K-Means clustering and ID3 decision tree learning methods. IEEE Trans. Knowl. Data Eng. 19(3), 345–354 (2007)

    Article  Google Scholar 

  18. Fraley, C., Raftery, A.E.: Model-based clustering, discriminant analysis, and density estimation. J. Am. Stat. Assoc. 97, 611–631 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Automatic SQL injection and database takeover tool. http://sqlmap.org/

  20. OWASP Zed Attack Proxy Project. https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project

  21. Apache log analyzer for security. https://code.google.com/p/apache-scalp/

Download references

Acknowledgments

This work was partially supported by Applied Research Programme (PBS) of the National Centre for Research and Development (NCBR) funds allocated for the Research Project number PBS1/A3/14/2012 (SECOR).

Author information

Authors and Affiliations

  1. Institute of Telecommunications and Computer Science, UTP University of Science and Technology, Bydgoszcz, Poland

    Rafał Kozik, Michał Choraś & Witold Hołubowicz

  2. ITTI Ltd., Poznan, Poland

    Rafał Kozik, Michał Choraś & Rafał Renk

  3. Adam Mickiewicz University, UAM, Poznan, Poland

    Rafał Renk & Witold Hołubowicz

Authors
  1. Rafał Kozik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michał Choraś
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafał Renk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Witold Hołubowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafał Kozik .

Editor information

Editors and Affiliations

  1. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Robert Burduk

  2. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Konrad Jackowski

  3. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Marek Kurzyński

  4. Dept. of Systems and Computer Networks, Wrocław University of Technology, Wroclaw, Poland

    Michał Woźniak

  5. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Andrzej Żołnierek

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Kozik, R., Choraś, M., Renk, R., Hołubowicz, W. (2016). Semi-unsupervised Machine Learning for Anomaly Detection in HTTP Traffic. In: Burduk, R., Jackowski, K., Kurzyński, M., Woźniak, M., Żołnierek, A. (eds) Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015. Advances in Intelligent Systems and Computing, vol 403. Springer, Cham. https://doi.org/10.1007/978-3-319-26227-7_72

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26227-7_72

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26225-3

  • Online ISBN: 978-3-319-26227-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation