Skip to main content

You Surf so Strange Today: Anomaly Detection in Web Services via HMM and CTMC

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10232))

Abstract

In recent years, with the increasing number of attacks against user privacy in web services, researchers put a significant effort on realizing more and more sophisticated Intrusion Detection Systems in order to identify potentially malicious activities. Among such systems, Anomaly Detection Systems rely on a baseline given by a normal behavior and consider every deviation from such behavior as an intrusion.

In this paper, we propose a novel Anomaly Detection System to detect intrusions in users’ private areas in on-line web services. Such services usually record logs of user activity from different points: access, actions in a session and system responses. We design an ad-hoc mathematical model for each of these logs to build a profile for a normal behavior. In particular, we model users’ accesses through a Hidden Markov Model (HMM) and Users’ activity with a Continuous Time Markov Chain (CTMC). We propose a novel Anomaly Detection System algorithm that takes into consideration the deviation from the above Markov Processes. Finally, we evaluate our proposal with a thorough set of experiments, which results confirm the feasibility and effectiveness of our solution.

This is a preview of subscription content, log in via an institution.

References

  1. Ariu, D., Giacinto, G.: HMMPayl: an application of HMM to the analysis of the HTTP payload. In: WAPA (2010)

    Google Scholar 

  2. Bilmes, J.A. et al.: A gentle tutorial of the EM algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. International Computer Science Institute (1998)

    Google Scholar 

  3. Chen, Y., Gupta, M.R.: EM demystified: an expectation-maximization tutorial. In: Electrical Engineering, Citeseer (2010)

    Google Scholar 

  4. Conti, M., Mancini, L.V., Spolaor, R., Verde, N.V.: Analyzing android encrypted network traffic to identify user actions. In: IEEE TIFS (2016)

    Google Scholar 

  5. Gao, D., Reiter, M.K., Song, D.: Behavioral distance measurement using hidden markov models. In: Zamboni, D., Kruegel, C. (eds.) RAID 2006. LNCS, vol. 4219, pp. 19–40. Springer, Heidelberg (2006). doi:10.1007/11856214_2

    Chapter  Google Scholar 

  6. Hoang, X., Hu, J.: An efficient hidden markov model training scheme for anomaly intrusion detection of server applications based on system calls. In: Proceedings of IEEE ICON (2004)

    Google Scholar 

  7. Joshi, S.S., Phoha, V.V.: Investigating hidden markov models capabilities in anomaly detection. In: Proceedings of ACM SE (2005)

    Google Scholar 

  8. Khanna, R., Liu, H.: System approach to intrusion detection using hidden markov model. In: Proceedings of ACM IWCMC (2006)

    Google Scholar 

  9. Liu, Y.-Y., Li, S., Li, F., Song, L., Rehg, J.M.: Efficient learning of continuous-time hidden markov models for disease progression. In: Advances in Neural Information Processing Systems (2015)

    Google Scholar 

  10. Norris, J.R.: Markov Chains, vol. 2. Cambridge University Press, Cambridge (1998)

    MATH  Google Scholar 

  11. Ourston, D., Matzner, S., Stump, W., Hopkins, B.: Applications of hidden markov models to detecting multi-stage network attacks. In Proceedings of IEEE HICSS (2003)

    Google Scholar 

  12. Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. In: Proceedings of the IEEE (1989)

    Google Scholar 

  13. Rahimi, A.: An erratum for “a tutorial on hidden markov models and selected applications in speech recognition” (2000). On-line article

    Google Scholar 

  14. Sperotto, A., Sadre, R., Boer, P.-T., Pras, A.: Hidden markov model modeling of SSH brute-force attacks. In: Bartolini, C., Gaspary, L.P. (eds.) DSOM 2009. LNCS, vol. 5841, pp. 164–176. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04989-7_13

    Chapter  Google Scholar 

  15. Tan, X., Xi, H.: Hidden semi-markov model for anomaly detection. Appl. Math. Comput. 205, 562–567 (2008)

    MathSciNet  MATH  Google Scholar 

  16. Taylor, V.F., Spolaor, R., Conti, M., Martinovic, I.: AppScanner: automatic fingerprinting of smartphone apps from encrypted network traffic. In: Proceedings of IEEE EuroS&P (2016)

    Google Scholar 

  17. Zraiaa, M.: Hidden markov models: a continuous-time version of the Baum-Welch algorithm. Imperial College London (2010)

    Google Scholar 

Download references

Acknowledgment

Mauro Conti is supported by a Marie Curie Fellowship funded by the European Commission (agreement PCIG11-GA-2012-321980). This work is also partially supported by the EU TagItSmart! Project (agreement H2020-ICT30-2015-688061), the EU-India REACH Project (agreement ICI+/2014/342-896), “Physical-Layer Security for Wireless Communication”, and “Content Centric Networking: Security and Privacy Issues” funded by the University of Padua. This work is partially supported by the grant no. 2017-166478 (3696) from Cisco University Research Program Fund and Silicon Valley Community Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Riccardo Spolaor .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Favaretto, M., Spolaor, R., Conti, M., Ferrante, M. (2017). You Surf so Strange Today: Anomaly Detection in Web Services via HMM and CTMC. In: Au, M., Castiglione, A., Choo, KK., Palmieri, F., Li, KC. (eds) Green, Pervasive, and Cloud Computing. GPC 2017. Lecture Notes in Computer Science(), vol 10232. Springer, Cham. https://doi.org/10.1007/978-3-319-57186-7_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57186-7_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57185-0

  • Online ISBN: 978-3-319-57186-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics