Skip to main content
SpringerLink
Log in

Detection of Application-Layer Tunnels with Rules and Machine Learning

  • Conference paper
  • First Online:
Security, Privacy, and Anonymity in Computation, Communication, and Storage (SpaCCS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11611))

Abstract

Application-layer tunnels are often used to construct covert channels in order to transmit secret data, which is often applied to raise network threats in recent years. Detection of application-layer tunnels can assist identifying a variety of network threats, thus has high research significance. In this paper, we explore application-layer tunnel detection and propose a generic detection method by applying both rules and machine learning. Our detection method mainly consists of two parts: rule-based domain name filtering for Domain Generation Algorithm (DGA) based on a trigram model and a machine learning model based on our proposed generic feature extraction framework for tunnel detection. The rule-based DGA domain name filtering can eliminate some obvious tunnels in order to reduce the amount of data processed by machine learning-based detection, thereby, the detection efficiency can be improved. The generic feature extraction framework comprehensively integrates previous research results by combining multiple detection methods, supporting multiple layers and performing multiple feature extraction. We take the three most common application-layer tunnels, i.e., DNS tunnel, HTTP tunnel and HTTPS tunnel as examples to analyze and test our detection method. The experimental results show that the proposed method is generic and efficient, compared with other existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Nuojua, V., David, G., Hämäläinen, T.: DNS tunneling detection techniques – classification, and theoretical comparison in case of a real APT campaign. In: Galinina, O., Andreev, S., Balandin, S., Koucheryavy, Y. (eds.) NEW2AN/ruSMART/NsCC 2017. LNCS, vol. 10531, pp. 280–291. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67380-6_26

    Chapter  Google Scholar 

  2. Borders, K., Prakash, A.: Web tap: detecting covert web traffic. In: Proceedings of the 11th ACM Conference on Computer and Communications Security, pp. 110–120. ACM, New York (2004)

    Google Scholar 

  3. Crotti, M., Dusi, M., Gringoli, F., Salgarelli, L.: Detecting http tunnels with statistical mechanisms. In: 2007 IEEE International Conference on Communications, Glasgow, pp. 6162–6168. IEEE (2007)

    Google Scholar 

  4. Dusi, M., Crotti, M., Gringoli, F., Salgarelli, L.: Tunnel hunter: detecting application-layer tunnels with statistical fingerprinting. Comput. Netw. 53(1), 81–97 (2009)

    Article  Google Scholar 

  5. Do, V.T., Engelstad, P., Feng, B., van Do, T.: Detection of DNS tunneling in mobile networks using machine learning. In: Kim, K., Joukov, N. (eds.) ICISA 2017. LNEE, vol. 424, pp. 221–230. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-4154-9_26

    Chapter  Google Scholar 

  6. Almusawi, A., Amintoosi, H.: DNS Tunneling detection method based on multilabel support vector machine. In: Security and Communication Networks 2018 (2018)

    Google Scholar 

  7. Qi, C., Chen, X., Xu, C., Shi, J., Liu, P.: A bigram based real time DNS tunnel detection approach. Procedia Comput. Sci. 17, 852–860 (2013)

    Article  Google Scholar 

  8. Aiello, M., Mongelli, M., Papaleo, G.: DNS tunneling detection through statistical fingerprints of protocol messages and machine learning. Int. J. Commun. Syst. 28(14), 1987–2002 (2015)

    Article  Google Scholar 

  9. Liu, J., Li, S., Zhang, Y., Xiao, J., Chang, P., Peng, C.: Detecting DNS tunnel through binary-classification based on behavior features. In: IEEE Trustcom/BigDataSE/ICESS, Sydney, pp. 339–346. IEEE (2017)

    Google Scholar 

  10. Ding, Y.J., Cai, W.D.: A method for HTTP-tunnel detection based on statistical features of traffic. In: 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi’an, pp. 247–250. IEEE (2011)

    Google Scholar 

  11. Piraisoody, G., Huang, C., Nandy, B., Seddigh, N.: Classification of applications in HTTP tunnels. In: 2013 IEEE 2nd International Conference on Cloud Networking (CloudNet), San Francisco, pp. 67–74. IEEE (2013)

    Google Scholar 

  12. Li, S., Yun, X., Zhang, Y.: Anomaly-based model for detecting HTTP-tunnel traffic using network behavior analysis. High Technol. Lett. 20(1), 63–69 (2014)

    Google Scholar 

  13. Mujtaba, G., Parish, D.J.: Detection of applications within encrypted tunnels using packet size distributions. In: 2009 International Conference for Internet Technology and Secured Transactions (ICITST), London, pp. 1–6. IEEE (2009)

    Google Scholar 

  14. Wang, F., Huang, L., Chen, Z., Miao, H., Yang, W.: A novel web tunnel detection method based on protocol behaviors. In: Zia, T., Zomaya, A., Varadharajan, V., Mao, M. (eds.) SecureComm 2013. LNICST, vol. 127, pp. 234–251. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-04283-1_15

    Chapter  Google Scholar 

  15. Allard, F., Dubois, R., Gompel, P., Morel, M.: Tunneling activities detection using machine learning techniques. J. Telecommun. Inf. Technol. 2011(1), 37–42 (2011)

    Google Scholar 

  16. Buczak, A.L., Guven, E.: A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun. Surv. Tutor. 18(2), 1153–1176 (2016)

    Article  Google Scholar 

  17. Mishra, P., Varadharajan, V., Tupakula, U., Pilli, E.S.: A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Commun. Surv. Tutor. 21(1), 686–728 (2018)

    Article  Google Scholar 

  18. Wang, T.S., Lin, H.T., Cheng, W.T., Chen, C.Y.: DBod: Clustering and detecting DGA-based botnets using DNS traffic analysis. Comput. Secur. 64, 1–15 (2017)

    Article  Google Scholar 

  19. Khehra, G., Sofat, S.: BotScoop: scalable detection of DGA based botnets using DNS traffic. In: 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Bangalore, pp. 1–6. IEEE (2018)

    Google Scholar 

  20. Alexa Top 1 Million Sites. http://www.alexa.com/topsites. Accessed 20 Jan 2019

  21. 360 Netlab Open Data DGA. https://data.netlab.360.com/dga/. Accessed 20 Jan 2019

  22. Jing, X., Yan, Z., Pedrycz, W.: Security data collection and data analytics in the Internet: a survey. IEEE Commun. Surv. Tutor. 21(1), 586–618 (2019)

    Article  Google Scholar 

  23. Lin, H., Yan, Z., Fu, Y.: Adaptive security-related data collection with context awareness. J. Netw. Comput. Appl. 126, 88–103 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This work is sponsored by the National Key Research and Development Program of China (Grant 2016YFB0800700), the National Natural Science Foundation of China (Grants 61672410 and U1536202), the Academy of Finland (Grants 308087 and 314203), the open grant of the Tactical Data Link Lab of the 20th Research Institute of China Electronics Technology Group Corporation (grant No. CLDL-20182119), the Key Lab of Information Network Security, Ministry of Public Security (Grant C18614).

Author information

Authors and Affiliations

  1. State Key Lab of ISN, School of Cyber Engineering, Xidian University, Xi’an, 710071, China

    Huaqing Lin, Gao Liu & Zheng Yan

  2. Department of Communications and Networking, Aalto University, 02150, Espoo, Finland

    Zheng Yan

Authors
  1. Huaqing Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zheng Yan .

Editor information

Editors and Affiliations

  1. Guangzhou University, Guangzhou, China

    Guojun Wang

  2. Huazhong University of Science and Technology, Wuhan, China

    Jun Feng

  3. Fordham University, New York City, NY, USA

    Md Zakirul Alam Bhuiyan

  4. University of New Brunswick, Fredericton, NB, Canada

    Rongxing Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lin, H., Liu, G., Yan, Z. (2019). Detection of Application-Layer Tunnels with Rules and Machine Learning. In: Wang, G., Feng, J., Bhuiyan, M., Lu, R. (eds) Security, Privacy, and Anonymity in Computation, Communication, and Storage. SpaCCS 2019. Lecture Notes in Computer Science(), vol 11611. Springer, Cham. https://doi.org/10.1007/978-3-030-24907-6_33

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-24907-6_33

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-24906-9

  • Online ISBN: 978-3-030-24907-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation