Abstract
Efficiently and quickly identifying malware in encrypted network traffic is a major challenge. This research presents and evaluates a novel approach for sampling encrypted network traffic, called ENS-RFMC. Initially, the method employs a rule-based strategy for extracting relevant metadata features of encrypted traffic from network connections, SSL and certificates log files. Subsequently, a multi-hierarchical clustering algorithm is utilized to divide the dataset into smaller clusters, with benign clusters subsequently eliminated for streamline data processing. Classifier models such as Random Forests, XGBoost and SVM are then utilized to detect attacks and assess performance. Experimental results indicate that the proposed ENS-RFMC sampling method is effective, with the encrypted traffic intrusion detection model employing ENS-RFMC sampling exhibits enhanced accuracy and efficiency in identifying attacks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen LC, Gao S, Liu BX, et al. THS-IDPC: A three-stage hierarchical sampling method based on improved density peaks clustering algorithm for encrypted malicious traffic detection[J]. The Journal of Supercomputing, 2020, 76(9): 7489-7518
Hou J, Lu H, Liu FA, et al. Detection and countermeasure of encrypted malicious traffic: A survey[J]. Journal of Software, 2023, 35(1): 333-355
De Carné X, Mannan M. Killed by proxy: Analyzing client-end TLS interception software[C]. the 23rd Annual Network and Distributed Systems Security Symp. NDSS, 2016
Han J, Kim S, Ha J, et al. SGX-Box: Enabling visibility on encrypted traffic using a secure middlebox module[C]. the 1st ACM Asia-Pacific Workshop on Networking, 2017: 99–105
Goltzsche D, Rüsch S, Nieke M, et al. EndBox: Scalable middlebox functions using client-side trusted execution[C]. The 48th Annual IEEE/IFIP Int Conf. on Dependable Systems and Networks. 2018: 386–397
Justine S, Lan C, Popa RA, et al. BlindBox: Deep packet inspection over encrypted traffic[J]. ACM SIGCOMM Computer Communication Review, 2015, 45(4): 213-226
Lan C, Sherry J, Popa RA. Embark: Securely outsourcing middleboxes to the cloud[C]. the 13th USENIX Conf. on Networked Systems Design and Implementation. 2016: 255–273
B. Anderson, S. Paul, D. McGrew. Deciphering malware’s use of TLS (without decryption) [J]. Journal of Computer Virology and Hacking Techniques, 2018, 14(3): 195211
Shekhawat AS, Troia FD, Stamp M. Feature analysis of encrypted malicious traffic[J]. Expert Systems with Applications, 2019, 125: 130-141
Liu JY, Zeng YZ, Shi JY, et al. MalDetect: A structure of encrypted malware traffic detection[J]. Computers, Materials & Continua, 2019, 60(2): 721-739
Claffy K, Polyzos G, Braun H. Application of sampling methodologies to network traffic characterization[C]. ACM SIGCOMM Comput Commun Rev. 1993, 23(4): 194-203
He G, Hou JC. On sampling self-similar internet traffic[J]. Computer Networks, 2006, 50 (16): 2919-2936
Raspall F. Efficient packet sampling for accurate traffic measurements[J]. Computer Networks, 2012, 56(6):1667-1684
Duffield N, Lund C. Predicting resource usage and estimation accuracy in an IP flow measurement collection infrastructure[C]. the 3rd ACM SIGCOMM conference, 2003: 179–191
Su L, Yao Y, Li N, et al. Hierarchical clustering based network traffic data reduction for improving suspicious flow detection[C]. The 17th IEEE TrustCom/BigDataSE Conference,
-753.
16. Wang Z, Fok KW, Thing VLL. Machine learning for encrypted malicious traffic detection: Approaches, datasets and comparative study[J]. Computers & security, 2022,113:102542
Stratosphere IPS. Malware Capture Facility Project. URL https://www.stratosphereips.org/datasets-malware
A source for pcap files and malware samples, 2013. Retrieved March 13, 2020, from https://www.malware-traffic-analysis.net/
18. Chen LC, Gao S, Liu BX. An improved density peaks clustering algorithm based on grid screening and mutual neighborhood degree for network anomaly detection[J]. Scientific Reports, 2022,12(1):1-14
19. Lu Y, Chai S, Suo Y, et al. Intrusion detection for industrial internet of things based on deep learning[J]. Neurocomputing, 2024, 564(7): 126886
20. Seyedi SA, Lotfi A, Moradi P, et al. Dynamic graph-based label propagation for density peaks clustering[J]. Expert Syst Appl, 2019, 115: 314-328
21. Ding S, Li C, Xu X, et al. A sampling-based density peaks clustering algorithm for large-scale data[J]. Pattern Recognition, 2023, 136: 109238
Acknowledgement
This research has been supported by the National Key Research and Development Program of China (No.2023YFB2603800), the National Statistical Science Research Project of China National Bureau of Statistics (No. 2022LY005), Research Funds for Key Laboratory of Network Assessment Technology of Chinese Academy of Sciences (No. KFKT2022–003), General Research Project of China University of Labor Relations (No. 23XYJS016), Teaching Reform Project of China University of Labor Relations (No. JG24037). This research was also partially supported by Key Laboratory of Network Assessment Technology, Chinese Academy of Sciences and Beijing Key Laboratory of Network Security and Protection Technology.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, LC., Gao, S., Wei, ZX., Liu, BX., Zhang, XY. (2025). ENS-RFMC: An Encrypted Network Traffic Sampling Method Based on Rule-Based Feature Extraction and Multi-hierarchical Clustering for Intrusion Detection. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15324. Springer, Cham. https://doi.org/10.1007/978-3-031-78383-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-78383-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78382-1
Online ISBN: 978-3-031-78383-8
eBook Packages: Computer ScienceComputer Science (R0)