Abstract
Protocol’s dormant behavior are the behaviors that cannot be observed during dynamic analysis, they can only be triggered under particular conditions. In the last few years, several approaches have been proposed to detect anomalous behaviors in malwares, but protocol’s dormant behaviors are rare studied due to the large invisibility and variability. This paper presents a novel approach to the analysis of protocol’s instruction sequences that uses a novel instruction sequences clustering technique to mine dormant behaviors in unknown protocols. Our technique provides a more precise solutions to mine unknown protocol’s dormant behaviors, and in addition, it is able to analyze encryption protocols and mine the behavior instruction sequences protected by obfuscation technologies. Experimental results show that the present method can mining the protocol’s dormant behaviors accurately and effectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
L. C. Li Xiang-Dong, “A survey on methods of automatic protocol reverse engineering[C],” Proceedings of the 2011 Seventh International Conference on Computational Intelligence and Security., pp. 685-689, 2011.
D. V. B. D. B. Hoang, “Multi-layer security analysis and experimentation of high speed protocol data transfer for GRID,” International Journal of Grid and Utility Computing, vol. 3, pp. 81-88, 2012.
W. Chang, A. Mohaisen, A. Wang, and S. Chen, “Measuring Botnets in the Wild: Some New Trends,” presented at the Proceedings of the 10th ACM Symposium on Information, Computer and Communications Security, Singapore, Republic of Singapore, 2015.
L.-z. G. Ying WANGa, Zhong-xian LIb, Yi-xian YANGa, “Protocol reverse engineering through dynamic and static binary analysis,” The Journal of China Universities of Posts and Telecommunications, vol. 20, pp. 75–79, 2013.
J. Narayan, S. K. Shukla, and T. C. Clancy, “A Survey of Automatic Protocol Reverse Engineering Tools,” ACM Comput. Surv., vol. 48, pp. 1-26, 2015.
J. L. S. Yu, “Position-based automatic reverse engineering of network protocols,” Journal of Network & Computer Applications, vol. 3, pp. 1070-1077, 2013.
Y. W. Meijian Li, Shangjie Jin, Peidai Xie, “Reverse extraction of protocol model from network applications,” International Journal of Internet Protocol Technology, vol. 7, pp. 228 - 245, 2013.
D. C. C. Mansour, “Security challenges in the internet of things,” International Journal of Space-Based and Situated Computing, vol. 5, pp. 141-149, 2015.
S. K. T. G. X. F. B. L. C. Chen, “Optimizing Communication for Multi-Join Query Processing in Cloud Data Warehouses,” International Journal of High Performance Computing and Networking, vol. 4, pp. 113-130, 2015.
M. Ficco, “Security Event Correlation Approach for Cloud Computing,” International Journal of High Performance Computing and Networking, vol. 3, pp. 173-185, 2013.
W. Q.-Y. Zhang Zhao, Tang Wen, “Survey of mining protocol specifications[J],” Computer Engineering and Applications., 2013.
M. Fanzhi Meng, Yuan Liu, Chunrui Zhang, Tong Li “Inferring protocol state machine for binary communication protocol[C],” Advanced Research and Technology in Industry Applications (WARTIA), 2014., pp. 870 - 874 2014
J. Newsome, D. Brumley, J. Franklin, and D. Song, “Replayer: automatic protocol replay by binary analysis,” presented at the Proceedings of the 13th ACM conference on Computer and communications security, Alexandria, Virginia, USA, 2006.
B. Anderson, C. Storlie, and T. Lane, “Improving malware classification: bridging the static/dynamic gap,” presented at the Proceedings of the 5th ACM workshop on Security and artificial intelligence, Raleigh, North Carolina, USA, 2012.
Z. L. X. J. D. X. X. Zhang, “Automatic protocol format reverse engineering through context-aware monitored execution,” Network & Distributed System Security Symposium, 2008, 2008.
C. Leita, M. Dacier, and F. Massicotte, “Automatic Handling of Protocol Dependencies and Reaction to 0-Day Attacks with ScriptGen Based Honeypots,” in Recent Advances in Intrusion Detection: 9th International Symposium, RAID 2006 Hamburg, Germany, September 20-22, 2006 Proceedings, D. Zamboni and C. Kruegel, Eds., ed Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 185-205.
C. L. K. M. M. Dacier, “ScriptGen: an automated script generation tool for Honeyd,” Computer Security Applications Conference, vol. 12, pp. 203-214, 2006.
P. McMinn, M. Harman, D. Binkley, and P. Tonella, “The species per path approach to SearchBased test data generation,” presented at the Proceedings of the 2006 international symposium on Software testing and analysis, Portland, Maine, USA, 2006.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Hu, YJ. (2017). Instruction Sequences Clustering and Analysis of Network Protocol’s Dormant Behaviors. In: Xhafa, F., Barolli, L., Amato, F. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2016. Lecture Notes on Data Engineering and Communications Technologies, vol 1. Springer, Cham. https://doi.org/10.1007/978-3-319-49109-7_61
Download citation
DOI: https://doi.org/10.1007/978-3-319-49109-7_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49108-0
Online ISBN: 978-3-319-49109-7
eBook Packages: EngineeringEngineering (R0)