skip to main content
survey

A Survey on Advanced Persistent Threat Detection: A Unified Framework, Challenges, and Countermeasures

Published: 11 November 2024 Publication History

Abstract

In recent years, frequent Advanced Persistent Threat (APT) attacks have caused disastrous damage to critical facilities, leading to severe information leakages, economic losses, and even social disruptions. Via sophisticated, long-term, and stealthy network intrusions, APT attacks are often beyond the capabilities of traditional intrusion detection methods. Existing methods employ various techniques to enhance APT detection at different stages, but this makes it difficult to fairly and objectively evaluate the capability, value, and orthogonality of available techniques. Overly focusing on hardening specific APT detection stages cannot address some essential challenges from a global perspective, which would result in severe consequences. To holistically tackle this problem and explore effective solutions, we abstract a unified framework that covers the complete process of APT attack detection, with standardized summaries of state-of-the-art solutions and analysis of feasible techniques. Further, we provide an in-depth discussion of the challenges and countermeasures faced by each component of the detection framework. In addition, we comparatively analyze public datasets and outline the capability criteria to provide a reference for standardized evaluations. Finally, we discuss insights into potential areas for future research.

References

[1]
Manar Abu Talib, Qassim Nasir, Ali Bou Nassif, Takua Mokhamed, Nafisa Ahmed, and Bayan Mahfood. 2022. APT beaconing detection: A systematic review. Comput. Secur. 122 (2022), 24 pages.
[2]
Umut Acar, Peter Buneman, James Cheney, Jan Van Den Bussche, Natalia Kwasnikowska, and Stijn Vansummeren. 2010. A graph model of data and workflow provenance. In Proceedings of the 2nd Conference on Theory and Practice of Provenance (TAPP’10). USENIX Association, USA, 8.
[3]
Atif Ahmad, Jeb Webb, Kevin C. Desouza, and James Boorman. 2019. Strategically-motivated advanced persistent threat: Definition, process, tactics and a disinformation model of counterattack. Comput. Secur. 86 (2019), 402–418.
[4]
Abdulellah Alsaheel, Yuhong Nan, Shiqing Ma, Le Yu, Gregory Walkup, Z Berkay Celik, Xiangyu Zhang, and Dongyan Xu. 2021. ATLAS: A sequence-based learning approach for attack investigation. In Proceedings of the 30th USENIX Conference on Security Symposium (USENIX Security’21). USENIX Association, Anaheim, CA, 3005–3022. https://www.usenix.org/system/files/sec21summer_alsaheel.pdf
[5]
Adel Alshamrani, Sowmya Myneni, Ankur Chowdhary, and Dijiang Huang. 2019. A survey on advanced persistent threats: Techniques, solutions, challenges, and research opportunities. IEEE Commun. Surv. Tutor. 21, 2 (2019), 1851–1877.
[6]
Md Monowar Anjum, Shahrear Iqbal, and Benoit Hamelin. 2021. Analyzing the usefulness of the DARPA OpTC dataset in cyber threat detection research. In Proceedings of the 26th ACM Symposium on Access Control Models and Technologies (SACMAT’21). Association for Computing Machinery, New York, NY, 27–32.
[7]
Asad Arfeen, Saad Ahmed, Muhammad Asim Khan, and Syed Faraz Ali Jafri. 2021. Endpoint detection & response: A malware identification solution. In Proceedings of the International Conference on Cyber Warfare and Security (ICCWS’21). IEEE, USA, 1–8.
[8]
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fus. 58 (2020), 82–115.
[9]
Mathieu Barre, Ashish Gehani, and Vinod Yegneswaran. 2019. Mining data provenance to detect advanced persistent threats. In Proceedings of the 11th USENIX Conference on Theory and Practice of Provenance (TAPP’19). USENIX Association, Anaheim, CA, 6.
[10]
BITSIGHT. 2020. The Financial Impact of SolarWinds Breach. Retrieved from https://www.bitsight.com/blog/bitsight-analysis-of-solarwinds-orion-part-1-prevalence
[11]
Chianson Threat Intelligence Center. 2023. Global Advanced Persistent Threats. Retrieved from https://www.qianxin.com/threat/reportdetail?report_id=295.
[12]
Zhiyan Chen, Jinxin Liu, Yu Shen, Murat Simsek, Burak Kantarci, Hussein T. Mouftah, and Petar Djukic. 2022. Machine learning-enabled IoT security: Open issues and challenges under advanced persistent threats. Comput. Surv. 55, 5, Article 105 (2022), 37 pages.
[13]
Nisha Dhanraj Dewani, Zubair Ahmed Khan, Aarushi Agarwal, Mamta Sharma, and Shaharyar Asaf Khan. 2022. Handbook of Research on Cyber Law, Data Protection, and Privacy. IGI Global, Hershey, PA, USA.
[14]
Hailun Ding, Juan Zhai, Dong Deng, and Shiqing Ma. 2023. The case for learned provenance graph storage systems. In Proceedings of the 32nd USENIX Conference on Security Symposium (USENIX Security’23). USENIX Association, Anaheim, CA.
[15]
Feng Dong, Liu Wang, Xu Nie, Fei Shao, Haoyu Wang, Ding Li, Xiapu Luo, and Xusheng Xiao. 2023. DISTDET: A cost-effective distributed cyber threat detection system. In Proceedings of the 32nd USENIX Conference on Security Symposium (USENIX Security’23). USENIX Association, 6575–6592.
[16]
Birhanu Eshete, Rigel Gjomemo, Md Nahid Hossain, Sadegh Momeni, R. Sekar, Scott Stoller, V. N. Venkatakrishnan, and Junao Wang. 2016. Attack analysis results for adversarial engagement 1 of the darpa transparent computing program. arXiv:1610.06936. Retrieved from https://arxiv.org/abs/1610.06936
[17]
Pengcheng Fang, Peng Gao, Changlin Liu, Erman Ayday, Kangkook Jee, Ting Wang, Yanfang (Fanny) Ye, Zhuotao Liu, and Xusheng Xiao. 2022. Back-propagating system dependency impact for attack investigation. In Proceedings of the 31st USENIX Conference on Security Symposium (USENIX Security’22). USENIX Association, 2461–2478.
[18]
Peng Fei, Zhou Li, Zhiying Wang, Xiao Yu, Ding Li, and Kangkook Jee. 2021. SEAL: Storage-efficient causality analysis on enterprise logs with query-friendly compression. In Proceedings of the 30th USENIX Conference on Security Symposium (USENIX Security’21). USENIX Association, 2987–3004.
[19]
FiveDirections. 2021. Darpa OpTC (Darpa Operationally Transparent Cyber (OpTC) Dataset). Retrieved from https://paperswithcode.com/dataset/darpa-optc
[20]
Center for Internet Security. 2021. The SolarWinds Cyber-Attack: What You Need to Know. Retrieved from https://www.cisecurity.org/solarwinds
[21]
Anmin Fu, Weijia Ding, Boyu Kuang, Qianmu Li, Willy Susilo, and Yuqing Zhang. 2022. FH-CFI: Fine-grained hardware-assisted control flow integrity for ARM-based IoT devices. Comput. Secur. 116 (2022), 12 pages.
[22]
Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. 2018. Ai2: Safety and robustness certification of neural networks with abstract interpretation. In Proceedings of the IEEE Symposium on Security and Privacy (SP’18). IEEE Computer Society, Los Alamitos, CA, 3–18.
[23]
Joshua Glasser and Brian Lindauer. 2013. Bridging the gap: A pragmatic approach to generating insider threat data. In Proceedings of the IEEE Security and Privacy Workshops (SPW’13). IEEE Computer Society, Los Alamitos, CA, 98–104.
[24]
Solomon W. Golomb, Basil Gordon, and Lloyd R. Welch. 1958. Comma-free codes. Can. J. Math. 10 (1958), 202–209.
[25]
Mohit Goyal, Kedar Tatwawadi, Shubham Chandak, and Idoia Ochoa. 2019. DeepZip: Lossless data compression using recurrent neural networks. In Proceedings of the Data Compression Conference (DCC’19), Snowbird, UT, USA, 575–575.
[26]
D. Gunning. 2017. Explainable Artificial Intelligence (xAI). Technical Report. Defense Advanced Research Projects Agency (DARPA).
[27]
David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. 2019. XAI—explainable artificial intelligence. Sci. Robot. 4, 37 (2019), eaay7120.
[28]
Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. 2018. LEMNA: Explaining deep learning based security applications. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’18). Association for Computing Machinery, New York, NY, 364–379.
[29]
Steffen Haas, Robin Sommer, and Mathias Fischer. 2020. Zeek-osquery: Host-network correlation for advanced monitoring and intrusion detection. In Proceedings of the 35th IFIP TC 11 International Conference ICT Systems Security and Privacy Protection (SEC’20). Vol. 580. Springer International, 248–262.
[30]
Xueyuan Han, Thomas Pasquier, Adam Bates, James Mickens, and Margo Seltzer. 2020. UNICORN: Runtime provenance-based detector for advanced persistent threats. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’20). Internet Society, The Internet Society, Reston, VA, 1–18.
[31]
Xueyuan Han, Thomas Pasquier, and Margo Seltzer. 2018. Provenance-based intrusion detection: Opportunities and challenges. In Proceedings of the 10th USENIX Conference on Theory and Practice of Provenance (TaPP’18). USENIX Association, USA, 3.
[32]
Wajih Ul Hassan, Adam Bates, and Daniel Marino. 2020. Tactical provenance analysis for endpoint detection and response systems. In Proceedings of the IEEE Symposium on Security and Privacy (SP’20). IEEE Computer Society, Los Alamitos, CA, 1172–1189.
[33]
Wajih Ul Hassan, Shengjian Guo, Ding Li, Zhengzhang Chen, Kangkook Jee, Zhichun Li, and Adam Bates. 2019. Nodoze: Combatting threat alert fatigue with automated provenance triage. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’19). The Internet Society, Reston, VA.
[34]
Wajih Ul Hassan, Ding Li, Kangkook Jee, Xiao Yu, Kexuan Zou, Dawei Wang, Zhengzhang Chen, Zhichun Li, Junghwan Rhee, Jiaping Gui, and Adam Bates. 2020. This is why we can’t cache nice things: Lightning-fast threat hunting using suspicion-based hierarchical storage. In Annual Computer Security Applications Conference (ACSAC’20). Association for Computing Machinery, New York, NY, 165–178.
[35]
Wajih Ul Hassan, Mohammad Ali Noureddine, Pubali Datta, and Adam Bates. 2020. OmegaLog: High-fidelity attack investigation via transparent multi-layer log analysis. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’20). The Internet Society, Reston, VA.
[36]
Pinjia He, Jieming Zhu, Shilin He, Jian Li, and Michael R Lyu. 2017. Towards automated log parsing for large-scale log data analysis. IEEE Trans. Depend. Secure Comput. 15, 6 (2017), 931–944.
[37]
Shilin He, Pinjia He, Zhuangbin Chen, Tianyi Yang, Yuxin Su, and Michael R. Lyu. 2021. A survey on automated log analysis for reliability engineering. ACM Comput. Surv. 54, 6 (2021), 1–37.
[38]
Md Nahid Hossain, Sadegh M. Milajerdi, Junao Wang, Birhanu Eshete, Rigel Gjomemo, R. Sekar, Scott D. Stoller, and V. N. Venkatakrishnan. 2017. SLEUTH: Real-time attack scenario reconstruction from COTS audit data. In Proceedings of the 26th USENIX Conference on Security Symposium (USENIX Security’17). USENIX Association, Berkeley, CA, 487–504.
[39]
Md Nahid Hossain, Sanaz Sheikhi, and R. Sekar. 2020. Combating dependence explosion in forensic analysis using alternative tag propagation semantics. In Proceedings of the IEEE Symposium on Security and Privacy (SP’20). IEEE Computer Society, Los Alamitos, CA, 1139–1155.
[40]
Md Nahid Hossain, Junao Wang, R. Sekar, and Scott D. Stoller. 2018. Dependence-preserving data compaction for scalable forensic analysis. In Proceedings of the 27th USENIX Conference on Security Symposium (USENIX Security’18). USENIX Association, Berkeley, CA, 1723–1740.
[41]
Hassaan Irshad, Gabriela Ciocarlie, Ashish Gehani, Vinod Yegneswaran, Kyu Hyung Lee, Jignesh Patel, Somesh Jha, Yonghwi Kwon, Dongyan Xu, and Xiangyu Zhang. 2021. Trace: Enterprise-wide provenance tracking for real-time apt detection. IEEE Trans. Inf. Forens. Secur. 16 (2021), 4363–4376.
[42]
Isaca. 2013. Responding to Targeted Cyberattacks. ISA.
[43]
Yang Ji, Sangho Lee, Mattia Fazzini, Joey Allen, Evan Downing, Taesoo Kim, Alessandro Orso, and Wenke Lee. 2018. Enabling refinable cross-host attack investigation with efficient data flow tagging and tracking. In Proceedings of the 27th USENIX Conference on Security Symposium (USENIX Security 18). USENIX Association, Berkeley, CA, 1705–1722.
[44]
Herbert Jordan, Bernhard Scholz, and Pavle Subotić. 2016. Soufflé: On synthesis of program analyzers. In Proceedings of the 28th International Conference on Computer Aided Verification. Springer, 422–430.
[45]
Leila Karimi, Maryam Aldairi, James Joshi, and Mai Abdelhakim. 2022. An automatic attribute-based access control policy extraction from access logs. IEEE Trans. Depend. Sec. Comput. 19, 4 (2022), 2304–2317.
[46]
Mahmoud Khonji, Youssef Iraqi, and Andrew Jones. 2013. Phishing detection: A literature survey. IEEE Commun. Surv. Tutor. 15, 4 (2013), 2091–2121.
[47]
Boyu Kuang, Anmin Fu, Lu Zhou, Willy Susilo, and Yuqing Zhang. 2020. DO-RA: Data-oriented runtime attestation for IoT devices. Comput. Secur. 97 (2020), 11 pages.
[48]
Yonghwi Kwon, Fei Wang, Weihang Wang, Kyu Hyung Lee, Wen-Chuan Lee, Shiqing Ma, Xiangyu Zhang, Dongyan Xu, Somesh Jha, Gabriela Ciocarlie, Ashish Gehani, and Vinod Yegneswaran. 2018. MCI: Modeling-based causality inference in audit logging for attack investigation. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’18), Vol. 2. The Internet Society, Reston, VA, 4.
[49]
Yonghwi Kwon, Weihang Wang, Jinho Jung, Kyu Hyung Lee, and Roberto Perdisci. 2021. C2SR: Cybercrime scene reconstruction for post-mortem forensic analysis. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’21). The Internet Society, Reston, VA.
[50]
Kyu Hyung Lee, Xiangyu Zhang, and Dongyan Xu. 2013. High accuracy attack provenance via binary-based execution partition. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’13), Vol. 16. The Internet Society, Reston, VA.
[51]
Teng Li, Ya Jiang, Chi Lin, Mohammad S. Obaidat, Yulong Shen, and Jianfeng Ma. 2023. DeepAG: Attack graph construction and threats prediction with bi-directional deep learning. IEEE Trans. Depend. Secure Comput. 20, 1 (2023), 740–757.
[52]
Zhenyuan Li, Qi Alfred Chen, Runqing Yang, Yan Chen, and Wei Ruan. 2021. Threat detection and investigation with system-level provenance graphs: A survey. Comput. Secur. 106 (2021), 16 pages.
[53]
Brian Lindauer, Joshua Glasser, Mitch Rosen, and Kurt C. Wallnau. 2014. Generating test data for insider threat detectors. J. Wireless Mobile Netw. Ubiq. Comput. Depend. Appl. 5, 2 (2014), 80–94.
[54]
Fucheng Liu, Yu Wen, Dongxue Zhang, Xihe Jiang, Xinyu Xing, and Dan Meng. 2019. Log2vec: A heterogeneous graph embedding based approach for detecting cyber threats within enterprise. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’19). Association for Computing Machinery, New York, NY, 1777–1794.
[55]
Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). Curran Associates Inc., Red Hook, NY, 4768–4777.
[56]
James Sadowski Maddie Stone. 2024. A Review of Aero-day In-the-wild Exploits in 2023. Retrieved from https://blog.google/technology/safety-security/a-review-of-zero-day-in-the-wild-exploits-in-2023/.
[57]
Mandiant. 2022. Zero Tolerance: More Zero-Days Exploited in 2021 Than Ever Before. Retrieved from https://cloud.google.com/blog/topics/threat-intelligence/zero-days-exploited-2021/.
[58]
Mandiant. 2023. Move, Patch, Get Out the Way: 2022 Zero-Day Exploitation Continues at an Elevated Pace. Retrieved from https://cloud.google.com/blog/topics/threat-intelligence/zero-days-exploited-2022/.
[59]
Emaad Manzoor, Sadegh M. Milajerdi, and Leman Akoglu. 2016. Fast memory-efficient anomaly detection in streaming heterogeneous graphs. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16). Association for Computing Machinery, New York, NY, 1035–1044.
[60]
Sadegh M. Milajerdi, Birhanu Eshete, Rigel Gjomemo, and V. N. Venkatakrishnan. 2019. POIROT: Aligning attack behavior with kernel audit records for cyber threat hunting. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS’19). Association for Computing Machinery, New York, NY, 1795–1812.
[61]
Sadegh M. Milajerdi, Rigel Gjomemo, Birhanu Eshete, Ramachandran Sekar, and V. N. Venkatakrishnan. 2019. Holmes: Real-time apt detection through correlation of suspicious information flows. In Proceedings of the IEEE Symposium on Security and Privacy (SP’19). IEEE Computer Society, Los Alamitos, CA, 1137–1152.
[62]
Eduardo Mosqueira-Rey, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal. 2023. Human-in-the-loop machine learning: A state of the art. Artif. Intell. Rev. 56, 4 (2023), 3005–3054.
[63]
Dr Nour Moustafa. 2021. The UNSW-NB15 Dataset. Retrieved from https://research.unsw.edu.au/projects/unsw-nb15-dataset.
[64]
Nour Moustafa and Jill Slay. 2015. UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In Proceedings of the Military Communications and Information Systems Conference (MilCIS’15). IEEE, Los Alamitos, CA, 1–6.
[65]
Nour Moustafa and Jill Slay. 2016. The evaluation of network anomaly detection systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set. Inf. Secur. J.: Global Perspect. 25, 1-3 (2016), 18–31.
[66]
The Hacker News. 2023. Researchers Uncover Years-Long Cyber Espionage on Foreign Embassies in Belarus. https://thehackernews.com/2023/08/researchers-uncover-decade-long-cyber.html.
[67]
University of California. 2015. KDD Cup 1999 Data. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html.
[68]
University of New Brunswick. 2009. NSL-KDD dataset. https://www.unb.ca/cic/datasets/nsl.html.
[69]
Bofeng Pan, Natalia Stakhanova, and Suprio Ray. 2023. Data provenance in security and privacy. Comput. Surv. 55, 14s, Article 323 (Jul.2023), 35 pages.
[70]
Cheolhee Park, Jonghoon Lee, Youngsoo Kim, Jong-Geun Park, Hyunjin Kim, and Dowon Hong. 2023. An enhanced AI-based network intrusion detection system using generative adversarial networks. IEEE IoT J. 10, 3 (2023), 2330–2345.
[71]
Thomas F.J.-M. Pasquier, Jatinder Singh, David Eyers, and Jean Bacon. 2015. CamFlow: Managed data-sharing for cloud services. IEEE Trans. Cloud Comput. 5, 3 (2015), 472–484.
[72]
Qianxin. 2024. Global Advanced Persistent Threats 2023 Annual Report. Retrieved from https://www.qianxin.com/threat/reportdetail?report_id=310.
[73]
Przemek Shem Radzikowski. 2015. Cybersecurity: Origins of the Advanced Persistent Threat (APT). Retrieved from https://drshem.com/2015/10/08/cybersecurity-origins-of-the-advanced-persistent-threat-apt/.
[74]
Md Rayhanur Rahman, Rezvan Mahdavi Hezaveh, and Laurie Williams. 2023. What are the attackers doing now? Automating cyberthreat intelligence extraction from text on pace with the changing threat landscape: A survey. Comput. Surv. 55, 12 (2023), 1–36.
[75]
ranok. 2020. Transparent- computing. https://www.darpa.mil/program/transparent-computing.
[76]
Sathyanarayanan Revathi and A. Malathi. 2013. A detailed analysis on NSL-KDD dataset using various machine learning techniques for intrusion detection. Int. J. Eng. Res. Technol. 2, 12 (2013), 1848–1853.
[77]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ”Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16). Association for Computing Machinery, New York, NY, 1135–1144.
[78]
Artur Rot and Boguslaw Olszewski. 2017. Advanced persistent threats attacks in cyberspace. threats, vulnerabilities, methods of protection. In FedCSIS (Position Papers). 113–117.
[79]
Hassan Saif, Yulan He, Miriam Fernandez, and Harith Alani. 2016. Contextual semantics for sentiment analysis of Twitter. Inf. Process. Manage. 52, 1 (2016), 5–19. Emotion and Sentiment in Social and Expressive Media.
[80]
Nikolai Samteladze and Ken Christensen. 2012. DELTA: Delta encoding for less traffic for apps. In Proceedings of the 37th Annual IEEE Conference on Local Computer Networks. IEEE, Los Alamitos, CA, 212–215.
[81]
Matthias J. Sax. 2018. Apache Kafka. Springer International Publishing, Cham, 1–8.
[82]
Khalid Sayood. 2018. Introduction to Data Compression (Fifth Edition). Morgan Kaufmann.
[83]
Daniel Schlette, Marco Caselli, and Günther Pernul. 2021. A comparative study on cyber threat intelligence: The security incident response perspective. IEEE Commun. Surv. Tutor. 23, 4 (2021), 2525–2556.
[84]
Iman Sharafaldin, Arash Habibi Lashkari, and Ali A. Ghorbani. 2018. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the International Conference on Information Systems Security and Privacy (ICISSP’18), Vol. 1. 108–116.
[85]
Yun Shen and Gianluca Stringhini. 2019. ATTACK2VEC: Leveraging temporal word embeddings to understand the evolution of cyberattacks. In Proceedings of the 28th USENIX Conference on Security Symposium (USENIX Security’19). USENIX Association, Anaheim, CA, 905–921.
[86]
Branka Stojanović, Katharina Hofer-Schmitz, and Ulrike Kleb. 2020. APT datasets and attack modeling for automated detection methods: A review. Comput. Secur. 92 (2020), 19 pages.
[87]
Blake E. Strom, Andy Applebaum, Doug P. Miller, Kathryn C. Nickels, Adam G. Pennington, and Cody B. Thomas. 2018. MITRE ATT and CK (Trademark): Design and Philosophy. Technical Report. MITRE CORP MCLEAN VA.
[88]
Sridevi Subbiah, Kalaiarasi Sonai Muthu Anbananthen, Saranya Thangaraj, Subarmaniam Kannan, and Deisy Chelliah. 2022. Intrusion detection technique in wireless sensor network using grid search random forest with Boruta feature selection algorithm. J. Commun. Netw. 24, 2 (2022), 264–273.
[89]
Nan Sun, Ming Ding, Jiaojiao Jiang, Weikang Xu, Xiaoxing Mo, Yonghang Tai, and Jun Zhang. 2023. Cyber threat intelligence mining for proactive cybersecurity defense: A survey and new perspectives. IEEE Commun. Surv. Tutor. 25, 3 (Jul.2023), 1748–1774.
[90]
Yutao Tang, Ding Li, Zhichun Li, Mu Zhang, Kangkook Jee, Xusheng Xiao, Zhenyu Wu, Junghwan Rhee, Fengyuan Xu, and Qun Li. 2018. NodeMerge: Template based efficient data reduction for big-data causality analysis. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’18). Association for Computing Machinery, New York, NY, 1324–1337.
[91]
Ankit Thakkar and Ritika Lohiya. 2022. A survey on intrusion detection system: Feature selection, model, performance measures, application perspective, challenges, and future research directions. Artif. Intell. Rev. 55, 1 (Jan.2022), 453–563.
[92]
Benjamin E. Ujcich, Samuel Jero, Richard Skowyra, Adam Bates, William H. Sanders, and Hamed Okhravi. 2021. Causal analysis for software-defined networking attacks. In Proceedings of the 30th USENIX Conference on Security Symposium (USENIX Security’21). USENIX Association, Berkeley, CA, 3183–3200.
[93]
Gaël Varoquaux and Veronika Cheplygina. 2022. Machine learning for medical imaging: Methodological failures and recommendations for the future. NPJ Digit. Med. 5, 1 (2022), 48.
[94]
Jai Vijayan. 2018. Silence APT group broadens attacks on banks, gets more dangerous. Dark Read. 5 (2018).
[95]
J. Vukalović and D. Delija. 2015. Advanced persistent threats - detection and defense. In Proceedings of the 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE, Los Alamitos, CA, 1324–1330.
[96]
Cynthia Wagner, Alexandre Dulaunoy, Gérard Wagener, and Andras Iklody. 2016. MISP: The design and implementation of a collaborative threat intelligence sharing platform. In Proceedings of the 2016 ACM on Workshop on Information Sharing and Collaborative Security (WISCS’16). Association for Computing Machinery, New York, NY, 49–56.
[97]
Thomas D. Wagner, Khaled Mahbub, Esther Palomar, and Ali E. Abdallah. 2019. Cyber threat intelligence sharing: Survey and research directions. Comput. Secur. 87 (2019), 13 pages.
[98]
Shang Wang, Yansong Gao, Anmin Fu, Zhi Zhang, Yuqing Zhang, Willy Susilo, and Dongxi Liu. 2023. CASSOCK: Viable backdoor attacks against DNN in the wall of source-specific backdoor defences. In Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security (ASIA CCS’23). Association for Computing Machinery, New York, NY, 938–950.
[99]
Su Wang, Zhiliang Wang, Tao Zhou, Hongbin Sun, Xia Yin, Dongqi Han, Han Zhang, Xingang Shi, and Jiahai Yang. 2022. THREATRACE: Detecting and tracing host-based threats in node level through provenance graph learning. IEEE Trans. Inf. Forens. Secur. 17 (2022), 3972–3987.
[100]
Feng Wei, Hongda Li, Ziming Zhao, and Hongxin Hu. 2023. XNIDS: Explaining deep learning-based network intrusion detection systems for active intrusion responses. In Proceedings of the 32nd USENIX Conference on Security Symposium (USENIX Security’23). USENIX Association, Berkeley, CA, Article 243, 18 pages.
[101]
Yafeng Wu, Yulai Xie, Xuelong Liao, Pan Zhou, Dan Feng, Lin Wu, Xuan Li, Avani Wildani, and Darrell Long. 2023. Paradise: Real-time, generalized, and distributed provenance-based intrusion detection. IEEE Trans. Depend. Secure Comput. 20, 2 (2023), 1624–1640.
[102]
Chunlin Xiong, Tiantian Zhu, Weihao Dong, Linqi Ruan, Runqing Yang, Yueqiang Cheng, Yan Chen, Shuai Cheng, and Xutong Chen. 2022. CONAN: A practical real-time APT detection system with high accuracy and efficiency. IEEE Trans. Depend. Secure Comput. 19, 1 (2022), 551–565.
[103]
Zhiqiang Xu, Pengcheng Fang, Changlin Liu, Xusheng Xiao, Yu Wen, and Dan Meng. 2022. Depcomm: Graph summarization on system audit logs for attack investigation. In Proceedings of the IEEE Symposium on Security and Privacy (SP’22). IEEE Computer Society, Los Alamitos, CA, 540–557.
[104]
Zhang Xu, Zhenyu Wu, Zhichun Li, Kangkook Jee, Junghwan Rhee, Xusheng Xiao, Fengyuan Xu, Haining Wang, and Guofei Jiang. 2016. High fidelity data reduction for big data security dependency analyses. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’16). Association for Computing Machinery, New York, NY, 504–516.
[105]
Carter Yagemann, Simon P. Chung, Brendan Saltaformaggio, and Wenke Lee. 2023. PUMM: Preventing use-after-free using execution unit partitioning. In Proceedings of the 32nd USENIX Conference on Security Symposium (USENIX Security 23). USENIX Association, Berkeley, CA, 823–840.
[106]
Carter Yagemann, Mohammad A. Noureddine, Wajih Ul Hassan, Simon Chung, Adam Bates, and Wenke Lee. 2021. Validating the integrity of audit logs against execution repartitioning attacks. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’ 21). Association for Computing Machinery, New York, NY, 3337–3351.
[107]
Fan Yang, Jiacen Xu, Chunlin Xiong, Zhou Li, and Kehuan Zhang. 2023. PROGRAPHER: An anomaly detection system based on provenance graph embedding. In Proceedings of the 32nd USENIX Conference on Security Symposium (SEC’23). USENIX Association, USA, Article 244, 18 pages.
[108]
Jian Yang, Qi Zhang, Xiaofeng Jiang, Shuangwu Chen, and Feng Yang. 2021. Poirot: Causal correlation aided semantic analysis for advanced persistent threat detection. IEEE Trans. Depend. Secure Comput. 19, 5 (082021), 3546–3563.
[109]
Keping Yu, Liang Tan, Shahid Mumtaz, Saba Al-Rubaye, Anwer Al-Dulaimi, Ali Kashif Bashir, and Farrukh Aslam Khan. 2021. Securing critical infrastructures: Deep-learning-based threat detection in IIoT. IEEE Commun. Mag. 59, 10 (2021), 76–82.
[110]
Le Yu, Shiqing Ma, Zhuo Zhang, Guanhong Tao, Xiangyu Zhang, Dongyan Xu, Vincent E. Urias, Han Wei Lin, Gabriela F. Ciocarlie, Vinod Yegneswaran, et al. 2021. ALchemist: Fusing application and audit logs for precise attack provenance without instrumentation. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’21). The Internet Society, Reston, VA.
[111]
Mohammed J. Zaki. 2001. SPADE: An efficient algorithm for mining frequent sequences. Mach. Learn. 42 (2001), 31–60.
[112]
Jun Zeng, Zheng Leong Chua, Yinfang Chen, Kaihang Ji, Zhenkai Liang, and Jian Mao. 2021. WATSON: Abstracting behaviors from audit logs via aggregation of contextual semantics. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’21). The Internet Society, Reston, VA.
[113]
Lei Zeng, Yang Xiao, and Hui Chen. 2015. Linux auditing: Overhead and adaptation. In Proceedings of the IEEE International Conference on Communications (ICC’15). IEEE, USA, 7168–7173.
[114]
Jun Zengy, Xiang Wang, Jiahao Liu, Yinfang Chen, Zhenkai Liang, Tat-Seng Chua, and Zheng Leong Chua. 2022. Shadewatcher: Recommendation-guided cyber threat analysis using system audit records. In Proceedings of the IEEE Symposium on Security and Privacy (SP’22). IEEE Computer Society, Los Alamitos, CA, 489–506.
[115]
Lefeng Zhang, Tianqing Zhu, Farookh Khadeer Hussain, Dayong Ye, and Wanlei Zhou. 2023. A game-theoretic method for defending against advanced persistent threats in cyber systems. IEEE Trans. Inf. Forens. Secur. 18 (2023), 1349–1364.
[116]
Chunyi Zhou, Yansong Gao, Anmin Fu, Kai Chen, Zhiyang Dai, Zhi Zhang, Minhui Xue, and Yuqing Zhang. 2023. PPA: Preference profiling attack against federated learning. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS’23). The Internet Society, Reston, VA.
[117]
Tiantian Zhu, Jiayu Wang, Linqi Ruan, Chunlin Xiong, Jinkai Yu, Yaosheng Li, Yan Chen, Mingqi Lv, and Tieming Chen. 2021. General, efficient, and real-time data compaction strategy for apt forensic analysis. IEEE Trans. Inf. Forens. Secur. 16 (042021), 3312–3325.
[118]
Tiantian Zhu, Jinkai Yu, Chunlin Xiong, Wenrui Cheng, Qixuan Yuan, Jie Ying, Tieming Chen, Jiabo Zhang, Mingqi Lv, Yan Chen, Ting Wang, and Yuan Fan. 2023. APTSHIELD: A stable, efficient and real-time APT detection system for linux hosts. IEEE Trans. Depend. Secure Comput. 20, 6 (2023), 5247–5264.
[119]
Michael Zipperle, Florian Gottwalt, Elizabeth Chang, and Tharam Dillon. 2022. Provenance-based intrusion detection systems: A survey. ACM Comput. Surv. 55, 7, Article 135 (Dec.2022), 36 pages.

Index Terms

  1. A Survey on Advanced Persistent Threat Detection: A Unified Framework, Challenges, and Countermeasures

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Computing Surveys
        ACM Computing Surveys  Volume 57, Issue 3
        March 2025
        984 pages
        EISSN:1557-7341
        DOI:10.1145/3697147
        Issue’s Table of Contents

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 11 November 2024
        Online AM: 16 October 2024
        Accepted: 25 September 2024
        Revised: 15 September 2024
        Received: 21 January 2024
        Published in CSUR Volume 57, Issue 3

        Check for updates

        Author Tags

        1. Advanced persistent threat
        2. intrusion detection
        3. system security
        4. causality analysis
        5. artificial intelligence

        Qualifiers

        • Survey

        Funding Sources

        • National Natural Science Foundation of China
        • Open Foundation of the State Key Laboratory of Integrated Services Networks
        • Postdoctoral Fellowship Program of CPSF

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 547
          Total Downloads
        • Downloads (Last 12 months)547
        • Downloads (Last 6 weeks)106
        Reflects downloads up to 14 Feb 2025

        Other Metrics

        Citations

        View Options

        Login options

        Full Access

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        Full Text

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media