Abdullahi Chowdhury
ABSTRACT
We have developed a novel ’Teacher-Student with human feedback’ model for Human-Artificial Intelligence (AI) collaborations in cybersecurity tasks. In our model, AI furnishes sufficient information about its decision-making process to enable human agents to provide feedback to improve the model. Our key innovations include: enhancing the interpretability of AI models by analyzing falsely detected samples using LIME and SHAP values; developing a novel posthoc explanation-based dynamic teacher-student model to address concept drift or concept shift; integrating human experts’ feedback on falsely detected samples to increase accuracy, precision, and recall values, without retraining the entire model; establishing a list of attack-based feature values for human experts to promote reproducibility. We show in experiments with real data and threat detection tasks that our model significantly improves the accuracy of existing AI algorithms for these tasks.
- Sabrina Caldwell, Penny Sweetser, Nicholas O’Donnell, Matthew J Knight, Matthew Aitchison, Tom Gedeon, Daniel Johnson, Margot Brereton, Marcus Gallagher, and David Conroy. 2022. An Agile New Research Framework for Hybrid Human-AI Teaming: Trust, Transparency, and Transferability. ACM Transactions on Interactive Intelligent Systems (TiiS) 12, 3 (2022), 1–36.Google Scholar
Digital Library
- Tristan Carrier, Princy Victor, Ali Tekeoglu, and Arash Habibi Lashkari. 2022. Detecting Obfuscated Malware using Memory Feature Engineering.. In 8th International Conference on Information Systems Security and Privacy (ICISSP 2022). Science and Technology Publications, Lda., 177–188.Google ScholarCross Ref
- Kamaluddeen Usman Danyaro, Haizatul Hafizah Hussain, Mujaheed Abdullahi, MS Liew, Lim Eu Shawn, and Mustapha Yusuf Abubakar. 2022. Development and Integration of Metocean Data Interoperability for Intelligent Operations and Automation Using Machine Learning: A Review. Applied Sciences 12, 11 (2022), 5690.Google ScholarCross Ref
- Ilhan Firat Kilincer, Fatih Ertam, and Abdulkadir Sengur. 2021. Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Computer Networks 188 (2021), 107840.Google ScholarCross Ref
- Samaneh Mahdavifar, Dima Alhadidi, and Ali A Ghorbani. 2022. Effective and efficient hybrid android malware classification using pseudo-label stacked auto-encoder. Journal of network and systems management 30 (2022), 1–34.Google ScholarDigital Library
- Jacob Nilsson, Fredrik Sandin, and Jerker Delsing. 2019. Interoperability and machine-to-machine translation model with mappings to machine learning tasks. In 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), Vol. 1. IEEE, 284–289.Google ScholarCross Ref
- Yanou Ramon, David Martens, Foster Provost, and Theodoros Evgeniou. 2020. A comparison of instance-level counterfactual explanation algorithms for behavioral and textual data: SEDC, LIME-C and SHAP-C. Advances in Data Analysis and Classification 14 (2020), 801–819.Google ScholarDigital Library
- Sagar Samtani, Murat Kantarcioglu, and Hsinchun Chen. 2020. Trailblazing the artificial intelligence for cybersecurity discipline: a multi-disciplinary research roadmap., 19 pages.Google Scholar
- Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. 2020. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 180–186.Google ScholarDigital Library
Recommendations
From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That Can Work Together with People
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing SystemsArtificial Intelligent (AI) and Machine Learning (ML) algorithms are coming out of research labs into the real-world applications, and recent research has focused a lot on Human-AI Interaction (HAI) and Explainable AI (XAI). However, Interaction is not ...
The Impact of Imperfect XAI on Human-AI Decision-Making
CSCWExplainability techniques are rapidly being developed to improve human-AI decision-making across various cooperative work settings. Consequently, previous research has evaluated how decision-makers collaborate with imperfect AI by investigating ...
Human-in-the-loop XAI-enabled vulnerability detection, investigation, and mitigation
ASE '21: Proceedings of the 36th IEEE/ACM International Conference on Automated Software EngineeringThe need for cyber resilience is increasingly important in our technology-dependent society, where computing systems, devices and data will continue to be the target of cyber attackers. Hence, we propose a conceptual framework called 'Human-in-the-Loop ...
Comments