Abstract
Weakly-supervised Anomaly Detection (AD) has achieved significant performance improvement compared to unsupervised methods by harnessing very little additional labeling information. However, most existing methods ignore anomalies in unlabeled data by simply treating the whole unlabeled set as normal; that is, they fail to resist such noise that may considerably disturb the learning process, and more importantly, they cannot extract key anomaly features from these unlabeled anomalies, which are complementary to those labeled ones. To solve this problem, a spiking reinforcement learning framework for weakly-supervised AD is proposed, named ADSD. Compared with artificial neural networks, the spiking neural network can effectively resist input perturbations due to its unique coding methods and neuronal characteristics. From this point of view, by using spiking neurons with noise filtering and threshold adaptation, as well as a multi-weight evaluation method to discover the most suspicious anomalies in unlabeled data, ADSD achieves end-to-end optimization for the utilization of a few labeled anomaly data and rare unlabeled anomalies in complex environments. The agent in ADSD has robustness and adaptability when exploring potential anomalies in the unknown space. Extensive experiments show that our method ADSD significantly outperforms four popular baselines in various environments while maintaining good robustness and generalization performance.
This work was supported in part by National Key Research and Development Program of China (2022ZD0210500), the National Natural Science Foundation of China under Grants U21A20491/62332019/61972067, and the Distinguished Young Scholars Funding of Dalian (No. 2022RJ01).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, C.C.: An Introduction to Outlier Analysis. In: Outlier Analysis, pp. 1–34. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-47578-3_1
Asrori, S.S., Wang, L., Ozawa, S.: Permissioned blockchain-based XGBoost for multi banks fraud detection. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds.) Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, 22–26 November 2022, Proceedings, Part III. LNCS, vol. 13625, pp. 683–692. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-30111-7_57
Bhuyan, M.H., Bhattacharyya, D.K., Kalita, J.K.: Network anomaly detection: methods, systems and tools. IEEE Commun. Surv. Tutorials 16(1), 303–336 (2013)
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)
Burkitt, A.N.: A review of the integrate-and-fire neuron model: I. homogeneous synaptic input. Biol. Cybern. 95, 1–19 (2006)
Ding, J., et al.: Biologically inspired dynamic thresholds for spiking neural networks. Adv. Neural. Inf. Process. Syst. 35, 6090–6103 (2022)
Friedrich, J., Urbanczik, R., Senn, W.: Spatio-temporal credit assignment in neuronal population learning. PLoS Comput. Biol. 7(6), e1002092 (2011)
Gerstner, W., Kistler, W.M., Naud, R., Paninski, L.: Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, Cambridge (2014)
Han, S., Hu, X., Huang, H., Jiang, M., Zhao, Y.: ADBench: anomaly detection benchmark. Adv. Neural. Inf. Process. Syst. 35, 32142–32159 (2022)
Hessel, M., et al.: Rainbow: combining improvements in deep reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Hu, L., Liu, Y., Qiu, W.: A deep spiking neural network anomaly detection method. Comput. Intell. Neurosci. CIN 2022, 6391750 (2022)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: An introduction to reinforcement learning. In: The Biology and Technology of Intelligent Autonomous Agents, pp. 90–127 (1995)
Kim, Y., Li, Y., Park, H., Venkatesha, Y., Panda, P.: Neural architecture search for spiking neural networks. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, 23–27 October 2022, Proceedings, Part XXIV. LNCS, vol. 13684, pp. 36–56. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20053-3_3
Kozma, R., Pino, R.E., Pazienza, G.E.: Advances in Neuromorphic Memristor Science and Applications, vol. 4. Springer, Cham (2012). https://doi.org/10.1007/978-94-007-4491-2
Lavin, A., Ahmad, S.: Evaluating real-time anomaly detection algorithms-the Numenta anomaly benchmark. In: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 38–44. IEEE (2015)
Li, Z., Zhao, Y., Botta, N., Ionescu, C., Hu, X.: COPOD: copula-based outlier detection. In: 2020 IEEE International Conference on Data Mining (ICDM), pp. 1118–1123. IEEE (2020)
Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422. IEEE (2008)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Pang, G., Cao, L., Chen, L., Liu, H.: Learning representations of ultrahigh-dimensional data for random distance-based outlier detection. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2041–2050 (2018)
Pang, G., van den Hengel, A., Shen, C., Cao, L.: Toward deep supervised anomaly detection: reinforcement learning from partially labeled anomaly data. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1298–1308 (2021)
Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep learning for anomaly detection: a review. ACM Comput. Surv. (CSUR) 54(2), 1–38 (2021)
Pang, G., Shen, C., van den Hengel, A.: Deep anomaly detection with deviation networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 353–362 (2019)
Ruff, L., et al.: Deep semi-supervised anomaly detection. arXiv preprint arXiv:1906.02694 (2019)
Savinov, N., et al.: Episodic curiosity through reachability. arXiv preprint arXiv:1810.02274 (2018)
Tamersoy, A., Roundy, K., Chau, D.H.: Guilt by association: large scale malware detection by mining file-relation graphs. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1524–1533 (2014)
Tang, G., Shah, A., Michmizos, K.P.: Spiking neural network on neuromorphic hardware for energy-efficient unidimensional SLAM. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4176–4181. IEEE (2019)
Ukil, A., Bandyoapdhyay, S., Puri, C., Pal, A.: IoT healthcare analytics: the importance of anomaly detection. In: 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), pp. 994–997. IEEE (2016)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Xu, H., Pang, G., Wang, Y., Wang, Y.: Deep isolation forest for anomaly detection. IEEE Trans. Knowl. Data Eng. 1–14 (2023). https://doi.org/10.1109/TKDE.2023.3270293
Xu, H., Wang, Y., Wei, J., Jian, S., Li, Y., Liu, N.: Fascinating supervisory signals and where to find them: deep anomaly detection with scale learning. In: Proceedings of the International Conference on Machine Learning (2023)
Yoon, J., Sohn, K., Li, C.L., Arik, S.O., Lee, C.Y., Pfister, T.: Self-trained one-class classification for unsupervised anomaly detection. arXiv e-prints pp. arXiv-2106 (2021)
Zhang, D., Zhang, T., Jia, S., Xu, B.: Multi-scale dynamic coding improved spiking actor network for reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 59–67 (2022)
Zhang, H., et al.: In the blink of an eye: event-based emotion recognition. In: ACM SIGGRAPH 2023 Conference Proceedings, pp. 1–11 (2023)
Zhang, J., et al.: Spiking transformers for event-based single object tracking. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 8801–8810 (2022)
Zhang, J., et al.: Frame-event alignment and fusion network for high frame rate tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9781–9790, June 2023
Zhou, Y., Song, X., Zhang, Y., Liu, F., Zhu, C., Liu, L.: Feature encoding with autoencoders for weakly supervised anomaly detection. IEEE Trans. Neural Networks Learn. Syst. 33(6), 2454–2465 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Jin, A., Wu, Z., Zhu, L., Xia, Q., Yang, X. (2024). Spiking Reinforcement Learning for Weakly-Supervised Anomaly Detection. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14451. Springer, Singapore. https://doi.org/10.1007/978-981-99-8073-4_14
Download citation
DOI: https://doi.org/10.1007/978-981-99-8073-4_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8072-7
Online ISBN: 978-981-99-8073-4
eBook Packages: Computer ScienceComputer Science (R0)