Abstract
Outlier detection in the high-dimensional data stream is a challenging data mining task. In high-dimensional data, the distance-based measures of outlierness become less effective and unreliable. Angle-based outlier detection ABOD technique was proposed as a more suitable scheme for high-dimensional data. However, ABOD is designed for static datasets and its naive application on a sliding window over data streams will result in poor performance. In this research, we propose two incremental algorithms for fast outlier detection based on an outlier threshold value in high-dimensional data streams: IncrementalVOA and \(VOA^{*}\). IncrementalVOA is a basic incremental algorithm for computing outlier factor of each data point in each window. \(VOA^{*}\) enhances the incremental computation by using a bound-based pruning method and a retrospect-based incremental computation technique. The effectiveness and efficiency of the proposed algorithms are experimentally evaluated on synthetic and real world datasets where \(VOA^{*}\) outperformed other methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, C.C., Philip, S.Y., Han, J., Wang, J.: A framework for clustering evolving data streams. In: Proceedings of 2003 Very Large Data Bases Conference, pp. 81–92. Elsevier (2003)
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)
Cao, F., Estert, M., Qian, W., Zhou, A.: Density-based clustering over an evolving data stream with noise. In: Proceedings of 2006 SIAM International Conference on Data Mining, pp. 328–339. SIAM (2006)
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–58 (2009)
Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Hawkins, D.M.: Identification of Outliers, vol. 11. Springer, Heidelberg (1980). https://doi.org/10.1007/978-94-015-3994-4
Hinneburg, A., Aggarwal, C.C., Keim, D.A.: What is the nearest neighbor in high dimensional spaces? In: Proceedings of 26th International Conference on Very Large Data Bases, pp. 506–515 (2000)
Ishida, K., Kitagawa, H.: Detecting current outliers: continuous outlier detection over time-series data streams. In: Bhowmick, S.S., Küng, J., Wagner, R. (eds.) DEXA 2008. LNCS, vol. 5181, pp. 255–268. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85654-2_26
Kieu, T., Yang, B., Jensen, C.S.: Outlier detection for multidimensional time series using deep neural networks. In: Proceedings of 2018 19th IEEE International Conference on Mobile Data Management, pp. 125–134. IEEE (2018)
Knorr, E.M., Ng, R.T., Tucakov, V.: Distance-based outliers: algorithms and applications. VLDB J. 8(3–4), 237–253 (2000)
Kontaki, M., Gounaris, A., Papadopoulos, A.N., Tsichlas, K., Manolopoulos, Y.: Continuous monitoring of distance-based outliers over data streams. In: Proceedings of 2011 IEEE 27th International Conference on Data Engineering, pp. 135–146. IEEE (2011)
Kriegel, H.P., Schubert, M., Zimek, A.: Angle-based outlier detection in high-dimensional data. In: Proceedings of 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 444–452 (2008)
Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1975–1981. IEEE (2010)
Papadimitriou, S., Kitagawa, H., Gibbons, P.B., Faloutsos, C.: LOCI: fast outlier detection using the local correlation integral. In: Proceedings of 19th International Conference on Data Engineering, pp. 315–326. IEEE (2003)
Pham, N., Pagh, R.: A near-linear time approximation algorithm for angle-based outlier detection in high-dimensional data. In: Proceedings of 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 877–885 (2012)
Pokrajac, D., Lazarevic, A., Latecki, L.J.: Incremental local outlier detection for data streams. In: 2007 IEEE Symposium on Computational Intelligence and Data Mining, pp. 504–515. IEEE (2007)
Salehi, M., Leckie, C., Bezdek, J.C., Vaithianathan, T., Zhang, X.: Fast memory efficient local outlier detection in data streams. IEEE Trans. Knowl. Data Eng. 28(12), 3246–3260 (2016)
Shaikh, S.A., Kitagawa, H.: Continuous outlier detection on uncertain data streams. In: Proceedings of 2014 IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, pp. 1–7. IEEE (2014)
Tran, L., Fan, L., Shahabi, C.: Distance-based outlier detection in data streams. PVLDB Endow. 9(12), 1089–1100 (2016)
Ye, H., Kitagawa, H., Xiao, J.: Continuous angle-based outlier detection on high-dimensional data streams. In: Proceedings of 19th International Database Engineering and Applications Symposium, pp. 162–167 (2015)
Yoon, S., Lee, J.G., Lee, B.S.: NETS: extremely fast outlier detection from a data stream via set-based processing. PVLDB Endow. 12(11), 1303–1315 (2019)
Acknowledgement
This work was partly supported by JSPS KAKENHI Grant Number JP19H04114.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Khalique, V., Kitagawa, H. (2021). VOA*: Fast Angle-Based Outlier Detection over High-Dimensional Data Streams. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12712. Springer, Cham. https://doi.org/10.1007/978-3-030-75762-5_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-75762-5_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75761-8
Online ISBN: 978-3-030-75762-5
eBook Packages: Computer ScienceComputer Science (R0)