Abstract
In this work we introduce hierarchical outliers that extend the notion of distance-based outliers for handling hierarchical data domains. We present a novel framework that permits us to detect hierarchical outliers in a consistent manner, providing a desired monotonicity property, which implies that a data observation that finds enough support so as to be disregarded as an outlier at a level of the hierarchy, will not be labelled as an outlier when examined at a more coarse-grained level above. This way, we enable users to grade how suspicious a data observation is, depending on the number of hierarchical levels for which the observation is found to be an outlier. Our technique utilizes an innovative locality sensitive hashing indexing scheme, where data points sharing the same hash value are being clustered. The computed centroids are maintained by our frameworkâs scheme index while detailed data descriptors are discarded. This results in reduced storage space needs, execution time and number of distance evaluations compared to utilizing a straightforward LSH index.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: FOCS, pp. 459â468. IEEE Computer Society (2006)
Bhaduri, K., Matthews, B.L., Giannella, C.R.: Algorithms for speeding up distance-based outlier detection. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 859â867. ACM (2011)
Breunig, M., Kriegel, H., Ng, R., Sander, J., et al.: LOF: identifying density-based local outliers. Sigmod Rec. 29(2), 93â104 (2000)
Charikar, M.: Similarity estimation techniques from rounding algorithms. In: STOC (2002)
Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: Proccedings of SCG (2004)
Georgoulas, K., Kotidis, Y.: Distributed similarity estimation using derived dimensions. VLDB J. 21(1), 25â50 (2012)
Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: STOC (1998)
Jin, W., Tung, A.K.H., Han, J., Wang, W.: Ranking outliers using symmetric neighborhood relationship. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 577â593. Springer, Heidelberg (2006)
Knorr, E., Ng, R., Tucakov, V.: Distance-based outliers: algorithms and applications. VLDB J. 8(3), 237â253 (2000)
Kriegel, H., Zimek, A., et al.: Angle-based outlier detection in high-dimensional data. In: Proceeding of ACM SIGKDD, pp. 444â452 (2008)
Lv, Q., Josephson, W., Wang, Z., Charikar, M., Li, K.: Multi-probe LSH: efficient indexing for high-dimensional similarity search. In: VLDB, pp. 950â961 (2007)
OLAP Council APB-1 Benchmark. http://www.olapcouncil.org/research/resrchly.htm
Papadimitriou, S., Kitagawa, H., Gibbons, P., Faloutsos, C.: LOCI: fast outlier detection using the local correlation integral. In: Proceedings of ICDE, pp. 315â326 (2003)
Slaney, M., Lifshits, Y., He, J.: Optimal parameters for locality-sensitive hashing. In: Proceedings of the IEEE, pp. 2604â2623 (2012)
Sugiyama, M., Borgwardt, K.: Rapid distance-based outlier detection via sampling. In: Advances in Neural Information Processing Systems, pp. 467â475 (2013)
Tang, J., Chen, Z., Fu, A.W., Cheung, D.W.: Enhancing effectiveness of outlier detections for low density patterns. In: Chen, M.-S., Yu, P.S., Liu, B. (eds.) PAKDD 2002. LNCS (LNAI), vol. 2336, pp. 535â548. Springer, Heidelberg (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Âİ 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Georgoulas, K., Kotidis, Y. (2016). Hierarchically Clustered LSH for Hierarchical Outliers Detection. In: Hartmann, S., Ma, H. (eds) Database and Expert Systems Applications. DEXA 2016. Lecture Notes in Computer Science(), vol 9827. Springer, Cham. https://doi.org/10.1007/978-3-319-44403-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-44403-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44402-4
Online ISBN: 978-3-319-44403-1
eBook Packages: Computer ScienceComputer Science (R0)