Abstract
This paper introduces a novel unsupervised outlier detection method, namely WOD, for identifying outliers in categorical data. Existing subspace-based methods are challenged by overwhelming irrelevant features and their performance sometimes heavily depends on the setting of subspace size. Feature selection-based methods may omit the relevant information when removing features. In contrast, WOD works on value-level subspace exploring, i.e., separating irrelevant values based on value cluster structure, which avoids the dilemma of setting subspace size. Value outlierness is estimated by modeling weighted value couplings between relevant value set and value full set to further eliminate the interference from noisy features. We show that (i) WOD significantly outperforms five state-of-the-art outlier detectors on 12 real-world data sets with different levels of noisy features; (ii) WOD obtains good scalability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The code of WOD is available at https://sites.google.com/site/hongzuoxu/.
References
Aggarwal, C.C.: Outlier Analysis. Springer, New York (2017). https://doi.org/10.1007/978-1-4614-6396-2
Akoglu, L., Tong, H., Vreeken, J., Faloutsos, C.: Fast and reliable anomaly detection in categorical data. In: CIKM, pp. 415–424. ACM (2012)
Angiulli, F., Fassetti, F., Palopoli, L.: Detecting outlying properties of exceptional objects. ACM Trans. Database Syst. 34(1), 7 (2009)
Blondel, V.D., Guillaume, J.L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech.: Theory Exp. 2008(10), P10008 (2008)
Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. ACM SIGMOD Rec. 29(2), 93–104 (2000)
Campos, G.O., et al.: On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data Min. Knowl. Discov. 30(4), 891–927 (2016)
Cao, L.: Coupling learning of complex interactions. Inf. Process. Manag. 51(2), 167–186 (2015)
Chen, Y., Dang, X., Peng, H., Bart, H.L.: Outlier detection with the kernelized spatial depth function. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 288–305 (2009)
He, Z., Xu, X., Huang, Z.J., Deng, S.: FP-outlier: frequent pattern based outlier detection. Comput. Sci. Inf. Syst. 2(1), 103–118 (2005)
Li, J., et al.: Feature selection: A data perspective. CoRR abs/1601.07996 (2016)
Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation-based anomaly detection. ACM Trans. Knowl. Discov. Data 6(1), 1–39 (2012)
Pang, G., Cao, L., Chen, L.: Outlier detection in complex categorical data by modelling the feature value couplings. In: IJCAI, pp. 1902–1908. AAAI Press (2016)
Pang, G., Cao, L., Chen, L., Liu, H.: Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings. In: ICDM, pp. 410–419. IEEE (2016)
Pang, G., Cao, L., Chen, L., Liu, H.: Learning homophily couplings from non-IID data for joint feature selection and noise-resilient outlier detection. In: IJCAI, pp. 2585–2591. AAAI Press (2017)
Pang, G., Ting, K.M., Albrecht, D., Jin, H.: Zero++: harnessing the power of zero appearances to detect anomalies in large-scale data sets. J. Artif. Intell. Res. 57, 593–620 (2016)
Pang, G., Xu, H., Cao, L., Zhao, W.: Selective value coupling learning for detecting outliers in high-dimensional categorical data. In: CIKM, pp. 807–816. ACM (2017)
Sathe, S., Aggarwal, C.C.: Subspace outlier detection in linear time with randomized hashing. In: ICDM, pp. 459–468. IEEE (2016)
Acknowledgements
This work is supported by NSFC No.61472439, National Natural Science Foundation of China under Grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, H., Wang, Y., Wu, Z., Ma, X., Qin, Z. (2018). Combine Value Clustering and Weighted Value Coupling Learning for Outlier Detection in Categorical Data. In: Hartmann, S., Ma, H., Hameurlain, A., Pernul, G., Wagner, R. (eds) Database and Expert Systems Applications. DEXA 2018. Lecture Notes in Computer Science(), vol 11030. Springer, Cham. https://doi.org/10.1007/978-3-319-98812-2_40
Download citation
DOI: https://doi.org/10.1007/978-3-319-98812-2_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98811-5
Online ISBN: 978-3-319-98812-2
eBook Packages: Computer ScienceComputer Science (R0)