Jian Yang
ABSTRACT
Peculiarity oriented mining (POM), aiming to discover peculiarity rules hidden in a dataset, is a new data mining method. In the past few years, many results and applications on POM have been reported. However, there is still a lack of theoretical analysis. In this paper, we prove that the peculiarity factor (PF), one of the most important concepts in POM, can accurately characterize the peculiarity of data with respect to the probability density function of a normal distribution, but is unsuitable for more general distributions. Thus, we propose the concept of local peculiarity factor (LPF). It is proved that the LPF has the same ability as the PF for a normal distribution and is the so-called µ-sensitive peculiarity description for general distributions. To demonstrate the effectiveness of the LPF, we apply it to outlier detection problems and give a new outlier detection algorithm called LPF-Outlier. Experimental results show that LPF-Outlier is an effective outlier detection algorithm.
- N. Abe, B. Zadrozny, and J. Langford. Outlier detection by active learning. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 504--509, 2006. Google Scholar
Digital Library
- N. L. Bhamidipati and S. K. Pal. Comparing rank-inducing scoring systems. Proceedings of the 18th International Conference on Pattern Recognition, pages 300--303, 2006. Google ScholarDigital Library
- L. Breiman. Bagging predictors. Machine Learning, 24:123--140, 1996. Google ScholarDigital Library
- M. M. Breunig, H. P. Kriegel, R. T. Ng, and J. Sander. Lof: identifying density-based local outliers. Proceedings of the 6th ACM SIGMOD International Conference on Management of Data, pages 93--104, 2000. Google ScholarDigital Library
- N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. Smote: Synthetic minority oversampling technique. Journal of Artificial Intelligence Research, 16:321--357, 2002. Google ScholarDigital Library
- L. Ertoz. Similarity Measures. Ph.D. Dissertation, University of Minnesota, 2005.Google Scholar
- S. Harkins, H. He, G. J. Willams, and R. A. Baster. Outlier detection using replicator neural networks. Proceedings of the 4th International Conference on Data Warehousing and Knowledge Discovery, pages 170--180, 2002. Google ScholarDigital Library
- Z. Y. He, X. F. Xu, and S. C. Deng. Discovering cluster based local outliers. Pattern Recognition Letters, 24:164--1650, 2003. Google ScholarDigital Library
- Z. Y. He, X. F. Xu, Z. X. Huang, and S. C. Deng. A frequent pattern discovery method for outlier detection. Proceedings of the 5th International Conference on Web-Age Information Management, pages 726--732, 2004.Google ScholarCross Ref
- E. Knorr and R. Ng. Algorithms for mining distance-based outliers in large datasets. Proceedings of the 12th International Conference on Very Large Data Bases, pages 392--403, 1998. Google ScholarDigital Library
- A. Lazarevic and V. Kumar. Feature bagging for outlier detection. Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 157--166, 2005. Google ScholarDigital Library
- B. Liu, W. Hsu, and Y. Ma. Integrating classification and association rule mining. Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining, pages 80--86, 1998.Google Scholar
- K. Mcgarry. A survey of interestingness measures for knowledge discovery. The Knowledge Engineering Review, 20:39--61, 2005. Google ScholarDigital Library
- G. Merz and P. Murphy. Uci repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html, 1996.Google Scholar
- M. Ohshima, N. Zhong, Y. Y. Yao, and C. Liu. Relational peculiarity oriented mining. Data Mining and Knowledge Discovery, 15:249--273, 2007. Google ScholarDigital Library
- M. Ohshima, N. Zhong, Y. Y. Yao, and S. Murata. Peculiarity oriented analysis in multi-people tracking images. Advances in Knowledge Discovery and Data Mining, pages 508--518, 2004.Google ScholarCross Ref
- S. Ramaswamy, R. Rastogi, and S. Kyuseok. Efficient algorithms for mining outliers from large data sets. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pages 427--438, 2000. Google ScholarDigital Library
- D. Saso and L. Nada. An introduction to inductive logic programming. Relational Data Mining, pages 48--73, 2001.Google Scholar
- Y. Y. Yao, F. Y. Wang, J. Wang, and D. D. Zeng. Rule + exception strategies for security information analysis. IEEE Intelligent Systems, 20:52--57, 2005. Google ScholarDigital Library
- Y. Y. Yao and N. Zhong. An analysis of peculiarity oriented data mining. Proceedings of the 2002 IEEE International Conference on Data Mining Workshop on the Foundation of Data Mining and Discovery, pages 185--188, 2002.Google Scholar
- N. Zhong, C. Liu, Y. Y. Yao, M. Ohshima, M. X. Huang, and J. J. Huang. Relational peculiarity oriented data mining. Proceedings of the 2004 IEEE International Conference on Data Mining, pages 575--578, 2004. Google ScholarDigital Library
- N. Zhong, M. Ohshima, and S. Ohsuga. Peculiarity oriented mining and its application for knowledge discovery in amino-acid data. Advances in Knowledge Discovery and Data Mining, pages 260--269, 2001. Google ScholarDigital Library
- N. Zhong, Y. Yao, and M. Ohshima. Peculiarity oriented multi-database mining. IEEE Transactions on Knowledge and Data Engineering, 15:952--960, 2003. Google ScholarDigital Library
- N. Zhong, Y. Y. Yao, M. Ohshima, and S. Ohsuga. Interestingness, peculiarity, and multi-database mining. Proceedings of the 2001 IEEE International Conference on Data Mining, pages 566--573, 2001. Google ScholarDigital Library
Index Terms
- Local peculiarity factor and its application in outlier detection
Recommendations
Peculiarity Analysis for Classifications
ICDM '09: Proceedings of the 2009 Ninth IEEE International Conference on Data MiningPeculiarity-oriented mining (POM) is a new data mining method consisting of peculiar data identification and peculiar data analysis. Peculiarity factor (PF) and local peculiarity factor (LPF) are important concepts employed to describe the peculiarity ...
MiFI-Outlier: Minimal infrequent itemset-based outlier detection approach on uncertain data stream
AbstractMassive outlier detection approaches have been proposed for static datasets in the past twenty years, and they have acquired good achievements. In real life, uncertain data stream is more and more common, but most existing outlier ...
Outlier detection based on cluster outlier factor and mutual density
Outlier detection is an important task in data mining with numerous applications. Recent years, the study on outlier detection is very active, many algorithms were proposed including based on clustering. However, most outlier detection algorithms based on ...
Comments