Abstract
In recent years, the scale of data stream is becoming much larger in real life. However, the anomaly data often exists in the collected data stream, while the existence of anomaly is a main reason for the decrease of the accuracy of data-based operations. The anomaly data have two main characteristics, that is, appear rarely and deviate much from most data elements, thus, the anomaly detection methods should accurately detect the anomaly data by considering these two attributes. Because the data stream is continuously generated and constantly flowing, thus, the previous static anomaly detection methods are not suitable for processing data streams. In addition, the large amount of data stream makes the time consumption and memory occupation of rare itemset mining phase very high. To effectively solve these problems, this paper first proposes an efficient MRI-Mine method for mining minimum rare itemsets, and then proposes an accurately anomaly detection method called MRI-AD based on anomaly index to identify the implicit anomaly data. The experiments indicate the proposed MRI-Mine method can mine the minimum rare itemsets in less time consumption and memory occupation, and the detection accuracy of MRI-AD method is also competitive.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, Z., Wu, P., Han, W., Yu, W.: Remote monitoring system for agricultural information based on wireless sensor network. J. Chin. Inst. Eng. 40(1), 75–81 (2017)
Okazaki, T., Orii, T., Ueda, A., Kuramitz, H.: A reusable fiber optic sensor for the real-time sensing of CaCO3 scale formation in geothermal water. IEEE Sens. J. 17(5), 1207–1208 (2017)
Yuan, J., Wang, Z., Sun, Y., Zhang, W., Jiang, J.: An effective pattern-based Bayesian classifier for evolving data stream. Neurocomputing 295, 17–28 (2018)
Hawkins, D.M.: Identification of Outliers, vol. 11. Chapman and Hall, London (1980)
Huang, J., Zhu, Q., Yang, L., Cheng, D., Wu, Q.: A novel outlier cluster detection algorithm without top-n parameter. Knowl.-Based Syst. 121, 32–40 (2017)
Ramaswamy, S., Rastogi, R., Shim, K.: Efficient algorithms for mining outliers from large data sets. In: ACM SIGMOD Record, Dallas, USA, vol. 29, no. 2, pp. 427–438 (2000)
Zhang, L., Lin, J., Karim, R.: Adaptive kernel density-based anomaly detection for nonlinear systems. Knowl.-Based Syst. 139, 50–63 (2018)
He, Z., Xu, X., Huang, Z., Deng, S.: FP-outlier: frequent pattern based outlier detection. Comput. Sci. Inf. Syst. 2(1), 103–118 (2005)
Cai, S., Sun, R., Li, J., Deng, C., Li, S.: Abnormal detecting over data stream based on maximal pattern mining technology. In: Sun, Y., Lu, T., Xie, X., Gao, L., Fan, H. (eds.) ChineseCSCW 2018. CCIS, vol. 917, pp. 371–385. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3044-5_27
Feng, L., Wang, L., Jin, B.: Research on maximal frequent pattern outlier factor for online high dimensional time-series outlier detection. J. Converg. Inf. Technol. 5(10), 66–71 (2010)
Hao, S., Cai, S., Sun, R., Li, S.: An efficient frequent closed itemset-based outlier detecting approach on data stream. In: CCF Conference on Computer Supported Cooperative Work and Social Computing, Guilin, China, pp. 371–385 (2018)
Hao, S., Cai, S., Sun, R., Li, S.: An efficient outlier detection approach over uncertain data stream based on frequent itemset mining. J. Inf. Technol. Control 48(1), 34–46 (2019)
Cai, S., Sun, R., Hao, S., Li, S., Yuan, G.: Minimal weighted infrequent itemset mining-based outlier detection approach on uncertain data stream. Neural Comput. Appl. 9, 1–21 (2018)
Zhang, W., Wu, J., Yu, J.: An improved method of outlier detection based on frequent pattern. In: WASE International Conference on Information Engineering (ICIE), Washington, USA, pp. 3–6 (2010)
Dallachiesa, M., Jacques-Silva, G., Gedik, B., Wu, K., Palpanas, T.: Sliding windows over uncertain data streams. Knowl. Inf. Syst. 45(1), 159–190 (2015)
Yang, G.: The complexity of mining maximal frequent itemsets and maximal frequent patterns. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, pp. 344–353 (2004)
Szathmary, L., Napoli, A., Valtchev, P.: Towards rare itemset mining. In: International Conference on Tools with Artificial Intelligence (ICTAI), Patras, Greece, pp. 305–312 (2007)
Gupta, A., Mittal, A., Bhattacharya, A.: Minimally infrequent itemset mining using pattern-growth paradigm and residual trees. In: International Conference on Management of Data, Bangalore, India, pp. 1–14 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Cai, S., Sun, R., Mu, H., Shi, X., Yuan, G. (2019). A Minimum Rare-Itemset-Based Anomaly Detection Method and Its Application on Sensor Data Stream. In: Sun, Y., Lu, T., Yu, Z., Fan, H., Gao, L. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2019. Communications in Computer and Information Science, vol 1042. Springer, Singapore. https://doi.org/10.1007/978-981-15-1377-0_9
Download citation
DOI: https://doi.org/10.1007/978-981-15-1377-0_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1376-3
Online ISBN: 978-981-15-1377-0
eBook Packages: Computer ScienceComputer Science (R0)
