Skip to main content
SpringerLink
Log in

An enhanced local outlier detection using random walk on grid information graph

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Outlier detection is a hot issue in data mining, which has plenty of practical applications. Local Outlier Factor algorithm, as a pioneering work of local outlier detection, has been paid much attention. However, it needs to perform the neighbor search with high time complexity and ignores the local distribution of an object within its neighbor. In this work, a novel local outlier detection method based on grid random walk is proposed, which uses random walk to obtain stationary distribution vector of grid information graph. Some grids with small values of stationary distribution vector will be considered as candidate outliers. The outlier detection is performed only on candidate outliers to improve the running efficiency. Then, considering the local distribution of an object within its neighbor, a new local outlier factor is constructed to estimate the abnormal degree of each object. The experimental results indicate that the proposed algorithm has better performance and lower running time than the others.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Jin F, Chen M, Zhang W, Yuan Y, Wang S (2021) Intrusion detection on internet of vehicles via combining log-ratio oversampling, outlier detection and metric learning. Inf Sci 579:814–831

    Article  MathSciNet  Google Scholar 

  2. Li H, Wang B (2020) Xie X (2020) An improved content-based outlier detection method for ics intrusion detection. EURASIP J Wirel Commun Netw 1:1–15

    Google Scholar 

  3. Vasudevan AR, Selvakumar S (2016) Local outlier factor and stronger one class classifier based hierarchical model for detection of attacks in network intrusion detection dataset. Front Comp Sci 10:755–766

    Article  Google Scholar 

  4. Sandosh S, Govindasamy V, Akila G (2020) Enhanced intrusion detection system via agent clustering and classification based on outlier detection. Peer-to-Peer Netw Appl 13(3):1038–1045

    Article  Google Scholar 

  5. Hauskrecht M, Batal I, Valko M, Visweswaran S, Cooper GF, Clermont G (2013) Outlier detection for patient monitoring and alerting. J Biomed Inform 46(1):47–55

    Article  Google Scholar 

  6. Hauskrecht M, Batal I, Hong C, Nguyen Q, Cooper GF, Visweswaran S, Clermont G (2016) Outlier-based detection of unusual patient-management actions: an icu study. J Biomed Inform 64:211–221

    Article  Google Scholar 

  7. Presbitero A, Quax R, Krzhizhanovskaya V, Sloot P (2017) Anomaly detection in clinical data of patients undergoing heart surgery. Proc Comput Sci 108, 99–108 (2017), international Conference on Computational Science, ICCS 2017, 12-14 June 2017, Zurich, Switzerland

  8. Cai L, Thornhill NF, Kuenzel S, Pal BC (2017) Real-time detection of power system disturbances based on \(k\) -nearest neighbor analysis. IEEE Access 5:5631–5639

    Article  Google Scholar 

  9. Anagnostou G, Boem F, Kuenzel S, Pal BC, Parisini T (2018) Observer-based anomaly detection of synchronous generators for power systems monitoring. IEEE Trans Power Syst 33(4):4228–4237

    Article  Google Scholar 

  10. Zhang Q, Wan S, Wang B, Gao DW, Ma H (2019) Anomaly detection based on random matrix theory for industrial power systems. J Syst Architect 95:67–74

    Article  Google Scholar 

  11. Wang B, Mao Z (2019) Outlier detection based on gaussian process with application to industrial processes. Appl Soft Comput 76:505–516

    Article  Google Scholar 

  12. Breunig MM, Kriegel HP, Ng RT, Sander J (2000) Lof: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. pp 93–104. SIGMOD ’00, Association for Computing Machinery, New York, NY, USA

  13. Kim S, Cho NW, Kang B, Kang SH (2011) Fast outlier detection for very large log data. Expert Syst Appl 38(8):9587–9596

    Article  Google Scholar 

  14. Shen J, Liu J, Zhao R, Lin X (2011) A kd-tree-based outlier detection method for airborne lidar point clouds. In: 2011 international symposium on image and data fusion. pp 1–4

  15. Zhang F, Yin F, Huang G (2020) An optimized lof algorithm based on tree structure. In: 2020 3rd international conference on artificial intelligence and big data (ICAIBD). pp 167–171

  16. Li Z, Luo Q, Liu J (2018) Vdod:distributed outlier detection algorithm based on kd-tree. Comput Digit Eng

  17. Pamula R, Deka JK, Nandi S (2011) An outlier detection method based on clustering. In: 2011 second international conference on emerging applications of information technology. pp 253–256

  18. Su S, Xiao L, Ruan L, Gu F, Li S, Wang Z, Xu R (2019) An efficient density-based local outlier detection approach for scattered data. IEEE Access 7:1006–1020

    Article  Google Scholar 

  19. Wang X, Wang X, Wilkes M (2021) Enhancing outlier detection by filtering out core points and border points, pp 173–193. Springer Singapore, Singapore

  20. Ester M, Kriegel HP, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the second international conference on knowledge discovery and data mining. p. 226-231. KDD’96, AAAI Press

  21. Schubert E, Zimek A, Kriegel HP (2014) Local outlier detection reconsidered: a generalized view on locality with applications to spatial, video, and network outlier detection. Data Min Knowl Disc 28(1):190–237

    Article  MathSciNet  Google Scholar 

  22. Wang YF, Jiong Y, Su GP, Qian YR (2019) A new outlier detection method based on optics. Sustain Cities Soc 45:197–212

    Article  Google Scholar 

  23. El Meziati M, Ziyati H (2018) Fast outlier detection method based on rough set. In: 2018 9th international symposium on signal, image, video and communications (ISIVC). pp 60–66

  24. Cai S, Sun R, Hao S, Li S, Yuan G (2019) An efficient outlier detection approach on weighted data stream based on minimal rare pattern mining. China Commun 16(10):83–99

    Article  Google Scholar 

  25. Du H, Ye Q, Sun Z, Liu C, Xu W (2021) Fast-odt: a lightweight outlier detection scheme for categorical data sets. IEEE Trans Netw Sci Eng 8(1):13–24

    Article  Google Scholar 

  26. Goldstein M (2012) Fastlof: an expectation-maximization based local outlier detection algorithm. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012). pp 2282–2285

  27. Tang J, Chen Z, Fu AWC, Cheung DW (2002) Enhancing effectiveness of outlier detections for low density patterns. In: Chen MS, Yu PS, Liu B (eds) Advances in knowledge discovery and data mining. Springer, Berlin Heidelberg, pp 535–548

    Chapter  Google Scholar 

  28. Zhang K, Hutter M, Jin H (2009) A new local distance-based outlier detection approach for scattered real-world data. In: Theeramunkong T, Kijsirikul B, Cercone N, Ho TB (eds) Advances in knowledge discovery and data mining. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 813–822

    Chapter  Google Scholar 

  29. Jin W, Tung AKH, Han J, Wang W (2006) Ranking outliers using symmetric neighborhood relationship. In: Ng WK, Kitsuregawa M, Li J, Chang K (eds) Advances in knowledge discovery and data mining. Springer, Berlin Heidelberg, pp 577–593

    Chapter  Google Scholar 

  30. Yang P, Wang D, Wei Z, Du X, Li T (2019) An outlier detection approach based on improved self-organizing feature map clustering algorithm. IEEE Access 7:115914–115925

    Article  Google Scholar 

  31. Dua D, Graff C (2017) UCI machine learning repository, http://archive.ics.uci.edu/ml

Download references

Acknowledgements

This work was supported by Chongqing University Innovation Research Group funding (No. CXQT20015), the Key Science and Technology Research Program of Chongqing Municipal Education Commission (No. KJZD-K201900505), and Research Project of Chongqing Normal University (No. YKC20032).

Author information

Authors and Affiliations

  1. College of Computer and Information Science, Chongqing Normal University, Chongqing, 401331, China

    Chunyan She & Shaohua Zeng

  2. Chongqing Center of Engineering Technology Research on Digital Agricultural and Service, Chongqing, 401331, China

    Chunyan She & Shaohua Zeng

Authors
  1. Chunyan She
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaohua Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaohua Zeng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

She, C., Zeng, S. An enhanced local outlier detection using random walk on grid information graph. J Supercomput 78, 14530–14547 (2022). https://doi.org/10.1007/s11227-022-04459-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04459-7

Keywords

Search

Navigation