Skip to main content
SpringerLink
Log in

Using multiple classifier behavior to develop a dynamic outlier ensemble

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Outlier ensembles that use more base detectors recently become an attractive approach to solving problems of single detectors. However, existing outlier ensembles often assume that base detectors make independent errors, which is difficult to satisfy in practical applications. To this end, this paper proposes a dynamic outlier ensemble to loose this error independence assumption. In our method, it is desired that the most competent base detector(s) can be singled out by the dynamic selection mechanism for each test pattern. The usage of the concept of multiple classifier behavior (MCB) has two purposes. One is to generate artificial outlier examples used for competence estimates. This strategy is different from other methods since we do not make any assumption regarding the data distribution. On the other hand, MCB is used to refine validation sets initialized by the K-nearest neighbors (KNN) rule. It is desired that objects in the refined validation sets are more representative than those found by KNN. With the refined validation sets, competences of all base detectors will be estimated by a probabilistic method, before which we have transformed outputs of base detectors into a probabilistic form. Finally, a switching mechanism that determines whether one detector should be nominated to make the decision or a fusion method should be applied instead is proposed in order to achieve a robust detection result. We carry out experiments on 20 benchmark data sets to verify the effectiveness of our detection method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. http://homepage.tudelft.nl/n9d04/functions/Contents.html.

References

  1. Aggarwal CC, Sathe S (2015) Theoretical foundations and algorithms for outlier ensembles. ACM SIGKDD Explor Newsl 17(1):24–47

    Article  Google Scholar 

  2. Aggarwal CC, Sathe S (2017) Outlier ensembles. Springer, Berlin

    Book  Google Scholar 

  3. Ando S, Thanomphongphan T, Seki Y, Suzuki E (2015) Ensemble anomaly detection from multi-resolution trajectory features. Data Min Knowl Disc 29(1):39–83

    Article  MathSciNet  Google Scholar 

  4. Campos GO, Zimek A, Sander J, Campello RJGB, Micenková B, Schubert E, Houle ME (2016) On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data Min Knowl Disc 30(4):891–927

    Article  MathSciNet  Google Scholar 

  5. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3):1–58

    Article  Google Scholar 

  6. Christou IT, Gekas G, Kyrikou A (2012) A classifier ensemble approach to the TV-viewer profile adaptation problem. Int J Mach Learn Cybern 3(4):313–326

    Article  Google Scholar 

  7. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(1):1–30

    MathSciNet  MATH  Google Scholar 

  8. Fan W, Miller M, Miller M (2001) Using artificial anomalies to detect unknown and known network intrusions. In: Paper presented at the International Conference on Data Mining

  9. Fernandez A, Garcia S, Jesus MJD, Herrera F (2008) A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets. Fuzzy Sets Syst 159(18):2378–2398

    Article  MathSciNet  Google Scholar 

  10. Gao J, Tan PN (2006) Converting output scores from outlier detection algorithms into probability estimates. In: Paper presented at the Sixth International Conference on Data Mining

  11. Giacinto G, Roli F (1999) Methods for dynamic classifier selection. In: Paper presented at the Proceedings of 10th International Conference on Image Analysis and Processing

  12. Giacinto G, Roli F (2001) Dynamic classifier selection based on multiple classifier behaviour. Pattern Recogn 34(9):1879–1881

    Article  Google Scholar 

  13. Hempstalk K, Frank E, Witten IH (2008) One-class classification by combining density and class probability estimation. In: Paper presented at the Joint European Conference on Machine Learning and Knowledge Discovery in Databases

  14. Huang J, Ling CX (2005) Using AUC and accuracy in evaluating learning algorithms. IEEE Trans Knowl Data Eng 17(3):299–310

    Article  Google Scholar 

  15. Huang YS, Suen CY (1995) A method of combining multiple experts for the recognition of unconstrained handwritten numerals. Pattern Anal Mach Intell IEEE Trans 17(1):90–94

    Article  Google Scholar 

  16. Iman RL, Davenport JM (1979) Approximations of the critical region of the Friedman statistic. Commun Stat 9(6):571–595

    Article  Google Scholar 

  17. Krawczyk B (2015) One-class classifier ensemble pruning and weighting with firefly algorithm. Neurocomputing 150(150):490–500

    Article  Google Scholar 

  18. Krawczyk B (2016) Dynamic classifier selection for one-class classification. Knowl-Based Syst 107:43–53

    Article  Google Scholar 

  19. Krawczyk B, Woźniak M, Cyganek B (2014) Clustering-based ensembles for one-class classification. Inf Sci 264(6):182–195

    Article  MathSciNet  Google Scholar 

  20. Kuncheva LI (2002) Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans Syst Man Cybern Part B: Cybern 32(2):146–156

    Article  Google Scholar 

  21. Kuncheva LI, Whitaker CJ (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2):181–207

    Article  Google Scholar 

  22. Oliveira DVR, Cavalcanti GDC, Sabourin R (2017) Online pruning of base classifiers for dynamic ensemble selection. Pattern Recogn 72:44–58

    Article  Google Scholar 

  23. Parhizkar E, Abadi M (2015) BeeOWA: a novel approach based on ABC algorithm and induced OWA operators for constructing one-class classifier ensembles. Neurocomputing 166:367–381

    Article  Google Scholar 

  24. Rätsch G, Mika S, Schölkopf B, Müller K-R (2002) Constructing boosting algorithms from SVMs: An application to one-class classification. IEEE Trans Pattern Anal Mach Intell 9:1184–1199

    Article  Google Scholar 

  25. Rayana S, Zhong W, Akoglu L (2016) Sequential ensemble learning for outlier detection: A bias-variance perspective. In: Paper presented at the 2016 IEEE 16th International Conference on Data Mining (ICDM)

  26. Salehi M, Zhang X, Bezdek JC, Leckie C (2016). Smart sampling: a novel unsupervised boosting approach for outlier detection. In: Paper presented at the Australasian Joint Conference on Artificial Intelligence

  27. Sun Y, Wong AKC, Kamel MS (2009) Classification of imbalanced data: a review. Int J Pattern Recognit Artif Intell 23(04):687–719

    Article  Google Scholar 

  28. Tax DM, Duin RP (2001) Uniform object generation for optimizing one-class classifiers. J Mach Learn Res 2:155–173

    MATH  Google Scholar 

  29. Tax DMJ, Breukelen MV, Duin RPW, Kittler J (2000) Combining multiple classifiers by averaging or by multiplying? Pattern Recogn 33(9):1475–1485

    Article  Google Scholar 

  30. Tax DMJ, Duin RPW (2001) Combining One-Class Classifiers. In: Paper presented at the International Workshop on Multiple Classifier Systems

  31. Wang B, Mao Z (2017) One-class classifiers ensemble based anomaly detection scheme for process control systems. Trans Inst Measur Control 40(12):3466–3476

    Article  Google Scholar 

  32. Wang B, Mao Z (2019) Outlier detection based on a dynamic ensemble model: applied to process monitoring. Inf Fusion 51:244–258

    Article  Google Scholar 

  33. Wang B, Mao Z, Huang K (2017) Detecting outliers in complex nonlinear systems controlled by predictive control strategy. Chaos Solitons Fract 103:588–595

    Article  Google Scholar 

  34. Zhang H, Gang L, Chow TWS, Wenyin L (2011) Textual and visual content-based anti-phishing: a Bayesian approach. IEEE Trans Neural Netw 22(10):1532–1546

    Article  Google Scholar 

  35. Zhao Q-L, Jiang Y-H, Xu M (2009) A fast ensemble pruning algorithm based on pattern mining process. Data Min Knowl Disc 19(2):277–292

    Article  MathSciNet  Google Scholar 

  36. Zimek A, Campello RJ, Sander J (2014) Ensembles for unsupervised outlier detection: challenges and research questions a position paper. ACM SIGKDD Explor Newsl 15(1):11–22

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (Grant no. 51634002) and National Key R & D Program of China (Grant no. 2017YFB0304104).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Northeastern University, 110819, Shenyang, China

    Ping Yuan & Zhizhong Mao

  2. School of Automation, Shenyang Aerospace University, 110136, Shenyang, China

    Biao Wang

Authors
  1. Ping Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Biao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhizhong Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Biao Wang.

Ethics declarations

Conflict of interest

No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, P., Wang, B. & Mao, Z. Using multiple classifier behavior to develop a dynamic outlier ensemble. Int. J. Mach. Learn. & Cyber. 12, 501–513 (2021). https://doi.org/10.1007/s13042-020-01183-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01183-7

Keywords

Search

Navigation