Skip to main content
SpringerLink
Log in

Pinball loss support vector data description for outlier detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Support vector data description (SVDD) has been widely used in outlier detection. The conventional SVDD employs the hinge loss function and the sphere classifier is decided by only a small amount of data around the sphere surface (namely support vectors), which makes it sensitive to noise and unstable for re-sampling. In this paper, we put forward a novel support vector data description method with pinball loss (pin-SVDD). In our method, all the training data, including those lying inside the sphere, is decisive to the sphere classifier. A small amount of noisy data has little influence on the classifier, which makes our method more robust to noise and achieve scatter minimization in the sphere center. Pin-SVDD has two main merits. (1) Different from the conventional SVDD which employs the hinge loss function and is sensitive to noise, pin-SVDD applies the pinball loss which makes our method more robust to noise and achieve scatter minimization in the sphere center. (2) Distinguished from the existing anti-noise SVDD methods which are based on weight varying and need an extra preprocessing time to generate the instance weights, pin-SVDD does not need preprocessing time and has the same time complexity with the conventional SVDD. Hence, pin-SVDD shows better robustness than the conventional SVDD, but has the same time complexity. The experiment result shows that pin-SVDD has better outlier detection performance than state-of-the-art SVDD-based outlier detection methods, and needs less time on training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Since the conventional SVDD employs the hinge loss function, the conventional SVDD is also referred to as hinge loss SVDD in this paper.

  2. The UCI datasets used in our experiments are available online from http://homepage.tudelft.nl/n9d04/occ/index.html

References

  1. David MJ, TaxRobert P, Duin W Support vector data description, Machine Learning

  2. Bernhard S (2003) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

    Google Scholar 

  3. Navia-Vazquez A, Gutierrez-Gonzalez D, Parrado-Hernandez E, Navarro-Abellan JJ (2006) Distributed support vector machines. IEEE Trans Neural Netw 17(4):1091–1097

    Article  Google Scholar 

  4. Chandola V, Banerjee A, Kumar V Anomaly detection: A survey, ACM Computing Surveys 41 (3)

  5. Li H, Yuan Y, Fan Z-P, Liu Y (2014) A fta-based method for risk decision-making in emergency response, 42, 49–57

  6. Giacinto G, Perdisci R, Rio MD, Roli F Intrusion detection in computer networks by a modular ensemble of one-class classifiers, Information Fusion

  7. Lee K, Kim DW, Lee KH, Lee D (2007) Density-induced support vector data description. IEEE Trans Neural Netw 18:284– 289

    Article  Google Scholar 

  8. Aggarwal CC, Yu PS (2008) Outlier detection with uncertain data. In: DBLP

  9. Bi J, Zhang T (2004) Support vector classification with input data uncertainty. 17

  10. Cha M, Kim JS, Baek JG (2014) Density weighted support vector data description. Expert Syst Appl 41(7):3343–3350

    Article  Google Scholar 

  11. Chen G, Zhang X, Wang ZJ, Li F (2015) Robust support vector data description for outlier detection with noise or uncertain data. Knowl-Based Syst 90(DEC):129–137

    Article  Google Scholar 

  12. Liu B, Xiao Y, Cao L, Hao Z, Deng F (2013) Svdd-based outlier detection on uncertain data. Knowledge Inf Syst 34(3):597– 618

    Article  Google Scholar 

  13. Wang CD, Lai JH (2013) Position regularized support vector domain description. Pattern Recogn 46(3):875–884

    Article  Google Scholar 

  14. Liu B, Xiao Y, Yu PS, Hao Z, Cao L (2014) An efficient approach for outlier detection with imperfect data labels. Knowledge & Data Engineering IEEE Transactions on 26(7):1602–1616

    Article  Google Scholar 

  15. Ergen T, Kozat SS (2020) Unsupervised anomaly detection with lstm neural networks. IEEE Trans Neural Netw Learn Syst 31(8):3127–3141. https://doi.org/10.1109/TNNLS.2019.2935975

    Article  MathSciNet  Google Scholar 

  16. Görnitz N, Lima LA, Müller K-R, Kloft M, Nakajima S (2018) Support vector data descriptions and k -means clustering: One class? IEEE Trans Neural Netw Learn Syst 29(9):3994–4006. https://doi.org/10.1109/TNNLS.2017.2737941

    Article  MathSciNet  Google Scholar 

  17. Du W, Tian Y, Qian F (2014) Monitoring for nonlinear multiple modes process based on ll-svdd-mrda. IEEE Trans Autom Sci Eng 11(4):1133–1148. https://doi.org/10.1109/TASE.2013.2285571

    Article  Google Scholar 

  18. Turkoz M, Kim S, Son Y, Jeong MK, Elsayed EA (2019) Generalized support vector data description for anomaly detection. Pattern Recog 100:107119

    Article  Google Scholar 

  19. Koenker R, Hall R, Jones C, Roberts J, Samuelson L, Lothgren M, Tambour M, Diewert E, Theil H, Thirlwall AP (2004) Quantile regression for longitudinal data, Academic Press Inc.

  20. Christmann A, Steinwart I (2008) How svms can estimate quantiles and the median. In: Platt J, Koller D, Singer Y, Roweis S (eds) Advances in neural information processing systems, vol 20. Curran Associates, Inc

  21. Steinwart I, Christmann A (2011) Estimating conditional quantiles with the help of the pinball loss. Bernoulli 17(1):211–225. https://doi.org/10.3150/10-BEJ267

    Article  MathSciNet  MATH  Google Scholar 

  22. Huang X, Shi L, Suykens JAK (2014) Support vector machine classifier with pinball loss. IEEE Trans Pattern Anal Mach Intell 36(5):984–997

    Article  Google Scholar 

  23. Xu Y, Yang Z, Pan X (2017) A novel twin support-vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370

    Article  MathSciNet  Google Scholar 

  24. Xu Y, Yang Z, Zhang Y, Pan X, Wang L (2016) A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification. Knowledge Based Systems 95(Mar.1):75–85

    Article  Google Scholar 

  25. Xu Y, Wang Q, Pang X, Tian Y (2018) Maximum margin of twin spheres machine with pinball loss for imbalanced data classification. Appl Intell 48(1):1–12

    Article  Google Scholar 

  26. Gong R, Wu C, Chu M, Wang H (2016) Twin pinball loss support vector hyper-sphere classifier for pattern recognition. In: Control & decision conference

  27. Wang K, Lan H (2020) Robust support vector data description for novelty detection with contaminated data. Eng Appl Artif Intell 91:103554

    Article  Google Scholar 

  28. Liu B, Yin J, Xiao Y, Cao L, Yu PS (2011) Exploiting local data uncertainty to boost global outlier detection. In: IEEE International conference on data mining

  29. Lazarevic A, Kumar V (2005) Feature bagging for outlier detection. In: Eleventh acm sigkdd international conference on knowledge discovery in data mining

  30. Wu M, Ye J (2009) A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans Pattern Anal Mach Intell 31(11):2088–2092

    Article  Google Scholar 

  31. Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning the. MIT Press

  32. Michael S (2008) Handbook of parametric and nonparametric statistical procedures (4th ed.)., Am Stat

  33. Demiar J, Schuurmans D (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(1):1–30

    MathSciNet  Google Scholar 

  34. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701

    Article  MATH  Google Scholar 

  35. Nemenyi PB (1963) Distribution-free multiple comparisons. Princeton University

Download references

Acknowledgements

The authors would like to thank the reviewers for their very useful comments and suggestions. This work was supported in part by the Natural Science Foundation of China under Grant 61876044 and Grant 62076074, in part by Guangdong Natural Science Foundation under Grant 2020A1515010670 and 2020A1515011501 in part by the Science and Technology Planning Project of Guangzhou under Grant 202002030141.

Author information

Authors and Affiliations

  1. Guangdong University of Technology - Panyu Campus, No.100 Waihuanxi Road, Higher Education Mega Center, Guangzhou, Guangdong, 510006, China

    Guangzheng Zhong, Yanshan Xiao, Bo Liu, Liang Zhao & Xiangjun Kong

Authors
  1. Guangzheng Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanshan Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangjun Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanshan Xiao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, G., Xiao, Y., Liu, B. et al. Pinball loss support vector data description for outlier detection. Appl Intell 52, 16940–16961 (2022). https://doi.org/10.1007/s10489-022-03237-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03237-5

Keywords

Search

Navigation