Skip to main content
SpringerLink
Log in

Efficient support vector data descriptions for novelty detection

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Support vector data description (SVDD) is a very attractive kernel method in many novelty detection problems. However, the decision function of SVDD is expressed in terms of the kernel expansion, which results in a run-time complexity linear in the number of support vectors (SVs). In this paper, an efficient SVDD (E-SVDD) is proposed to improve the prediction speed of SVDD. This proposed E-SVDD first finds some crucial feature vectors by the partitioning-entropy-based kernel fuzzy c-means (KFCM) cluster technique and then uses the images of the preimages of these feature vectors to reexpress the center of SVDD. Hence, the decision function of E-SVDD only contains some crucial kernel terms, and the complexity of E-SVDD is linear in the number of the clusters. The experimental results on several benchmark datasets indicate that E-SVDD not only obtains fast prediction speed but also shows better stable generalization than other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Available at: http://www.mathworks.co.

  2. Available from http://ict.ewi.tudelft.nl, http://davidt/occ/index.html and http://www.liacc.up.pt/ML/old/statlog/datasets.htm.

References

  1. Markou M, Singh S (2003) Novelty detection: a review, part I: statistical approaches. Signal Process 83(12):2481–2497

    Article  MATH  Google Scholar 

  2. Markou M, Singh S (2003) Novelty detection: a review, part II: neural network based approaches. Signal Process 83(12):2499–2521

    Article  MATH  Google Scholar 

  3. Lee K, Kim D, Lee K, Lee D (2007) Density-induced support vector data desciption. IEEE Trans Neural Netw 18(1):284–289

    Article  Google Scholar 

  4. Vapnik V (1995) The natural of statistical learning theory. Springer, New York

    Google Scholar 

  5. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  6. Campbell C, Bennett K (2001) A linear programming approach to novelty detection. In: Advances in neural information processing systems, vol 13. MIT Press, Cambridge, MA

  7. Tax D, Duin R (1999) Support vector domain description. Pattern Recogn Lett 20:1191–1199

    Article  Google Scholar 

  8. Tax D, Duin R (2004) Support vector data description. Mach Learn 54:45–66

    Article  MATH  Google Scholar 

  9. Roberts S, Tarassenko L (1994) A probabilistic resource allocation network for novelty detection. Neural Comput Appl 6:270–284

    Article  Google Scholar 

  10. Crammer K, Chechik G (2004) A needle in a haystack: local one-class optimization. In: Proceedings of 21th international conference on machine learning, p 26

  11. Hoffmann H (2007) Kernel PCA for novelty detection. Pattern Recogn Lett 40(3):863–874

    Article  MATH  Google Scholar 

  12. Lanckriet G, Ghaoui L, Jordan M (2003) Robust novelty detection with single-class MPM. In: Advances in neural information processing systems, vol 15. MIT Press, Cambridge, MA, pp 92–936

  13. Schölkopf B, Mika S, Burges C, Knirsch P, Müller K, Rätsch G, Smola A (1999) Input space vs. feature space in kernel-based methods. IEEE Trans Neural Netw 10(5):1000–1017

    Article  Google Scholar 

  14. Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, Cambridge

    Google Scholar 

  15. Scott C, Nowak R (2006) Learning minimum volume sets. J Mach Learn Res 7:665–704

    MathSciNet  MATH  Google Scholar 

  16. Steinwart I, Hush D, Scovel C (2005) A classification framework for anomaly detection. J Mach Learn Res 6:211–232

    MathSciNet  MATH  Google Scholar 

  17. Vert R, Vert J (2006) Consistency and convergence rates of one-class SVM and related algorithms. J Mach Learn Res 7:817–854

    MathSciNet  MATH  Google Scholar 

  18. Liu Y, Lin S, Hsueh Y, M.L. L (2009) Automatic target defect identification for TFT-LCD array process inspection using kernel FCM-based fuzzy SVDD ensemble. Expert Syst Appl 36(2):1978–1998

    Article  Google Scholar 

  19. Park J, Kang D, Kim J, Kwok J, Tsang I (2007) Svdd-baded pattern denoising. Neural Comput Appl 19(7):1919–1938

    Article  MathSciNet  MATH  Google Scholar 

  20. Nanni L (2006) Machine learning algorithms for T-cell epitopes prediction. Neurocomputing 69(7–9):866–868

    Article  Google Scholar 

  21. Banerjee A, Burlina P, Diehl C (2006) A support vector method for anomaly detection in hyperspectral imagery. IEEE Trans Geosci Remote Sens 44(8):2282–2291

    Article  Google Scholar 

  22. Burges C (1996) Simplified support vector decision rules. In: Proceedings of the 13th international conference on machine learning, pp 71–77

  23. Thies T, Weber F (2004) Optimal reduced-set vectors for support vector machines with a quadratic kernel. Neural Comput Appl 16:1769–1777

    Article  MATH  Google Scholar 

  24. Downs T, Gates K, Masters A (2002) Exact simplification of support vector solutions. J Mach Learn Res 2:293–297

    MathSciNet  MATH  Google Scholar 

  25. Guo J, Takahashi N, Nishi T (2005) A learning algorithm for improving the classification speed of support vector machines. In: Proceedings of 2005 European conference on circuit theory and design

  26. Jiao L, Bo L, Wang L (2007) Fast sparse approximation for least squares support vector machine. IEEE Trans Neural Netw 18:685–697

    Article  Google Scholar 

  27. Suykens J, Lukas L, van Dooren P, De Moor B, Vandewalle J (1999) Least squares support vector machine classifiers: a large scale algorithm. In: Proceedings of European conference of circuit theory design, pp 839–842

  28. Suykens J, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  MathSciNet  Google Scholar 

  29. Bo L, Wang L, Jiao L (2007) Selecting a reduced set for building sparse support vector regression in the primal. In: Advances in knowledge discovery and data mining, pp 35–46

  30. Liang X, Chen R, Guo X (2008) Pruning support vector machines without altering performances. IEEE Trans Neural Netw 19(10):1792–1803

    Article  Google Scholar 

  31. Li Q, Jiao L, Hao Y (2007) Adaptive simplification of solution for support vector machine. Pattern Recogn Lett 40:972–980

    MATH  Google Scholar 

  32. Joachims T, Yu C (2009) Sparse kernel SVMs via cutting-plane training. Mach Learn 76(2-3):179–193

    Article  Google Scholar 

  33. Liu Y, Liu Y, Chen Y (2010) Fast support vector data description for novelty detection. IEEE Trans Neural Netw 21(8):1296–1313

    Article  Google Scholar 

  34. Mercer J (1909) Functions of positive and negative type and the connection with the theory of integal equations. Philos Trans R Soc Lond Ser A 209:415–446

    Article  MATH  Google Scholar 

  35. Bakir G, Weston J, Schölkopf B (2004) Learning to find pre-images. In: Thrun J, Saul L, Schölkopf B (eds) Advances in neural information processing systems. vol 16, MIT Press, Cambridge, MA, pp 449–456

    Google Scholar 

  36. Burges C, Schölkopf B (1997) Improving the accuracy and speed of support vector learning machines. In: Mozer M, Jordan M, Petsche T (eds) Advances in neural information processing systems. vol 9, MIT Press, Cambridge, MA, pp 375–381

    Google Scholar 

  37. Kwok J, Tsang I (2004) The pre-image problem in kernel methods. IEEE Trans Neural Netw 15(6):1517–1525

    Article  Google Scholar 

  38. Mika S, Schölkopf B, Smola A, Müller K, Scholz M, Rätsch G (1998) Kernel pca and de-noising in feature space. In: Kearns M, Solla S, Cohn D (eds) Advances in neural information processing systems, vol 11. Morgan Kaufmann, San Mateo, CA

  39. Bezdek J (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York

    Book  Google Scholar 

  40. Wu Z, Xie W (2003) Fuzzy c-means clustering algorithm based on kernel method. In: Proceedings of fifth international conference on computational intelligence and multimedia applications, pp 49–54

  41. Lin C, Lee C (1996) Neural fuzzy systems: a neuro-fuzzy synergism to intelligent system. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  42. Höppner F, Klawonn F, Kruse R, Runkler T (1999) Fuzzy cluster analysis: methods for classification, data analysis and image recognition. Wiley, New York

    MATH  Google Scholar 

  43. Huang H, Liu Y (2002) Fuzzy support vector machines for pattern recognition and data mining. Int J Fuzzy Syst 4:826–835

    MathSciNet  Google Scholar 

  44. Meyer C (2000) Matrix analysis and applied linear algebra. SIAM, Philadelphia

  45. Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. In: Proceedings of 14th international conference on machine learning

  46. Wu G, Chang E (2003) Class-boundary alignment for imbalanced dataset learning. In: Proceedings of international conference on machine learning workshop learning from imbalanced datasets

Download references

Acknowledgments

Supported by the Innovative Project of Shanghai Municipal Education Commission (11YZ81), the Natural Science Foundation of SHNU (SK200937, SK201030), and the Shanghai Leading Academic Discipline Project (S30405).

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai Normal University, 200234, Shanghai, People’s Republic of China

    Xinjun Peng

  2. Scientific Computing Key Laboratory of Shanghai Universities, 200234, Shanghai, People’s Republic of China

    Xinjun Peng

  3. Department of Mathematics, Shanghai Normal University, 200234, Shanghai, People’s Republic of China

    Dong Xu

Authors
  1. Xinjun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinjun Peng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Peng, X., Xu, D. Efficient support vector data descriptions for novelty detection. Neural Comput & Applic 21, 2023–2032 (2012). https://doi.org/10.1007/s00521-011-0625-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0625-3

Keywords

Search

Navigation