Skip to main content
SpringerLink
Log in

A Method for Reducing the Number of Support Vectors in Fuzzy Support Vector Machine

  • Conference paper
  • First Online:
Advanced Computational Methods for Knowledge Engineering

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 453))

  • 618 Accesses

Abstract

We offer an efficient method to reduce the number of support vectors for Fuzzy Support Vector Machine. Firstly, we consider the Fuzzy Support Vector Machine model which was proposed by Lin and Wang. For the reducing the number of support vectors, we apply the \(l_0\) regularization term to the dual form of this model. The resulting optimization problem is non-smooth and non-convex. The \(l_0\) is then replaced by an approximation function. An algorithm which is based on DC programming and DCA is then investigated to solve this problem. Numerical results on real-world datasets show the efficiency and the superiority of our method versus the standard algorithm on both support vector reduction and classification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Bradley, P.S., Mangasarian, O.L.: Feature selection via concave minimization and support vector machines. In: Shavlik, J. (ed.) Machine Learning Proceedings of the Fifteenth International Conferences (ICML’98), pp. 82–90. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  2. Brank, J., Grobelnik, M., Milic-Frayling, N., Mladenic, D.: Feature selection using linear support vector machines. In: Proceedings of the 3rd International Conference on Data Mining Methods and Databases for Engineering (2002)

    Google Scholar 

  3. Burges, C.J.: Simplified support vector decision rules. In: Machine learning-International Workshop then Conference, pp. 71–77 (1996)

    Google Scholar 

  4. Downs, T., Gates, K.E., Masters, A.: Exact simplification of support vector solutions. J. Mach. Learn. Res. 2, 293–297 (2002)

    MathSciNet  MATH  Google Scholar 

  5. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its Oracle Properties. J. Am. Stat. Assoc. 96, 1348–1360 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  6. Figueiredo, M.A.: Adaptive sparseness for supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1150–1159 (2003)

    Article  Google Scholar 

  7. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  8. Hui, Z.: The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 101(476), 1418–1429 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Huang, J., Ma, S., Zhang, C.H.: Adaptive lasso for sparse high-dimentional regression models. Stat. Sinica 18, 1603–1618 (2008)

    MathSciNet  MATH  Google Scholar 

  10. Huang, K., Zheng, D., Sun, J., Hotta, Y., Fujimoto, K., Naoi, S.: Sparse learning for support vector classification. Pattern Recogn. Lett. 31, 1944–1951 (2010)

    Article  Google Scholar 

  11. Huang, K., King, I., Lyu, M.R.: Direct zero-norm Optimization for feature selection. In: Eighth IEEE International Conference on Data Mining, pp. 840–850 (2008)

    Google Scholar 

  12. Le Thi, H.A., Pham Dinh, T.: The DC (Difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133, 23–46 (2005)

    Google Scholar 

  13. Le Thi, H.A., Belghiti, T., Pham Dinh, T.: A new efficient algorithm based on DC programming and DCA for Clustering. J. Global Optim. 37, 593–608 (2006)

    Google Scholar 

  14. Le Thi, H.A., Le Hoai, M., Pham Dinh, T.: Optimization based DC programming and DCA for Hierarchical Clustering. Eur. J. Oper. Res. 183, 1067–1085 (2007)

    Google Scholar 

  15. Le Thi, H.A., Le Hoai, M., Nguyen, V.V., Pham Dinh, T.: A DC Programming approach for feature selection in support vector machines learning. J. Adv. Data Anal. Classif. 2(3), 259–278 (2008)

    Google Scholar 

  16. Le Thi, H.A., Nguyen, V.V., Ouchani, S.: Gene selection for cancer classification using DCA. In: Tang, C., Ling, C.X., Zhou, X., Cercone, N.J., Li, X. (eds.) ADMA 2008. LNCS (LNAI), vol. 5139, pp. 62–72. Springer, Heidelberg (2008)

    Google Scholar 

  17. Le Thi, H.A.: A new approximation for the \(\ell _{0}\)-norm. Research Report LITA EA 3097, University of Lorraine (2012)

    Google Scholar 

  18. Le Thi, H.A., Pham Dinh, T., Le Hoai, M., Vo, X.T.: DC approximation approaches for sparse optimization. Eur. J. Oper. Res. 244(1), 26–46 (2015)

    Google Scholar 

  19. Le Hoai, M., Le Thi, H.A., Nguyen, M.C.: Sparse semi-supervised support vector machines by DC programming and DCA. Neurocomputing 153, 62–76 (2015)

    Article  Google Scholar 

  20. Le Thi, H.A., Nguyen, M.C.: Efficient algorithms for feature selection in multi-class support vector machine. In: Advanced Computational Methods for Knowledge Engineering, Studies in Computational Intelligence, vol. 479, pp. 41–52, Springer (2013)

    Google Scholar 

  21. Le Thi, H.A., Huynh, V.N., Pham Dinh, T.: Exact penalty and error bounds in DC programming. J. Global Optim. Dedic. Reiner Horst. ISSN 0925-5001. doi:10.1007/s10898-011-9765-3 (2011)

  22. Lee, J.Y., Mangasarian, O.L.: RSVM: reduced support vector machines. In: Proceedings of the First SIAM International Conference on Data Mining, pp. 5–7 (2001)

    Google Scholar 

  23. Lee, Y.J., Huang, S.Y.: Reduced support vector machines: a statistical theory. IEEE Trans. Neural Networks 18, 1–13 (2007)

    Article  Google Scholar 

  24. Lee, G.H., Taur, J.S., Tao, C.W.: A Robust Fuzzy support vector machine for two-class pattern classification. Int. J. Fuzzy Syst. 8(2), 76–86 (2006)

    Google Scholar 

  25. Li, Q., Jiao, L., Hao, Y.: Adaptive simplification of solution for support vector machine. Pattern Recogn. 40, 972–980 (2007)

    Article  MATH  Google Scholar 

  26. Li, Y., Zhang, W., Lin, C.: Simplify support vector machines by iterative learning. Neural Inf. Process. Lett. Rev. 10, 11–17 (2006)

    Google Scholar 

  27. Lin, C.F., Wang, S.D.: Fuzzy support vector machines. IEEE Trans. Neural Networks 13(2), 464–471 (2002). doi:10.1109/72.991432

    Article  Google Scholar 

  28. Jian, L., Xia, Z.: Fuzzy support vector machines based algorithm for peptide identification from Tandem mass spectra. Int. J. Pure Appl. Math. 76(3), 439–447 (2012)

    Google Scholar 

  29. Nguyen, D., Ho, T.: An efficient method for simplifying support vector machines. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 617–624 (2005)

    Google Scholar 

  30. Ong, C.S., Le Thi, H.A.: Learning sparse classifiers with difference of Convex functions Algorithms. Optim. Methods Softw. 28, 4 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. Dinh, P., T., Le Thi, H.A.: Convex analysis approach to d.c. programming: theory, algorithm and applications. Acta Mathematica Vietnamica 22, 289–355 (1997)

    Google Scholar 

  32. Pham Dinh, T., Le Thi, H.A.: Optimization algorithms for solving the trust region subproblem. SIAM J. Optim. 2, 476–505 (1998)

    Google Scholar 

  33. Pham Dinh, T., Le Thi, H.A.: Recent advances on DC programming and DCA. Trans. Comput. Intell. XIII Lect. Notes Comput. Sci. 8342, 1–37 (2014)

    Google Scholar 

  34. He, Q., Congxin, W.: Membership evaluation and feature selection for fuzzy support vector machine based on fuzzy rough sets. Soft Comput. 15, 1105–1114 (2011). doi:10.1007/s00500-010-0577-z

    Article  Google Scholar 

  35. Rakotomamonjy, A.: Variable selection using SVM-based criteria. J. Mach. Learn. Res. 3, 1357–1370 (2003)

    MathSciNet  MATH  Google Scholar 

  36. Keerthi, S.S., Chapelle, O., DeCoste, D.: Building support vector machines with reduced classifier complexity. J. Mach. Learn. Res. 7, 1493–1515 (2006)

    Google Scholar 

  37. Scholkopf, B., Mika, S., Burges, C.J., Knirsch, P., Muller, K.R., Ratsch, G.: Input space versus feature space in kernel-based methods. IEEE Trans. Neural Networks 10, 1000–1017 (1999)

    Article  Google Scholar 

  38. Shifei, D.I.N.G., Yaxiang, G.U.: A fuzzy support vector machine algorithm with dual membership based on hypersphere. J. Comput. Inf. Syst. 7(6), 2028–2034 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Hanoi University of Industry - Vietnam, Hanoi, Vietnam

    Nguyen Manh Cuong & Nguyen Van Thien

Authors
  1. Nguyen Manh Cuong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nguyen Van Thien
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nguyen Manh Cuong .

Editor information

Editors and Affiliations

  1. International Institute for Applied Systems Analysis (IIASA), Laxenburg, Austria

    Thanh Binh Nguyen

  2. Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary

    Tien van Do

  3. Laboratory of Theoretical and Applied Computer Science (LITA), UFR MIM, University of Lorraine, Ile du Saulcy, Metz, France

    Hoai An Le Thi

  4. Institute of Informatics, Wrocław University of Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Manh Cuong, N., Van Thien, N. (2016). A Method for Reducing the Number of Support Vectors in Fuzzy Support Vector Machine. In: Nguyen, T.B., van Do, T., An Le Thi, H., Nguyen, N.T. (eds) Advanced Computational Methods for Knowledge Engineering. Advances in Intelligent Systems and Computing, vol 453. Springer, Cham. https://doi.org/10.1007/978-3-319-38884-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-38884-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-38883-0

  • Online ISBN: 978-3-319-38884-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation