Skip to main content
Springer Nature Link
Log in

DCA Based Algorithms for Feature Selection in Semi-supervised Support Vector Machines

  • Conference paper
Machine Learning and Data Mining in Pattern Recognition (MLDM 2013)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7988))

  • 4442 Accesses

Abstract

In this paper, we develop an efficient method for feature selection in Semi-Supervised Support Vector Machine (S3VM). Using an appropriate continuous approximation of the l 0 − norm, we reformulate the feature selection S3VM problem as a DC (Difference of Convex functions) program. DCA (DC Algorithm), an innovative approach in nonconvex programming is then developed to solve the resulting problem. Computational experiments on several real-world datasets show the efficiency and the scalability of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Amaldi, E., Kann, V.: On the approximability of minimizing non zero variables or unsatisfied relations in linear systems. Theoretical Computer Science 209, 237–260 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bennett, K.P., Demiriz, A.: Semi-supervised support vector machines. In: Proceedings of the Conference on Advances in Neural Information Processing Systems II, Cambridge, MA, USA, pp. 368–374 (1999)

    Google Scholar 

  3. Bradley, P.S., Mangasarian, O.L.: Feature Selection via concave minimization and support vector machines. In: Proceeding of ICML 1998, SF, CA, USA, pp. 82–90 (1998)

    Google Scholar 

  4. Chapelle, O., Zien, A.: A, Semi-supervised classification by low density separation. In: Proc. 10th Internat. Workshop on Artificial Intelligence and Statistics, Barbados, pp. 57–64 (2005)

    Google Scholar 

  5. Chapelle, O., Sindhwani, V., Keerthi, S.: Branch and bound for semi-supervised support vector machines. In: Advances in Neural Information Processing Systems, vol. 17, pp. 217–224. MIT Press, Cambridge (2006)

    Google Scholar 

  6. Chapelle, O., Sindhwani, V., Keerthi, S.S.: Optimization Techniques for Semi-Supervised Support Vector Machines. Journal of Machine Learning Research 9, 203–233 (2008)

    MATH  Google Scholar 

  7. Collobert, R., Sinz, F., Weston, J., Bottou, L.: Large scale transductive SVMs. J. Machine Learn. 7, 1687–1712 (2006)

    MathSciNet  MATH  Google Scholar 

  8. Fung, G., Mangasarian, O.: Semi-supervised support vector machines for unlabeled data classification. Optimization Methods and Software 15, 29–44 (2001)

    Article  MATH  Google Scholar 

  9. Joachims, T.: Transductive inference for text classification using support vector machines. In: 16th Inter. Conf. on Machine Learning, SF, USA, pp. 200–209 (1999)

    Google Scholar 

  10. Krause, N., Singer, Y.: Leveraging the margin more carefully. In: Proceeding of ICML 2004, NY, USA, pp. 63–71 (2004)

    Google Scholar 

  11. H.A. Le Thi, Contribution à l’optimisation non convexe et l’optimisation globale: Théorie, Algoritmes et Applications, Habilitation à Diriger des Recherches, Université de Rouen (1997).

    Google Scholar 

  12. Le Thi, H.A., Dinh, T.P.: Solving a class of linearly constrained indefinite quadratic problems by DC algorithms. Journal of Global Optimization 11(3), 253–285 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  13. Le Thi, H.A., Dinh, T.P.: The DC (difference of convex functions) Programming and DCA revisited with DC models of real world nonconvex optimization problems. Annals of Operations Research 133, 23–46 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Le Thi, H.A., Belghiti, T., Dinh, T.P.: A new efficient algorithm based on DC programming and DCA for Clustering. Journal of Global Optimization 37, 593–608 (2006)

    Google Scholar 

  15. Le Thi, H.A., Le Hoai, M., Dinh, T.P.: Optimization based DC programming and DCA for Hierarchical Clustering. European Journal of Operational Research 183, 1067–1085 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Le Thi, H.A., Le Hoai, M., Nguyen, N.V., Dinh, T.P.: A DC Programming approach for Feature Selection in Support Vector Machines learning. Journal of Advances in Data Analysis and Classification 2(3), 259–278 (2008)

    Article  Google Scholar 

  17. Le Thi, H.A.: DC Programming and DCA, http://lita.sciences.univ-metz.fr/~lethi

  18. Liu, Y., Shen, X., Doss, H.: Multicategory ψ-Learning and Support Vector Machine: Computational Tools. Journal of Computational and Graphical Statistics 14, 219–236 (2005)

    Article  MathSciNet  Google Scholar 

  19. Neumann, J., Schnörr, C., Steidl, G.: Combined SVM-based feature selection and classification. Machine Learning 61(1–3), 129–150 (2005)

    Article  MATH  Google Scholar 

  20. Thiao, M., Dinh, T.P., Le Thi, H.A.: DC programming approach for a class of nonconvex programs involving l 0 norm. In: Le Thi, H.A., Bouvry, P., Pham Dinh, T. (eds.) MCO 2008. CCIS, vol. 14, pp. 348–357. Springer, Heidelberg (2008)

    Google Scholar 

  21. Rakotomamonjy, A.: Variable Selection Using SVM-based Criteria. Journal of Machine Learning Research 3, 1357–1370 (2003)

    MathSciNet  MATH  Google Scholar 

  22. Ronan, C., Fabian, S., Jason, W., Lé, B.: Trading Convexity for Scalability. In: Proceedings of the 23rd International Conference on Machine Learning ICML 2006, Pittsburgh, Pennsylvania, pp. 201–208 (2006)

    Google Scholar 

  23. Yuille, A.L., Rangarajan, A.: The Convex Concave Procedure. Neural Computation 15(4), 915–936 (2003)

    Article  MATH  Google Scholar 

  24. Sindhwani, V., Keerthi, S., Chapelle, O.: Deterministic annealing for semi-supervised kernel machines. In: Proceedings of ICML 2006, NY, USA, pp. 841–848 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Theoretical and Applied Computer Science - LITA EA 3097, University of Lorraine, Ile du Saulcy, 57045, Metz, France

    Hoai Minh Le, Hoai An Le Thi & Manh Cuong Nguyen

Authors
  1. Hoai Minh Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hoai An Le Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manh Cuong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Vision and Applied Computer Sciences, IBaI, Leipzig, Germany

    Petra Perner

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Le, H.M., Le Thi, H.A., Nguyen, M.C. (2013). DCA Based Algorithms for Feature Selection in Semi-supervised Support Vector Machines. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2013. Lecture Notes in Computer Science(), vol 7988. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39712-7_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39712-7_41

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39711-0

  • Online ISBN: 978-3-642-39712-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics