Skip to main content
SpringerLink
Log in

Kernel projection algorithm for large-scale SVM problems

  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Support Vector Machine (SVM) has become a very effective method in statistical machine learning and it has proved that training SMV is to solve Nearest Point pair Problem (NPP) between two disjoint closed convex sets. Later Keerthi pointed out that it is difficult to apply classical excellent geometric algorithms direcly to SVM and so designed a new geometric algorithm for SVM. In this article, a new algorithm for geometrically solving SVM, Kernel Projection Algorithm, is presented based on the theorem on fixed-points of projection mapping. This new algorithm makes it easy to apply classical geometric algorithms to solving SVM and is more understandable than Keerthi’s. Experiments show that the new algorithm can also handle large-scale SVM problems. Geometric algorithms for SVM, such as Keerthi’s algorithm, require that two closed convex sets be disjoint and otherwise the algorithms are meaningless. In this article, this requirement will be guaranteed in theory by using the theoretic result on universal kernel functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Vapnik V. Statistical Learning Theory. Addison-Wiley, 1998.

  2. Bennett R, Bredensteiner E J. Geometry in Learning. Tech. Report, Department of Mathematical Sciences, Rennselaer Polytechnic Institutes, New York, 1996.

    Google Scholar 

  3. Keerthi S S, Shevade S K, Bhattacharyya Cet al. A fast iterative nearest point algorithm for support vector machine classifier desing.IEE Transactions on Neural Networks, 2000, 11(1): 124–136.

    Article  Google Scholar 

  4. Crisp D J, Burges C J C A geometric interpretation of μ-SVM classifiers.NIPS 12, 2000, pp. 244–250.

    Google Scholar 

  5. Tao Qing, Sun Dermin, Fan Jinsong. The maximal margin linear classifier based on the contraction of the closed convex hull. To appear inJournal of Software.

  6. Llanas B, de Sevilla M F. An interative algorithm for finding a nearest pair of points in two convex subsets ofR n.Computers and Mathematics with Applications, 2000, 40: 971–983.

    Article  MATH  MathSciNet  Google Scholar 

  7. Platt J Fast training of support vector machines using sequential minimal optimization in Advances in Kernel Methods—Support Vector Learning. Scholkopf B, Burges C J C, Smola A J (eds.), Cambridge, MA: MIT Press, 1999, pp.185–208.

    Google Scholar 

  8. Gilbert E G Minimizing the quadratic form on a convex set.SIAM J. Contr., 1966, 4: 61–79.

    Article  MATH  Google Scholar 

  9. Mitchell B F, Dem’yanov V F, Malozemov V N. Finding the point of a polyhedron closest to the origin.SIAM J. Contr., 1974, 12: 19–26.

    Article  MATH  MathSciNet  Google Scholar 

  10. Gilbert E G, Johnson D W, Keerthi S S. A fast procedure computing the distance between complex objects in three dimension space.IEEE J. Robot. Automat., 1988, 4: 193–203.

    Article  Google Scholar 

  11. Muller K Ret al. An introduction to kernel-based learning algorithms.IEEE Trans. Neural Networks, 2001, 12(2): 181–201.

    Article  Google Scholar 

  12. Tao Qing, Wu Gaowei, Wang Jue. The theoretical analysis of kernel technique and kernel covering approach. Technique Report, Institute of Automation, Chinese Academy of Sciences, 2001.

  13. Tao Qing, Wang Jiaqi, Wang Jue. Soft kernel projection algorithm for support vector machines. Technique Report, Institute of Automation, Chinese Academy of Sciences, 2001.

Download references

Author information

Authors and Affiliations

  1. Institute of Automation, The Chinese Academy of Sciences, 10080, Beijing, P.R. China

    Wang Jiaqi, Tao Qing & Wang Jue

Authors
  1. Wang Jiaqi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Qing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wang Jue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wang Jiaqi.

Additional information

This research is supported by the NKBRSF of China (Grant No. G1998030508), the National Natural Science Foundation of in China (Grant No. 60175032) and the Pilot Program of the Knowledge Innovation Project of Chinese Academy of Sciences.

WANG Jiaqi received his B.S. degree from Beijing Polytechnic University in 1998. He is currently a graduate student in Institute of Automation, Chinese Academy of Sciences, P.R. China. His research interests are data mining, machine learning and kernel method.

TAO Qing received the M.S. degree in mathematics from Southwest Normal University in 1989 and the Ph.D. degree from the University of Science and Technology of China in 1999. From June 1999 to June 2001, he was a Postdoctoral Fellow in the University of Science and Technology of China. He is currently a Postdoctoral Fellow in Institute of Automation, Chinese Academy of Sciences. His research interests are neural networks, nonlinear function analysis and SVM theory.

WANG Jue is a professor in Institute of Automation, Chinese Academy of Science. His research interests are ANN, GA, multi-agent system, machine learning and data mining.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, J., Tao, Q. & Wang, J. Kernel projection algorithm for large-scale SVM problems. J. Comput. Sci. & Technol. 17, 556–564 (2002). https://doi.org/10.1007/BF02948824

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02948824

Keywords

Search

Navigation