Abstract
We consider a parallel decomposition technique for solving the large quadratic programs arising in training the learning methodology Support Vector Machine. At each iteration of the technique a subset of the variables is optimized through the solution of a quadratic programming subproblem. This inner subproblem is solved in parallel by a special gradient projection method. In this paper we consider some improvements to the inner solver: a new algorithm for the projection onto the feasible region of the optimization subproblem and new linesearch and steplength selection strategies for the gradient projection scheme. The effectiveness of the proposed improvements is evaluated, both in terms of execution time and relative speedup, by solving large-scale benchmark problems on a parallel architecture.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Chang, C.C., Lin, C.J.: LIBSVM: a Library for Support Vector Machines (2002), www.csie.ntu.edu.tw/~cjlin/libsvm
Collobert, R., Benjo, S.: SVMTorch: Support Vector Machines for Large-Scale Regression Problems. Journal of Machine Learning Research 1, 143–160 (2001)
Dai, Y.H., Fletcher, R.: New Algorithms for Singly Linearly Constrained Quadratic Programs Subject to Lower and Upper Bounds, Research Report NA/216. Department of Mathematics, University of Dundee (2003)
Joachims, T.: Making Large-Scale SVM Learning Practical. In: Schölkopf, B., et al. (eds.) Advances in Kernel Methods. MIT Press, Cambridge (1998)
Lin, C.J.: On the Convergence of the Decomposition Method for Support Vector Machines. IEEE Transactions on Neural Networks 12(6), 1288–1298 (2001)
Pardalos, P.M., Kovoor, N.: An Algorithm for a Singly Constrained Class of Quadratic Programs Subject to Upper and Lower Bounds. Math. Programming 46, 321–328 (1990)
Ruggiero, V., Zanni, L.: A Modified Projection Algorithm for Large Strictly Convex Quadratic Programs. J. Optim. Theory Appl. 104(2), 281–299 (2000)
Ruggiero, V., Zanni, L.: Variable Projection Methods for Large Convex Quadratic Programs. In: Trigiante, D. (ed.) Recent Trends in Numerical Analysis. Advances in the Theory of Computational Mathematics 3, pp. 299–313. Nova Science Publ. (2000)
Serafini, T.: Gradient Projection Methods for Quadratic Programs and Applications in Training Support Vector Machines. Ph.D. Thesis, Dept. of Mathematics, University of Modena and Reggio Emilia (2005)
Serafini, T., Zanghirati, G., Zanni, L.: Gradient Projection Methods for Large Quadratic Programs and Applications in Training Support Vector Machines. Optim. Meth. and Soft. 20, 353–378 (2005)
Serafini, T., Zanghirati, G., Zanni, L.: Parallel Decomposition Approaches for Training Support Vector Machines. In: Joubert, G.R., Nagel, W.E., Peters, F.J., Walter, W.V. (eds.) Parallel Computing: Software Technology, Algorithms, Architectures and Applications. Advances in Parallel Computing, vol. 13, pp. 259–266. Elsevier, Amsterdam (2004)
Serafini, T., Zanni, L.: On the working set selection in Gradient Projection-based Decomposition Techniques for Support Vector Machines.Optim. Meth. and Soft. (2005) (to appear), http://cdm.unimo.it/home/matematica/zanni.luca/
Vapnik, V.N.: Statistical Learning Theory. John Wiley and Sons, New York (1998)
Zanghirati, G., Zanni, L.: A Parallel Solver for Large Quadratic Programs in Training Support Vector Machines. Parallel Computing 29, 535–551 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Serafini, T., Zanni, L., Zanghirati, G. (2005). Some Improvements to a Parallel Decomposition Technique for Training Support Vector Machines. In: Di Martino, B., Kranzlmüller, D., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2005. Lecture Notes in Computer Science, vol 3666. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11557265_7
Download citation
DOI: https://doi.org/10.1007/11557265_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29009-4
Online ISBN: 978-3-540-31943-6
eBook Packages: Computer ScienceComputer Science (R0)