Abstract
Recently, Support Vector Machines with the ramp loss (RLM) have attracted attention from the computational point of view. In this technical note, we propose two heuristics, the first one based on solving the continuous relaxation of a Mixed Integer Nonlinear formulation of the RLM and the second one based on the training of an SVM classifier on a reduced dataset identified by an integer linear problem. Our computational results illustrate the ability of our heuristics to handle datasets of much larger size than those previously addressed in the literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
IBM ILOG CPLEX (2012). Www-01.ibm.com/software/integration/optimization/cplex-optimizer.
Apte, C.: The big (data) dig. OR/MS Today p. 24 (2003)
Baesens, B., Setiono, R., Mues, C., Vanthienen, J.: Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science 49(3), 312–329 (2003)
Bertsimas, D., Bjarnadóttir, M., Kane, M., Kryder, J., Pandey, R., Vempala, S., Wang, G.: Algorithmic prediction of health-care costs. Operations Research 56(6), 1382–1392 (2008)
Blake, C., Merz, C.: UCI Repository of Machine Learning Databases. University of California, Irvine, Department of Information and Computer Sciences. http://www.ics.uci.edu/~mlearn/MLRepository.html (1998)
Brooks, J.P.: Support vector machines with the ramp loss and the hard margin loss. Operations Research 59(2), 467–479 (2011)
Carrizosa, E., Martín-Barragán, B., Romero Morales, D.: Variable neighborhood search for parameter tuning in support vector machines. Tech. rep. (2012)
Carrizosa, E., Romero Morales, D.: Supervised classification and mathematical optimization. Computers and Operations Research 40, 150–165 (2013)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 1–27 (2011)
Chaovalitwongse, W.A., Fan, Y.J., Sachdeo, R.C.: Novel optimization models for abnormal brain activity classification. Operations Research 56(6), 1450–1460 (2008)
Collobert, R., Sinz, F., Weston, J., Bottou, L.: Trading convexity for scalability. In: Proceedings of the 23rd International Conference on Machine Learning, ICML06, pp. 201–208. New York, USA (2006)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Ertekin, S., Bottou, L., Giles, C.L.: Nonconvex online support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(2), 368–381 (2011)
Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Machine Learning 46, 389–422 (2002)
Han, J., Altman, R., Kumar, V., Mannila, H., Pregibon, D.: Emerging scientific applications in data mining. Communications of the ACM 45(8), 54–58 (2002)
Hand, H., Mannila, H., Smyth, P.: Principles of Data Mining. MIT Press, Cambridge (2001)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)
Liu, Y., Wu, Y.: Optimizing \(\psi \)-learning via mixed integer programming. Statistica Sinica 16, 441–457 (2006)
Loveman, G.: Diamonds in the data mine 81(5), 109–113 (2003)
Mangasarian, O., Street, W., Wolberg, W.: Breast cancer diagnosis and prognosis via linear programming. Operations Research 43(4), 570–577 (1995)
Maniezzo, V., Stützle, T., Voss, S. (eds.): Matheuristics: Hybridizing Metaheuristics and Mathematical Programming, Annals of Information Systems, vol. 10. Springer (2009)
Orsenigo, C., Vercellis, C.: Multivariate classification trees based on minimum features discrete support vector machines. IMA Journal of Management Mathematics 14(3), 221–234 (2003)
Shen, X., Tseng, G.C., Zhang, X., Wong, W.H.: On \(\psi \)-learning. Journal of the American Statistical Association 98, 724–734 (2003)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Berlin (1995)
Vapnik, V.: Statistical Learning Theory. Willey, London (1998)
Wang, L., Jia, H., Li, J.: Training robust support vector machine with smooth ramp loss in the primal space. Neurocomputing 71(13–15), 3020–3025 (2008)
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.H., Steinbach, M., Hand, D.J., Steinberg, D.: Top 10 algorithms in data mining. Knowledge and Information Systems 14, 1–37 (2007)
Wu, Y., Liu, Y.: Robust truncated hinge loss support vector machines. Journal of the American Statistical Association 102(479), 974–983 (2007)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been partially supported by projects MTM2012-36163, Ministerio de Economía y Competitividad, Spain, and FQM-329 of Junta de Andalucía, Spain, both with EU ERD Funds.
Rights and permissions
About this article
Cite this article
Carrizosa, E., Nogales-Gómez, A. & Romero Morales, D. Heuristic approaches for support vector machines with the ramp loss. Optim Lett 8, 1125–1135 (2014). https://doi.org/10.1007/s11590-013-0630-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11590-013-0630-9