Abstract
Support vector machines (SVM) is one of the well known supervised classes of learning algorithms. Basic SVM models are dealing with the situation where the exact values of the data points are known. This paper studies SVM when the data points are uncertain. With some properties known for the distributions, chance-constrained SVM is used to ensure the small probability of misclassification for the uncertain data. As infinite number of distributions could have the known properties, the robust chance-constrained SVM requires efficient transformations of the chance constraints to make the problem solvable. In this paper, robust chance-constrained SVM with second-order moment information is studied and we obtain equivalent semidefinite programming and second order cone programming reformulations. The geometric interpretation is presented and numerical experiments are conducted. Three types of estimation errors for mean and covariance information are studied in this paper and the corresponding formulations and techniques to handle these types of errors are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abe, S. (2010). Support vector machines for pattern classification. Berlin: Springer.
Ben-Hur, A., & Weston, J. (2010). A users guide to support vector machines. In O. Carugo & F. Eisenhaber (Eds.), Data mining techniques for the life sciences (pp. 223–239). Berlin: Springer.
Ben-Tal, A., Bhadra, S., Bhattacharyya, C., & Nath, J. S. (2011). Chance constrained uncertain classification via robust optimization. Mathematical Programming, 127(1), 145–173.
Bertsimas, D., & Popescu, I. (2005). Optimal inequalities in probability theory: A convex optimization approach. Siam Journal on Optimization, 15(3), 780–804.
Bhattacharyya, C., Grate, L. R., Jordan, M. I., El Ghaoui, L., & Mian, I. S. (2004). Robust sparse hyperplane classifiers: Application to uncertain molecular profiling data. Journal of Computational Biology, 11(6), 1073–1089.
Bi, J., & Zhang, T. (2005). Support vector classification with input data uncertainty. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems 17: Proceedings of the 2004 conference. Cambridge: MIT Press.
Burges, C. J. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167.
Chang, C. C., & Lin, C. J. (2011). Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 27.
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
Fan, N., Sadeghi, E., & Pardalos, P. M. (2014). Robust support vector machines with polyhedral uncertainty of the input data. In P. M. Pardalos, M. G. C. Resende, C. Vogiatzis, & J. L. Walteros (Eds.), Learning and intelligent optimization (pp. 291–305). Berlin: Springer.
Ghaoui, L. E., Lanckriet, G. R., & Natsoulis, G. (2003). Robust classification with interval data. Technical report UCB/CSD-03-1279, Computer Science Division, University of California, Berkeley.
Ghaoui, L. E., Oks, M., & Oustry, F. (2003). Worst-case value-at-risk and robust portfolio optimization: A conic programming approach. Operations Research, 51(4), 543–556.
Isii, K. (1960). The extrema of probability determined by generalized moments (i) bounded random variables. Annals of the Institute of Statistical Mathematics, 12(2), 119–134.
Lanckriet, G. R., Ghaoui, L. E., Bhattacharyya, C., & Jordan, M. I. (2002). A robust minimax approach to classification. Journal of Machine Learning Research, 3, 555–582.
Marshall, A. W., & Olkin, I. (1960). Multivariate chebyshev inequalities. The Annals of Mathematical Statistics, 31(4), 1001–1014.
Pant, R., Trafalis, T. B., & Barker, K. (2011). Support vector machine classification of uncertain and imbalanced data using robust optimization. In Proceedings of the 15th WSEAS international conference on computers (pp. 369–374). World Scientific and Engineering Academy and Society (WSEAS).
Pólik, I., & Terlaky, T. (2007). A survey of the s-lemma. SIAM Review, 49(3), 371–418.
Shivaswamy, P. K., Bhattacharyya, C., & Smola, A. J. (2006). Second order cone programming approaches for handling missing and uncertain data. Journal of Machine Learning Research, 7, 1283–1314.
Tian, Y., Shi, Y., & Liu, X. (2012). Recent advances on support vector machines research. Technological and Economic Development of Economy, 18(1), 5–33.
Trafalis, T. B., & Alwazzi, S. A. (2010). Support vector machine classification with noisy data: A second order cone programming approach. International Journal of General Systems, 39(7), 757–781.
Trafalis, T. B., & Gilbert, R. C. (2006). Robust classification and regression using support vector machines. European Journal of Operational Research, 173(3), 893–909.
Trafalis, T. B., & Gilbert, R. C. (2007). Robust support vector machines for classification and computational issues. Optimization Methods and Software, 22(1), 187–198.
Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.
Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5), 988–999.
Wang, X., & Pardalos, P. M. (2014). A survey of support vector machines with uncertainties. Annals of Data Science, 1(3–4), 293–309.
Xanthopoulos, P., Guarracino, M. R., & Pardalos, P. M. (2014). Robust generalized eigenvalue classifier with ellipsoidal uncertainty. Annals of Operations Research, 216(1), 327–342.
Xanthopoulos, P., Pardalos, P. M., & Trafalis, T. B. (2012). Robust data mining. Berlin: Springer.
Yakubovich, V. A. (1971). S-procedure in nonlinear control theory. Vestnik Leningrad University, 1, 62–77.
Zymler, S., Kuhn, D., & Rustem, B. (2013). Distributionally robust joint chance constraints with second-order moment information. Mathematical Programming, 137(1–2), 167–198.
Acknowledgments
We are grateful to Danial Kuhn and Berç Rustem for their valuable discussions. We would like to thank the anonymous reviewers for their helpful comments. Research was conducted at National Research University, Higher School of Economics, and supported by RSF grant 14-41-00039.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, X., Fan, N. & Pardalos, P.M. Robust chance-constrained support vector machines with second-order moment information. Ann Oper Res 263, 45–68 (2018). https://doi.org/10.1007/s10479-015-2039-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-015-2039-6