Abstract
We consider the problem of finding the optimal specification of hyper-parameters in Support Vector Machines (SVMs). We sample the hyper-parameter space and then use Bézier curves to approximate the performance surface. This geometrical approach allows us to use the information provided by the surface and find optimal specification of hyper-parameters. Our results show that in most cases the specification found by the proposed algorithm is very close to actual optimal point(s). The results suggest that our algorithm can serve as a framework for hyper-parameter search, which is precise and automatic.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Gärtner, B., Jaggi, M., Maria, C.: An exponential lower bound on the complexity of regularization paths. J. Comput. Geom. 3(1), 168–195 (2012)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27 (2011). http://www.csie.ntu.edu.tw/cjlin/libsvm
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407–499 (2004)
Giesen, J., Jaggi, M., Laue, S.: Approximating parameterized convex optimization problems. ACM Trans. Algorithms 9(1), 10:1–10:17 (2012). http://doi.acm.org/10.1145/2390176.2390186
Giesen, J., Laue, S., Wieschollek, P.: Robust and efficient kernel hyperparameter paths with guarantees. In: Jebara, T., Xing, E.P. (eds.) Proceedings of the 31st International Conference on Machine Learning (ICML 2014) and JMLR Workshop and Conference Proceedings, pp. 1296–1304 (2014). http://jmlr.org/proceedings/papers/v32/giesen14.pdf
Giesen, J., Mueller, J., Laue, S., Swiercy, S.: Approximating concavely parameterized optimization problems. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems 25, pp. 2105–2113. Curran Associates, Inc. (2012). http://papers.nips.cc/paper/4578-approximating-concavely-parameterized-optimization-problems.pdf
Harlow, J., Sainudiin, R., Tucker, W.: Mapped regular pavings. Reliable Comput. 16, 252–282 (2012)
Hastie, T., Rosset, S., Tibshirani, R., Zhu, J.: The entire regularization path for the support vector machine. J. Mach. Learn. Res. 5, 1391–1415 (2004). http://dl.acm.org/citation.cfm?id=1005332.1044706
Keerthi, S.S., Lin, C.J.: Asymptotic behaviors of support vector machines with gaussian kernel. Neural Comput. 15(7), 1667–1689 (2003). http://dx.doi.org/10.1162/089976603321891855
Prautzsch, H., Boehm, W., Paluszny, M.: Bezier and B-Spline Techniques. Springer, Heidelberg (2002)
Steinwart, I., Christmann, A.: Support Vector Machines, 1st edn. Springer, New York (2008)
Vapnik, V.N.: Statistical Learning Theory, 1st edn. Wiley, New York (1998)
Yan, Z., Yang, Y., Yunjing, D.: An experimental study of the hyper-parameters distribution region and its optimization method for support vector machine with gaussian kernel. Int. J. Signal Process. Image Process. Pattern Recogn. 6(5), 437–446 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Yamada, S., Neshatian, K., Sainudiin, R. (2015). Optimal Hyper-Parameter Search in Support Vector Machines Using Bézier Surfaces. In: Pfahringer, B., Renz, J. (eds) AI 2015: Advances in Artificial Intelligence. AI 2015. Lecture Notes in Computer Science(), vol 9457. Springer, Cham. https://doi.org/10.1007/978-3-319-26350-2_55
Download citation
DOI: https://doi.org/10.1007/978-3-319-26350-2_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26349-6
Online ISBN: 978-3-319-26350-2
eBook Packages: Computer ScienceComputer Science (R0)