Abstract
Hinge loss is one-sided function which gives optimal solution than that of squared error (SE) loss function in case of classification. It allows data points which have a value greater than 1 and less than \(-1\) for positive and negative classes, respectively. These have zero contribution to hinge function. However, in the most classification tasks, least square (LS) method such as ridge regression uses SE instead of hinge function. In this paper, a simple projection method is used to minimize hinge loss function through LS methods. We modify the ridge regression and its kernel based version i.e. kernel ridge regression so that it can adopt to hinge function instead of using SE in case of classification problem. The results show the effectiveness of hinge loss projection method especially on imbalanced data sets in terms of geometric mean (GM).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The Matlab code implementation of HLP is available in the author’s repository.
- 2.
Available in http://sci2s.ugr.es/keel/imbalanced.php#sub20.
References
Alcalá, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Multiple-Valued Log. Soft Comput. 17(11), 255–287 (2011)
Barandela, R., Sánchez, J.S., Garcıa, V., Rangel, E.: Strategies for learning in class imbalance problems. Pattern Recogn. 36(3), 849–851 (2003)
Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning. Springer Series in Statistics, vol. 1. Springer, Berlin (2001)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Lang, K.J., Witbrock, M.J.: Learning to tell two spirals apart. In: Proceeding of 1988 Connectionist Models Summer School (1988)
Rosasco, L., De Vito, E., Caponnetto, A., Piana, M., Verri, A.: Are loss functions all the same? Neural Comput. 16(5), 1063–1076 (2004)
Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In: (ICML-1998) Proceedings of the 15th International Conference on Machine Learning, pp. 515–521. Morgan Kaufmann (1998)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Zhang, S., Hu, Q., Xie, Z., Mi, J.: Kernel ridge regression for general noise model with its application. Neurocomputing 149, 836–846 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Alfarozi, S.A.I., Woraratpanya, K., Pasupa, K., Sugimoto, M. (2016). Hinge Loss Projection for Classification. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9948. Springer, Cham. https://doi.org/10.1007/978-3-319-46672-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-46672-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46671-2
Online ISBN: 978-3-319-46672-9
eBook Packages: Computer ScienceComputer Science (R0)