Abstract
The concept of a large margin is central to support vector machines and it has recently been adapted and applied for nearest neighbour classification. In this paper, we suggest a modification of this method in order to be used for regression problems. The learning of a distance metric is performed by means of an evolutionary algorithm. Our technique allows the use of a set of prototypes with different distance metrics, which can increase the flexibility of the method especially for problems with a large number of instances. The proposed method is tested on a real world problem – the prediction of the corrosion resistance of some alloys containing titanium and molybdenum – and provides very good results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D.W., Goldstone, R.L.: Concept learning and flexible weighting. In: Proceedings of the 14th Annual Conference of the Cognitive Science Society, pp. 534–539 (1992)
Assi, K.C., Labelle, H., Cheriet, F.: Modified Large Margin Nearest Neighbor Metric Learning for Regression. IEEE Signal Processing Letters 21(3), 292–296 (2014). doi:10.1109/LSP.2014.2301037
Atkeson, C.G., Moore, A.W., Schaal, S.: Locally weighted learning. Artificial Intelligence Review 11(1-5), 11–73 (1997)
Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2005, pp. 349–356, San Diego, CA, USA (2005)
Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Transactions in Information Theory, IT-13, 21–27 (1967)
Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Advances in Neural Information Processing Systems, vol. 17, pp. 513–520. MIT Press, Cambridge (2005)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA Data Mining Software: An Update. ACM SIGKDD Explorations 11(1), 10–18 (2009)
Hansen, B.E.: Nearest Neighbor Methods. NonParametric Econometrics. University of Wisconsin-Madison, pp. 84-88 (2009). http://www.ssc.wisc.edu/~bhansen/718/NonParametrics10.pdf (accessed March 15, 2015)
Kumar, M.P., Torr, P.H.S., Zisserman, A.: An invariant large margin nearest neighbour classifier. In: Proceedings of the IEEE 11th International Conference on Computer Vision, ICCV 2007, Rio de Janeiro, Brazil (2007). doi: 10.1109/iccv.2007.4409041
Leon, F., Piuleac, C.G., Curteanu, S., Poulios, I.: Instance-Based Regression with Missing Data Applied to a Photocatalytic Oxidation Process. Central European Journal of Chemistry 10(4), 1149–1156 (2012). doi:10.2478/s11532-012-0038-x
Mareci, D., Sutiman, D., Chelariu, R., Leon, F., Curteanu, S.: Evaluation of the corrosion resistance of new TiZr binary alloys by experiment and simulation based on regression model with incomplete data. Corrosion Science, Elsevier 73, 106–122 (2013). doi:10.1016/j.corsci.2013.03.030
Moore, R.C., DeNero, J.: L1 and L2 regularization for multiclass hinge loss models. In: Proceedings of the Symposium on Machine Learning in Speech and Language Processing (2011)
Navot, A., Shpigelman, L., Tishby, N., Vaadia, E.: Nearest neighbor based feature selection for regression and its application to neural activity. In: Advances in Neural Information Processing Systems, vol. 19 (2005)
Shalev-Shwartz, S., Singer, Y., Ng, A.Y.: Online and batch learning of pseudo-metrics. In: Proceedings of the 21st International Conference on Machine Learning, ICML 2004, pp. 94–101, Banff, Canada (2004)
Shental, N., Hertz, T., Weinshall, D., Pavel, M.: Adjustment learning and relevant component analysis. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 776–790. Springer, Heidelberg (2002)
Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. In: Advances in Neural Information Processing Systems, vol. 18, pp. 1473–1480. MIT Press, Cambridge (2006)
Weinberger, K.Q., Saul, L.K.: Fast solvers and efficient implementations for distance metric learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1160–1167, Helsinki, Finland (2008)
Weinberger, K.Q., Saul, L.K.: Distance Metric Learning for Large Margin Nearest Neighbor Classification. Journal of Machine Learning Research 10, 207–244 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Leon, F., Curteanu, S. (2015). Evolutionary Algorithm for Large Margin Nearest Neighbour Regression. In: Núñez, M., Nguyen, N., Camacho, D., Trawiński, B. (eds) Computational Collective Intelligence. Lecture Notes in Computer Science(), vol 9329. Springer, Cham. https://doi.org/10.1007/978-3-319-24069-5_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-24069-5_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24068-8
Online ISBN: 978-3-319-24069-5
eBook Packages: Computer ScienceComputer Science (R0)