Abstract
The purpose of this paper is to learn a specific distance function for the Nadayara Watson estimator to be applied as a non-linear classifier. The idea of transforming the predictor variables and learning a kernel function based on Mahalanobis pseudo distance througth an low rank structure in the distance function will help us to lead the development of this problem. In context of metric learning for kernel regression, we introduce an Accelerated Proximal Gradient to solve the non-convex optimization problem with better convergence rate than gradient descent. An extensive experiment and the corresponding discussion tries to show that our strategie its a competitive solution in relation to previously proposed approaches. Preliminary results suggest that this line of work can deal with others regularization approach in order to improve the kernel regression problem.
This work has been partially funded by FEDER and Spanish MEC through project TIN2014-59641-C2-1-P and INV17-01-15-03.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Benedetti, J.K.: On the nonparametric estimation of regression functions. J. R. Stat. Soc. Ser. B (Methodological) 23, 248–253 (1977)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
García, S., Fernández, A., Luengo, J., Herrera, F.: A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput. 13(10), 959–977 (2009)
Ghadimi, S., Lan, G.: Accelerated gradient methods for nonconvex nonlinear and stochastic programming. Math. Program. 156(1–2), 59–99 (2016)
Härdle, W.K., Müller, M., Sperlich, S., Werwatz, A.: Nonparametric and Semiparametric Models. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-17146-8
Hu, J., Sun, C.S., Lam, K.M.: Semi-supervised metric learning for image classification. In: Qiu, G., Lam, K.M., Kiya, H., Xue, X.-Y., Kuo, C.-C.J., Lew, M.S. (eds.) PCM 2010. LNCS, vol. 6298, pp. 728–735. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15696-0_67
Huang, K., Ying, Y., Campbell, C.: GSML: a unified framework for sparse metric learning. In: Ninth IEEE International Conference on Data Mining, ICDM 2009, pp. 189–198. IEEE (2009)
Huang, R., Sun, S.: Kernel regression with sparse metric learning. J. Intell. Fuzzy Syst. 24(4), 775–787 (2013)
Kulis, B.: Metric learning: a survey. Found. Trends Mach. Learn. 5(4), 287–364 (2012)
Kumar, S., Kolluru, P.K., Chowdary, E.D., Babu, J.B.: Image denoising using combined FFT, kernel regression and local content metrics. J. Adv. Res. Dyn. Control Syst. 2, 1339–1347 (2017)
Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: Advances in Neural Information Processing Systems, pp. 379–387 (2015)
Neal, R.M.: Assessing relevance determination methods using DELVE. Nato Asi Ser. F Comput. Syst. Sci. 168, 97–132 (1998)
Schultz, M., Joachims, T.: Learning a distance metric from relative comparisons. In: Advances in Neural Information Processing Systems (NIPS), p. 41 (2004)
Sun, S., Chen, Q.: Kernel regression with a mahalanobis metric for short-term traffic flow forecasting. In: Fyfe, C., Kim, D., Lee, S.-Y., Yin, H. (eds.) IDEAL 2008. LNCS, vol. 5326, pp. 9–16. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88906-9_2
Tibshirani, R., Wainwright, M., Hastie, T.: Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman and Hall/CRC, Boca Raton (2015)
Wang, Q., Wan, J., Yuan,Y.: Deep metric learning for crowdedness regression. IEEE Trans. Circuits Syst. Video Technol., 10 (2018). https://doi.org/10.1109/TCSVT.2017.2703920
Wang, Q., Wan, J., Yuan, Y.: Locality constraint distance metric learning for traffic congestion detection. Pattern Recognit. 75, 272–281 (2018)
Weinberger, K., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbor classification. In: Advances in Neural Information Processing Systems, vol. 18, p. 1473 (2006)
Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009)
Weinberger, K.Q., Tesauro, G.: Metric learning for kernel regression. In: Artificial Intelligence and Statistics, pp. 612–619 (2007)
Ying, Y., Huang, K., Campbell, C.: Sparse metric learning via smooth optimization. In: Advances in Neural Information Processing Systems, pp. 2214–2222 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Gonzalez, H., Morell, C., Ferri, F.J. (2018). Accelerated Proximal Gradient Descent in Metric Learning for Kernel Regression. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2018. Lecture Notes in Computer Science(), vol 11047. Springer, Cham. https://doi.org/10.1007/978-3-030-01132-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-01132-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01131-4
Online ISBN: 978-3-030-01132-1
eBook Packages: Computer ScienceComputer Science (R0)