Abstract
Classified labels are expensive by virtue of the utilization of field knowledge while the unlabeled data contains significant information, which can not be explored by supervised learning. The Manifold Regularization (MR) based semi-supervised learning (SSL) could explores information from both labeled and unlabeled data. Moreover, the model selection of MR seriously affects its predictive performance due to the inherent additional geometry regularizer of SSL. In this paper, a leave-one-out cross-validation based PRESS criterion is first presented for model selection of MR to choose appropriate regularization coefficients and kernel parameters. The Manifold regularization and model selection algorithm are employed to a real-life benchmark dataset. The proposed approach, leveraged by effectively exploiting the embedded intrinsic geometric manifolds, outperforms the original MR and supervised learning approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Zhu, X.: Semi-supervised learning literature survey, Tech. rep. (2008), http://pages.cs.wisc.edu/~jerryzhu/research/ssl/semireview.html
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research 7, 2399–2434 (2006)
Wei, L., Keogh, E.: Semi-supervised time series classification. In: KDD 2006: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 748–753. ACM, New York (2006)
Jansen, A., Niyogi, P.: A geometric perspective on speech sounds, Tech. rep., University of Chicago (2005)
Pan, J.J., Yang, Q., Chang, H., Yeung, D.Y.: A manifold regularization approach to calibration reduction for sensor-network based tracking. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, Boston, United States, pp. 988–993 (2006)
Williams, O., Blake, A., Cipolla, R.: Sparse and semi-supervised visual mapping with the s 3gp. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 230–237 (2006)
Li, M., Xu, J., Yang, J., Yang, D., Wang, D.: Multiple manifolds analysis and its application to fault diagnosis. Mechanical Systems and Signal Processing 23(8), 2500–2509 (2009)
Sindhwani, V., Niyogi, P., Belkin, M.: Beyond the point cloud: from transductive to semi-supervised learning. In: ICML 2005, New York, NY, USA, pp. 824–831 (2005)
Chapelle, O., Zien, A.: Semi-Supervised Classification by Low Density Separation. In: AISTAT (2005)
Sindhwani, V.: On semi-supervised kernel methods, Ph.D. thesis, Chicago, IL, USA (2007)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Cawley, G.: Leave-one-out cross-validation based model selection criteria for weighted ls-svms. In: Neural Networks, IJCNN 2006, pp. 1661–1668 (2006)
Allen, D.M.: The relationship between variable selection and prediction. Technometrics 16, 125–127 (1974)
Bo, L., Wang, L., Jiao, L.: Feature scaling for kernel fisher discriminant analysis using leave-one-out cross validation. Neural Computation 18, 961–978 (2006)
Yuan, J., Wang, K., Yu, T., Fang, M.: Reliable multi-objective optimization of high-speed wedm process based on gaussian process regression. International Journal of Machine Tools and Manufacture 48(1), 47–60 (2008)
Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the nelder-mead simplex algorithm in low dimensions. SIAM Journal of Optimization 9, 112–147 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yuan, J., Li, YM., Liu, CL., Zha, X.F. (2010). Leave-One-Out Cross-Validation Based Model Selection for Manifold Regularization. In: Zhang, L., Lu, BL., Kwok, J. (eds) Advances in Neural Networks - ISNN 2010. ISNN 2010. Lecture Notes in Computer Science, vol 6063. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13278-0_59
Download citation
DOI: https://doi.org/10.1007/978-3-642-13278-0_59
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13277-3
Online ISBN: 978-3-642-13278-0
eBook Packages: Computer ScienceComputer Science (R0)