Abstract
Manifold regularization (MR) provides a powerful framework for semi-supervised classification using both the labeled and unlabeled data. It constrains that similar instances over the manifold graph should share similar classification outputs according to the manifold assumption. It is easily noted that MR is built on the pairwise smoothness over the manifold graph, i.e., the smoothness constraint is implemented over all instance pairs and actually considers each instance pair as a single operand. However, the smoothness can be pointwise in nature, that is, the smoothness shall inherently occur “everywhere” to relate the behavior of each point or instance to that of its close neighbors. Thus in this paper, we attempt to develop a pointwise MR (PW_MR for short) for semi-supervised learning through constraining on individual local instances. In this way, the pointwise nature of smoothness is preserved, and moreover, by considering individual instances rather than instance pairs, the importance or contribution of individual instances can be introduced. Such importance can be described by the confidence for correct prediction, or the local density, for example. PW_MR provides a different way for implementing manifold smoothness. Finally, empirical results show the competitiveness of PW_MR compared to pairwise MR.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhou Z H, Li M. Semi-supervised learning by disagreement. Knowledge and Information Systems, 2010, 24(3): 415–439
Zhu X J, Goldberg A B. Introduction to semi-supervised learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2009, 3(1): 1–130
Zhu X J. Semi-supervised learning literature survey. Technical Report, 2005
Chapelle O, Schölkopf B, Zien A. Semi-supervised Learning. Cambridge, MA: MIT Press, 2006
Mallapragada P K, Jin R, Jain A K, Liu Y. Semiboost: boosting for semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 31(11): 2000–2014
Joachims T. Transductive inference for text classification using support vector machines. In: Proceedings of the 16th Annual International Conference on Machine Learning. 1999, 200–209
Fung G, Mangasarian O L. Semi-supervised support vector machines for unlabeled data classification. Optimization Methods and Software, 2001, 15(1): 29–44
Collobert R, Sinz F, Weston J, Bottou L. Large scale transductive SVMs. Journal of Machine Learning Research, 2006, 7(8): 1687–1712
Li Y F, Kwok J T, Zhou Z H. Semi-supervised learning using label mean. In: Proceedings of the 26th Annual International Conference on Machine Learning. 2009, 633–640
Bengio Y, Delalleau O, Roux N L. Label propagation and quadratic criterion. In: Chapelle O, Schölkopf B, Zien A, eds. Semi-supervised Learning. Cambridge, MA: MIT Press, 2006, 193–216
Zhu X J, Ghahramani Z. Learning from labeled and unlabeled data with label propagation. Technical Report, 2002
Blum A, Chawla S. Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the 18th Annual International Conference on Machine Learning. 2001, 19–26
Belkin M, Niyogi P, Sindhwani V. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 2006, 7(11): 2399–2434
Chen K, Wang S H. Semi-supervised learning via regularized boosting working on multiple semi-supervised assumptions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(1): 129–143
He X F. Laplacian regularized D-optimal design for active learning and its application to image retrieval. IEEE Transactions on Image Processing, 2009, 19(1): 254–263
Abernethy J, Chapelle O, Castillo C. Web spam identification through content and hyperlinks. In: Proceedings of the 4th International Workshop on Adversarial Information Retrieval on the Web. 2008, 41–44
Fang Y, Chang K C C, Lauw H W. Graph-based semi-supervised learning: realizing pointwise smoothness probabolistically. In: Proceedings of the 31st Annual International Conference on Machine Learning. 2014, 406–414
Singh A, Nowak R, Zhu J. Unlabeled data: now it helps, now it doesn’t. In: Proceedings of Advances in Neural Information Processing Systems. 2009, 1513–1520
Wasserman L, Lafferty J D. Statistical analysis of semi-supervised regression. In: Proceedings of Advances in Neural Information Processing Systems. 2008, 801–808
Rigollet P. Generlization error bounds in semi-supervised classification under the cluster assumption. Journal of Machine Learning Research, 2007, 8(7): 1369–1392
Wang H, Wang S B, Li Y F. Instance selection method for improving graph-based semi-supervised learning. Frontiers of Computer Science, 2018, 12(4): 725–735
Gan H, Li Z, Wu W, Luo Z, Huang R. Safety-aware graph-based semi-supervised learning. Expert Systems with Applications, 2018, 107: 243–254
Wang Y, Meng Y, Li Y, Chen S C, Fu Z Y, Xue H. Semi-supervised manifold regularization with adaptive graph construction. Pattern Recognition Letters, 2017, 98: 90–95
Gan H T, Luo Z Z, Sun Y, Xi X G, Sang N, Huang R. Towards designing risk-based safe laplacian regularized least squares. Expert Systems with Applicaions, 2016, 45: 1–7
Quang M H, Bazzani L, Murino V. A unifying framework for vector-valued manifold regularization and multi-view learning. In: Proceedings of the 30th Annual International Conference on Machine Learning. 2013, 100–108
Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant No. 61876091), and China Postdoctoral Science Foundation (2019M651918).
Author information
Authors and Affiliations
Corresponding author
Additional information
Yunyun Wang received the PhD degree in Computer Science and Technology from Nanjing University of Aeronautics and Astronautics, China in 2012. She is currently with the School of Computer Science and Technology in Nanjing University of Posts and Telecommunications, China. Her current research interests include pattern recognition, machine learning and neural computing.
Jiao Han received the BS degree in Internet of things engineering from Nanjing University of Information Science & Technology, China in 2018. She is currently pursuing a master degree in the School of Computer Science and Technology in Nanjing University of Posts and Telecommunications, China. Her main research interests are pattern recognition, machine learning and neural computing.
Yating Shen, Lecturer, Nanjing University of Science and Technology Zi Jin College, China. She received the BS degree in software engineering from Yancheng Teachers University, China in 2015. She received the master degree in the software engineering from Nanjing University of Posts and Telecommunications, China in 2018. Her research interests include pattern recognition, machine learning and neural computing.
Hui Xue received her MS degree in mathematics from Nanjing University of Aeronautics & Astronautics (NUAA), China in 2005. And she also received her PhD degree in computer application technology at NUAA in 2008. Since 2009, she has been with the School of Computer Science & Engineering at Southeast University, China. Her research interests include pattern recognition, machine learning and neural computing.
Supporting information
Rights and permissions
About this article
Cite this article
Wang, Y., Han, J., Shen, Y. et al. Pointwise manifold regularization for semi-supervised learning. Front. Comput. Sci. 15, 151303 (2021). https://doi.org/10.1007/s11704-019-9115-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-019-9115-z