Abstract
In semi-supervised learning, co-training is successfully in augmenting the training data with predicted pseudo-labels. With two independently trained regressors, a co-trainer iteratively exchanges their selected instances coupled with pseudo-labels. However, some low-quality pseudo-labels may significantly decrease the prediction accuracy. In this paper, we propose a self-paced safe co-training for regression (SPOR) algorithm to enrich the training data with unlabeled instances and their pseudo-labels. First, a safe mechanism is designed to enhance the quality of pseudo-labels without side effects. Second, a self-paced learning technique is designed to select “easy” instances in the current situation. Third, a “qualifier-based” treatment is designed to remove “weak” instances selected in previous rounds. Experiments were undertaken on nine benchmark datasets. The results show that SPOR is superior to both popular co-training regression methods and state-of-the-art semi-supervised regressors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balcan, M.F., Blum, A.: A discriminative model for semi-supervised learning. J. ACM 57(3), 1–46 (2010)
Balcan, M.F., Blum, A., Yang, K.: Co-training and expansion: towards bridging theory and practice. In: NIPS, vol. 17, pp. 89–96 (2004)
Balsubramani, A., Freund, Y.: Optimally combining classifiers using unlabeled data. In: COLT, vol. 21, pp. 211–225 (2015)
Bao, L., Yuan, X.F., Ge, Z.Q.: Co-training partial least squares model for semi-supervised soft sensor development. Chemometr Intell. Lab. Syst. 147, 75–85 (2015)
Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: COLT, pp. 41–48 (2009)
Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100 (1998)
Brefeld, U., Scheffer, T.: CO-EM support vector learning. In: ICML, p. 16 (2004)
Fazakis, N., Karlos, S., Kotsiantis, S., Sgarbas, K.: A multi-scheme semi-supervised regression approach. Pattern Recogn. Lett. 125, 758–765 (2019)
Gu, Y., Yang, H., Zhou, C.: SelectNet: self-paced learning for high-dimensional partial differential equations. J. Comput. Phys. 441, 110444 (2021)
Hady, M.F.A., Schwenker, F., Palm, G.: Semi-supervised learning for regression with co-training by committee. In: ICANN, pp. 121–130 (2009)
Jiang, L., Meng, D.Y., Mitamura, T., Hauptmann, A.G.: Easy samples first: self-paced reranking for zero-example multimedia search. In: ACM MM, pp. 547–556 (2014)
Jiang, L., Meng, D.Y., Zhao, Q., Shan, S.G., Hauptmann, A.G.: Self-paced curriculum learning. In: AAAI, vol. 29, pp. 2694–2700 (2015)
Kumar, M.P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: NIPS, vol. 23, pp. 1189–1197 (2010)
Li, Y.F., Tsang, I.W., Kwok, J.T., Zhou, Z.H.: Convex and scalable weakly labeled SVMs. J. Mach. Learn. Res. 14(7), 2151–2188 (2013)
Li, Y.F., Wang, S.B., Zhou, Z.H.: Graph quality judgement: a large margin expedition. In: IJCAI, pp. 1725–1731 (2016)
Li, Y.F., Zha, H.W., Zhou, Z.H.: Learning safe prediction for semi-supervised regression. In: AAAI, vol. 31, pp. 2217–2223 (2017)
Li, Y.F., Zhou, Z.H.: Towards making unlabeled data never hurt. IEEE Trans. Pattern Anal. 37(1), 175–188 (2014)
Ma, F., Meng, D.Y., Xie, Q., Li, Z.N., Dong, X.Y.: Self-paced co-training. In: ICML, vol. 70, pp. 2275–2284 (2017)
Ma, F., Meng, D., Dong, X., Yang, Y.: Self-paced multi-view co-training. J. Mach. Learn. Res. 21(57), 1–38 (2020)
Supancic, J.S., Ramanan, D.: Self-paced learning for long-term tracking. In: CVPR, pp. 2379–2386 (2013)
Timilsina, M., Figueroa, A., d’Aquin, M., Yang, H.: Semi-supervised regression using diffusion on graphs. Appl. Soft Comput. 104, 107188 (2021)
Wang, W., Zhou, Z.H.: A new analysis of co-training. In: ICML, vol. 12, pp. 1135–1142 (2010)
Wang, W., Zhou, Z.H.: Co-training with insufficient views. In: ACML, pp. 467–482 (2013)
Yu-Feng, L., Lan-Zhe, G., Zhi-Hua, Z.: Towards safe weakly supervised learning. IEEE Trans. Pattern Anal. 43(1), 334–346 (2019)
Zhou, Z.H., Li, M.: Semi-supervised regression with co-training. In: IJCAI, vol. 5, pp. 908–913 (2005)
Zhu, X.J., Goldberg, A.B.: Introduction to semi-supervised learning. Synth. Lect. Artif. Intell. Mach. Learn. 3(1), 1–130 (2009)
Acknowledgment
This work is supported in part by the Central Government Funds of Guiding Local Scientific and Technological Development (No. 2021ZYD0003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Min, F., Li, Y., Liu, L. (2022). Self-paced Safe Co-training for Regression. In: Gama, J., Li, T., Yu, Y., Chen, E., Zheng, Y., Teng, F. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2022. Lecture Notes in Computer Science(), vol 13281. Springer, Cham. https://doi.org/10.1007/978-3-031-05936-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-05936-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05935-3
Online ISBN: 978-3-031-05936-0
eBook Packages: Computer ScienceComputer Science (R0)