Abstract
Training a binary classifier only on positive and unlabeled examples (i.e., the PU learning) is an important yet challenging issue, widely seen in many problems in which it is difficult to obtain negative examples. Existing methods for handling this challenge often perform unsatisfactorily, since they often ignore the relations between positive and unlabeled examples and are also limited to the traditional shallow learning frameworks. Therefore, this work proposes a new approach: Deep Positive and Unlabeled learning with Manifold Regularization (Deep-PUMR), which integrates the manifold regularization with deep neural networks to address the above issues with classic PU learning. Deep-PUMR holds two major advantages: (i) Our method exploits the manifold properties of data distribution to capture the relationship of positive and unlabeled examples; (ii) The adopted deep network enables Deep-PUMR with strong learning ability, especially on large-scale datasets. Extensive experiments on five diverse datasets demonstrate that Deep-PUMR achieves the state-of-the-art performance in comparison with classic PU learning algorithms and risk estimators.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
\(\mathbf {L}^i\) represents the sub-matrix of \(\mathbf {L}\) corresponding to \(\mathcal {D}^i\). For example, if the i-th mini-batch corresponds to the \(l_1\)-th examples to \(l_2\)-th examples, \(\mathbf {L}^i=\mathbf { L}_{l_1:l_2,l_1:l_2}\) in matlab formation.
References
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. JMLR 7(1), 2399–2434 (2006)
Belkin, M., Niyogi, P., Sindhwani, V.: On manifold regularization. In: AISTATS, p. 1 (2005)
Du Plessis, M., Niu, G., Sugiyama, M.: Convex formulation for learning from positive and unlabeled data. In: ICML, pp. 1386–1394 (2015)
Du Plessis, M.C., Niu, G., Sugiyama, M.: Analysis of learning from positive and unlabeled data. In: NIPS, pp. 703–711 (2014)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12(Jul), 2121–2159 (2011)
Elhamifar, E., Vidal, R.: Sparse manifold clustering and embedding. In: NIPS, pp. 55–63 (2011)
Elkan, C., Noto, K.: Learning classifiers from only positive and unlabeled data. In: KDD, pp. 213–220 (2008)
Gong, C., Liu, T., Tao, D., Fu, K., Tu, E., Yang, J.: Deformed graph laplacian for semisupervised learning. TNNLS 26(10), 2261–2274 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Hsieh, C.J., Natarajan, N., Dhillon, I.: PU learning for matrix completion. In: JMLR, pp. 2445–2453 (2015)
Jaafar, H.F., Nandi, A.K., Al-Nuaimy, W.: Automated detection and grading of hard exudates from retinal fundus images. In: EUSIPCO, pp. 66–70. IEEE (2011)
Kiryo, R., Niu, G., du Plessis, M.C., Sugiyama, M.: Positive-unlabeled learning with non-negative risk estimator. NIPS (2017)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lee, W.S., Liu, B.: Learning with positive and unlabeled examples using weighted logistic regression. In: ICML, pp. 448–455 (2003)
Li, X.L., Yu, P.S., Liu, B., Ng, S.K.: Positive unlabeled learning for data stream classification. In: SDM, pp. 259–270 (2009)
Li, X., Liu, B.: Learning to classify texts using positive and unlabeled data. In: IJCAI, pp. 587–592 (2003)
Liu, B., Lee, W.S., Yu, P.S., Li, X.: Partially supervised classification of text documents. In: ICML, pp. 387–394 (2002)
Menon, A., Van Rooyen, B., Ong, C.S., Williamson, B.: Learning from corrupted binary labels via class-probability estimation. In: ICML, pp. 125–134 (2015)
Nadler, B., Lafon, S., Kevrekidis, I., Coifman, R.R.: Diffusion maps, spectral clustering and eigenfunctions of fokker-planck operators. In: NIPS, pp. 955–962 (2006)
Niu, G., du Plessis, M.C., Sakai, T., Ma, Y., Sugiyama, M.: Theoretical comparisons of positive-unlabeled learning against positive-negative learning. In: NIPS, pp. 1199–1207 (2016)
Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: CVPR (2016)
Rothe, R., Timofte, R., Gool, L.V.: Deep expectation of real and apparent age from a single image without facial landmarks. IJCV 126, 144–157 (2016)
Ruder, S.: An overview of gradient descent optimization algorithms. arXiv (2016)
Acknowledgments
This research is partly supported by NSFC, China (No: 61572315, 6151101179) and 973 Plan, China (No. 2015CB856004).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, X., Liu, F., Tu, E., Cao, L., Yang, J. (2018). Deep-PUMR: Deep Positive and Unlabeled Learning with Manifold Regularization. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11301. Springer, Cham. https://doi.org/10.1007/978-3-030-04167-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-04167-0_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04166-3
Online ISBN: 978-3-030-04167-0
eBook Packages: Computer ScienceComputer Science (R0)