Abstract
Labeled examples are scarce while there are numerous unlabeled examples in real-world. Manual labeling these unlabeled examples is often expensive and inefficient. Active learning paradigm seeks to handle this problem by identifying the most informative examples from the unlabeled examples to label. In this paper, we present two novel active learning approaches based on non-parallel support vector machines and twin support vector machines which adopt the margin sampling method and the manifold-preserving graph reduction algorithm to select the most informative examples. The manifold-preserving graph reduction is a sparse subset selecting algorithm which exploits the structural space connectivity and spatial diversity among examples. In each iteration, an active learner draws the informative and representative candidates from the subset instead of the whole unlabeled data. This strategy can keep the manifold structure and reduce noisy points and outliers in the whole unlabeled data. Experimental results on multiple datasets validate the effective performance of the proposed methods.
Similar content being viewed by others
References
Olsson F (2009) A literature survey of active machine learning in the context of natural language processing. Computer Scineces Technical Report
Settles B (2009) Active learning literature survey. University of Wisconsin Madison, Computer Scineces Technical Report, p 1648
Tuia D, Ratle F, Pacifici F, Kanevski M, Emery W (2009) Active learning methods for remote sensing image classification. IEEE Trans Geosci Remote Sens 47:2218–2232
Lewis D, Gale W (1994) A sequential algorithm for training text classifiers. In: Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 3–12
Roy N, McCallum A (2001) Toward optimal active learning through sampling estimation of error reduction. In: Proceedings of the International Conference on Machine Learning, pp 441–448
Campbell C, Cristianini N, Smola A (2000) Query learning with large margin classifiers. In: Proceedings of the International Conference on Machine Learning, pp 111–118
Silva C, Ribeiro B (2004) Margin-based active learning and background knowledge in text mining. In: Proceedinds of International Conference on Hybird Intelligent Systems, pp 8–13
Schohn G, Cohn D (2000) Less is more: Active learning with support vectors machines. In: Proceedings of the International conference on machine learning, pp 839–846
Mitra P, Murphy C, Pal S (2004) A probabilistic active support vector learning algorithm. IEEE Trans Pattern Anal Mach Intell 26:413–418
Tong S, Koller D (2002) Support vector machine active learning with applications to text classification. J Mach Learn Res 2:45–66
Freund Y, Seung H, Shamir E, Tishby N (1997) Selective sampling using the query by committee algorithm. Mach Learn 28:133–168
Zhou Y, Goldman S (2004) Democratic co-learning. In: Proceedings of the IEEE International Conference on Tools with Artificial Intelligence, pp 594–602
Sun S, Hardoon D (2010) Active learning with extremely sparse labeled examples. Neurocomputing 73:2980–2988
Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 74:905–910
Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44:1–12
Tang J, Li D, Tian Y, Liu D (2018) Multi-view learning based on nonparallel support vector machine. Knowl Based Syst 158:94–108
Tang J, Tian Y (2017) A multi-kernel framework with nonparallel support vector machine. Neurocomputing 266:226–238
Tang L, Tian Y, Yang C (2018) Nonparallel support vector regression model and its SMO-type solver. Neural Netw 105:431–446
Qi Z, Wang B, Meng F, Niu L (2017) Learning with label proportions via NPSVM. IEEE Trans Cybern 47:3293–3305
Chen Z, Qi Z, Wang B, Cui L, Meng F (2017) Learning with label proportions based on nonparallel support vector machines. Knowl Based Syst 119:126–141
Tian Y, Zhang Q, Liu D (2014) \(\nu \)-Nonparallel support vector machine for pattern classification. Neural Comput Appl 25:1007–1020
Tian Y, Ju X, Shi Y (2016) A divide-and-combine method for large scale nonparallel support vector machines. Neural Netw 75:12–21
Liu D, Shi Y, Tian Y (2015) Ramp loss nonparallel support vector machine for pattern classification. Knowl Based Syst 85:224–233
Ju X, Tian Y (2018) A divide-and-conquer method for large scale \(\nu \)-nonparallel support vector machines. Neural Comput Appl 29:497–509
Chen D, Tian Y, Liu X (2016) Structural nonparallel support vector machine for pattern recognition. Pattern Recognit 60:296–305
Zhao X, Chen Z, Shi Y (2014) Active learning with nonparallel support vector machine for binary classification. In: Proceedings of the IEEE International Conference on Data Mining Workshop, pp 101–104
Sun S, Hussain Z, Taylor J (2013) Manifold-preserving graph reduction for sparse semi-supervised learning. Neurocomputing 124:13–21
Zhou J, Sun S (2015) Gaussian process versus margin sampling active learning. Neurocomputing 167:122–131
Shawe-Taylor J, Sun S (2011) A review of optimization methodologies in support vector machines. Neurocomputing 74:3609–3618
Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evolut Comput 1:3–18
Acknowledgements
This work is supported by Ningbo University talent project 421703670 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. It is also supported by NSFC 61906101, 62071260 and 62006131, the Zhejiang Provincial Natural Science Foundation of China under Project LQ18F020001 and LQ20F020013, the Natural Science Foundation of Ningbo city of Zhejiang Province of China under Project 2018A610155 and 2019A610102, the Zhejiang Provincial Public Welfare Technology Research Project (No. LGF18F020007).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xie, X. Sampling Active Learning Based on Non-parallel Support Vector Machines. Neural Process Lett 53, 2081–2094 (2021). https://doi.org/10.1007/s11063-021-10494-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-021-10494-x