Abstract
Active learning reduces the annotation cost of machine learning by selecting and querying informative unlabeled samples. Semi-supervised active learning methods can considerably utilize the regional information of unlabeled samples, and thus, more effectively select valuable samples. Existing semi-supervised batch active learning algorithms frequently exhibit poor robustness due to their high computational complexity, making handling large-scale datasets a difficult task. However, existing active learning algorithms based on high-performance semi-supervised learners adopt a single-sample selection mode, under which the model requires multiple rounds of iterative processes, significantly reducing the overall efficiency of the algorithm and affecting its practicality. To address these issues, we propose a new semi-supervised batch active learning algorithm called approximate error reduction based on mutual information (MIAER). First, we use hierarchical anchor graph regularization (HAGR) as the semi-supervised learner. HAGR exhibits good robustness and only involves a small-scale reduced Laplacian matrix in its optimization process, enabling rapid processing of large-scale datasets. Second, we propose a batch sampling strategy based on mutual information and error reduction in the sample selection stage. This strategy, which is based on hierarchical anchor graphs, first measures the uncertainty of samples by using approximate error reduction, considerably reducing computational overhead. Then, it uses mutual information to measure the diversity of samples in category space while removing redundant batch samples, preserving samples with high uncertainty as much as possible. Comparative experiments with several advanced active learning methods on a large number of datasets fully demonstrate the effectiveness and stability of MIAER.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
Data sharing not applicable.
Code Availability
Code availability not applicable.
References
Vekkot S, Gupta D (2022) Fusion of spectral and prosody modelling for multilingual speech emotion conversion. Knowl-Based Syst 242:108360
Buddenkotte T, Sanchez LE, Crispin-Ortuzar M et al (2023) Calibrating ensembles for scalable uncertainty quantification in deep learning-based medical image segmentation. Comput Biol Med 163:107096
He Z, Yuan S, Zhao J et al (2022) A novel myocardial infarction localization method using multi-branch densenet and spatial matching-based active semi-supervised learning. Inf Sci 606:649–668
Jin Q, Yuan M, Li S et al (2022) Cold-start active learning for image classification. Inf Sci 616:16–36
Jin Q, Yuan M, Qiao Q et al (2022) One-shot active learning for image segmentation via contrastive learning and diversity-based sampling. Knowl-Based Syst 241:108278
Settles B (2009) Active learning literature survey
Roy N, McCallum A (2001) Toward optimal active learning through monte carlo estimation of error reduction. Icml, williamstown 2(441–448):4
Macskassy SA (2009) Using graph-based metrics with empirical risk minimization to speed up active learning on networked data. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 597–606
Huang SJ, Jin R, Zhou ZH (2010) Active learning by querying informative and representative examples. Adv Neural Inf Process Syst 23
Hua W, Zhang Y, Liu H et al (2024) Multichannel semi-supervised active learning for polsar image classification. Int J Appl Earth Obs Geoinf 127:103706
Ju W, Mao Z, Qiao Z et al (2024) Focus on informative graphs! semi-supervised active learning for graph-level classification. Pattern Recogn 153:110567
Li J, Li Y, Tan J et al (2024) Bridging the gap with grad: Integrating active learning into semi-supervised domain generalization. Neural Netw 171:186–199
Wang Z, Xu R, Nie T et al (2023) Semi-supervised active learning hypothesis verification for improved geometric expression in three-dimensional object recognition. Eng Appl Artif Intell 120:105956
Fan C, Wu Q, Zhao Y et al (2024) Integrating active learning and semi-supervised learning for improved data-driven hvac fault diagnosis performance. Appl Energy 356:122356
Fu W, Wang M, Hao S et al (2018) Scalable active learning by approximated error reduction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1396–1405
He G, Li B, Wang H et al (2020) Cost-effective active semi-supervised learning on multivariate time series data with crowds. IEEE Trans Syst Man Cybern: Systems 52(3):1437–1450
Zhao Y, Lin J, Lin J et al (2023) Batch-mode active learning of gaussian process regression with maximum model change. IEEE Trans Syst Man Cybern: Systems
Wang M, Zhang YY, Min F et al (2020) A two-stage density clustering algorithm. Soft Comput 24(23):17797–17819
Yan X, Nazmi S, Gebru B et al (2022) A clustering-based active learning method to query informative and representative samples. Appl Intell 52(11):13250–13267
Wang M, Min F, Zhang ZH et al (2017) Active learning through density clustering. Expert Syst Appl 85:305–317
Ji X, Ye W, Li X et al (2023) Adaptive active learning through k-nearest neighbor optimized local density clustering. Appl Intell 53(12):14892–14902
Min F, Zhang SM, Ciucci D et al (2020) Three-way active learning through clustering selection. Int J Mach Learn Cybern 11:1033–1046
Wang Z, Ye J (2015) Querying discriminative and representative samples for batch mode active learning. ACM Trans Knowl Discov Data (TKDD) 9(3):1–23
Ghafarian SH (2023) Local variational probabilistic minimax active learning. Expert Syst Appl 211:118538
Collobert R, Sinz F, Weston J et al (2006) Large scale transductive svms. J Mach Learn Res 7(8)
Gu B, Zhai Z, Deng C et al (2020) Efficient active learning by querying discriminative and representative samples and fully exploiting unlabeled data. IEEE Trans Neural Netw Learn Syst 32(9):4111–4122
Wang M, Fu W, Hao S et al (2017) Learning on big graph: label inference and regularization with anchor hierarchy. IEEE Trans Knowl Data Eng 29(5):1101–1114
Wang M, Fu W, Hao S et al (2016) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877
Liu W, He J, Chang SF (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the 27th international conference on machine learning (ICML-10). Citeseer, pp 679–686
Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell 36(11):2227–2240
Bemporad A (2023) Active learning for regression by inverse distance weighting. Inf Sci 626:275–292
Jose A, de Mendonça JPA, Devijver E et al (2024) Regression tree-based active learning. Data Min Knowl Disc 38(2):420–460
Kottke D, Herde M, Sandrock C et al (2021) Toward optimal probabilistic active learning using a bayesian approach. Mach Learn 110(6):1199–1231
Cho JW, Kim DJ, Jung Y et al (2022) Mcdal: maximum classifier discrepancy for active learning. IEEE Trans Neural Netw Learn Syst
Wang M, Feng T, Shan Z et al (2022) Attribute and label distribution driven multi-label active learning. Appl Intell 52(10):11131–11146
Caramalau R, Bhattarai B, Kim TK (2021) Sequential graph convolutional network for active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9583–9592
Lu S, Zheng J, Li Z et al (2024) Wmbal: weighted minimum bounds for active learning. Appl Intell, pp 1–13
Roth D, Small K (2006) Margin-based active learning for structured output spaces. In: Machine Learning: ECML 2006: 17th European Conference on Machine Learning Berlin, Germany, September 18-22, 2006 Proceedings 17. Springer, pp 413–424
Joshi AJ, Porikli F, Papanikolopoulos NP (2012) Scalable active learning for multiclass image classification. IEEE Trans Pattern Anal Mach Intell 34(11):2259–2273
Wang D, Shang Y (2014) A new active labeling method for deep learning. In: 2014 International joint conference on neural networks (IJCNN). IEEE, pp 112–119
Zhou Z, Shin JY, Gurudu SR et al (2021) Active, continual fine tuning of convolutional neural networks for reducing annotation efforts. Med Image Anal 71:101997
Zhu X, Lafferty J, Ghahramani Z (2003) Combining active learning and semi-supervised learning using gaussian fields and harmonic functions. In: ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, pp 58–65
Mac Aodha O, Campbell ND, Kautz J et al (2014) Hierarchical subquery evaluation for active learning on a graph. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 564–571
Zhang Y, Zhao T, Miao D et al (2021) Granular multilabel batch active learning with pairwise label correlation. IEEE Trans Syst Man Cybern: Systems 52(5):3079–3091
Li H, Wang Y, Li Y et al (2021) Batch mode active learning via adaptive criteria weights. Appl Intell 51:3475–3489
Wang M, Fu K, Min F et al (2020) Active learning through label error statistical methods. Knowl-Based Syst 189:105140
Hoi SC, Jin R, Zhu J et al (2009) Semisupervised svm batch mode active learning with applications to image retrieval. ACM Trans Inf Syst (TOIS) 27(3):1–29
Hoi SC, Jin R, Zhu J et al (2006) Batch mode active learning and its application to medical image classification. In: Proceedings of the 23rd international conference on Machine learning, pp 417–424
Nguyen HT, Smeulders A (2004) Active learning using pre-clustering. In: Proceedings of the twenty-first international conference on Machine learning, p 79
Parvaneh A, Abbasnejad E, Teney D et al (2022) Active learning by feature mixing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12237–12246
Patra S, Bruzzone L (2012) A cluster-assumption based batch mode active learning technique. Pattern Recogn Lett 33(9):1042–1048
Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423
Blake CL (1998) Uci repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html
Breviglieri P, Erdem T, Eken S (2021) Predicting smart grid stability with optimized deep models. SN Comput Sci 2:1–12
Liu Zi-Ang J, Xue W (2021) Unsupervised pool-based active learning for linear regression. Acta Automatica Sinica 47(12):2771–2783
Hinton G, Van Der Maaten L (2008) Visualizing data using t-sne journal of machine learning research. J Mach Learn Res 9:2579–2605
Yu H, Kim S (2010) Passive sampling for regression. In: 2010 IEEE international conference on data mining. IEEE, pp 1151–1156
Acknowledgements
This work was supported in part by the Natural Science Foundation of China under Grant 62176001, 62076002, and, in part by the Natural Science Foundation of Anhui Province under Grant 2008085MF194 and 2108085MF212, and in part by the Key Project of Natural Science Foundation of Anhui Provincial Department of Education under Grant KJ2020A0041 and KJ2021A0043.
Funding
the Natural Science Foundation of China under Grant 62176001, 62076002, and, in part by the Natural Science Foundation of Anhui Province under Grant 2008085MF194 and 2108085MF212, and in part by the Key Project of Natural Science Foundation of Anhui Provincial Department of Education under Grant KJ2020A0041 and KJ2021A0043.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ji, X., Wang, L. & Fang, X. Semi-supervised batch active learning based on mutual information. Appl Intell 55, 117 (2025). https://doi.org/10.1007/s10489-024-05962-5
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05962-5