Skip to main content

Advertisement

Semi-supervised batch active learning based on mutual information

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Active learning reduces the annotation cost of machine learning by selecting and querying informative unlabeled samples. Semi-supervised active learning methods can considerably utilize the regional information of unlabeled samples, and thus, more effectively select valuable samples. Existing semi-supervised batch active learning algorithms frequently exhibit poor robustness due to their high computational complexity, making handling large-scale datasets a difficult task. However, existing active learning algorithms based on high-performance semi-supervised learners adopt a single-sample selection mode, under which the model requires multiple rounds of iterative processes, significantly reducing the overall efficiency of the algorithm and affecting its practicality. To address these issues, we propose a new semi-supervised batch active learning algorithm called approximate error reduction based on mutual information (MIAER). First, we use hierarchical anchor graph regularization (HAGR) as the semi-supervised learner. HAGR exhibits good robustness and only involves a small-scale reduced Laplacian matrix in its optimization process, enabling rapid processing of large-scale datasets. Second, we propose a batch sampling strategy based on mutual information and error reduction in the sample selection stage. This strategy, which is based on hierarchical anchor graphs, first measures the uncertainty of samples by using approximate error reduction, considerably reducing computational overhead. Then, it uses mutual information to measure the diversity of samples in category space while removing redundant batch samples, preserving samples with high uncertainty as much as possible. Comparative experiments with several advanced active learning methods on a large number of datasets fully demonstrate the effectiveness and stability of MIAER.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

Data sharing not applicable.

Code Availability

Code availability not applicable.

References

  1. Vekkot S, Gupta D (2022) Fusion of spectral and prosody modelling for multilingual speech emotion conversion. Knowl-Based Syst 242:108360

    Article  MATH  Google Scholar 

  2. Buddenkotte T, Sanchez LE, Crispin-Ortuzar M et al (2023) Calibrating ensembles for scalable uncertainty quantification in deep learning-based medical image segmentation. Comput Biol Med 163:107096

    Article  MATH  Google Scholar 

  3. He Z, Yuan S, Zhao J et al (2022) A novel myocardial infarction localization method using multi-branch densenet and spatial matching-based active semi-supervised learning. Inf Sci 606:649–668

    Article  MATH  Google Scholar 

  4. Jin Q, Yuan M, Li S et al (2022) Cold-start active learning for image classification. Inf Sci 616:16–36

    Article  MATH  Google Scholar 

  5. Jin Q, Yuan M, Qiao Q et al (2022) One-shot active learning for image segmentation via contrastive learning and diversity-based sampling. Knowl-Based Syst 241:108278

    Article  MATH  Google Scholar 

  6. Settles B (2009) Active learning literature survey

  7. Roy N, McCallum A (2001) Toward optimal active learning through monte carlo estimation of error reduction. Icml, williamstown 2(441–448):4

    MATH  Google Scholar 

  8. Macskassy SA (2009) Using graph-based metrics with empirical risk minimization to speed up active learning on networked data. In: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 597–606

  9. Huang SJ, Jin R, Zhou ZH (2010) Active learning by querying informative and representative examples. Adv Neural Inf Process Syst 23

  10. Hua W, Zhang Y, Liu H et al (2024) Multichannel semi-supervised active learning for polsar image classification. Int J Appl Earth Obs Geoinf 127:103706

    MATH  Google Scholar 

  11. Ju W, Mao Z, Qiao Z et al (2024) Focus on informative graphs! semi-supervised active learning for graph-level classification. Pattern Recogn 153:110567

    Article  MATH  Google Scholar 

  12. Li J, Li Y, Tan J et al (2024) Bridging the gap with grad: Integrating active learning into semi-supervised domain generalization. Neural Netw 171:186–199

    Article  MATH  Google Scholar 

  13. Wang Z, Xu R, Nie T et al (2023) Semi-supervised active learning hypothesis verification for improved geometric expression in three-dimensional object recognition. Eng Appl Artif Intell 120:105956

    Article  MATH  Google Scholar 

  14. Fan C, Wu Q, Zhao Y et al (2024) Integrating active learning and semi-supervised learning for improved data-driven hvac fault diagnosis performance. Appl Energy 356:122356

    Article  MATH  Google Scholar 

  15. Fu W, Wang M, Hao S et al (2018) Scalable active learning by approximated error reduction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1396–1405

  16. He G, Li B, Wang H et al (2020) Cost-effective active semi-supervised learning on multivariate time series data with crowds. IEEE Trans Syst Man Cybern: Systems 52(3):1437–1450

    Article  MATH  Google Scholar 

  17. Zhao Y, Lin J, Lin J et al (2023) Batch-mode active learning of gaussian process regression with maximum model change. IEEE Trans Syst Man Cybern: Systems

  18. Wang M, Zhang YY, Min F et al (2020) A two-stage density clustering algorithm. Soft Comput 24(23):17797–17819

    Article  MATH  Google Scholar 

  19. Yan X, Nazmi S, Gebru B et al (2022) A clustering-based active learning method to query informative and representative samples. Appl Intell 52(11):13250–13267

    Article  MATH  Google Scholar 

  20. Wang M, Min F, Zhang ZH et al (2017) Active learning through density clustering. Expert Syst Appl 85:305–317

    Article  MATH  Google Scholar 

  21. Ji X, Ye W, Li X et al (2023) Adaptive active learning through k-nearest neighbor optimized local density clustering. Appl Intell 53(12):14892–14902

    Article  Google Scholar 

  22. Min F, Zhang SM, Ciucci D et al (2020) Three-way active learning through clustering selection. Int J Mach Learn Cybern 11:1033–1046

    Article  MATH  Google Scholar 

  23. Wang Z, Ye J (2015) Querying discriminative and representative samples for batch mode active learning. ACM Trans Knowl Discov Data (TKDD) 9(3):1–23

    MATH  Google Scholar 

  24. Ghafarian SH (2023) Local variational probabilistic minimax active learning. Expert Syst Appl 211:118538

    Article  Google Scholar 

  25. Collobert R, Sinz F, Weston J et al (2006) Large scale transductive svms. J Mach Learn Res 7(8)

  26. Gu B, Zhai Z, Deng C et al (2020) Efficient active learning by querying discriminative and representative samples and fully exploiting unlabeled data. IEEE Trans Neural Netw Learn Syst 32(9):4111–4122

    Article  MathSciNet  Google Scholar 

  27. Wang M, Fu W, Hao S et al (2017) Learning on big graph: label inference and regularization with anchor hierarchy. IEEE Trans Knowl Data Eng 29(5):1101–1114

    Article  MATH  Google Scholar 

  28. Wang M, Fu W, Hao S et al (2016) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877

    Article  MATH  Google Scholar 

  29. Liu W, He J, Chang SF (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the 27th international conference on machine learning (ICML-10). Citeseer, pp 679–686

  30. Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell 36(11):2227–2240

    Article  MATH  Google Scholar 

  31. Bemporad A (2023) Active learning for regression by inverse distance weighting. Inf Sci 626:275–292

    Article  MATH  Google Scholar 

  32. Jose A, de Mendonça JPA, Devijver E et al (2024) Regression tree-based active learning. Data Min Knowl Disc 38(2):420–460

    Article  MathSciNet  MATH  Google Scholar 

  33. Kottke D, Herde M, Sandrock C et al (2021) Toward optimal probabilistic active learning using a bayesian approach. Mach Learn 110(6):1199–1231

    Article  MathSciNet  MATH  Google Scholar 

  34. Cho JW, Kim DJ, Jung Y et al (2022) Mcdal: maximum classifier discrepancy for active learning. IEEE Trans Neural Netw Learn Syst

  35. Wang M, Feng T, Shan Z et al (2022) Attribute and label distribution driven multi-label active learning. Appl Intell 52(10):11131–11146

    Article  MATH  Google Scholar 

  36. Caramalau R, Bhattarai B, Kim TK (2021) Sequential graph convolutional network for active learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9583–9592

  37. Lu S, Zheng J, Li Z et al (2024) Wmbal: weighted minimum bounds for active learning. Appl Intell, pp 1–13

  38. Roth D, Small K (2006) Margin-based active learning for structured output spaces. In: Machine Learning: ECML 2006: 17th European Conference on Machine Learning Berlin, Germany, September 18-22, 2006 Proceedings 17. Springer, pp 413–424

  39. Joshi AJ, Porikli F, Papanikolopoulos NP (2012) Scalable active learning for multiclass image classification. IEEE Trans Pattern Anal Mach Intell 34(11):2259–2273

    Article  MATH  Google Scholar 

  40. Wang D, Shang Y (2014) A new active labeling method for deep learning. In: 2014 International joint conference on neural networks (IJCNN). IEEE, pp 112–119

  41. Zhou Z, Shin JY, Gurudu SR et al (2021) Active, continual fine tuning of convolutional neural networks for reducing annotation efforts. Med Image Anal 71:101997

    Article  Google Scholar 

  42. Zhu X, Lafferty J, Ghahramani Z (2003) Combining active learning and semi-supervised learning using gaussian fields and harmonic functions. In: ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, pp 58–65

  43. Mac Aodha O, Campbell ND, Kautz J et al (2014) Hierarchical subquery evaluation for active learning on a graph. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 564–571

  44. Zhang Y, Zhao T, Miao D et al (2021) Granular multilabel batch active learning with pairwise label correlation. IEEE Trans Syst Man Cybern: Systems 52(5):3079–3091

    Article  MATH  Google Scholar 

  45. Li H, Wang Y, Li Y et al (2021) Batch mode active learning via adaptive criteria weights. Appl Intell 51:3475–3489

    Article  MATH  Google Scholar 

  46. Wang M, Fu K, Min F et al (2020) Active learning through label error statistical methods. Knowl-Based Syst 189:105140

    Article  MATH  Google Scholar 

  47. Hoi SC, Jin R, Zhu J et al (2009) Semisupervised svm batch mode active learning with applications to image retrieval. ACM Trans Inf Syst (TOIS) 27(3):1–29

    Article  MATH  Google Scholar 

  48. Hoi SC, Jin R, Zhu J et al (2006) Batch mode active learning and its application to medical image classification. In: Proceedings of the 23rd international conference on Machine learning, pp 417–424

  49. Nguyen HT, Smeulders A (2004) Active learning using pre-clustering. In: Proceedings of the twenty-first international conference on Machine learning, p 79

  50. Parvaneh A, Abbasnejad E, Teney D et al (2022) Active learning by feature mixing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12237–12246

  51. Patra S, Bruzzone L (2012) A cluster-assumption based batch mode active learning technique. Pattern Recogn Lett 33(9):1042–1048

    Article  MATH  Google Scholar 

  52. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423

    Article  MathSciNet  MATH  Google Scholar 

  53. Blake CL (1998) Uci repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html

  54. Breviglieri P, Erdem T, Eken S (2021) Predicting smart grid stability with optimized deep models. SN Comput Sci 2:1–12

    Article  Google Scholar 

  55. Liu Zi-Ang J, Xue W (2021) Unsupervised pool-based active learning for linear regression. Acta Automatica Sinica 47(12):2771–2783

    MATH  Google Scholar 

  56. Hinton G, Van Der Maaten L (2008) Visualizing data using t-sne journal of machine learning research. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  57. Yu H, Kim S (2010) Passive sampling for regression. In: 2010 IEEE international conference on data mining. IEEE, pp 1151–1156

Download references

Acknowledgements

This work was supported in part by the Natural Science Foundation of China under Grant 62176001, 62076002, and, in part by the Natural Science Foundation of Anhui Province under Grant 2008085MF194 and 2108085MF212, and in part by the Key Project of Natural Science Foundation of Anhui Provincial Department of Education under Grant KJ2020A0041 and KJ2021A0043.

Funding

the Natural Science Foundation of China under Grant 62176001, 62076002, and, in part by the Natural Science Foundation of Anhui Province under Grant 2008085MF194 and 2108085MF212, and in part by the Key Project of Natural Science Foundation of Anhui Provincial Department of Education under Grant KJ2020A0041 and KJ2021A0043.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xia Ji.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, X., Wang, L. & Fang, X. Semi-supervised batch active learning based on mutual information. Appl Intell 55, 117 (2025). https://doi.org/10.1007/s10489-024-05962-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05962-5

Keywords