Skip to main content
SpringerLink
Log in

An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

An incremental online semi-supervised active learning algorithm, which is based on a self-organizing incremental neural network (SOINN), is proposed. This paper describes improvement of the two-layer SOINN to a single-layer SOINN to represent the topological structure of input data and to separate the generated nodes into different groups and subclusters. We then actively label some teacher nodes and use such teacher nodes to label all unlabeled nodes. The proposed method can learn from both labeled and unlabeled samples. It can query the labels of some important samples rather than selecting the labeled samples randomly. It requires neither prior knowledge, such as the number of nodes, nor the number of classes. It can automatically learn the number of nodes and teacher vectors required for a current task. Moreover, it can realize online incremental learning. Experiments using artificial data and real-world data show that the proposed method performs effectively and efficiently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Cohen I, Cozman FG, Sebe N, Cirelo MC, Huang TS (2004) Semisupervised learning of classifiers: theory, algorithms, and their application to human-computer interaction. IEEE Trans Pattern Anal Mach Intell 26(12):1553–1566

    Article  Google Scholar 

  2. Baraldi A, Bruzzone L, Blonda P (2006) A multiscale expectation-maximization semisupervised classifier suitable for badly posed image classification. IEEE Trans Image Process 15(8):2208–2225

    Article  Google Scholar 

  3. Yeung D-Y, Chang H (2007) A kernel approach for semisupervised metric learning. IEEE Trans Neural Netw 18(1):141–149

    Article  Google Scholar 

  4. Zaki SM, Yin H (2008) A semi-supersized learning algorithm for growing neural gas in face recognition. J Math Model Algorithms 7:425–435

    Article  MATH  Google Scholar 

  5. Freund Y, Seung H, Shamir E, Tishby N (1997) Selective sampling using the query by committee algorithm. Mach Learn 28:133–168

    Article  MATH  Google Scholar 

  6. Muslea I, Minston S, Knoblock C (2000) Selective sampling with redundant views. In: Proceedings of the national conference on artificial intelligence, pp 621–626

  7. Kothari R, Jain V (2003) Learning from labeled and unlabeled data using a minimal number of queries. IEEE Trans Neural Netw 14(6):1496–1505

    Article  Google Scholar 

  8. Mingkun L, Sethi IK (2006) Confidence-based active learning. IEEE Trans Pattern Anal Mach Intell 28(8):1251–1261

    Article  Google Scholar 

  9. Muslea I, Minston S, Knoblock C (2002) Active + semi-supervised learning = robust multi-view learning. In: Proceedings of ICML-02, 19th international conference on machine learning, pp 435–442

  10. Nigam K, McCallum AK, Thrum S, Mitchell T (2000) Text classification from labeled and unlabeled documents using em. Mach Learn 39:103–134

    Article  MATH  Google Scholar 

  11. Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the conference on computational learning theory, pp 92–100

  12. Bennett K, Demiriz A (1999) Semi-supervised support vector machines. Adv Neural Inf Process Syst 11:368–374

    Google Scholar 

  13. Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: ICML-03, 20th international conference on machine learning, pp 912–919

  14. Zhu X, Lafferty J, Ghahramani Z (2003) Combining active learning and semi-supervised learning using gaussian fields and harmonic functions. In: ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, pp 58–65

  15. Wang Z, Song Y, Zhang C (2009) Efficient active learning with boosting. In: SDM 2009, pp 1230–1241

  16. Huang R, Lam W (2009) An active learning framework for semi-supervised document clustering with language modeling. Data Knowl Eng 68(1):49–67

    Article  Google Scholar 

  17. Hoi SCH, Jin R, Zhu J, Lyu MR (2009) Semi-supervised SVM batch mode active learning with applications to image retrieval. ACM Trans Inf Syst 27(3):16:1–16:29

    Article  Google Scholar 

  18. Carpenter GA, Grossberg S (1988) The art of adaptive pattern recognition by a self-organizing neural network. IEEE Comput 21:77–88

    Google Scholar 

  19. Shen F, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19:90–106

    Article  MATH  Google Scholar 

  20. Kamiya Y, Shen F, Hasegawa O (2007) An incremental neural network for online supervised learning and topology learning. J Adv Comput Intell Intell Inform 11(1):87–95

    Google Scholar 

  21. Shen F, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw 21:1537–1547

    Article  Google Scholar 

  22. Sudo A, Sato A, Hasegawa O (2007) Associative memory for online incremental learning ina noisy environment. In: The 2007 international joint conference on neural networks

  23. Shen F, Sudo A, Hasegawa O (2010) An online incremental learning pattern-based reasoning system. Neural Netw 23(1):135–143

    Article  Google Scholar 

  24. Shen F, Ogura T, Hasegawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20:893–903

    Article  MATH  Google Scholar 

  25. He X, Kojima R, Hasegawa O (2007) Developmental word grounding through a growing neural network with a humanoid robot. IEEE Trans Syst Man Cybern B 37(2):451–462

    Article  Google Scholar 

  26. He X, Ogura T, Satou A, Hasegawa O (2007) Developmental word acquisition and grammar learning by humanoid robots through a self-organizing incremental neural network. IEEE Trans Syst Man Cybern B 37(5):1357–1372

    Article  Google Scholar 

  27. Merz C, Murphy M (1996) Uci repository of machine learning database. University of California Department of Information, Irvine

  28. Chang C-C, Lin C-J (2001) IBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm

  29. Fritzke B (1995) A growing neural gas network learns topology. In: Advances in neural information processing systems 7. MIT, Cambridge

Download references

Acknowledgements

This work was supported in part by the Fund of the National Natural Science Foundation of China (Grant No. 60975047, 60723003, 60721002), 973 Program (2010CB327903), and Jiangsu NSF grant (\#BK2009080). This study was also supported in part by the New Energy and Industrial Technology Development Organization (NEDO) of Japan.

Author information

Authors and Affiliations

  1. The State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, People’s Republic of China

    Furao Shen & Hui Yu

  2. Jiangyin Information Technology Research Institute, Nanjing University, Nanjing, People’s Republic of China

    Furao Shen & Hui Yu

  3. Imaging Science and Engineering Laboratory, Tokyo Institute of Technology, Tokyo, Japan

    Keisuke Sakurai & Osamu Hasegawa

Authors
  1. Furao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keisuke Sakurai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Osamu Hasegawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Furao Shen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shen, F., Yu, H., Sakurai, K. et al. An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network. Neural Comput & Applic 20, 1061–1074 (2011). https://doi.org/10.1007/s00521-010-0428-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-010-0428-y

Keywords

Search

Navigation