Skip to main content

Advertisement

Springer Nature Link
Log in

Semi-supervised Bayesian ARTMAP

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper proposes a semi-supervised Bayesian ARTMAP (SSBA) which integrates the advantages of both Bayesian ARTMAP (BA) and Expectation Maximization (EM) algorithm. SSBA adopts the training framework of BA, which makes SSBA adaptively generate categories to represent the distribution of both labeled and unlabeled training samples without any user’s intervention. In addition, SSBA employs EM algorithm to adjust its parameters, which realizes the soft assignment of training samples to categories instead of the hard assignment such as winner takes all. Experimental results on benchmark and real world data sets indicate that the proposed SSBA achieves significantly improved performance compared with BA and EM-based semi-supervised learning method; SSBA is appropriate for semi-supervised classification tasks with large amount of unlabeled samples or with strict demands for classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge. http//www.kyb.tuebingen.mpg.de/ssl-book/

    Google Scholar 

  2. Zhu X (2006) Semi-supervised learning literature survey. Computer Sciences, University of Wisconsin-Madison, Technical report http://pages.cs.wisc.edu/~jerryzhu/research/ssl/semireview.html

  3. Cohen I, Cozman FG, Sebe N, Cirelo MC, Huang TS (2004) Semisupervised learning of classifiers: theory, algorithms, and their application to human-computer interaction. IEEE Trans Pattern Anal Mach Intell 26(12):1553–1567

    Article  Google Scholar 

  4. Lu Y, Tian Q, Liu F, Sanchez M, Wang Y (2007) Interactive semisupervised learning for microarray analysis. IEEE/ACM Trans Comput Biol Bioinf 4(2):190–203

    Article  Google Scholar 

  5. Mann GS, McCallum A (2007) Simple, robust, scalable semi-supervised learning via expectation regularization. In: Proceedings of the 24th international conference on machine learning, pp 593–600

  6. Ando RK, Tong Z (2007) Two-view feature generation model for semi-supervised learning. In: Proceedings of the 24th international conference on machine learning, pp 25–32

  7. Weston J, Leslie C, Ie E, Zhou DY, Elisseeff A, Noble WS (2005) Semi-supervised protein classification using cluster kernels. Bioinformatics 21(15):3241–3247

    Article  Google Scholar 

  8. Klose A, Kruse R (2005) Semi-supervised learning in knowledge discovery. Fuzzy Sets Syst 149(1):209–233

    Article  MATH  MathSciNet  Google Scholar 

  9. Li CH (2005) Guided Cluster Discovery with Markov Model. Appl Intell 22(1):37–46

    Article  MATH  Google Scholar 

  10. Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the 16th international conference on machine learning, pp 200–209

  11. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64

  12. Chapelle O, Chi M, Zien A (2006) A continuation method for semi-supervised SVMs. In: Proceedings of the 23th international conference on machine learning, pp 185–192

  13. Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712

    MathSciNet  Google Scholar 

  14. Astorino A, Fuduli A (2007) Nonsmooth optimization techniques for semi-supervised classification. IEEE Trans Pattern Anal Mach Intell 29(12):2135–2142

    Article  Google Scholar 

  15. Sindhwani V, Keerthi SS (2006) Large scale semi-supervised linear SVMs. In: Proceedings of the 29th international ACM SIGIR conference on research and development in information retrieval, pp 477–484

  16. Chapelle O, Sindhwani V, Keerthi SS (2006) Branch and bound for semi-supervised support vector machines. In: Proceedings of the 19th NIPS conference, pp 217–224

  17. Johnson R, Zhang T (2008) Graph-based semi-supervised learning and spectral kernel design. IEEE Trans Inf Theory 54(1):275–288

    Article  MathSciNet  Google Scholar 

  18. Belkin M, Niyogi P (2004) Semi-supervised learning on Riemannian manifolds. Mach Learn 56(1–3):209–239

    Article  MATH  Google Scholar 

  19. Zhu X, Ghahramani Z (2002) Learning from labeled and unlabeled data with label propagation. Technical report CMU-CALD-02-107, Carnegie Mellon University

  20. Szummer M, Jaakkola T (2002) Partially labeled classification with Markov random walks. In: Proceedings of the 14th NIPS conference, pp 945–952

  21. Blum A, Chawla S (2001) Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the 18th international conference on machine learning, pp 19–26

  22. Joachims T (2003) Transductive learning via spectral graph partitioning. In: Proceedings of the 20th international conference on machine learning, pp 290–297

  23. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64

  24. Pelckmans K, Shawe-Taylor J, Suykens JAK, De Moor B (2007) Margin based transductive graph cuts using linear programming. In: Proceedings of the 11th international conference on artificial intelligence, pp 360–367

  25. Bie TD, Cristianini N (2006) Fast SDP relaxations of graph cut clustering, transduction, and other combinatorial problems. J Mach Learn Res 7:1409–1436

    MathSciNet  Google Scholar 

  26. Mallapragada P, Jin R, Jain A, Liu Y SemiBoost: boosting for semi-supervised learning, IEEE Trans Pattern Anal Mach Intell. doi:10.1109/TPAMI.2008.235

  27. Deng C, Guo M (2006) Tri-training and data editing based semi-supervised clustering algorithm. In: MICAI 2006: advances in artificial intelligence, pp 641–651

  28. Nigam K, Ghani R (2000) Analyzing the effectiveness and applicability of co-training. In: Proceedings of the 9th international conference on information and knowledge management, pp 86–93

  29. Blum A, Mitchell T (1998) Combining Labeled and Unlabeled Data with Co-Training. In: Proceedings of the 11th annual conference on computational learning theory, pp 92–100

  30. Wang W, Zhou ZH (2007) Analyzing co-training style algorithms. In: Machine learning: ECML 2007, pp 454–465

  31. Goldman S, Zhou Y (2000) Enhancing supervised learning with unlabeled data. In: Proceedings of the 17th international conference on machine learning, pp 327–334

  32. Zhou ZH, Li M (2005) Tri-training: exploiting unlabeled data using three classifiers. IEEE Trans Knowl Data Eng 17(11):1529–1541

    Article  Google Scholar 

  33. Mavroeidis D, Chaidos K, Pirillos S, Christopoulos D, Vazirgiannis M (2006) Using tri-training and support vector machines for addressing the ECML/PKDD 2006 discovery challenge. In: Proceedings of the ECML-PKDD discovery challenge workshop, pp 39–47

  34. Nigam K, McCallum AK, Thrun S, Mitchell T (2000) Text classification from labeled and unlabeled documents using EM. Mach Learn 39(2–3):103–134

    Article  MATH  Google Scholar 

  35. Dong A, Bhanu B (2005) Active concept learning in image databases. IEEE Trans Syst Man Cybern B 35(3):450–466

    Article  Google Scholar 

  36. Song Y, Zhang C (2008) Content-based information fusion for semi-supervised music genre classification. IEEE Trans Multimedia 10(1):145–152

    Article  Google Scholar 

  37. Zhong S (2006) Semi-supervised model-based document clustering: a comparative study. Mach Learn 65(1):3–29

    Article  Google Scholar 

  38. Duda RO, Hart PE, Stork DG (2000) Pattern classification. Wiley-Interscience, Hoboken

    Google Scholar 

  39. Christopher M (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  40. Jank W (2006) EM algorithm, its randomized implementation and global optimization: some challenges and opportunities for operations research. In: Perspectives in operations research. Springer, New York, pp 367–392

    Chapter  Google Scholar 

  41. Carpenter GA, Grossberg S, Markuzon N (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5), 698–713

    Article  Google Scholar 

  42. Amorim DG, Delgado MF, Ameneiro SB (2007) Polytope ARTMAP: pattern classification without vigilance based on general geometry categories. IEEE Trans Neural Netw 18(5):1306–1325

    Article  Google Scholar 

  43. Williamson JR (1996) Gaussian ARTMAP: a neural network for fast incremental learning of noisy multidimensional maps. Neural Netw 9(5):881–897

    Article  Google Scholar 

  44. Murphey YL, Guo H, Feldkamp LA (2004) Neural learning from unbalanced data. Appl Intell 21(2):117–128

    Article  MATH  Google Scholar 

  45. Azouaoui O, Chohra A (2002) Soft computing based pattern classifiers for the obstacle avoidance behavior of intelligent autonomous vehicles (IAV). Appl Intell 16(3):249–272

    Article  MATH  Google Scholar 

  46. Sánchez EG, Dimitriadis YA, Cano-Izquierdo JM (2002) μ-ARTMAP: use of mutual information for category reduction in fuzzy ARTMAP. IEEE Trans Neural Netw 13(1):58–69

    Article  Google Scholar 

  47. Suzuki Y (1995) Self-organizing QRS-wave recognition in ECG using neural networks. IEEE Trans Neural Netw 6(6):1469–1477

    Article  Google Scholar 

  48. Xu R, Anagnostopoulos GC, Wunsch DC (2007) Multiclass cancer classification using semisupervised ellipsoid ARTMAP and particle swarm optimization with gene expression data. IEEE–ACM Trans Comput Biol Bioinf 4(1):65–77

    Article  Google Scholar 

  49. Vigdor B, Lerner B (2006) Accurate and fast off and online fuzzy ARTMAP-based image classification with application to genetic abnormality diagnosis. IEEE Trans Neural Netw 17(5):1288–1300

    Article  Google Scholar 

  50. Ng SK, McLachlan GJ (2003) On the choice of the number of blocks with the incremental EM algorithm for the fitting of normal mixtures. Stat Comput 13(1):45–55

    Article  MathSciNet  Google Scholar 

  51. Ng SK, McLachlan GJ (2003) On some variants of EM algorithm for the fitting of finite mixture models. Austrian J Stat 32:143–161

    Google Scholar 

  52. Vigdor B, Lerner B (2007) The Bayesian ARTMAP. IEEE Trans Neural Netw 18(6):1628–1644

    Article  Google Scholar 

  53. Xu L, Jordan MI (1996) On convergence properties of the EM algorithm for Gaussian mixtures. Neural Comput 8(1):129–151

    Article  Google Scholar 

  54. Ma JW, Fu SQ (2005) On the correct convergence of the EM algorithm for Gaussian mixtures. Pattern Recognit 38(12):2602–2611

    Article  Google Scholar 

  55. Byrne W, Gunawardana A (2000) Comments on efficient training algorithms for HMM’s using incremental estimation. IEEE Trans Speech Audio Process 8(6):751–754

    Article  Google Scholar 

  56. Gunawardana A, Byrne W (2005) Convergence theorems for generalized alternating minimization procedures. J Mach Learn Res 6:2049–2073

    MathSciNet  Google Scholar 

  57. Al-Daraiseh A, Kaylani A, Georgiopoulos M, Mollaghasemi M, Wu AS, Anagnostopoulos G (2007) GFAM: evolving fuzzy ARTMAP neural networks. Neural Netw 20(8):874–892

    Article  MATH  Google Scholar 

  58. Zhong MY, Rosander B, Georgiopoulos M, Anagnostopoulos GC, Mollaghasemi M, Richie S (2007) Experiments with safe mu ARTMAP: effect of the network parameters on the network performance. Neural Netw 20(2):245–259

    Article  MATH  Google Scholar 

  59. Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233

    Google Scholar 

  60. Kuroda M, Sakakihara M (2006) Accelerating the convergence of the EM algorithm using the vector ε algorithm. Comput Stat Data Anal 51(3):1549–1561

    Article  MATH  MathSciNet  Google Scholar 

  61. Ng SK, McLachlan GJ (2004) Speeding up the EM algorithm for mixture model-based segmentation of magnetic resonance images. Pattern Recognit 37(8):1573–1589

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Dalian University of Technology, 2 Linggong Lu, Ganjingzi Qu, Dalian, 116023, China

    Xiao-liang Tang & Min Han

Authors
  1. Xiao-liang Tang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Min Han
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Min Han.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tang, Xl., Han, M. Semi-supervised Bayesian ARTMAP. Appl Intell 33, 302–317 (2010). https://doi.org/10.1007/s10489-009-0167-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-009-0167-x

Keywords