Skip to main content
SpringerLink
Log in

Application of fuzzy ARTMAP and fuzzy c-means clustering to pattern classification with incomplete data

  • Original Article
  • Published:
Neural Computing & Applications Aims and scope Submit manuscript

Abstract

In this paper, a hybrid neural network that is capable of incremental learning and classification of patterns with incomplete data is proposed. Fuzzy ARTMAP (FAM) is employed as the constituting network for pattern classification while fuzzy c-means (FCM) clustering is used as the underlying algorithm for processing training as well as test samples with missing features. To handle an incomplete training set, FAM is first trained using complete samples only. Missing features of the training samples are estimated and replaced using two FCM-based strategies. Then, network training is conducted using all the complete and estimated samples. To handle an incomplete test set, a non-substitution FCM-based strategy is employed so that a predicted output can be produced rapidly. The performance of the proposed hybrid network is evaluated using a benchmark problem, and its practical applicability is demonstrated using a medical diagnosis task. The results are compared, analysed and quantified statistically with the bootstrap method. Implications of the proposed network for pattern classification tasks with incomplete data are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Kahl F, Heyden A, Quan L (2001) Minimal projective reconstruction including missing data. IEEE Trans Pattern Anal Machine Intell 23:418–423

    Article  Google Scholar 

  2. Huang X, Zhu Q (2002) A pseudo-nearest-neighbor approach for missing data recovery on Gaussian random data sets. Pattern Recogn Lett 23:1613–1622

    Article  MATH  Google Scholar 

  3. Trucco E, Fusiello A, Roberto V (1999) Robust motion and correspondence of noisy 3-D point sets with missing data. Pattern Recogn Lett 20:889–898

    Article  Google Scholar 

  4. King G, Honaker J, Joseph A, Scheve K (2001) Analyzing incomplete political science data: an alternative algorithm for multiple imputation. Am Political Sci Rev 95:49–69

    MATH  Google Scholar 

  5. Little R, Rubin D (1987) Statistical analysis with missing data. Wiley, New York

    Google Scholar 

  6. Tresp V, Neuneier R, Ahmad S (1995) Efficient methods for dealing with missing data in supervised learning. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems 7. Morgan Kaufman, San Mateo, California, pp 689–696

    Google Scholar 

  7. Ishibuchi H, Miyazaki A, Kwon K, Tanaka H (1993) Learning from incomplete training data with missing values and medical application. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’93), Nagoya, Japan, October 1993, pp 1871–1874

  8. Ghahramani Z, Jordan MI (1994) Supervised learning from incomplete data via an EM approach. In: Cowan JD, Tesauro G, Alspector J (eds) Advances in neural information processing systems 6. Morgan Kaufmann, San Mateo, California, pp 120–127

    Google Scholar 

  9. Nijman MJ, Kappen HJ (1997) Symmetry breaking and training from incomplete data with radial basis Boltzmann machines. Int J Neural Syst 8:301–316

    Article  Google Scholar 

  10. McMichael D, Liu L, Pan H (1999) Estimating the parameters of mixed Bayesian networks from incomplete data. In: Proceedings of the conference on information, decision control (IDC’99), Adelaide, Australia, February 1999, pp 1–6

    Google Scholar 

  11. Carpenter GA, Grossberg S (1987) A massively parallel architecture for a self-organising neural pattern recognition machine. Comput Vis Graph Image Process 37:54–115

    MATH  Google Scholar 

  12. Carpenter GA, Grossberg S (1988) The ART of adaptive pattern recognition by a self-organising neural network. IEEE Comput 21:77–88

    Google Scholar 

  13. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3:698–712

    Article  Google Scholar 

  14. Cano Izquierdo J, Dimitriadis Y, Gómez Sánchez E, López-Coronado J (2001) Learning from noisy information in FasArt and FasBack neuro-fuzzy systems. Neural Netw 14:407–425

    Article  Google Scholar 

  15. Carpenter GA, Ross W (1995) ART-EMAP: a neural network architecture for object recognition by evidence accumulation. IEEE Trans Neural Netw 6:805–818

    Article  Google Scholar 

  16. Williamson JR (1996) Gaussian ARTMAP: a neural network for fast incremental learning of noisy multidimensional maps. Neural Netw 9:881–897

    Article  Google Scholar 

  17. Carpenter GA, Milenova BL, Noeske BW (1998) Distributed ARTMAP: a neural network for fast distributed supervised learning. Neural Netw 11:793–813

    Article  Google Scholar 

  18. Granger E, Rubin MA, Grossberg S, Lavoie P (2001) Classification of incomplete data using the Fuzzy ARTMAP neural network. In: Proceedings of the INNS-IEEE international joint conference on neural networks (IJCNN 2001), Washington, DC, July 2001, 6:35–40

  19. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York

    Google Scholar 

  20. Hathaway RJ, Bezdek JC (2001) Fuzzy c-means clustering of incomplete data. IEEE Trans Syst Man Cybern B Cybern 31:735–744

    Article  Google Scholar 

  21. Lim CP, Kuan MM, Harrison RF (2003) An ART-based hybrid network for medical pattern classification tasks with missing data. Proceedings of the 7th international conference on knowledge-based intelligent information and engineering systems (KES 2003), Oxford, UK, September 2003, vol 2, pp 7–15

  22. Carpenter GA, Grossberg S, Rosen DB (1991) Fuzzy ART: fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Netw 4:759–771

    Article  Google Scholar 

  23. Xie XL, Beni G (1991) A validity measure for fuzzy clustering. IEEE Trans Pattern Anal Machine Intell 13:841–847

    Article  Google Scholar 

  24. Pal NR, Bezdek JC (1995) On cluster validity for the fuzzy c-means model. IEEE Trans Fuzzy Syst 3:370–379

    Article  Google Scholar 

  25. Lim CP, Harrison RF (1997) An incremental adaptive network for on-line supervised learning and probability estimation. Neural Netw 10:925–939

    Article  Google Scholar 

  26. Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Available at http://www.ics.uci.edu/~mlearn/MLRepository.html, University of California, Department of Information and Computer Science, Irvine, California

  27. Yoon SY, Lee SY (1999) Training algorithm with incomplete data for feed-forward neural networks. Neural Processing Lett 10:171–179

    Article  Google Scholar 

  28. Efron B (1979) Bootstrap methods: another look at the jackknife. Ann Statist 7:1–26

    MATH  Google Scholar 

Download references

Acknowledgements

The corresponding author gratefully acknowledges the research grants provided by Universiti Sains Malaysia and the Ministry of Science, Technology and Innovation Malaysia (no. 06-02-05-8002 and no. 04-02-05-0010, respectively) that have, in part, resulted in this article.

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, University of Science Malaysia, Engineering Campus, 14300, Nibong Tebal, Penang, Malaysia

    Chee Peng Lim & Mei Ming Kuan

  2. Department of Automatic Control and Systems Engineering, The University of Sheffield, Mappin Street, Sheffield, S1 3JD, UK

    Robert F. Harrison

Authors
  1. Chee Peng Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mei Ming Kuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert F. Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chee Peng Lim.

Appendix

Appendix

The notation used in accordance with the fuzzy c-means clustering algorithm are:

  • X=(x1,..., x n ), a data set with n data vectors

  • ?≡missing features; c=number of clusters; ε>0 stopping criterion; r=0, 1, 2,..., iteration counter

  • x k =(x p , x m ), k-th s-dimensional data vector, 1≤k≤n, where x p and x m , respectively, are the present and missing portions of x k

  • x kj =j-th feature value of the k-th data vector, 1≤j≤s, 1≤k≤n

  • X W ={x k ∈X|x k is a complete datum} (the whole-data subset of X)

  • X P ={x kj for 1≤j≤s, 1≤k≤n|the value for x kj is present in X}

  • X M ={x kj =? for 1≤j≤s, 1≤k≤n|the value for x kj is missing from X}

  • v i =i-th cluster centre, for 1≤i≤c

  • \( D = {\left\| {\mathbf{z}} \right\|}^{2}_{A} = {\mathbf{z}}^{{\text{T}}} A{\mathbf{z}}, \) the vector A-norm (e.g. Euclidean distance if A is a diagonal matrix)

  • U=the fuzzy membership (partition) matrix with elements U ik , 1≤i≤c,1≤k≤n

  • m>1, the fuzzification (weighting) parameter

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lim, C.P., Kuan, M.M. & Harrison, R.F. Application of fuzzy ARTMAP and fuzzy c-means clustering to pattern classification with incomplete data. Neural Comput & Applic 14, 104–113 (2005). https://doi.org/10.1007/s00521-004-0445-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-004-0445-9

Keywords

Search

Navigation