Skip to main content
SpringerLink
Log in

Fusing Neural Networks Through Space Partitioning and Fuzzy Integration

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. Aggregation weights assigned to neural networks or groups of networks can be the same in the entire data space or can be different (data dependent) in various regions of the space. In this paper, we propose a method for obtaining data dependent aggregation weights. The proposed approach is tested in two aggregation schemes, namely aggregation through neural network selection, and aggregation by the Choquet integral with respect to the λ-fuzzy measure. The effectiveness of the approach is demonstrated on two artificial and three real data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Battiti, R. and Colla, M.: Democracy in neural nets: Voting schemes for classification. Neural Networks, 7(4) (1994), 691–707.

    Google Scholar 

  2. Bezdek, J. C.: Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, 1981.

    Google Scholar 

  3. Breiman, L.: Bagging predictors. Technical report 421, Statistics Departament, University of California, Berkeley, 1994.

    Google Scholar 

  4. Breiman, L.: Half & Half bagging and hard boundary points. Technical report 534, Statistics Departament, University of California, Berkeley, 1998. www.stat.berkley.edu/users/ breiman

    Google Scholar 

  5. Chen, D. and Cheng, X.: An asymptotic analysis of some expert fusion methods. Pattern Recognition Letters, 22 (2001), 901–904.

    Google Scholar 

  6. Chen, W., Gader, P. D., Shi, H.: Improved dynamic programming-based handwritten word recognition using optimal order statistics. In Proceedings of the International Conference 'Statistical and Stochastic Methods in Image Processing II', pages 246–256, San Diego, 1997.

  7. Delve. http://www.cs.toronto.edu/~delve/

  8. Efron, B. and Tibshirani, R.: An introduction to the bootstrap. Chapman and Hall, London, 1993.

    Google Scholar 

  9. Freund, Y. and Schapire, R. E.: Experiments with a new boosting algorithm. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 148–156, 1996.

  10. Freund, Y. and Schapire, R. E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55 (1997), 119–139.

    Google Scholar 

  11. Gader, P. D., Mohamed M. A. and Keller, J. M.: Fusion of handwritten word classifiers. Pattern Recognition Letters, 17 (1996), 577–584.

    Google Scholar 

  12. Grabisch, M. and Nicolas, J.-M.: Classification by fuzzy integral: Performance and tests. Fuzzy Sets and Systems, 65 (1994), 255–271.

    Google Scholar 

  13. Hashem, S.: Optimal linear combinations of neural networks. Neural Networks, 10(4) (1997), 599–614.

    Google Scholar 

  14. Ho, T. K., Hull, J. J. and Srihari, S. N.: Decision combination in multiple classifier systems. IEEE Trans. Pattern Analysis and Machine Intelligence, 6(1) (1994), 66–75.

    Google Scholar 

  15. Huang, Y. S. and Suen, C. Y.: A method of combining multiple classifiers-A neural network approach. In Proceedings of the 12th International Conference on Pattern Recognition, pages 473–475, Jerusalem, 1994.

  16. Jordan, M. I. and Xu, L.: Convergence results of the EM approach to mixtures of experts architectures. Neural Networks, 8 (1995), 1409–1431.

    Google Scholar 

  17. Jutten, C. et al.: ESPRIT basic research project number 6891 ELENA. ftp.dice.ucl.ac.be/pub/neural-net/ELENA/databases

  18. Keller, J. M., Gader, P. Tahani, H. Chiang, J. H. and Mohamed, M.: Advances in fuzzy integration for pattern-recognition. Fuzzy Sets and Systems, 65(2-3) (1994), 273–283.

    Google Scholar 

  19. Kittler, J., Hatef, M. Duin, R. P. W. and Matas, J.: On combining classifiers. IEEE Trans Pattern Analysis and Machine Intelligence, 20(3) (1998), 226–239.

    Google Scholar 

  20. Kuncheva, L. I., Bezdek, J. C. and Duin, R. P. W.: Decision templates for multiple classifier fusion. Pattern Recognition, 34(2) (2001), 299–314.

    Google Scholar 

  21. MacKay, D. J.: Bayesian interpolation. Neural Computation, 4 (1992), 415–447.

    Google Scholar 

  22. Optitz, D. W. and Shavlik, J. W.: Generating accurate and diverse members of a neuralnetwork ensemble. In: Touretzky, D. S. Mozer, M. C. and Hasselmo, M. E. (eds):, Advances in Neural Information Processing Systems 8, pages 535–541. MIT Press, 1996.

  23. Perrone, M. P. and Cooper, L. N.: When networks disagree: Ensamble method for neural networks. In: Mammone, R. J. (ed.), Neural Networks for Speech and Image Processing, Chapman-Hall, 1993.

  24. Sollich, P. and Krogh, A.: Learning with ensembles: How over-fitting can be useful. In: Touretzky, D. S. Mozer, M. C. and Hasselmo, M. E. (eds.), Advances in Neural Information Processing Systems 8, pages 190–197. MIT Press, 1996.

  25. Sugeno, M.: Fuzzy measures and fuzzy integrals: A survey. In: Automata and Decision making, pages 89–102. Amsterdam, North Holland, 1977.

    Google Scholar 

  26. Taniguchi, M., and Tresp, V.: Averaging regularized estimators. Neural Computation 9 (1997), 1163–1178.

    Google Scholar 

  27. Tresp, V. and Taniguchi, M.: Combining estimators using non-constant weighting functions. In: Tesauro, G. Touretzky, D. S. and Leen, T. K. (eds.), Advances in Neural Information Processing Systems 7, MIT Press, 1996.

  28. Tsao, C. E, Bezdek, J. C. and Pal, N. R.: Fuzzy Kohonen Clustering Networks. Pattern Recognition, 29 (1996), 757–764.

    Google Scholar 

  29. Tumer, K., and Ghosh, J.: Linear and order statistics combiners for pattern classification. In: Sharkey, A. J. C.: (ed.), Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pages 127–162. Springer-Verlag, 1999.

  30. Verikas, A., Bachauskiene, M. Vilunas, S. and Skaisgiris, D.: Adaptive character recognition system. Pattern Recognition Letters, 13 (1992), 207–212.

    Google Scholar 

  31. Verikas, A., Lipnickas, A. Malmqvist, K. Bacauskiene, M. and Gelzinis, A.: Soft combination of neural classifiers: A comparative study. Pattern Recognition Letters, 20 (1999), 429–444.

    Google Scholar 

  32. Verikas, A., and A. Gelzinis. Training neural networks by stochastic optimisation. Neurocomputing, 30 (2000), 153–172.

    Google Scholar 

  33. Verikas, A. et al.: Fusing neural networks through fuzzy integration. In: Bunke, H. Kandel, A. (eds.), Hybrid Methods in Pattern Recognition, World Scientific, 2001, in press.

  34. Waterhouse, S. and Cook, G.: Ensemble methods for phoneme classification. In: Mozer, M. C. Jordan, M. I. and Petsche, T. (eds.), Advances in Neural Information Processing Systems 9, pages 800–806. MIT Press, 1997.

  35. Woods, K., Kegelmeyer, W. P. and Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans Pattern Analysis Machine Intelligence, 19(4) (1997), 405–410.

    Google Scholar 

  36. Xu, L., Krzyzak, A., and Suen, C. Y.: Methods for combining multiple classifiers and their appli-cations to handwriting recognition. IEEE Trans. Systems, Man, and Cybernetics, 22(3) (1992), 418–435.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intelligent Systems Laboratory, Halmstad University, Box 823, S-301 18, Halmstad, Sweden

    Antonas Verikas

  2. Department of Applied Electronics, Kaunas University of Technology, LT-3031, Kaunas, Lithuania

    Arunas Lipnickas

Authors
  1. Antonas Verikas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arunas Lipnickas
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Verikas, A., Lipnickas, A. Fusing Neural Networks Through Space Partitioning and Fuzzy Integration. Neural Processing Letters 16, 53–65 (2002). https://doi.org/10.1023/A:1019703911322

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1019703911322

Search

Navigation