Skip to main content

An Approach to Aggregating Ensembles of Lazy Learners That Supports Explanation

  • Conference paper
  • First Online:
Advances in Case-Based Reasoning (ECCBR 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2416))

Included in the following conference series:

  • 958 Accesses

Abstract

Ensemble research has shown that the aggregated output of an ensemble of predictors can be more accurate than a single predictor. This is true also for lazy learning systems like Case-Based Reasoning (CBR) and k-Nearest-Neighbour. Aggregation is normally achieved by voting in classification tasks and by averaging in regression tasks. For CBR, this increased accuracy comes at the cost of interpretability however. If we consider the use of retrieved cases for explanation to be one of the advantages of CBR then this is lost in an ensemble. This is because a large number of cases will have been retrieved by the ensemble members. In this paper we present a new technique for aggregation that obtains excellent results and identifies a small number of cases for use in explanation. This new approach might be viewed as a transformation process whereby cases are transformed from their feature based representation to a representation based on the predictions of ensemble members. This new representation produces very accurate predictions and allows a small number of similar neighbours to be identified.

This research was carried out as part of the MediLink project funded under the PRTLI programme of the Irish Higher Education Authority.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Armengol, E., Palaudàries, A., Plaza, E., (2001). Individual Prognosis of Diabetes Long-term Risks: A CBR Approach. Methods of Information in Medicine. Special issue on prognostic models in Medicine. vol. 40, pp. 46–51.

    Google Scholar 

  • Breiman, L., (1996) Bagging predictors. Machine Learning, 24:123–140.

    MATH  MathSciNet  Google Scholar 

  • Cunningham, P., Carney, J., (2000) Diversity versus Quality in Classification Ensembles based on Feature Selection, 11th European Conference on Machine Learning (ECML 2000), Lecture Notes in Artificial Intelligence, R. López de Mántaras and E. Plaza, (eds) pp109–116, Springer Verlag.

    Google Scholar 

  • Cunningham, P., Zenobi, G., (2001) Case Representation Issues for Case-Based Reasoning from Ensemble Research, in Proceedings of 4th International Conference on Case-Based Reasoning eds D. W. Aha, I. Watson, LNAI 2080, pp146–157, Springer Verlag.

    Google Scholar 

  • Guerra-Salcedo, C., Whitley, D., (1999a). Genetic Approach for Feature Selection for Ensemble Creation. in GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, Banzhaf, W., Daida, J., Eiben, A. E., Garzon, M. H., Honavar, V., Jakiela, M., & Smith, R. E. (eds.). Orlando, Florida USA, pp236–243, San Francisco, CA: Morgan Kaufmann.

    Google Scholar 

  • Guerra-Salcedo, C., Whitley, D., (1999b). Feature Selection Mechanisms for Ensemble Creation: A Genetic Search Perspective, in Data Mining with Evolutionary Algorithms: Research Directions. Papers from the AAAI Workshop. Alex A. Freitas (Ed.) Technical Report WS-99-06. AAAI Press, 1999.

    Google Scholar 

  • Hansen, L.K., Salamon, P., (1990) Neural Network Ensembles, IEEE Pattern Analysis and Machine Intelligence, 1990. 12,10, 993–1001.

    Article  Google Scholar 

  • Heskes, T.M. (1998). Selecting weighting factors in logarithmic opinion pools. Advances in Neural Information Processing Systems, 10, 266–272.

    Google Scholar 

  • Ho, T.K., (1998) Nearest Neighbours in Random Subspaces, Proc. Of 2nd International Workshop on Statistical Techniques in Pattern Recognition, A. Amin, D. Dori, P. Puil, H. Freeman, (eds.) pp640–648, Springer Verlag LNCS 1451.

    Google Scholar 

  • Kohavi, P. Langley, Y. Yun, (1997) The Utility of Feature Weighting in NearestNeighbor Algorithms, European Conference on Machine Learning, ECML’97, Prague, Czech Republic, 1997, poster.

    Google Scholar 

  • Krogh, A., Vedelsby, J., (1995) Neural Network Ensembles, Cross Validation and Active Learning, in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretsky, T. K. Leen, eds., pp231–238, MIT Press, Cambridge MA.

    Google Scholar 

  • Jacobs, R.A., Jordan, M.I., Nowlan, S.J., & Hinton, G.E., (1991) Adaptive mixtures of local experts, Neural Computation, 3, 79–98.

    Article  Google Scholar 

  • Leake, D., B., (1996) CBR in Context: The Present and Future, in Leake, D.B. (ed) Case-Based Reasoning: Experiences, Lessons and Future Directions, pp3–30, MIT Press.

    Google Scholar 

  • Ong, L.S., Sheperd, B., Tong, L.C., Seow-Choen, F., Ho, Y.H., Tong, L.C., Ho Y.S, Tan, K. (1997) The Colorectal Cancer Recurrence Support (CARES) System. Artificial Intelligence in Medicine 11(3): 175–188.

    Article  Google Scholar 

  • Ricci, F., & Aha, D. W. (1998). Error-correcting output codes for local learners. Proceedings of the Tenth European Conference on Machine Learning (280-291). Chemnitz, Germany: Springer.

    Google Scholar 

  • van de Laar, P., Heskes, T., (2000) Input selection based on an ensemble, Neurocomputing, 34:227–238.

    Article  MATH  Google Scholar 

  • Wettschereck, D., Aha, D. W., & Mohri, T. (1997). A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artificial Intelligence Review, 11, 273–314.

    Article  Google Scholar 

  • Zenobi, G., Cunningham, P., (2001) Using Diversity in Preparing Ensembles of Classifiers Based on Different Feature Subsets to Minimize Generalization Error, 12th European Conference on Machine Learning (ECML 2001), eds L. De Raedt & P. Flach, LNAI 2167, pp576–587, Springer Verlag.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zenobi, G., Cunningham, P. (2002). An Approach to Aggregating Ensembles of Lazy Learners That Supports Explanation. In: Craw, S., Preece, A. (eds) Advances in Case-Based Reasoning. ECCBR 2002. Lecture Notes in Computer Science(), vol 2416. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46119-1_32

Download citation

  • DOI: https://doi.org/10.1007/3-540-46119-1_32

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-44109-0

  • Online ISBN: 978-3-540-46119-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics