Skip to main content
SpringerLink
Log in

Building ensemble classifiers using belief functions and OWA operators

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

A pervasive task in many forms of human activity is classification. Recent interest in the classification process has focused on ensemble classifier systems. These types of systems are based on a paradigm of combining the outputs of a number of individual classifiers. In this paper we propose a new approach for obtaining the final output of ensemble classifiers. The method presented here uses the Dempster–Shafer concept of belief functions to represent the confidence in the outputs of the individual classifiers. The combing of the outputs of the individual classifiers is based on an aggregation process which can be seen as a fusion of the Dempster rule of combination with a generalized form of OWA operator. The use of the OWA operator provides an added degree of flexibility in expressing the way the aggregation of the individual classifiers is performed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahmadzadeh MR, Petrou M (2003) Use of Dempster–Shafer theory to combine classifiers which use different class boundaries. Pattern Anal Appl 6:41–46

    Article  MATH  MathSciNet  Google Scholar 

  • Al-Ani A, Deriche M (2002) A new technique for combining multiple classifiers using the Dempster–Shafer theory of evidence. J Artif Intell Res 17:333–361

    MATH  MathSciNet  Google Scholar 

  • Ali K (1995) A comparison of methods for learning and combining evidence from multiple models. Technical Report 95–47, Dept. of Information and Computer Science, University of California, Irvine

  • Ali K, Pazzani M (1996) Error reduction through learning multiple descriptions. Mach Learn 24:173–202

    Google Scholar 

  • Altincay H (2005) A Dempster–Shafer theoretic framework for boosting based ensemble design. Pattern Anal Appl 8:287–302

    Article  MathSciNet  Google Scholar 

  • Altincay H (2006) On the independence requirement in Dempster– Shafer theory for combining classifiers providing statistical evidence. Appl Intell 25:73–90

    Article  MATH  Google Scholar 

  • Altincay H, Demirekler M (2003) Speaker identification by combining multiple classifiers using Dempster–Shafer theory of evidence. Speech Commun 41:531–547

    Article  Google Scholar 

  • Binaghi E, Madella P (1999) Fuzzy Dempster–Shafer reasoning for rule-based classifiers. Int J Intell Syst 14:559–583

    Article  MATH  Google Scholar 

  • Breiman L (1996) Bagging predictors. Mach Learn 24:123–140

    MATH  MathSciNet  Google Scholar 

  • Buntine W (1990) A theory of learning classification rules. Ph.D. Dissertation, University of Technology, Sydney, Australia

  • Cios KJ, Pedrycz W, Swiniarski RW (1998) Data mining methods for knowledge discovery. Kluwer, Boston

    MATH  Google Scholar 

  • Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13:21–27

    Article  MATH  Google Scholar 

  • Dempster AP, Yager RR, Liu L (2007) Classic works on the Dempster– Shafer theory of belief functions. Studies in fuzziness & soft computing, vol. 219. Springer, Heidelberg

    Google Scholar 

  • Denoeux T (1995) A k-nearest neighbor classification rule based on Dempster–Shafer theory. IEEE Trans Syst Man Cybern 25(5):804–813

    Article  Google Scholar 

  • Denoeux T (1997) Analysis of evidence-theoretic decision rules for pattern classification. Pattern Recognit 30(7):1095–1107

    Article  Google Scholar 

  • Denoeux T (2000) A neural network Classifier Based on Dempster–Shafer Theory. IEEE Trans Syst Man Cybern A 30(2):131–150

    Article  MathSciNet  Google Scholar 

  • Dietterich TG (2000) Ensemble methods in machine learning. In: Kittler J, Roli F (eds) First international workshop on multiple classifier systems. Lecture Notes in Computer Science. Springer, New York, pp 1–15

    Chapter  Google Scholar 

  • Duda RO, Hart PE, Stork DG (2001) Pattern classification. Wiley Interscience, New York

    MATH  Google Scholar 

  • Dunham M (2003) Data mining. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Freund Y, Schapire RE (1996) Experiments with a new boosting algorithm. In: Proceedings of the 13th international conference on machine learning. Morgan Kaufmann, San Francisco, pp 148–156

  • Han J, Kamber M (2001) Data mining: Concepts And Techniques. Morgan Kaufmann, San Francisco

    Google Scholar 

  • Ho TK, Hull JJ, Srihari SS (1994) Decision combination in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell 16:66–75

    Article  Google Scholar 

  • Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20:226–239

    Article  Google Scholar 

  • Klement EP, Mesiar R, Pap E (2000) Triangular norms. Kluwer, Dordrecht

    MATH  Google Scholar 

  • Kononenko M, Kovacic M (1992) Learning as optimization: stochastic generation of multiple knowledge. In: Proceedings of 9th international workshop on machine learning, Aberdeen, UK, Morgan Kaufmann, pp 257–262

  • Kramosil I (2001) Dempster combination rule with boolean-like processed belief functions. Int J Uncertain Fuzziness Knowl Based Syst 9(1):105–121

    MATH  MathSciNet  Google Scholar 

  • Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms. Wiley, Hoboken

    MATH  Google Scholar 

  • Kwok S, Carter C (1990) Multiple Decision Trees. Uncertain Artif Intell 4:327–335

    Google Scholar 

  • Laha A, Pal NR, Das J (2006) Land cover classification using fuzzy rules and aggregation of contextual information through evidence theory. IEEE Trans Geosci Remote Sens 44(6):1633–1641

    Article  Google Scholar 

  • Lin TS, Yao YY, Zadeh LA (2002) Data mining, rough sets and granular computing. Physica-Verlag, Heidelberg

    MATH  Google Scholar 

  • Mandler E, Schurmann J (1988) Combining the classification results of independent classifiers based on the Dempster–Shafer theory of evidence. In: Gelsema E, Kanal L (eds) Pattern recognition and artificial intelligence, pp 381–393

  • O’Hagan M (1990) Using maximum entropy-ordered weighted averaging to construct a fuzzy neuron. In: Proceedings 24th Annual IEEE Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, pp 618–623

  • Reformat M (2005) A fuzzy-based meta-model system for reasoning about the number of software defects. Int J Intell Syst 20:1093–1115

    Article  MATH  Google Scholar 

  • Rogova G (1994) Combining the results of several neural network classifiers. Neural Netw 7:777–781

    Article  Google Scholar 

  • Roli F (2006) A gentle introduction to fusion of multiple pattern classifiers, in data fusion for situation monitoring, incident detection, alert and response management. In: Shahbazian E, Rogova G, Valin P (eds) IOS NATO Publication, Amsterdam, pp 23–34

  • Shafer G (1976) A mathematical theory of evidence. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Smets Ph (1988) Belief functions. In: Smets Ph, Mamdani A, Dubois D, Prade H (eds) Non standard logics for automated reasoning. Academic, London, pp 253–286

    Google Scholar 

  • Smets Ph, Kennes R (1994) The transferable belief model. Artif Intell 66:191–234

    Article  MATH  MathSciNet  Google Scholar 

  • Todorovski L, Dzeroski S (2000) Combining multiple models with meta decision trees. In: Proceedings of the 4th European conference on principles of data mining and knowledge discovery. Springer, Heidelberg, pp 54–64

  • Winer BJ, Brown DR, Michels KM (1991) Statistical principles in experimental design. McGraw-Hill, New York

    Google Scholar 

  • Yager RR (1993) Families of OWA operators. Fuzzy Sets Syst 59:125–148

    Article  MATH  MathSciNet  Google Scholar 

  • Yager RR (1996) Quantifier guided aggregation using OWA operators. Int J Intell Syst 11:49–73

    Article  Google Scholar 

  • Yager RR (1988) On ordered weighted averaging aggregation operators in multi-criteria decision making. IEEE Trans Syst Man Cybern 18:183–190

    Article  MATH  MathSciNet  Google Scholar 

  • Yager RR (2005) Extending multicriteria decision making by mixing t-norms and OWA operators. Int J Intell Syst 20:453–474

    Article  MATH  Google Scholar 

  • Yager RR (2006) Generalized naive Bayesian modeling. Inf Sci 176:577–588

    Article  MathSciNet  Google Scholar 

  • Zadeh LA (1983) A computational approach to fuzzy quantifiers in natural languages. Comput Math Appl 9:149–184

    Article  MATH  MathSciNet  Google Scholar 

  • Zouhal LM, Denoeux T (1998) An evidence-theoretic k-NN rule with parameter optimization. IEEE Trans Syst Man Cybern C 28(2):263–271

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. thinkS2 Thinking Software and System Laboratory, Electrical and Computer Engineering, University of Alberta, Edmonton, Canada, T6G 2Y7

    Marek Reformat

  2. Machine Intelligence Institute, Iona College, New Rochelle, NY, 10801, USA

    Ronald R. Yager

Authors
  1. Marek Reformat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ronald R. Yager
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ronald R. Yager.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Reformat, M., Yager, R.R. Building ensemble classifiers using belief functions and OWA operators. Soft Comput 12, 543–558 (2008). https://doi.org/10.1007/s00500-007-0227-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-007-0227-2

Keywords

Search

Navigation