Abstract
Evaluating the performance of classifiers is a difficult task in machine learning. Many criteria have been proposed and used in such a process. Each criterion measures some facets of classifiers. However, none is good enough for all cases. In this communication, we justify the use of discrimination measures for evaluating classifiers. The justification is mainly based on a hierarchical model for discrimination measures, which was introduced and used in the induction of decision trees.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alvarez, I.: Explaining the result of a decision tree to the end-user. In: Proceedings of the 16th European Conference on Artificial Intelligence, ECAI 2004, pp. 411–415 (2004)
Bradley, A.P.: The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7), 1145–1159 (1997)
Caruana, R., Joachims, T., Backstrom, L.: KDD-Cup 2004: Results and analysis. SIGKDD Explorations 6(2), 95–108 (2004)
Dang, T.H.: Entropies et leurs applications en apprentissage inductif (rapport de pré-soutenance). Technical report, Université Paris 6, France (2005)
Dang, T.H., Bouchon-Meunier, B., Marsala, C.: Measures of information for inductive learning. In: Proc. of Information Processing and Management of Uncertainty in Knownledge-Based Systems, IPMU 2004, Perugia - Italy, July 2004, pp. 1495–1502 (2004)
Fawcett, T.: Roc graphs: Notes and practical considerations for researchers. Machine Learning (1-2) (2004)
Kononenko, I., Bratko, I.: Information-based evaluation criterion for classifier’s performance. Machine Learning 6, 67–80 (1991)
Korb, K.B., Hope, L.R., Hughes, M.J.: The evaluation of predictive learners: Some theoretical and empirical results. In: EMCL 2001: Proceedings of the 12th European Conference on Machine Learning, London, UK, pp. 276–287. Springer, Heidelberg (2001)
Marsala, C.: Apprentissage inductif en présence de données imprécises: construction et utilisation d’arbres de décision flous. Ph.D thesis, Université Paris 6, France (1998)
Marsala, C., Bouchon-Meunier, B., Ramer, A.: Hierarchical model for discrimination measures. In: Proc. of the eight IFSA 1999 World Congres, Taipei - Taiwan, August 1999, pp. 339–343 (1999)
Provost, F.J., Fawcett, T.: Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: Knowledge Discovery and Data Mining, pp. 43–48 (1997)
Provost, F.J., Fawcett, T., Kohavi, R.: The case against accuracy estimation for comparing induction algorithms. In: ICML 1998: Proceedings of the Fifteenth International Conference on Machine Learning, pp. 445–453. Morgan Kaufmann Publishers Inc., San Francisco (1998)
Quinlan, J.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)
Ramer, A., Bouchon-Meunier, B., Marsala, C.: An alytical structure of hierarchical discrimination. In: Proc. of the IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE, Seoul - Korea, August 1999, pp. 1050–1053 (1999)
Hettich, C.B.S., Merz, C.: UCI repository of machine learning databases (1998)
Shannon, C.: A mathematical theory of communication. Bell System Technical Journal 27, 379–423, 623–656 (1948)
Stermann, F., Longuet, N.: Document technique de DTGen, rapport de stage de fin d’étude, DESS IA. Technical report, Laboratoire d’Informatique de Paris 6, Avril (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dang, T.H., Marsala, C., Bouchon-Meunier, B., Boucher, A. (2006). Discrimination-Based Criteria for the Evaluation of Classifiers. In: Larsen, H.L., Pasi, G., Ortiz-Arroyo, D., Andreasen, T., Christiansen, H. (eds) Flexible Query Answering Systems. FQAS 2006. Lecture Notes in Computer Science(), vol 4027. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11766254_47
Download citation
DOI: https://doi.org/10.1007/11766254_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34638-8
Online ISBN: 978-3-540-34639-5
eBook Packages: Computer ScienceComputer Science (R0)