Abstract
F-measure is an indicator which has been commonly used for 25 years to evaluate classification algorithms in textmining, based on precision and recall. For classification and information retrieval, some prefer to use the break even point. Nevertheless, these measures have some inconvenient: they use a binary logic and don’t allow to apply a user (judge) assessment. This paper proposes a new approach for evaluation. First, we distinguish classification and categorization from a semantic point of view. Then, we introduce a new measure: the K-measure, which is an overall of F-measure, and allows to apply user requirements. Finally, we propose a methodology for evaluation.
This work is partially financed by MENRT for the RNTS Rhea project.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Budanitsky, A., Hirst, G.: Semantic distance inWordNet: An experimental, application-oriented evaluation of five measures. Department of Computer Science Univ. of Toronto
Saracevic, T.: Introduction to Information Science. In: The concept of relevance in information science: A historical review, ch. 3, pp. 111–151. R.R. Bowker, New York (1970)
Sparck Jones, K.: Automatic language and information processing: Rethinking evaluation. Natural Language Engineering 7(1), 29–46 (2001)
Van Rijsbergen, K.: Information Retrieval, 2nd edn. Butterworths, London, http://www.dcs.gla.ac.uk/Keith/Preface.html
Voorhees, E.M.: Evaluating the Evaluation: Edmonton. In: Proceedings of HLT-NAACL 2003, May-June 2003, pp. 181–188 (2003) (Main Papers)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nakache, D., Metais, E., Timsit, J.F. (2005). Evaluation and NLP. In: Andersen, K.V., Debenham, J., Wagner, R. (eds) Database and Expert Systems Applications. DEXA 2005. Lecture Notes in Computer Science, vol 3588. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11546924_61
Download citation
DOI: https://doi.org/10.1007/11546924_61
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28566-3
Online ISBN: 978-3-540-31729-6
eBook Packages: Computer ScienceComputer Science (R0)