Abstract
While they follow similar procedures, evaluations of state of the art error correction systems always rely on different resources (collections of documents, evaluation metrics, dictionaries, ...). In this context, error correction approaches cannot be directly compared without being re-implemented from scratch every time they have to be compared with a new one. In other domains such as Information Retrieval this problem is solved through Cranfield like experiments such as TRECĀ [5] evaluation campaign. We propose a generic solution to overcome those evaluation difficulties through a modular evaluation platform which formalizes similarities between evaluation procedures and provides standard sets of instantiated resources for particular domains. While this was our main problem at first, in this article, the set of resources is dedicated to the evaluation of error correction systems. The idea is to provide the leanest way to evaluate error correction systems by implementing only the core algorithm and relying on the platform for everything else.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Atkinson, K.: Aspell Spellchecker. http://aspell.net (2012). Accessed 15 Jan 2012
Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)
Hirst, G., Budanitsky, A.: Correcting real-word spelling errors by restoring lexical cohesion. Nat. Lang. Eng. 11(1), 87ā111 (2005)
Hirst, G., St-Onge, D.: Lexical chains as representations of context for the detection and correction of malapropisms, Chapter 13. In: Fellbaum, C. (ed.) WordNet: An Electronic Lexical Database, vol. 305, pp. 305ā332. MIT Press, Cambridge (1998)
Kantor, P.B., Voorhees, E.M.: The TREC-5 confusion track: comparing retrieval methods for scanned text. Inf. Retrieval 2(2), 165ā176 (2000)
Kukich, K.: Techniques for automatically correcting words in text. ACM Comput. Surv. (CSUR) 24(4), 439 (1992)
Mays, E., Damerau, F.J., Mercer, R.L.: Context based spelling correction. Inf. Process. Manag. 27(5), 517ā522 (1991)
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39ā41 (1995)
Mitton, R.: Ordering the suggestions of a spellchecker without using context. Nat. Lang. Eng. 15(02), 173ā192 (2008)
Mudge, R.: After the Deadline. http://static.afterthedeadline.com (2012). Accessed 15 Jan 2012
OSGi-Alliance. Open Services Gateway initiative. http://www.osgi.org (2012). Accessed 15 Jan 2012
Pedler, J.: Computer correction of real-word spelling errors in dyslexic text. Ph.D. thesis, Birkbeck, London University (2007)
Rosnay, J., Revelli, C.: Pronetarian Revolution (2006)
Ruch, P.: Using contextual spelling correction to improve retrieval effectiveness in degraded text collections. In: Proceedings of the 19th International Conference on Computational Linguistics, vol. 1, p. 7. Association for Computational Linguistics (2002)
Shannon, C.: A mathematical theory of communication. Bell Sys. Tech. J. 27(379ā423), pp. 623ā656 (1948)
Subramaniam, L.V., Roy, S., Faruquie, T.A., Negi, S.: A Survey of Types of Text Noise and Techniques to Handle Noisy Text. Language, pp. 115ā122 (2009)
Varnhagen, C.K., McFall, G.P., Figueredo, L., Takach, B.S., Daniels, J., Cuthbertson, H.: Spelling and the web. J. App.l. Develop. Psychol. 30(4), 454ā462 (2009)
Voorhees, E.M., Garofolo, J.: The TREC-6 spoken document retrieval track. Bull. Am. Soc. Inf. Sci. Technol. 26(5), 18ā19 (2000)
Wikipedia Community. Wikipedia List of Common Misspellings. http://en.wikipedia.org/wiki/Wikipedia:Lists_of_common_misspellings (2012). Accessed 15 Jan 2012
Wiktionary Community. Wiktionary Online Collaborative Dictionary. http://en.wiktionary.org/wiki/Wiktionary:Main_Page (2012). Accessed 15 Jan 2012
Wilcox-OāHearn, A., Hirst, G., Budanitsky, A.: Real-word spelling correction with trigrams: a reconsideration of the Mays, Damerau, and Mercer model. In: Gelbukh, A. (ed.) CICLing 2008. LNCS, vol. 4919, pp. 605ā616. Springer, Heidelberg (2008)
Wong, W., Liu, W., Bennamoun, M.: Integrated scoring for spelling error correction, abbreviation expansion and case restoration in dirty text. In: 5th Australasian conference on Data mining and analystics (AusDMā06), Sydney, Australia, pp. 83ā89. Australian Computer Society (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Renard, A., Calabretto, S., Rumpler, B. (2013). Towards a Leaner Evaluation Process: Application to Error Correction Systems. In: Cordeiro, J., Maciaszek, L.A., Filipe, J. (eds) Enterprise Information Systems. Lecture Notes in Business Information Processing, vol 141. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40654-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-40654-6_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40653-9
Online ISBN: 978-3-642-40654-6
eBook Packages: Computer ScienceComputer Science (R0)