Abstract
Any NLP system needs enough data for training and testing purposes. They can be split into two datasets: correct and incorrect (erroneous) data. Usually, it is not a problem to find and get a set of correct data because the correct texts are available from different sources, although they may also contain some mistakes. On the other hand, it is a hard task to get data containing errors like typos, mistakes and misspellings. This kind of data is usually obtained by a lengthy manual process and it requires annotation by human. One way to get the incorrect dataset faster is to generate it. However, this creates a problem how to generate incorrect texts so that they correspond to real human mistakes. In this paper, we focused on getting the incorrect dataset by help of humans. We created an automated web application (a game) that allows to collect incorrect texts and misspellings from players for texts written in the Slovak language. Based on the obtained data, we built a model of common errors that can be used to generate a large amount of authentic looking erroneous texts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Rodrigues, P., Rytting, C.A.: Typing race games as a method to create spelling error corpora. In: International Conference on Language Resources and Evaluation (LREC), Istanbul (2012)
Tachibana, R., Komachi, M.: Analysis of English spelling errors in a word-typing game. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Portorož, Slovenia (2016)
The International Arcade Museum: The Typing Of The Dead, WebMagic Ventures. https://www.arcade-museum.com/game_detail.php?game_id=10244. Accessed 27 Sept 2019
TypeRacer. https://data.typeracer.com/misc/about. Accessed 27 Sept 2019
Szablewski, D.: ZType - Typing Game - Type to Shoot. https://zty.pe/. Accessed 27 Sept 2019
Typing.com: Typing Games. https://www.typing.com/student/games. Accessed 27 Sept 2019
Grundkiewicz, R., Junczys-Dowmunt, M.: The WikEd error corpus: a corpus of corrective wikipedia edits and its application to grammatical error correction. In: Przepiórkowski, A., Ogrodniczuk, M. (eds.) NLP 2014. LNCS (LNAI), vol. 8686, pp. 478–490. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10888-9_47
Ľ. Štúr Institute of Linguistics of the Slovak Academy of Sciences: Error Corpus of Slovak CHIBY, 05 Aug 2019. https://www.juls.savba.sk/errcorp_en.html
The Wikimedia Foundation: skwiki dump. https://dumps.wikimedia.org/skwiki/latest/
Attardi, G.: WikiExtractor - Python script that extracts and cleans text from a Wikipedia database dump. https://github.com/attardi/wikiextractor
Navarro, G.: A guided tour to approximate string matching. ACM Comput. Surv. (CSUR) 33(1), 31–88 (2001)
Acknowledgment
This article was created in the framework of the National project IT Academy – Education for the 21st Century, which is supported by the European Social Fund and the European Regional Development Fund in the framework of the Operational Programme Human Resources.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hrkút, P., Toth, Š., Ďuračík, M., Meško, M., Kršák, E., Mikušová, M. (2020). Data Collection for Natural Language Processing Systems. In: Sitek, P., Pietranik, M., Krótkiewicz, M., Srinilta, C. (eds) Intelligent Information and Database Systems. ACIIDS 2020. Communications in Computer and Information Science, vol 1178. Springer, Singapore. https://doi.org/10.1007/978-981-15-3380-8_6
Download citation
DOI: https://doi.org/10.1007/978-981-15-3380-8_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3379-2
Online ISBN: 978-981-15-3380-8
eBook Packages: Computer ScienceComputer Science (R0)