Abstract
Corpora are not easy to get a handle on. The usual way of getting to grips with text is to read it, but corpora are mostly too big to read (and not designed to be read). We show, with examples, how keyword lists (of one corpus vs. another) are a direct, practical and fascinating way to explore the characteristics of corpora, and of text types. Our method is to classify the top one hundred keywords of corpus1 vs. corpus2, and corpus2 vs. corpus1. This promptly reveals a range of contrasts between all the pairs of corpora we apply it to. We also present improved maths for keywords, and briefly discuss quantitative comparisons between corpora. All the methods discussed (and almost all of the corpora) are available in the Sketch Engine, a leading corpus query tool.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baroni, M., Bernardini, S., Ferraresi, A., Zanchetta, E.: The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation Journal 43, 209–226 (2009)
Pomikàlek, J., Rychlý, P., Kilgarriff, A.: Scaling to billion-plus word corpora. Advances in Computational Linguistics. Special Issue or Research in Computer Science 41 (2009)
Biber, D.: Variation across speech and writing. Cambridge University Press (1988)
Biber, D.: Dimensions of Register Variation: a cross-linguistic study. Cambridge University Press (2006)
Heylighen, F., Dewaele, J.M.: Formality of language: definition, measurement and behavioral determinants. Technical report, Free University of Brussels (1999)
Manning, C., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press (1999)
Kilgarriff, A.: Language is never ever ever random. Corpus Linguistics and Linguistic Theory 1, 263–276 (2005)
Heuboeck, A., Holmes, J., Nesi, H.: The BAWE corpus manual. Technical report, Universities of Warwick, Coventry and Reading (2007)
Partington, A.: Modern diachronic corpus-assisted discourse studies MD-CADS on UK newspapers: an overview of the project. Corpora 5, 83–108 (2010)
Baker, P., Gabrielatos, C., McEnery, T.: Discourse Analysis and Media Bias: The representation of Islam in the British Press. Cambridge University Press (2012)
Biber, D.: University Language: A corpus-based study of spoken and written registers. John Benjamins (2006)
Paquot, M.: Academic Vocabulary in Learner Writing. Continuum (2010)
Kosem, I.: Designing a model for a corpus-driven dictionary of Academic English. Ph.D. thesis, Aston University, UK (2010)
Keller, F., Lapata, M.: Using the web to obtain frequencies for unseen bigrams. Computational Linguistics 29, 459–484 (2003)
Sharoff, S.: Creating general-purpose corpora using automated search engine queries. In: Baroni, M., Bernardini, S. (eds.) WaCky! Working papers on the Web as Corpus, Gedit, Bologna (2006)
Leech, G.: New resources, or just better old ones? the holy grail of representativeness. In: Hundt, M., Nesselehauf, N., Biewer, C. (eds.) Corpus Linguistics and the Web, pp. 133–149. Rodopi, Amsterdam (2007)
Čermák, F., Schmiedtová, V., Křen, M.: Czech national corpus – syn. Technical report, Institute of the Czech National Corpus (Prague, Czech Republic) http://www.korpus.cz (accessed on June 08, 2012)
Kilgarriff, A.: Comparing corpora. Int. Jnl. Corpus Linguistics 6, 263–276 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kilgarriff, A. (2012). Getting to Know Your Corpus. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds) Text, Speech and Dialogue. TSD 2012. Lecture Notes in Computer Science(), vol 7499. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32790-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-32790-2_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32789-6
Online ISBN: 978-3-642-32790-2
eBook Packages: Computer ScienceComputer Science (R0)