Skip to main content
Springer Nature Link
Log in

Getting to Know Your Corpus

  • Conference paper
Text, Speech and Dialogue (TSD 2012)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7499))

Included in the following conference series:

Abstract

Corpora are not easy to get a handle on. The usual way of getting to grips with text is to read it, but corpora are mostly too big to read (and not designed to be read). We show, with examples, how keyword lists (of one corpus vs. another) are a direct, practical and fascinating way to explore the characteristics of corpora, and of text types. Our method is to classify the top one hundred keywords of corpus1 vs. corpus2, and corpus2 vs. corpus1. This promptly reveals a range of contrasts between all the pairs of corpora we apply it to. We also present improved maths for keywords, and briefly discuss quantitative comparisons between corpora. All the methods discussed (and almost all of the corpora) are available in the Sketch Engine, a leading corpus query tool.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Baroni, M., Bernardini, S., Ferraresi, A., Zanchetta, E.: The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation Journal 43, 209–226 (2009)

    Article  Google Scholar 

  2. Pomikàlek, J., Rychlý, P., Kilgarriff, A.: Scaling to billion-plus word corpora. Advances in Computational Linguistics. Special Issue or Research in Computer Science 41 (2009)

    Google Scholar 

  3. Biber, D.: Variation across speech and writing. Cambridge University Press (1988)

    Google Scholar 

  4. Biber, D.: Dimensions of Register Variation: a cross-linguistic study. Cambridge University Press (2006)

    Google Scholar 

  5. Heylighen, F., Dewaele, J.M.: Formality of language: definition, measurement and behavioral determinants. Technical report, Free University of Brussels (1999)

    Google Scholar 

  6. Manning, C., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press (1999)

    Google Scholar 

  7. Kilgarriff, A.: Language is never ever ever random. Corpus Linguistics and Linguistic Theory 1, 263–276 (2005)

    Article  Google Scholar 

  8. Heuboeck, A., Holmes, J., Nesi, H.: The BAWE corpus manual. Technical report, Universities of Warwick, Coventry and Reading (2007)

    Google Scholar 

  9. Partington, A.: Modern diachronic corpus-assisted discourse studies MD-CADS on UK newspapers: an overview of the project. Corpora 5, 83–108 (2010)

    Article  Google Scholar 

  10. Baker, P., Gabrielatos, C., McEnery, T.: Discourse Analysis and Media Bias: The representation of Islam in the British Press. Cambridge University Press (2012)

    Google Scholar 

  11. Biber, D.: University Language: A corpus-based study of spoken and written registers. John Benjamins (2006)

    Google Scholar 

  12. Paquot, M.: Academic Vocabulary in Learner Writing. Continuum (2010)

    Google Scholar 

  13. Kosem, I.: Designing a model for a corpus-driven dictionary of Academic English. Ph.D. thesis, Aston University, UK (2010)

    Google Scholar 

  14. Keller, F., Lapata, M.: Using the web to obtain frequencies for unseen bigrams. Computational Linguistics 29, 459–484 (2003)

    Article  Google Scholar 

  15. Sharoff, S.: Creating general-purpose corpora using automated search engine queries. In: Baroni, M., Bernardini, S. (eds.) WaCky! Working papers on the Web as Corpus, Gedit, Bologna (2006)

    Google Scholar 

  16. Leech, G.: New resources, or just better old ones? the holy grail of representativeness. In: Hundt, M., Nesselehauf, N., Biewer, C. (eds.) Corpus Linguistics and the Web, pp. 133–149. Rodopi, Amsterdam (2007)

    Google Scholar 

  17. Čermák, F., Schmiedtová, V., Křen, M.: Czech national corpus – syn. Technical report, Institute of the Czech National Corpus (Prague, Czech Republic) http://www.korpus.cz (accessed on June 08, 2012)

  18. Kilgarriff, A.: Comparing corpora. Int. Jnl. Corpus Linguistics 6, 263–276 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lexical Computing Ltd., Brighton, UK

    Adam Kilgarriff

Authors
  1. Adam Kilgarriff
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Informatics, Department of Computer Graphics and Design, Masaryk University, Botanická 68a, 602 00, Brno, Czech Republic

    Petr Sojka

  2. Faculty of Informatics, Department of Information Technologies, Masaryk University, Botanická 68a, 602 00, Brno, Czech Republic

    Aleš Horák , Ivan Kopeček  & Karel Pala ,  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kilgarriff, A. (2012). Getting to Know Your Corpus. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds) Text, Speech and Dialogue. TSD 2012. Lecture Notes in Computer Science(), vol 7499. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32790-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32790-2_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32789-6

  • Online ISBN: 978-3-642-32790-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics