Skip to main content
SpringerLink
Log in

Cbow Training Time and Accuracy Optimization Using SkipGram

  • Conference paper
  • First Online:
Advances in Computational Collective Intelligence (ICCCI 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1463))

Included in the following conference series:

  • 1144 Accesses

Abstract

Most word embedding techniques get their theoretical foundation from distributional semantics theory. They have been among the most popular trends of natural language processing for the last two decades. They have a large range of application. The present paper presents an overview of recent word embedding techniques. Furthermore, it proposes an optimized continuous bag of word (Cbow) model. The experiments we conducted show that the proposed approach outperforms the classic Cbow technique in terms of accuracy and training time.

The authors would like to thank the Natural Sciences and Engineering Research Council of Canada (NSERC) as well as the Canadian Social Sciences and Humanities Research Council (SSHRC) for funding this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Berners, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 284(5), 34–43 (2001)

    Article  Google Scholar 

  2. Roman, V., Yampolskiy R.V.: Turing test as a defining feature of AI-completeness. In: Yang, X.S. (eds.) Artificial Intelligence, Evolutionary Computing and Metaheuristics. Studies in Computational Intelligence, vol. 427. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-29694-9_1

  3. Bobrow, D.: Natural Language Input for a Computer Problem Solving System, Massachusetts Institute of Technology 201 Vassar Street, W59–200 Cambridge, MA, USA (1964)

    Google Scholar 

  4. Weizenbaum, J.: Computer Power and Human Reason, pp. 188–189. From Judgment to Calculation W. H. Freeman and Company, San Francisco (1976). ISBN 0-7167-0463-3

    Google Scholar 

  5. Schank, R.: A conceptual dependency parser for natural language. In: Proceedings of the 1969 Conference on Computational Linguistics, Sång-Säby, pp. 1–3. Sweden (1969)

    Google Scholar 

  6. Aaronson, D.: Computer use in cognitive psychology. Behav. Res. Meth. Instrum. Comput. 26, 81–93 (1994)

    Article  Google Scholar 

  7. Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. (1990)

    Google Scholar 

  8. Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing [archive]. Commun. ACM 18(11), 613–620 (1975)

    Article  Google Scholar 

  9. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality (2013)

    Google Scholar 

  10. Harris, Z.: Distributional structure. Word 10(23), 146–162 (1954)

    Article  Google Scholar 

  11. Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  12. Bojanowski, P., Grave, P., Joulin, E., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)

    Article  Google Scholar 

  13. Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5 an open multilingual graph of general knowledge. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  14. Speer, R., J Duda, J.: ConceptNet extending word embeddings with multilingual relational knowledge. In: SemEval-2017 (2017)

    Google Scholar 

  15. Faruqui, M., Sujay, J., Jauhar, K., Hovy, C.E., Smith, N.A.: Retrofitting word vectors to semantic lexicons. In: Proceedings of NAACL (2015)

    Google Scholar 

  16. Harris, Z.: Distributional structure. Word 10, 146–162 (1954). https://doi.org/10.1007/978-94-009-8467-7-1

    Article  Google Scholar 

  17. Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988)

    Article  Google Scholar 

  18. McDonald, S., Ramscar, M.: Testing the distributional hypothesis: the influence of context on judgements of semantic similarity. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 23(23) (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Quebec in Montreal, Montreal, QC, Canada

    Toufik Mechouma, Jean Guy Meunier & Alaidine Ben Ayed

  2. University of Quebec in Trois Rivieres, Trois Rivieres, QC, Canada

    Ismail Biskri

Authors
  1. Toufik Mechouma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ismail Biskri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean Guy Meunier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alaidine Ben Ayed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Toufik Mechouma , Ismail Biskri , Jean Guy Meunier or Alaidine Ben Ayed .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Krystian Wojtkiewicz

  2. VU Amsterdam, Amsterdam, The Netherlands

    Jan Treur

  3. University of the West of England, Bristol, UK

    Elias Pimenidis

  4. Wrocław University of Science and Technology, Wrocław, Poland

    Marcin Maleszka

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mechouma, T., Biskri, I., Meunier, J.G., Ayed, A.B. (2021). Cbow Training Time and Accuracy Optimization Using SkipGram. In: Wojtkiewicz, K., Treur, J., Pimenidis, E., Maleszka, M. (eds) Advances in Computational Collective Intelligence. ICCCI 2021. Communications in Computer and Information Science, vol 1463. Springer, Cham. https://doi.org/10.1007/978-3-030-88113-9_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88113-9_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88112-2

  • Online ISBN: 978-3-030-88113-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation