Skip to main content
SpringerLink
Log in

Comparison of Paragram and Glove Results for Similarity Benchmarks

  • Conference paper
  • First Online:
Multimedia and Network Information Systems (MISSI 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 833))

Included in the following conference series:

  • 699 Accesses

Abstract

Distributional Semantics Models (DSM) derive word space from linguistic items in context. In this paper we provide comparison between two methods for post process improvements to the baseline DSM vectors. The counter-fitting method which enforces antonymy and synonymy constraints into the Paragram vector space representations recently showed improvement in the vectors’ capability for judging semantic similarity. The second method is our novel RESM method applied to GloVe baseline vectors. By applying the hubness reduction method, implementing relational knowledge into the model by retrofitting synonyms and providing a new ranking similarity definition RESM that gives maximum weight to the top vector component values we equal the results for the ESL and TOEFL sets in comparison with our calculations using the Paragram and Paragram + Counter-fitting methods. The Paragram or our cosine retrofitting method are state-of-the-art results for the SIMLEX-999 gold standard. Apparently relational knowledge and counter-fitting is more important for judging semantic similarity than sense determination for words.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Semeval-2017 tasks. http://alt.qcri.org/semeval2017/index.php?id=tasks

  2. Bruni, E., Boleda, G., Baroni, M., Tran, N.-K.: Distributional semantics in technicolor. In: The 50th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Jeju Island, Korea, 8–14 July 2012, Volume 1: Long Papers, pp. 136–145 (2012). http://www.aclweb.org/anthology/P12-1015

  3. Bullinaria, J.A., Levy, J.P.: Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD. Behav. Res. Methods 44(3), 890–907 (2012)

    Article  Google Scholar 

  4. Dinu, G., Baroni, M.: Improving zero-shot learning by mitigating the hubness problem. CoRR, abs/1412.6568 (2014)

    Google Scholar 

  5. Faruqui, M., Dodge, J., Jauhar, S.K., Dyer, C., Hovy, E.H., Smith, N.A.: Retrofitting word vectors to semantic lexicons. CoRR (2014)

    Google Scholar 

  6. Feldbauer, R., Flexer, A.: Centering Versus Scaling for Hubness Reduction, pp. 175–183. Springer, Cham (2016)

    Google Scholar 

  7. Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., Ruppin, E.: Placing search in context: the concept revisited. In: Proceedings of the 10th International Conference on World Wide Web, WWW 2001, pp. 406–414. ACM, New York (2001). ISBN 1-58113-348-0. https://doi.acm.org/10.1145/371920.372094

  8. Ganitkevitch, J., Van Durme, B., Callison-burch, C.: PPDB: the paraphrase database. In: HLT-NAACL 2013 (2013)

    Google Scholar 

  9. Hara, K., Suzuki, I., Shimbo, M., Kobayashi, K., Fukumizu, K., Radovanovic, M.: Localized centering: reducing hubness in large-sample data. In: AAAI, pp. 2645– 2651. AAAI Press (2015)

    Google Scholar 

  10. Hill, F., Reichart, R., Korhonen, A.: Simlex-999: evaluating semantic models with (genuine) similarity estimation. Comput. Linguist. 41(4), 665–695 (2015). https://doi.org/10.1162/COLI_a_00237

    Article  MathSciNet  Google Scholar 

  11. Jarmasz, M., Szpakowicz, S.: Roget’s thesaurus and semantic similarity. CoRR (2012)

    Google Scholar 

  12. Kiela, D., Hill, F., Clark, S.: Specializing word embeddings for similarity or relatedness. In: EMNLP, pp. 2044–2048. The Association for Computational Linguistics (2015)

    Google Scholar 

  13. Landauer, T.K., Dumais, S.T.: A solution to plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104(2), 211–240 (1997)

    Article  Google Scholar 

  14. Levy, O., Goldberg, Y., Dagan, I.: Improving distributional similarity with lessons learned from word embeddings. TACL 3, 211–225 (2015). https://tacl2013.cs.columbia.edu/ojs/index.php/tacl/article/view/570

  15. Lu, C.-H., Ong, C.-S., Hsu, W.-L., Lee, H.-K.: Using filtered second order co-occurrence matrix to improve the traditional co-occurrence model (2011)

    Google Scholar 

  16. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR (2013)

    Google Scholar 

  17. Miller, G.A., Fellbaum, C.: Wordnet then and now. Lang. Resour. Eval. 41(2), 209–214 (2007)

    Article  Google Scholar 

  18. Mrksic, N., Séaghdha, D.O., Thomson, B., Gasic, M., Rojas-Barahona, L.M., Su, P.-H., Vandyke, D., Wen, T.-H., Young, S.J.: Counter-fitting word vectors to linguistic constraints. In: HLT-NAACL, pp. 142–148. The Association for Computational Linguistics (2016)

    Google Scholar 

  19. Osterlund, A., Odling, D., Sahlgren, M.: Factorization of latent variables in distributional semantic models. In: EMNLP (2015)

    Google Scholar 

  20. Pavlick, E., Rastogi, P., Ganitkevitch, J., Van Durme, B., CallisonBurch, C.: PPDB 2.0: better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification. In: ACL (2), pp. 425–430. The Association for Computer Linguistics (2015)

    Google Scholar 

  21. Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, October 2014, pp. 1532–1543. Association for Computational Linguistics (2014)

    Google Scholar 

  22. Pilehvar, M.T, Collier, N.: De-conflated semantic representations. In: EMNLP, pp. 1680–1690. The Association for Computational Linguistics

    Google Scholar 

  23. Pilehvar, M.T., Navigli, R.: From senses to texts: an all-in-one graph-based approach for measuring semantic similarity. Artif. Intell. 228, 95–128 (2015). https://doi.org/10.1016/j.artint.2015.07.005

    Article  MathSciNet  MATH  Google Scholar 

  24. Recski, G., Iklódi, E., Pajkossy, K., Kornai, A.: Measuring semantic similarity of words using concept networks. In: Proceedings of the 1st Workshop on Representation Learning for NLP, Berlin, Germany, pp. 193–200. Association for Computational Linguistics (2016)

    Google Scholar 

  25. Roberts, K., Gururaj, A.E., Chen, X., Pournejati, S., Hersh, W.R., Demner-Fushman, D., Ohno-Machado, L., Cohen, T., Xu, H.: Information retrieval for biomedical datasets: the 2016 biocaddie dataset retrieval challenge. Database, 2017:bax068 (2017)

    Google Scholar 

  26. Rubenstein, H., Goodenough, J.B.: Contextual correlates of synonymy. Commun. ACM 8(10), 627–633 (1965). https://doi.acm.org/10.1145/365628.365657

  27. Santus, E., Chiu, T.-S., Lu, Q., Lenci, A., Huang, C.-R.: What a nerd! beating students and vector cosine in the ESL and TOEFL datasets. CoRR, abs/1603.08701 (2016)

    Google Scholar 

  28. Turney, P.D.: Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL, pp. 491–502. Springer, Heidelberg (2001)

    Google Scholar 

  29. Wieting, J., Bansal, M., Gimpel, K., Livescu, K.: From paraphrase database to compositional paraphrase model and back. TACL 3, 345–358 (2015). https://tacl2013.cs.columbia.edu/ojs/index.php/tacl/article/view/571

Download references

Author information

Authors and Affiliations

  1. Poznan University of Technology, Plac Marii Skłodowskiej-Curie 5, 60-965, Poznań, Poland

    Jakub Dutkiewicz & Czesław Jędrzejek

Authors
  1. Jakub Dutkiewicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Czesław Jędrzejek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Czesław Jędrzejek .

Editor information

Editors and Affiliations

  1. Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Kazimierz Choroś

  2. Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Marek Kopel

  3. Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Elżbieta Kukla

  4. Faculty of Computer Science and Management, Wrocław University of Science and Technology, Wrocław, Poland

    Andrzej Siemiński

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dutkiewicz, J., Jędrzejek, C. (2019). Comparison of Paragram and Glove Results for Similarity Benchmarks. In: Choroś, K., Kopel, M., Kukla, E., Siemiński, A. (eds) Multimedia and Network Information Systems. MISSI 2018. Advances in Intelligent Systems and Computing, vol 833. Springer, Cham. https://doi.org/10.1007/978-3-319-98678-4_25

Download citation

Publish with us

Policies and ethics

Search

Navigation