Skip to main content
SpringerLink
Log in

Is Context All You Need? Non-contextual vs Contextual Multiword Expressions Detection

  • Conference paper
  • First Online:
Computational Science – ICCS 2022 (ICCS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13350))

Included in the following conference series:

  • 1109 Accesses

Abstract

Effective methods of the detection of multiword expressions are important for many technologies related to Natural Language Processing. Most contemporary methods are based on the sequence labeling scheme, while traditional methods use statistical measures. In our approach, we want to integrate the concepts of those two approaches. In this paper, we present a novel weakly supervised multiword expressions extraction method which focuses on their behaviour in various contexts. Our method uses a lexicon of Polish multiword units as the reference knowledge base and leverages neural language modelling with deep learning architectures. In our approach, we do not need a corpus annotated specifically for the task. The only required components are: a lexicon of multiword units, a large corpus, and a general contextual embeddings model. Compared to the method based on non-contextual embeddings, we obtain gains of 15% points of the macro F1-score for both classes and 30% points of the F1-score for the incorrect multiword expressions. The proposed method can be quite easily applied to other languages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/

  2. Agrawal, S., Sanyal, R., Sanyal, S.: Hybrid method for automatic extraction of multiword expressions. Int. J. Eng. Technol. 7, 33 (2018)

    Article  Google Scholar 

  3. Anke, L.E., Schockaert, S., Wanner, L.: Collocation classification with unsupervised relation vectors. In: Proceedings of the 57th Annual Meeting of the ACL (2019)

    Google Scholar 

  4. Berk, G., Erden, B., Güngör, T.: Deep-BGT at PARSEME shared task 2018: bidirectional LSTM-CRF model for verbal multiword expression identification. In: Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pp. 248–253. ACL (2018)

    Google Scholar 

  5. Bojanowski, P., Grave, E., Joulin, A., et al.: Enriching word vectors with subword information. Trans. ACL 5, 135–146 (2017)

    Google Scholar 

  6. Boros, T., Burtica, R.: GBD-NER at PARSEME shared task 2018: Multi-word expression detection using bidirectional long-short-term memory networks and graph-based decoding. In: Proceedings of the Joint Work. on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pp. 254–260. ACL (2018)

    Google Scholar 

  7. Broda, B., Derwojedowa, M., Piasecki, M.: Recognition of structured collocations in an inflective language. Syst. Sci. 34(4), 27–36 (2008)

    MATH  Google Scholar 

  8. Buljan, M., Šnajder, J.: Combining linguistic features for the detection of Croatian multiword expressions. In: Proc. of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 194–199. ACL (2017)

    Google Scholar 

  9. Chakraborty, S., Cougias, D., Piliero, S.: Identification of multiword expressions using transformers (2020)

    Google Scholar 

  10. Chawla, N.V., Bowyer, K.W., Hall, L.O., et al.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  Google Scholar 

  11. Dziob, A., Piasecki, M., Rudnicka, E.K.: plwordnet 4.1 - a linguistically motivated, corpus-based bilingual resource. In: Proceedings of the Tenth Global Wordnet Conference: 23–27 July 2019, Wrocław (Poland), pp. 353–362 (2019)

    Google Scholar 

  12. Evert, S.: The Statistics of Word Cooccurrences: Word Pairs and Collocations. Ph.D. thesis, Institut für maschinelle Sprachverarbeitung, Univ. of Stuttgart (2004)

    Google Scholar 

  13. Fu, R., Guo, J., et al.: Learning semantic hierarchies via word embeddings. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 1199–1209. Baltimore, Maryland (2014)

    Google Scholar 

  14. Green, S., de Marneffe, M.C., Manning, C.D.: Parsing models for identifying multiword expressions. Comput. Linguist. 39(1), 195–227 (2013)

    Article  Google Scholar 

  15. Han, H., Wang, W.Y., Mao, B.H.: Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International Conference on Intelligent Computing, pp. 878–887 (2005)

    Google Scholar 

  16. Handler, A., Denny, M., Wallach, H., et al.: Bag of what? simple noun phrase extraction for text analysis. In: NLP+CSS@EMNLP (2016)

    Google Scholar 

  17. He, H., Bai, Y., Garcia, E.A., Li, S.: Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE International Joint Conference on Neural Networks, pp. 1322–1328. IEEE (2008)

    Google Scholar 

  18. Hosseini, M.J., Smith, N.A., Lee, S.I.: UW-CSE at SemEval-2016 task 10: detecting multiword expressions and supersenses using double-chained conditional random fields. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 931–936. ACL (2016)

    Google Scholar 

  19. Klyueva, N., Doucet, A., Straka, M.: Neural networks for multi-word expression detection. In: Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 60–65. ACL (2017)

    Google Scholar 

  20. Kocoń, J., Gawor, M.: Evaluating kgr10 polish word embeddings in the recognition of temporal expressions using bilstm-crf. Schedae Informaticae 27 (2018)

    Google Scholar 

  21. Kurfalı, M.: TRAVIS at PARSEME shared task 2020: how good is (m)BERT at seeing the unseen? In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pp. 136–141. ACL (2020)

    Google Scholar 

  22. Levy, O., Remus, S., et al.: Do supervised distributional methods really learn lexical inference relations? In: Proceedings of the 2015 Conference of the North American Chapter of ACL: Human Language Technologies, pp. 970–976 (2015)

    Google Scholar 

  23. Maziarz, M., Szpakowicz, S., Piasecki, M.: A procedural definition of multi-word lexical units. In: Mitkov, R., Angelova, G., Boncheva, K. (eds.) Proceedings of the International Conference Recent Advances in Natural Language Processing - RANLP’2015, pp. 427–435. INCOMA Ltd. (2015)

    Google Scholar 

  24. Mikolov, T., Chen, K., Corrado, G., et al.: Efficient estimation of word representations in vector space. In: Bengio, Y., LeCun, Y. (eds.) 1st International Conference on Learning Representations, ICLR 2013, pp. 2–4. Arizona, USA, May, Scottsdale (2013)

    Google Scholar 

  25. Mroczkowski, R., Rybak, P., Wróblewska, A., et al.: HerBERT: efficiently pretrained transformer-based language model for Polish. In: Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing, pp. 1–10. ACL (2021)

    Google Scholar 

  26. Pedregosa, F., Varoquaux, G., Gramfort, A., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  27. Pečina, P.: Lexical association measures and collocation extraction. Lang. Res. Eval. 44, 137–158 (2010)

    Article  Google Scholar 

  28. Piasecki, M., Wendelberger, M., Maziarz, M.: Extraction of the multi-word lexical units in the perspective of the wordnet expansion. In: Proceedings of the International Conference Recent Advances in Natural Language Processing (2015)

    Google Scholar 

  29. Ramisch, C.: Multiword Expressions Acquisition. TANLP, Springer, Cham (2015). https://doi.org/10.1007/978-3-319-09207-2

    Book  Google Scholar 

  30. Ramisch, C., et al.: Edition 1.1 of the PARSEME shared task on automatic identification of verbal multiword expressions. In: Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (2018)

    Google Scholar 

  31. Ramisch, C., Savary, A., Guillaume, B., et al.: Edition 1.2 of the PARSEME shared task on semi-supervised identification of verbal multiword expressions. In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons (2020)

    Google Scholar 

  32. Ramshaw, L., Marcus, M.: Text chunking using transformation-based learning. In: Third Workshop on Very Large Corpora (1995)

    Google Scholar 

  33. Rohanian, O., Taslimipoor, S., Kouchaki, S., et al.: Bridging the gap: attending to discontinuity in identification of multiword expressions. In: Proceedings of the 2019 Conference of the North American Chapter of the ACL: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2692–2698. ACL (2019)

    Google Scholar 

  34. Saied, H.A., Candito, M., Constant, M.: Comparing linear and neural models for competitive MWE identification. In: Proceedings of the 22nd Nordic Conference on Computational Linguistics, pp. 86–96. Linköping University Electronic Press (2019)

    Google Scholar 

  35. Scholivet, M., Ramisch, C.: Identification of ambiguous multiword expressions using sequence models and lexical resources. In: Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 167–175 (2017)

    Google Scholar 

  36. Seretan, V.: Syntax-Based Collocation Extraction, Text, Speech and Language Technology, vol. 44. Springer, Netherlands (2011). https://doi.org/10.1007/978-94-007-0134-2

    Book  MATH  Google Scholar 

  37. Spasić, I., Owen, D., Knight, D., et al.: Unsupervised multi-word term recognition in Welsh. In: Proceedings of the Celtic Language Technology Workshop, pp. 1–6 (2019)

    Google Scholar 

  38. Taslimipoor, S., Bahaadini, S., Kochmar, E.: MTLB-STRUCT @Parseme 2020: capturing unseen multiword expressions using multi-task learning and pre-trained masked language models. In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pp. 142–148. ACL (2020)

    Google Scholar 

  39. Yirmibeşoğlu, Z., Güngör, T.: ERMI at PARSEME shared task 2020: embedding-rich multiword expression identification. In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pp. 130–135. ACL (2020)

    Google Scholar 

Download references

Acknowledgements

This work was financed by the National Science Centre, Poland, project no. 2019/33/B/HS2/02814.

Author information

Authors and Affiliations

  1. Department of Artificial Intelligence, Wrocław University of Science and Technology, Wrocław, Poland

    Maciej Piasecki & Kamil Kanclerz

Authors
  1. Maciej Piasecki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamil Kanclerz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maciej Piasecki .

Editor information

Editors and Affiliations

  1. Brunel University London, London, UK

    Derek Groen

  2. University of Amsterdam, Amsterdam, The Netherlands

    Clélia de Mulatier

  3. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  4. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  5. University of Tennessee at Knoxville, Knoxville, TN, USA

    Jack J. Dongarra

  6. University of Amsterdam, Amsterdam, The Netherlands

    Peter M. A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Piasecki, M., Kanclerz, K. (2022). Is Context All You Need? Non-contextual vs Contextual Multiword Expressions Detection. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13350. Springer, Cham. https://doi.org/10.1007/978-3-031-08751-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-08751-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-08750-9

  • Online ISBN: 978-3-031-08751-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation