Skip to main content
SpringerLink
Log in

Exploring Accurate and Generic Simile Knowledge from Pre-trained Language Models

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14232))

Included in the following conference series:

  • 260 Accesses

Abstract

A simile is an important linguistic phenomenon in daily communication and an important task in natural language processing (NLP). In recent years, pre-trained language models (PLMs) have achieved great success in NLP since they learn generic knowledge from a large corpus. However, PLMs still have hallucination problems that they could generate unrealistic or context-unrelated information. In this paper, we aim to explore more accurate simile knowledge from PLMs. To this end, we first fine-tune a single model to perform three main simile tasks (recognition, interpretation, and generation). In this way, the model gains a better understanding of the simile knowledge. However, this understanding may be limited by the distribution of the training data. To explore more generic simile knowledge from PLMs, we further add semantic dependency features in three tasks. The semantic dependency feature serves as a global signal and helps the model learn simile knowledge that can be applied to unseen domains. We test with seen and unseen domains after training. Automatic evaluations demonstrate that our method helps the PLMs to explore more accurate and generic simile knowledge for downstream tasks. Our method of exploring more accurate knowledge is not only useful for simile study but also useful for other NLP tasks leveraging knowledge from PLMs. Our code and data will be released on GitHub.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/realZsh/simile-tasks.

  2. 2.

    https://stanfordnlp.github.io/CoreNLP.

References

  1. Birke, J., Sarkar, A.: A clustering approach for nearly unsupervised recognition of nonliteral language. In: McCarthy, D., Wintner, S. (eds.) EACL 2006, 11st Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference, 3–7 April 2006, Trento, Italy. The Association for Computer Linguistics (2006). https://aclanthology.org/E06-1042/

  2. Bizzoni, Y., Lappin, S.: Predicting human metaphor paraphrase judgments with deep neural networks. In: Klebanov, B.B., Shutova, E., Lichtenstein, P., Muresan, S., Leong, C.W. (eds.) Proceedings of the Workshop on Figurative Language Processing, Fig-Lang@NAACL-HLT 2018, New Orleans, Louisiana, 6 June 2018, pp. 45–55. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/W18-0906

  3. Chakrabarty, T., Muresan, S., Peng, N.: Generating similes effortlessly like a pro: a style transfer approach for simile generation. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, 16–20 November 2020, pp. 6455–6469. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.emnlp-main.524

  4. Chen, W., et al.: Probing simile knowledge from pre-trained language models. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), ACL 2022, Dublin, Ireland, 22–27 May 2022, pp. 5875–5887. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.acl-long.404

  5. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019)

    Google Scholar 

  6. Dziri, N., Kamalloo, E., Mathewson, K.W., Zaïane, O.R.: Augmenting neural response generation with context-aware topical attention. CoRR abs/1811.01063 (2018). http://arxiv.org/abs/1811.01063

  7. Dziri, N., Milton, S., Yu, M., Zaïane, O.R., Reddy, S.: On the origin of hallucinations in conversational models: is it the datasets or the models? In: Carpuat, M., de Marneffe, M., Ruíz, I.V.M. (eds.) Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, 10–15 July 2022, pp. 5271–5285. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.naacl-main.387

  8. He, Q., Cheng, S., Li, Z., Xie, R., Xiao, Y.: Can pre-trained language models interpret similes as smart as human? In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), ACL 2022, Dublin, Ireland, 22–27 May 2022, pp. 7875–7887. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.acl-long.543

  9. Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J.R. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, 5–10 July 2020, pp. 7871–7880. Association for Computational Linguistics (2020). https://www.aclweb.org/anthology/2020.acl-main.703/

  10. Li, Y., Lin, C., Guerin, F.: CM-GEN: a neural framework for Chinese metaphor generation with explicit context modelling. In: Calzolari, N., et al. (eds.) Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, 12–17 October 2022, pp. 6468–6479. International Committee on Computational Linguistics (2022). https://aclanthology.org/2022.coling-1.563

  11. Liu, L., Hu, X., Song, W., Fu, R., Liu, T., Hu, G.: Neural multitask learning for simile recognition. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018, pp. 1543–1553. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/d18-1183

  12. Liu, T., et al.: A token-level reference-free hallucination detection benchmark for free-form text generation. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), ACL 2022, Dublin, Ireland, 22–27 May 2022, pp. 6723–6737. Association for Computational Linguistics (2022). https://doi.org/10.18653/v1/2022.acl-long.464

  13. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, 22–27 June 2014, Baltimore, MD, USA, System Demonstrations, pp. 55–60. The Association for Computer Linguistics (2014). https://doi.org/10.3115/v1/p14-5010

  14. Mohler, M., Brunson, M., Rink, B., Tomlinson, M.T.: Introducing the LCC metaphor datasets. In: Calzolari, N., et al. (eds.) Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorož, Slovenia, 23–28 May 2016. European Language Resources Association (ELRA) (2016). http://www.lrec-conf.org/proceedings/lrec2016/summaries/1156.html

  15. Niculae, V., Danescu-Niculescu-Mizil, C.: Brighter than gold: figurative language in user generated comparisons. In: Moschitti, A., Pang, B., Daelemans, W. (eds.) Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, 25–29 October 2014, Doha, Qatar, A Meeting of SIGDAT, a Special Interest Group of the ACL, pp. 2008–2018. ACL (2014). https://doi.org/10.3115/v1/d14-1215

  16. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019 (December), vol. 32, pp. 8–14, 2019, Vancouver, BC, Canada, pp. 8024–8035 (2019). https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html

  17. Paul, A.M.: Figurative language. In: Philosophy and Rhetoric, pp. 225–248 (1970)

    Google Scholar 

  18. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)

    Google Scholar 

  19. Shuster, K., Poff, S., Chen, M., Kiela, D., Weston, J.: Retrieval augmentation reduces hallucination in conversation. In: Moens, M., Huang, X., Specia, L., Yih, S.W. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event/Punta Cana, Dominican Republic, 16–20 November 2021, pp. 3784–3803. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.findings-emnlp.320

  20. Song, W., Guo, J., Fu, R., Liu, T., Liu, L.: A knowledge graph embedding approach for metaphor processing. IEEE ACM Trans. Audio Speech Lang. Process. 29, 406–420 (2021). https://doi.org/10.1109/TASLP.2020.3040507

  21. Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: an open multilingual graph of general knowledge. In: Singh, S., Markovitch, S. (eds.) Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 4–9 February 2017, San Francisco, California, USA, pp. 4444–4451. AAAI Press (2017). http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14972

  22. Steen, G.: A Method for Linguistic Metaphor Identification: From MIP to MIPVU, vol. 14. John Benjamins Publishing, Amsterdam (2010)

    Google Scholar 

  23. Stowe, K., Beck, N., Gurevych, I.: Exploring metaphoric paraphrase generation. In: Bisazza, A., Abend, O. (eds.) Proceedings of the 25th Conference on Computational Natural Language Learning, CoNLL 2021, Online, 10–11 November 2021, pp. 323–336. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.conll-1.26

  24. Su, C., Tian, J., Chen, Y.: Latent semantic similarity based interpretation of Chinese metaphors. Eng. Appl. Artif. Intell. 48, 188–203 (2016). https://doi.org/10.1016/j.engappai.2015.10.014

  25. Tsvetkov, Y., Boytsov, L., Gershman, A., Nyberg, E., Dyer, C.: Metaphor detection with cross-lingual model transfer. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, 22–27 June 2014, Baltimore, MD, USA, vol. 1: Long Papers, pp. 248–258. The Association for Computer Linguistics (2014). https://doi.org/10.3115/v1/p14-1024

  26. Zhang, J., et al.: Writing polishment with simile: Task, dataset and A neural approach. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, 2–9 February 2021, pp. 14383–14392. AAAI Press (2021). https://ojs.aaai.org/index.php/AAAI/article/view/17691

Download references

Acknowledgements

This research project is supported by the National Natural Science Foundation of China (61872402), Science Foundation of Beijing Language and Culture University (supported by “the Fundamental Research Funds for the Central Universities”) (18ZDJ03).

Author information

Authors and Affiliations

  1. School of Information Science, Beijing Language and Culture University, Beijing, China

    Shuhan Zhou & Yanqiu Shao

  2. Research Center for Social Computing and Information Retrieval, Faculty of Computing, Harbin Institute of Technology, Harbin, China

    Longxuan Ma

Authors
  1. Shuhan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Longxuan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanqiu Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanqiu Shao .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

    Maosong Sun

  2. Harbin Institute of Technology, Harbin, China

    Bing Qin

  3. Fudan University, Shanghai, China

    Xipeng Qiu

  4. School of Computing and Information, Singapore Management University, Singapore, Singapore

    Jiang Jing

  5. Institute of Software, Chinese Academy of Sciences, Beijing, China

    Xianpei Han

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

  7. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Yubo Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, S., Ma, L., Shao, Y. (2023). Exploring Accurate and Generic Simile Knowledge from Pre-trained Language Models. In: Sun, M., et al. Chinese Computational Linguistics. CCL 2023. Lecture Notes in Computer Science(), vol 14232. Springer, Singapore. https://doi.org/10.1007/978-981-99-6207-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-6207-5_22

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-6206-8

  • Online ISBN: 978-981-99-6207-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation