Skip to main content
SpringerLink
Log in

Multi-Granularity Neural Sentence Model for Measuring Short Text Similarity

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10177))

Included in the following conference series:

Abstract

Measuring the semantic similarities between short texts is a critical and fundamental task because it is the basis for many applications. Although existing methods have explored this problem through enriching the short text representations based on the pre-trained word embeddings, the performance is still far from satisfaction because of the limited feature information. In this paper, we present an effective approach that combines convolutional neural network and long short-term memory to exploit from character-level to sentence-level features for performing the semantic matching of short texts. The proposed approach nicely models the feature information of sentences with the multiple representations and captures the rich matching patterns at different levels. Our model is rather generic and can hence be applied to matching tasks in different language. We use both paraphrase identification and semantic similarity tasks for evaluating our approach. The experimental results demonstrate that the proposed multiple-granularity neural sentence model obtains a significant improvement on measuring short texts similarity compared with the existing benchmark approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aliguliyev, R.M.: A new sentence similarity measure and sentence based extractive technique for automatic text summarization. Expert Syst. Appl. 36(4), 7764–7772 (2009)

    Article  Google Scholar 

  2. Asli, E., Keller, B.: ASOBEK: twitter paraphrase identification with simple overlap features and SVMs. In: Proceedings of SemEval, pp. 64–69. ACL, Denver (2015)

    Google Scholar 

  3. Bertero, D., Fung, P.: HLTC-HKUST: a neural network paraphrase classifier using translation metrics, semantic roles and lexical similarity features. In: Proceedings of SemEval, pp. 23–28. ACL (2015)

    Google Scholar 

  4. Bicici, E.: RTM-DCU: predicting semantic similarity with referential translation machines. In: Proceedings of SemEval, pp. 56–63. ACL (2015)

    Google Scholar 

  5. Byeon, W., Breuel, T.M., Raue, F., Liwicki, M.: Scene labeling with LSTM recurrent neural networks. In: 2015 IEEE Conference on CVPR, pp. 3547–3555 (2015)

    Google Scholar 

  6. Das, D., Smith, N.A.: Paraphrase identification as probabilistic quasi-synchronous recognition. In: Proceedings of ACL and IJCNLP, pp. 468–476. ACL, Suntec, Singapore (2009)

    Google Scholar 

  7. Eyecioglu, A., Keller, B.: Twitter paraphrase identification with simple overlap features and SVMs. In: Proceedings of SemEval, pp. 64–69. ACL (2015)

    Google Scholar 

  8. Guo, W., Diab, M.: Modeling sentences in the latent space. In: Proceedings of ACL, pp. 864–872. ACL, Jeju Island (2012)

    Google Scholar 

  9. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  10. Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Proceedings of NIPS, pp. 2042–2050. Curran Associates Inc. (2014)

    Google Scholar 

  11. Ji, Y., Eisenstein, J.: Discriminative improvements to distributional sentence similarity. In: Proceedings of EMNLP, pp. 891–896. ACL, Seattle (2013)

    Google Scholar 

  12. Huang, J., Ji, D., Yao, S., Huang, W.: Character-aware convolutional neural networks for paraphrase identification. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9948, pp. 177–184. Springer, Cham (2016). doi:10.1007/978-3-319-46672-9_21

    Chapter  Google Scholar 

  13. Huang, J., Ji, D., Yao, S., Huang, W., Chen, B.: Learning phrase representations based on word and character embeddings. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9950, pp. 547–554. Springer, Cham (2016). doi:10.1007/978-3-319-46681-1_65

    Chapter  Google Scholar 

  14. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of ACL, pp. 655–665. ACL, Baltimore (2014)

    Google Scholar 

  15. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of EMNLP, pp. 1746–1751. ACL, Doha (2014)

    Google Scholar 

  16. Li, J., Luong, T., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. In: Proceedings of ACL, pp. 1106–1115. ACL, Beijing (2015)

    Google Scholar 

  17. Lyu, C., Lu, Y., Ji, D., Chen, B.: Deep learning for textual entailment recognition. In: Tools with Artificial Intelligence (ICTAI), pp. 154–161. IEEE (2015)

    Google Scholar 

  18. Mihalcea, R., Corley, C.D., Strapparava, C.: Corpus-based and knowledge-based measures of text semantic similarity. In: Proceedings of AAAI, pp. 775–780 (2006)

    Google Scholar 

  19. Milajevs, D., Kartsaklis, D., Sadrzadeh, M., Purver, M.: Evaluating neural word representations in tensor-based compositional settings. In: Proceedings of EMNLP, pp. 708–719. ACL, Doha (2014)

    Google Scholar 

  20. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of EMNLP, pp. 1532–1543. ACL, Doha (2014)

    Google Scholar 

  21. Ronan, C., Jason, W.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of ICML, pp. 160–167 (2008)

    Google Scholar 

  22. Severyn, A., Moschitti, A.: Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of SIGIR, pp. 373–382 (2015)

    Google Scholar 

  23. Shengxian, W., Yanyan, L., Jiafeng, G., Jun, X., Liang, P., Xueqi, C.: A deep architecture for semantic matching with multiple positional sentence representations. In: Proceedings of AAAI, Phoenix, Arizona, pp. 2835–2841 (2016)

    Google Scholar 

  24. Socher, R., Huang, E.H., Pennin, J., Manning, C.D., Ng, A.Y.: Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Proceedings of NIPS, pp. 801–809. Curran Associates Inc. (2011)

    Google Scholar 

  25. Stefanescu, D., Rus, V., Niraula, N., Banjade, R.: Combining knowledge and corpus-based measures for word-to-word similarity. In: Proceedings of the Twenty-Seventh International Florida Artificial Intelligence Research Society Conference, pp. 87–90 (2014)

    Google Scholar 

  26. Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of ACL. pp. 1556–1566. ACL, Beijing (2015)

    Google Scholar 

  27. Wei, X., Alan, R., Chris, C.B., William, B., Yangfeng, J.: Extracting lexically divergent paraphrases from twitter. TACL 2(1), 435–448 (2014)

    Google Scholar 

  28. Xiaodan, Z., Parinaz, S., Honhyu, G.: Long short-term memory over recursive structures. In: Proceedings of ICML, Beijing, China, pp. 1604–1612 (2015)

    Google Scholar 

  29. Xu, W., Callison-Burch, C., Dolan, B.: SemEval-2015 task 1: paraphrase and semantic similarity in twitter (PIT). In: Proceedings of SemEval, pp. 1–11. ACL (2015)

    Google Scholar 

  30. Yin, W., Schütze, H.: Convolutional neural network for paraphrase identification. In: Proceedings of NAACL, pp. 901–911. ACL (2015)

    Google Scholar 

  31. Yoon, K., Yacine, J., David, S., Alexander, M.: Character-aware neural language models. In: Proceedings of AAAI, pp. 2741–2749. AAAI (2016)

    Google Scholar 

  32. Zarrella, G., Henderson, J., Merkhofer, E.M., Strickhart, L.: MITRE: seven systems for semantic similarity in tweets. In: Proceedings of SemEval, pp. 12–17. ACL (2015)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under grant 61133012 and 61373108, and supported by Humanities and Social Science Foundation of Ministry of Education of China (16YJCZH004).

Author information

Authors and Affiliations

  1. Computer School, Wuhan University, Wuhan, 430072, China

    Jiangping Huang, Chen Lyu & Donghong Ji

  2. Language Technologies Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, 15213, USA

    Shuxin Yao

Authors
  1. Jiangping Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuxin Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chen Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Donghong Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Donghong Ji .

Editor information

Editors and Affiliations

  1. Arizona State University , Tempe - Phoenix, Arizona, USA

    Selçuk Candan

  2. Hong Kong University of Science and Tech , Hong Kong, China

    Lei Chen

  3. Aalborg University , Aalborg, Denmark

    Torben Bach Pedersen

  4. University of New South Wales , Sydney, New South Wales, Australia

    Lijun Chang

  5. The University of Queensland , Brisbane, Queensland, Australia

    Wen Hua

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Huang, J., Yao, S., Lyu, C., Ji, D. (2017). Multi-Granularity Neural Sentence Model for Measuring Short Text Similarity. In: Candan, S., Chen, L., Pedersen, T., Chang, L., Hua, W. (eds) Database Systems for Advanced Applications. DASFAA 2017. Lecture Notes in Computer Science(), vol 10177. Springer, Cham. https://doi.org/10.1007/978-3-319-55753-3_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55753-3_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55752-6

  • Online ISBN: 978-3-319-55753-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation