Abstract
Semantic Textual Similarity (STS) of low-resource language is a challenging research problem with practical applications. Traditional solutions employ machine translation techniques to translate the low-resource languages to some resource-rich languages such as English. Hence, the final performance is highly dependent on the quality of machine translation. To decouple the machine translation dependency while still take advantage of the data in resource-rich languages, this work proposes to jointly learn the low-resource language STS task and that of a resource-rich one, which only relies on multilingual word embeddings. In particular, we project the low-resource language word embeddings into the semantic space of the resource-rich language via a translation matrix. To make the projected word embeddings resemble that of the resource-rich language, a language discriminator is introduced as an adversarial teacher. Thus the parameters of sentence similarity neural networks of two tasks can be effectively shared. The plausibility of our model is demonstrated by extensive experimental results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In this paper, MTL and joint learning are interchangeable.
- 2.
The data is available at http://alt.qcri.org/semeval2017/task1/index.php?id=data-and-tools.
- 3.
- 4.
References
Artetxe, M., Labaka, G., Agirre, E.: Learning bilingual word embeddings with (almost) no bilingual data. In: Proceedings of ACL, pp. 451–462, July 2017
Béchara, H., Escartín, C.P., Orasan, C., Specia, L.: Semantic textual similarity in quality estimation. Baltic J. Mod. Comput. 4(2), 256 (2016)
Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L.: Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In: Proceedings of SemEval, pp. 1–14 (2017)
Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of ICML, pp. 160–167 (2008)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Proceedings of ICML, pp. 1180–1189 (2015)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Proceeding of NIPS, pp. 2672–2680 (2014)
He, H., Gimpel, K., Lin, J.J.: Multi-perspective sentence similarity modeling with convolutional neural networks. In: Proceedings of EMNLP, pp. 1576–1586 (2015)
Hermann, K.M., Blunsom, P.: Multilingual distributed representations without word alignment. In: Proceedings of ICLR (2014)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)
Lan, M., Wang, J., Wu, Y., Niu, Z.Y., Wang, H.: Multi-task attention-based neural networks for implicit discourse relationship representation and identification. In: Proceedings of EMNLP, pp. 1310–1319 (2017)
Lan, M., Wu, G., Xiao, C., Wu, Y., Wu, J.: Building mutually beneficial relationships between question retrieval and answer ranking to improve performance of community question answering. In: Proceedings of IJCNN (2016)
Li, Y., McLean, D., Bandar, Z.A., O’shea, J.D., Crockett, K.: Sentence similarity based on semantic nets and corpus statistics. IEEE Trans. Knowl. Data Eng. 18(8), 1138–1150 (2006)
Liu, P., Qiu, X., Chen, J., Huang, X.: Deep fusion lstms for text semantic matching. In: Proceedings of ACL (2016)
Liu, P., Qiu, X., Huang, X.: Deep multi-task learning with shared memory for text classification. In: Proceedings of EMNLP, pp. 118–127 (2016)
Liu, P., Qiu, X., Huang, X.: Adversarial multi-task learning for text classification. In: Proceeding of ACL, pp. 1–10 (2017)
Liu, Y., Li, S., Zhang, X., Sui, Z.: Implicit discourse relation classification via multi-task neural networks. arXiv preprint arXiv:1603.02776 (2016)
Lo, C.k., Wu, D.: Meant: an inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility via semantic frames. In: Proceedings of ACL, pp. 220–229 (2011)
Luong, M.T., Le, Q.V., Sutskever, I., Vinyals, O., Kaiser, L.: Multi-task sequence to sequence learning. In: Proceedings of ICLR (2016)
Mihalcea, R., Corley, C., Strapparava, C., et al.: Corpus-based and knowledge-based measures of text semantic similarity. In: Proceedings of AAAI (2006)
Mihalcea, R., Tarau, P.: Textrank: bringing order into texts. In: Proceedings of ACL (2004)
Mikolov, T., Le, Q.V., Sutskever, I.: Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168 (2013)
Mueller, J., Thyagarajan, A.: Siamese recurrent architectures for learning sentence similarity. In: Proceedings of AAAI, pp. 2786–2792 (2016)
Nagwani, N.K., Verma, S.: A frequent term and semantic similarity based single document text summarization algorithm (0975–8887). Int. J. Comput. Appl. 17, 36–40 (2011)
Park, G., Im, W.: Image-text multi-modal representation learning by adversarial backpropagation. arXiv preprint arXiv:1612.08354 (2016)
Smith, S.L., Turban, D.H.P., Hamblin, S., Hammerla, N.Y.: Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In: Proceedings of ICLR (2017)
Sultan, M.A., Bethard, S., Sumner, T.: Dls\(@\)cu: Sentence similarity from word alignment and semantic vector composition. In: Proceedings of SemEval (2015)
Tai, K.S., Socher, R., Manning, C.D.: Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of ACL (2015)
Šarić, F., Glavaš, G., Karan, M., Šnajder, J., Dalbelo Bašić, B.: Takelab: systems for measuring semantic text similarity. In: Proceedings of SemEval (2012)
Wang, B., Liu, K., Zhao, J.: Inner attention based recurrent neural networks for answer selection. In: Proceedings of ACL (2016)
Wieting, J., Gimpel, K.: Revisiting recurrent networks for paraphrastic sentence embeddings. In: Proceedings of ACL, pp. 2078–2088 (2017)
Yanaka, H., Mineshima, K., Martínez-Gómez, P., Bekki, D.: Determining semantic textual similarity using natural deduction proofs. In: Proceedings of EMNLP, pp. 681–691 (2017)
Zhang, M., Liu, Y., Luan, H., Sun, M.: Adversarial training for unsupervised bilingual lexicon induction. In: Proceedings of ACL, pp. 1959–1970 (2017)
Zhao, J., Zhu, T., Lan, M.: Ecnu: one stone two birds: ensemble of heterogenous measures for semantic relatedness and textual entailment. In: Proceedings of SemEval (2014)
Zou, W.Y., Socher, R., Cer, D., Manning, C.D.: Bilingual word embeddings for phrase-based machine translation. In: Proceedings of EMNLP (2013)
Acknowledgments
We would like to thank the reviewers for their valuable comments. This work is supported by grants from Science and Technology Commission of Shanghai Municipality (15ZR1410700), the Open Project of Shanghai Key Laboratory of Trustworthy Computing (No. 07dz22304201604).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Tian, J. et al. (2018). An Adversarial Joint Learning Model for Low-Resource Language Semantic Textual Similarity. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds) Advances in Information Retrieval. ECIR 2018. Lecture Notes in Computer Science(), vol 10772. Springer, Cham. https://doi.org/10.1007/978-3-319-76941-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-76941-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-76940-0
Online ISBN: 978-3-319-76941-7
eBook Packages: Computer ScienceComputer Science (R0)