Abstract
Quantum language models have attracted extensive attention in natural language processing tasks. Nevertheless, in this field measurement operators are mostly generated by randomly initialized parameter matrices, which cannot well explain the role of measurement operators in quantum theory. In this paper, we propose a Measurement-Based Quantum-like Language Model (MBQLM). Specifically, each word is considered a fundamental event in quantum probability space, which is a quantum state represented by a density matrix. We take the word density matrix in one sentence as a set of measurement operators to measure another sentence, which is consistent with the definition of measurement operators in quantum theory and has a specific semantic interpretation. The measured sentence state matrix representation effectively interact the feature information between sentences. To evaluate the performance of our model, we conduct experiments on WikiQA and TREC-QA datasets. The results show that our model achieves better performance than all benchmarks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Basile, I., Tamburini, F.: Towards quantum language models. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1840–1849 (2017)
Li, Q., Wang, B., Melucci, M.: CNM: an interpretable complex-valued network for matching. arXiv preprint arXiv:1904.05298 (2019)
Bruza, P., Kitto, K., McEvoy, D.: Entangling words and meaning. In: Quantum Interaction: Proceedings of the Second Quantum Interaction Symposium (QI-2008), pp. 118–124. College Publications (2008)
Aerts, D., Sozzo, S.: Quantum entanglement in concept combinations. Int. J. Theory Phys. 53(10), 3587–3603 (2014)
Wang, B., Zhang, P., Li, J., Song, D., Hou, Y., Shang, Z.: Exploration of quantum interference in document relevance judgement discrepancy. Entropy 18(4), 144 (2016)
Sordoni, A., Nie, J.Y., Bengio, Y.: Modeling term dependencies with quantum language models for IR. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 653–662 (2013)
Blacoe, W., Kashefi, E., Lapata, M.: A quantum-theoretic approach to distributional semantics. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 847–857 (2013)
Zhang, P., Niu, J., Su, Z., Wang, B., Ma, L., Song, D.: End-to-end quantum-like language models with application to question answering. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Cho, K., et al.: Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Von Neumann, J.: Mathematical Foundations of Quantum Mechanics. Princeton University Press, Princeton (2018)
Nielsen, M.A., Chuang, I.: Quantum Computation and Quantum Information (2002)
Zhang, P., Su, Z., Zhang, L., Wang, B., Song, D.: A quantum many-body wave function inspired language modeling approach. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 1303–1312 (2018)
Wang, M., Smith, N.A., Mitamura, T.: What is the jeopardy model? a quasi-synchronous grammar for QA. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp. 22–32 (2007)
Yang, Y., Yih, W.T., Meek, C.: WikiQA: a challenge dataset for open-domain question answering. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2013–2018 (2015)
Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015)
Yang, L., Ai, Q., Guo, J., Croft, W.B.: aNMM: ranking short answer texts with attention-based neural matching model. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 287–296 (2016)
He, H., Lin, J.: Pairwise word interaction modeling with deep neural networks for semantic similarity measurement. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 937–948 (2016)
Rao, J., He, H., Lin, J.: Noise-contrastive estimation for answer selection with deep neural networks. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 1913–1916 (2016)
dos Santos, C.N., Tan, M., Xiang, B., Zhou, B.: Attentive pooling networks. CoRR abs/1602.03609 (2016). http://arxiv.org/abs/1602.03609
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Chen, Y., Pan, Y., Dong, D.: Quantum language model with entanglement embedding for question answering. arXiv preprint arXiv:2008.09943 (2020)
Mindspore (2020). https://www.mindspore.cn/
Acknowledgement
This work is supported in part by the Natural Science Foundation of China (grant No. 62276188 and No. 61876129), TJU-Wenge joint laboratory funding, Tianjin Research Innovation ‘ Students (grant No. 2021YJSB167), and MindSpore (https://www.mindspore.cn/) [22].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, W., Gan, G., Gao, H., Zhang, P., Hui, W., Fan, Z. (2023). A Measurement-Based Quantum-Like Language Model for Text Matching. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)