Rongjunchen Zhang
Skip Abstract Section
Abstract
Multi-turn response selection is a key issue in retrieval-based chatbots and has attracted considerable attention in the NLP (Natural Language processing) field. So far, researchers have developed many solutions that can select appropriate responses for multi-turn conversations. However, these works are still suffering from the semantic mismatch problem when responses and context share similar words with different meanings. In this article, we propose a novel chatbot model based on Semantic Awareness Matching, called SAM. SAM can capture both similarity and semantic features in the context by a two-layer matching network. Appropriate responses are selected according to the matching probability made through the aggregation of the two feature types. In the evaluation, we pick 4 widely used datasets and compare SAM’s performance to that of 12 other models. Experiment results show that SAM achieves substantial improvements, with up to 1.5% R10@1 on Ubuntu Dialogue Corpus V2, 0.5% R10@1 on Douban Conversation Corpus, and 1.3% R10@1 on E-commerce Corpus.
- [1] . 2016. Standardized usability questionnaires: Features and quality focus. Electronic Journal of Computer Science and Information Technology 6, 1 (2016).Google Scholar
- [2] . 2009. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly Media, Inc.Google Scholar
- [3] . 1977. GUS, a frame-driven dialog system. Artificial Intelligence 8, 2 (1977), 155–173.Google ScholarDigital Library
- [4] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long and Short Papers), ACL, Minneapolis, MN, 4171–4186.Google Scholar
- [5] . 2020. Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland (
CIKM’20 ). ACM, 2041–2044.Google ScholarDigital Library - [6] . 2019. Improving text classification with weighted word embeddings via a multi-channel TextCNN model. Neurocomputing 363 (2019), 366–374.Google ScholarDigital Library
- [7] . 2014. Convolutional neural network architectures for matching natural language sentences. In Proceedings of the 27th International Conference on Neural Information Processing Systems, Vol. 2, MIT Press, Montreal, 2042–2050.Google Scholar
- [8] . 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, Baltimore MD, 655–665.Google ScholarCross Ref
- [9] . 2011. Performance analysis of various activation functions in generalized MLP architectures of neural networks. International Journal of Artificial Intelligence and Expert Systems 1, 4 (2011), 111–122.Google Scholar
- [10] . 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14), ACL, Doha, 1746–1751.Google ScholarCross Ref
- [11] . 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097–1105.Google ScholarDigital Library
- [12] . 2015. Deep learning. Nature 521, 7553 (2015), 436.Google ScholarCross Ref
- [13] . 1990. Handwritten digit recognition with a back-propagation network. In Advances in Neural Information Processing Systems. 396–404.Google ScholarDigital Library
- [14] . 2018. Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, Melbourne, 1437–1447.Google ScholarCross Ref
- [15] . 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019).Google Scholar
- [16] . 2015. The Ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, ACL, Prague, 285–294.Google ScholarCross Ref
- [17] . 2017. Training end-to-end dialogue systems with the Ubuntu dialogue corpus. Dialogue & Discourse 8, 1 (2017), 31–65.Google ScholarCross Ref
- [18] . 2019. Constructing interpretive spatio-temporal features for multi-turn responses selection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL, Florence, 44–50.Google ScholarCross Ref
- [19] . 2013. A deep architecture for matching short texts. In Advances in Neural Information Processing Systems. 1367–1375.Google Scholar
- [20] . 2018. Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. arXiv preprint arXiv:1804.08217 (2018).Google Scholar
- [21] . 2016. Neural belief tracker: Data-driven dialogue state tracking. arXiv preprint arXiv:1606.03777 (2016).Google Scholar
- [22] . 2018. Exemplar encoder-decoder for neural conversation generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, Melbourne, 1329–1338.Google ScholarCross Ref
- [23] . 2016. Text matching as image recognition. In 30th AAAI Conference on Artificial Intelligence, Vol. 30.Google ScholarCross Ref
- [24] . 2018. Deep dyna-Q: Integrating planning for task-completion dialogue policy learning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, Melbourne, 2182–2192.Google ScholarCross Ref
- [25] . 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14), ACL, Doha, 1532–1543.Google ScholarCross Ref
- [26] . 2016. SQuAD: 100,000+ Questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, ACL, Austin, 2383–2392.Google ScholarCross Ref
- [27] . 2003. Using TF-IDF to determine word relevance in document queries. In Proceedings of the 1st Instructional Conference on Machine Learning, Vol. 242, ACM, Alberta, 133–142.Google Scholar
- [28] . 2019. Bridging the gap between relevance matching and semantic matching for short text similarity modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), ACL, Hong Kong, 5373–5384.Google ScholarCross Ref
- [29] . 2011. Data-driven response generation in social media. In Proceedings of the 2011 Conference on Empirical Methods, ACL, Edinburgh, 583–593.Google Scholar
- [30] . 2010. Evaluation of pooling operations in convolutional architectures for object recognition. In 20th International Conference on Artificial Neural Networks, Springer, Thessaloniki, 92–101.Google Scholar
- [31] . 2015. Neural responding machine for short-text conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), ACL, Beijing, 1577–1586.Google ScholarCross Ref
- [32] . 2018. From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19, 1 (2018), 10–26.Google ScholarCross Ref
- [33] . 2015. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, ACL, Denver, 196–205.Google ScholarCross Ref
- [34] . 2019. Multi-representation fusion network for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining, ACM, Melbourne, 267–275.Google ScholarDigital Library
- [35] . 2019. One time of interaction may not be enough: Go deep with an interaction-over-interaction network for response selection in dialogues. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL, Florence, 1–11.Google ScholarCross Ref
- [36] . 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 (2015).Google Scholar
- [37] . 2013. A dataset for research on short-text conversations. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, ACL, Seattle, 935–945.Google Scholar
- [38] . 2015. Syntax-based deep matching of short texts. In 24th International Joint Conference on Artificial Intelligence, ACM, Buenos, 1354–1361.Google ScholarDigital Library
- [39] . 2017. Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, Vancouver, 496–505.Google ScholarCross Ref
- [40] . 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems. 5754–5764.Google Scholar
- [41] . 2019. Multi-hop selector network for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), ACL, Hong Kong, 111–120.Google ScholarCross Ref
- [42] . 2018. Modeling multi-turn conversation with deep utterance aggregation. In Proceedings of the 27th International Conference on Computational Linguistics, ACL, Santa Fe, 3740–3752.Google Scholar
- [43] . 2016. Multi-view response selection for human-computer conversation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, ACL, Austin, TX, 372–381.Google ScholarCross Ref
- [44] . 2018. Multi-turn response selection for chatbots with deep attention matching network. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, Melbourne, 1118–1127.Google ScholarCross Ref
Index Terms
- SAM: Multi-turn Response Selection Based on Semantic Awareness Matching
Recommendations
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge ManagementIn this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (SA-BERT), is proposed in order to make the model aware of the speaker ...
Hierarchical matching network for multi-turn response selection in retrieval-based chatbots
AbstractProper response selection is a crucial challenge in retrieval-based chatbots. The state-of-the-art methods match a response with the word sequence of a context, or match the response with each utterance in the context and then accumulate matching ...
Context-Aware Network for Multi-Turn Response Selection in Retrieval-Based Chatbots
ICHMI '21: Proceedings of the 2021 International Conference on Human-Machine InteractionMulti-turn response selection is a major challenge for chatbot dialogue systems. The existing methods either ignore the interactions among previous utterances for context modeling, or regard all the previous utterances of the same importance. In this ...
Comments