short-paper

Dialogue State Tracking Based on Hierarchical Slot Attention and Contrastive Learning

Authors:

Xueming QianAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 4737 - 4741

https://doi.org/10.1145/3511808.3557581

Published: 17 October 2022 Publication History

Abstract

Dialogue state is a key information in traditional task-oriented dialogue systems, which represents the user's dialogue intention at each moment through a set of (slot, value). The recent methods model the slot and the dialogue context to keep track of the state, but there is a lack of refinement of context information. They do not consider the influence of dialogue context in different scenarios. Our proposed approach utilizes a fine-grained representation of each slot at multiple levels and incorporates an interaction mechanism to obtain a weight of past memory, present utterance and relevance of the slots. Besides, to address the problem that the dialogue utterance is semantically distant from the corresponding slot value, we introduce the contrastive learning to make the utterance embedding mapped under each slot name more suitable with the ground truth value and away from other slot values. This improves the accuracy of mapping between feature space and semantic space. In the predefined ontology-based approaches, our model achieves leading results with both MultiWOZ2.0 and MultiWOZ2.1 datasets.

References

[1]

Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasić. 2018. MultiWOZ--A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. arXiv preprint arXiv:1810.00278 (2018).

[2]

Junfan Chen, Richong Zhang, Yongyi Mao, and Jie Xu. 2020. Parallel interactive networks for multi-domain dialogue state generation. arXiv preprint arXiv:2009.07616 (2020).

[3]

Lu Chen, Boer Lv, Chi Wang, Su Zhu, Bowen Tan, and Kai Yu. 2020. Schemaguided multi-domain dialogue state tracking with graph attention neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 7521--7528.

[4]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[5]

Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, and Dilek Hakkani-Tür. 2019. Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines. (2019).

[6]

Jinyu Guo, Kai Shuang, Jijie Li, and Zihan Wang. 2021. Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking. arXiv preprint arXiv:2107.12578 (2021).

[7]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9729--9738.

[8]

Michael Heck, Carel van Niekerk, Nurul Lubis, Christian Geishauser, Hsien-Chin Lin, Marco Moresi, and Milica Gasic. 2020. Trippy: A triple copy strategy for value independent neural dialog state tracking. arXiv preprint arXiv:2005.02877 (2020).

[9]

Sungdong Kim, Sohee Yang, Gyuwan Kim, and Sang-Woo Lee. 2019. Efficient dialogue state tracking by selectively overwriting memory. arXiv preprint arXiv:1911.03906 (2019).

[10]

Hwaran Lee, Jinsik Lee, and Tae-Yoon Kim. 2019. Sumbt: Slot-utterance matching for universal and scalable belief tracking. arXiv preprint arXiv:1907.07421 (2019).

[11]

Han Liu, Feng Zhang, Xiaotong Zhang, Siyang Zhao, and Xianchao Zhang. 2021. An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling. In Findings of the Association for Computational Linguistics: EMNLP 2021. 1945--1955.

[12]

Liliang Ren, Jianmo Ni, and Julian McAuley. 2019. Scalable and accurate dialogue state tracking via hierarchical sequence generation. arXiv preprint arXiv:1909.00754 (2019).

[13]

Yong Shan, Zekang Li, Jinchao Zhang, Fandong Meng, Yang Feng, Cheng Niu, and Jie Zhou. 2020. A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking. CoRR abs/2006.01554 (2020). arXiv:2006.01554 https://arxiv.org/abs/2006.01554

[14]

Hao Tang, Guoshuai Zhao, Yuxia Wu, and Xueming Qian. 2021. Multisamplebased Contrastive Loss for Top-k Recommendation. IEEE Transactions on Multimedia (2021), 1--1. https://doi.org/10.1109/TMM.2021.3126146

[15]

Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv e-prints (2018), arXiv--1807.

[16]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).

[17]

Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. arXiv preprint arXiv:1905.08743 (2019).

[18]

YuxiaWu, Lizi Liao, Gangyi Zhang,Wenqiang Lei, Guoshuai Zhao, Xueming Qian, and Tat-Seng Chua. 2022. State Graph Reasoning for Multimodal Conversational Recommendation. IEEE Transactions on Multimedia (2022), 1--1. https://doi.org/10.1109/TMM.2022.3155900

Digital Library

[19]

Fanghua Ye, Jarana Manotumruksa, Qiang Zhang, Shenghui Li, and Emine Yilmaz. 2021. Slot Self-Attentive Dialogue State Tracking. In Proceedings of the Web Conference 2021. 1598--1608.

Digital Library

[20]

Xuanzhi Zheng, Guoshuai Zhao, Li Zhu, Jihua Zhu, and Xueming Qian. 2022. What You Like, What I Am: Online Dating Recommendation via Matching Individual Preferences with Features. IEEE Transactions on Knowledge and Data Engineering (2022), 1--1. https://doi.org/10.1109/TKDE.2022.3148485

Digital Library

[21]

Jingyao Zhou, Haipang Wu, Zehao Lin, Guodun Li, and Yin Zhang. 2021. Dialogue state tracking with multi-level fusion of predicted dialogue states and conversations. arXiv preprint arXiv:2107.05168 (2021).

Cited By

Li BFei HLi FWu SLiao LWei YChua TJi D(2024)Revisiting Conversation Discourse for Dialogue DisentanglementACM Transactions on Information Systems10.1145/369819143:1(1-34)Online publication date: 30-Nov-2024
https://dl.acm.org/doi/10.1145/3698191
Cheng SMa ZYe J(2024)A Benchmark of Zero-Shot Cross-Lingual Task-Oriented Dialogue Based on Adversarial Contrastive Representation Learning2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10688251(1-6)Online publication date: 15-Jul-2024
https://doi.org/10.1109/ICME57554.2024.10688251
Chen QZhao GWu YQian X(2023)Fine-grained semantic textual similarity measurement via a feature separation networkApplied Intelligence10.1007/s10489-022-04448-653:15(18205-18218)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1007/s10489-022-04448-6

Index Terms

Dialogue State Tracking Based on Hierarchical Slot Attention and Contrastive Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics

Recommendations

Toward Real-life Dialogue State Tracking Involving Negative Feedback Utterances
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Recently, the research of dialogue systems has been widely concerned, especially task-oriented dialogue systems, which have received increased attention due to their wide application prospect. As a core component, dialogue state tracking (DST) plays a ...
Slot Self-Attentive Dialogue State Tracking
WWW '21: Proceedings of the Web Conference 2021

An indispensable component in task-oriented dialogue systems is the dialogue state tracker, which keeps track of users’ intentions in the course of conversation. The typical approach towards this goal is to fill in multiple pre-defined slots that are ...
Hierarchical temporal slot interactions for dialogue state tracking
Abstract
Dialogue state tracking (DST), as an essential component of task-oriented dialogue systems, refers to keeping track of the user’s intentions as a conversation progresses. Typical methods formulate it as a classification task with fixed pre-defined ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Postdoctoral Innovative Talents Support Program
National Natural Science Foundation of China
China Postdoctoral Science Foundation
Humanities and Social Sciences Foundation of Ministry of Education
Science and Technology Program of Xi?an

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
168
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li BFei HLi FWu SLiao LWei YChua TJi D(2024)Revisiting Conversation Discourse for Dialogue DisentanglementACM Transactions on Information Systems10.1145/369819143:1(1-34)Online publication date: 30-Nov-2024
https://dl.acm.org/doi/10.1145/3698191
Cheng SMa ZYe J(2024)A Benchmark of Zero-Shot Cross-Lingual Task-Oriented Dialogue Based on Adversarial Contrastive Representation Learning2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10688251(1-6)Online publication date: 15-Jul-2024
https://doi.org/10.1109/ICME57554.2024.10688251
Chen QZhao GWu YQian X(2023)Fine-grained semantic textual similarity measurement via a feature separation networkApplied Intelligence10.1007/s10489-022-04448-653:15(18205-18218)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1007/s10489-022-04448-6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten