research-article

Context-Aware Intent Identification in Email Conversations

Authors:

Saghar Hosseini,

Ahmed Hassan Awadallah,

Paul N. Bennett,

Chris QuirkAuthors Info & Claims

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 585 - 594

https://doi.org/10.1145/3331184.3331260

Published: 18 July 2019 Publication History

Abstract

Email continues to be one of the most important means of online communication. People spend a significant amount of time sending, reading, searching and responding to email in order to manage tasks, exchange information, etc. In this paper, we study intent identification in workplace email. We use a large scale publicly available email dataset to characterize intents in enterprise email and propose methods for improving intent identification in email conversations. Previous work focused on classifying email messages into broad topical categories or detecting sentences that contain action items or follow certain speech acts. In this work, we focus on sentence-level intent identification and study how incorporating more context (such as the full message body and other metadata) could improve the performance of the intent identification models. We experiment with several models for leveraging context including both classical machine learning and deep learning approaches. We show that modeling the interaction between sentence and context can significantly improve the performance.

Supplementary Material

MP4 File (cite2-17h40-d2.mp4)

Download
434.36 MB

References

[1]

2015. Email Statistics Report. The Radicati Group, INC. (2015). https://goo.gl/brmqrm.

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. CoRR, Vol. abs/1409.0473 (2014). arxiv: 1409.0473 http://arxiv.org/abs/1409.0473

[3]

Ron Bekkerman, Andrew Mccallum, and Gary Huang. 2004. Automatic categorization of email into folders: Benchmark experiments on Enron and SRI corpora. (01 2004).

[4]

Paul N. Bennett and Jaime Carbonell. 2005. Detecting Action-items in e-Mail. In SIGIR '05. ACM, New York, NY, USA, 585--586.

Digital Library

[5]

Vitor R. Carvalho and William W. Cohen. 2005. On the Collective Classification of Email "Speech Acts". In SIGIR '05. ACM, New York, NY, USA, 345--352.

Digital Library

[6]

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).

[7]

Michael Chui, James Manyika, Jacques Bughin, Richard Dobbs, Charles Roxburgh, Hugo Sarrazin, Georey Sands, and Magdalena Westergren. 2012. The social economy: Unlocking value and productivity through social technologies. McKinsey Global Institute. (2012).

[8]

William Cohen, Vitor Carvalho, and Tom Mitchell. 2004. Learning to Classify Email into "Speech Acts". Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.

[9]

William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. 2004. Learning to classify email into speech acts. In In Proceedings of Empirical Methods in Natural Language Processing.

[10]

Simon Corston-Oliver, Eric Ringger, Michael Gamon, and Richard Campbell. 2004. Task-focused summarization of email. Text Summarization Branches Out (2004).

[11]

Simon Corston-Oliver, Eric Ringger, Michael Gamon, and Richard Campbell. 2004. Task-focused summarization of email. In ACL.

[12]

Nick Craswell, Hugo Zaragoza, and Stephen Robertson. 2005. Microsoft Cambridge at TREC 14: Enterprise Track. In Proceedings of the Fourteenth Text REtrieval Conference, TREC 2005, Gaithersburg, Maryland, USA, November 15-18, 2005. http://trec.nist.gov/pubs/trec14/papers/microsoft-cambridge.enterprise.pdf

[13]

L. A. Dabbish and R. E. Kraut. 2006. Email overload at work: An analysis of factors associated with email strain. In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work (CSCW '06). 431--440.

Digital Library

[14]

Laura A. Dabbish, Robert E. Kraut, Susan Fussell, and Sara Kiesler. 2005. Understanding Email Use: Predicting Action on a Message. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). ACM, New York, NY, USA, 691--700.

Digital Library

[15]

Dotan Di Castro, Zohar Karnin, Liane Lewin-Eytan, and Yoelle Maarek. 2016. You'Ve Got Mail, and Here is What You Could Do With It! Analyzing and Predicting Actions on Email Messages. In WSDM '16. 307--316.

Digital Library

[16]

S. Dumais, E. Cutrell, J. J. Cadiz, G. Jancke, R. Sarin, and D. C. Robbins. 2003. Stuff I've seen: A system for personal information retrieval and re-use. In ACM SIGIR Forum, Vol. 49. ACM, 28--35.

Digital Library

[17]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. 249--256.

[18]

David Graus, David van Dijk, Manos Tsagkias, Wouter Weerkamp, and Maarten de Rijke. 2014. Recipient Recommendation in Enterprises Using Communication Graphs and Email Content. In SIGIR '14. ACM, New York, NY, USA, 1079--1082.

Digital Library

[19]

M. Grbovic, G. Halawi, Z. Karnin, and Y. Maarek. 2014. How many folders do you really need? Classifying email into a handful of categories. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM '14). Shanghai, China, 869--878.

Digital Library

[20]

A. F. Hayes and K Krippendorff. 2007. Answering the call for a standard reliability measure for coding data. In Communication Methods and Measures 1. 77--89.

[21]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014).

[22]

Bryan Klimt and Yiming Yang. 2004. The Enron Corpus: A New Dataset for Email Classification Research. In ECML'04. 217--226.

Digital Library

[23]

Farshad Kooti, Luca Maria Aiello, Mihajlo Grbovic, Kristina Lerman, and Amin Mantrach. 2015. Evolution of Conversations in the Age of Email Overload. In WWW '15. ACM, 603--613.

Digital Library

[24]

Andrew Lampert, Robert Dale, and Cecile Paris. 2010. Detecting Emails Containing Requests for Action. Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics (HLT '10). Association for Computational Linguistics, Stroudsburg, PA, USA, 984--992.

Digital Library

[25]

Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In ICML. 1188--1196.

Digital Library

[26]

Chu-Cheng Lin, Dongyeop Kang, Michael Gamon, Madian Khabsa, Ahmed Hassan Awadallah, and Patrick Pantel. 2017. Actionable Email Intent Modeling with Reparametrized RNNs. arXiv preprint arXiv:1712.09185 (2017).

[27]

Douglas Oard, William Webber, David Kirsch, and Sergey Golitsynskiy. 2015. Avocado Research Email Collection. DVD. (2015).

[28]

P. Ogilvie and J. Callan. 2005. Experiments with language models for known-item finding of e-mail messages. In TREC.

[29]

Byung-Won On, Ee-Peng Lim, Jing Jiang, Amruta Purandare, and Loo-Nin Teow. 2010. Mining Interaction Behaviors for Email Reply Order Prediction. In Proceedings of the 2010 International Conference on Advances in Social Networks Analysis and Mining (ASONAM '10). IEEE Computer Society, Washington, DC, USA, 306--310.

Digital Library

[30]

Christopher Pal and Andrew McCallum. 2006. CC Prediction with Graphical Models. In CEAS.

[31]

Pranav Ramarao, Suresh Iyengar, Pushkar Chitnis, Raghavendra Udupa, and Balasubramanyan Ashok. 2016. InLook: Revisiting Email Search Experience. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '16). ACM, New York, NY, USA, 1117--1120.

Digital Library

[32]

Maya Sappelli, Gabriella Pasi, Suzan Verberne, Maaike de Boer, and Wessel Kraaij. 2016. Assessing e-mail intent and tasks in e-mail messages. Information Sciences, Vol. 358 (2016), 1--17.

Digital Library

[33]

Wouter Weerkamp, Krisztian Balog, and Maarten Rijke. 2009. Using Contextual Information to Improve Search in Email Archives. In Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval (ECIR '09). Springer-Verlag, Berlin, Heidelberg, 400--411.

Digital Library

[34]

S. Whittaker and C. Sidner. 1996. Email overload: Exploring personal information management of email. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '96). Vancouver, British Columbia, Canada, 276--283.

Digital Library

[35]

Liu Yang, Susan T. Dumais, Paul N. Bennett, and Ahmed Hassan Awadallah. 2017. Characterizing and Predicting Enterprise Email Reply Behavior. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17). ACM, New York, NY, USA, 235--244.

Digital Library

Cited By

Khandaker FSenderovich AZhao JCohen EYu ECarbajales SChan A(2024)Transformer models for mining intents and predicting activities from emails in knowledge-intensive processesEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107450128(107450)Online publication date: Feb-2024
https://doi.org/10.1016/j.engappai.2023.107450
Farshidi SRezaee KMazaheri SRahimi ADadashzadeh AZiabakhsh MEskandari SJansen S(2024)Understanding user intent modeling for conversational recommender systems: a systematic literature reviewUser Modeling and User-Adapted Interaction10.1007/s11257-024-09398-xOnline publication date: 6-Jun-2024
https://doi.org/10.1007/s11257-024-09398-x
Yuvaraj SBenatallah BMotahari-Nezhad HRabhi F(2024)Intent Identification Using Few-Shot and Active Learning with User FeedbackWeb Information Systems Engineering – WISE 202410.1007/978-981-96-0573-6_4(49-59)Online publication date: 27-Nov-2024
https://doi.org/10.1007/978-981-96-0573-6_4
Show More Cited By

Index Terms

Context-Aware Intent Identification in Email Conversations
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Clustering and classification
  2. World Wide Web
    1. Web applications
      1. Internet communications tools
        Email

Recommendations

Scribee Experimentation - Early Statistics on Email Conversations
WI-IAT '11: Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 03

Information overload is a real challenge becoming more and more critical regarding the quite limited human capabilities. 107 trillion emails sent in 2010, that is more than we could even imagine so how to help the user to take advantage from this? That ...
Evolution of Conversations in the Age of Email Overload
WWW '15: Proceedings of the 24th International Conference on World Wide Web

Email is a ubiquitous communications tool in the workplace and plays an important role in social interactions. Previous studies of email were largely based on surveys and limited to relatively small populations of email users within organizations. In ...
Preventing Spam Email by Delivery Limitation in RMX
IDEAS '15: Proceedings of the 19th International Database Engineering & Applications Symposium

On the rule-based email exchange system called RMX, similar to general mailing lists, anyone can send emails by sending to an address unique to RMX. However, there is a security problem that we cannot prevent spam emails and accidentally sending email ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2019

1512 pages

ISBN:9781450361729

DOI:10.1145/3331184

General Chairs:
Benjamin Piwowarski
CNRS - Sorbonne Universite, France
,
Max Chevalier
Universite de Toulouse, CNRS, France
,
Eric Gaussier
Universite Grenoble Alpes, CNRS, France
,
Program Chairs:
Yoelle Maarek
Amazon Research, Israel
,
Jian-Yun Nie
University of Montreal, Canada
,
Falk Scholer
RMIT University, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '19

Sponsor:

SIGIR

SIGIR '19: The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 21 - 25, 2019

Paris, France

Acceptance Rates

SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
636
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)7

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khandaker FSenderovich AZhao JCohen EYu ECarbajales SChan A(2024)Transformer models for mining intents and predicting activities from emails in knowledge-intensive processesEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107450128(107450)Online publication date: Feb-2024
https://doi.org/10.1016/j.engappai.2023.107450
Farshidi SRezaee KMazaheri SRahimi ADadashzadeh AZiabakhsh MEskandari SJansen S(2024)Understanding user intent modeling for conversational recommender systems: a systematic literature reviewUser Modeling and User-Adapted Interaction10.1007/s11257-024-09398-xOnline publication date: 6-Jun-2024
https://doi.org/10.1007/s11257-024-09398-x
Yuvaraj SBenatallah BMotahari-Nezhad HRabhi F(2024)Intent Identification Using Few-Shot and Active Learning with User FeedbackWeb Information Systems Engineering – WISE 202410.1007/978-981-96-0573-6_4(49-59)Online publication date: 27-Nov-2024
https://doi.org/10.1007/978-981-96-0573-6_4
Aserkar SBhada S(2024)AI-Enabled Policy Content Modeling: A Systems ApproachThe Proceedings of the 2024 Conference on Systems Engineering Research10.1007/978-3-031-62554-1_25(401-412)Online publication date: 26-Jul-2024
https://doi.org/10.1007/978-3-031-62554-1_25
Qiao YGodwin OOuyang H(2024)On-Device Query Auto-completion for Email SearchAdvances in Information Retrieval10.1007/978-3-031-56027-9_18(295-309)Online publication date: 20-Mar-2024
https://doi.org/10.1007/978-3-031-56027-9_18
Shah HJaidka KUngar LFagan JGrosser T(2023)Building a Multimodal Classifier of Email Behavior: Towards a Social Network Understanding of Organizational CommunicationInformation10.3390/info1412066114:12(661)Online publication date: 14-Dec-2023
https://doi.org/10.3390/info14120661
Aldag LBallreich FBerens BVolkamer M(2023)A user-centred approach to facilitate locating company security policiesProceedings of Mensch und Computer 202310.1145/3603555.3603573(173-185)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603573
Xia HWang TGunturu AJiang PDuan WYao X(2023)CrossTalk: Intelligent Substrates for Language-Oriented Interaction in Video-Based Communication and CollaborationProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606773(1-16)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606773
Fang FYang MLi CXu RChen HDuh WHuang HKato MMothe JPoblete B(2023)Adversarial Meta Prompt Tuning for Open Compound Domain Adaptive Intent DetectionProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591945(1791-1795)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591945
Yuvaraj SBenatallah BMotahari-Nezhad HRabhi F(2023)Identification and Generation of Actions Using Pre-trained Language ModelsWeb Information Systems Engineering – WISE 202310.1007/978-981-99-7254-8_52(674-683)Online publication date: 21-Oct-2023
https://doi.org/10.1007/978-981-99-7254-8_52
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten