Hosein Azarbonyad
Ryen W. White
ABSTRACT
People often make commitments to perform future actions. Detecting commitments made in email (e.g., "I'll send the report by end of day'') enables digital assistants to help their users recall promises they have made and assist them in meeting those promises in a timely manner. In this paper, we show that commitments can be reliably extracted from emails when models are trained and evaluated on the same domain (corpus). However, their performance degrades when the evaluation domain differs. This illustrates the domain bias associated with email datasets and a need for more robust and generalizable models for commitment detection. To learn a domain-independent commitment model, we first characterize the differences between domains (email corpora) and then use this characterization to transfer knowledge between them. We investigate the performance of domain adaptation, namely transfer learning, at different granularities: feature-level adaptation and sample-level adaptation. We extend this further using a neural autoencoder trained to learn a domain-independent representation for training samples. We show that transfer learning can help remove domain bias to obtain models with less domain dependence. Overall, our results show that domain differences can have a significant negative impact on the quality of commitment detection models and that transfer learning has enormous potential to address this issue.
- Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational Linguistics, Vol. 34, 4 (2008), 555--596. Google Scholar
Digital Library
- Hosein Azarbonyad, Mostafa Dehghani, Kaspar Beelen, Alexandra Arkut, Maarten Marx, and Jaap Kamps. 2017. Words Are Malleable: Computing Semantic Shifts in Political and Media Discourse. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management . 1509--1518. Google ScholarDigital Library
- Victoria Bellotti, Nicolas Ducheneaut, Mark Howard, and Ian Smith. 2003. Taking email to task: the design and evaluation of a task management centered email tool. In Proceedings of the SIGCHI conference on Human factors in computing systems. 345--352. Google ScholarDigital Library
- Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A Theory of Learning from Different Domains. Machine Learning, Vol. 79, 1--2 (2010), 151--175. Google ScholarDigital Library
- Yoshua Bengio et almbox. 2009. Learning deep architectures for AI. Foundations and trends® in Machine Learning, Vol. 2, 1 (2009), 1--127. Google ScholarDigital Library
- Paul N Bennett and Jaime Carbonell. 2005. Detecting action-items in e-mail. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval . 585--586. Google ScholarDigital Library
- John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics. 440--447.Google Scholar
- Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. In Proceedings of Advances in Neural Information Processing Systems. 343--351. Google ScholarDigital Library
- Minmin Chen, Zhixiang Xu, Kilian Weinberger, and Fei Sha. 2012. Marginalized denoising autoencoders for domain adaptation. arXiv preprint arXiv:1206.4683 (2012). Google ScholarDigital Library
- Xilun Chen and Claire Cardie. 2018. Multinomial Adversarial Networks for Multi-Domain Text Classification. arXiv preprint arXiv:1802.05694 (2018).Google Scholar
- Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).Google Scholar
- William W Cohen, Vitor R Carvalho, and Tom M Mitchell. 2004. Learning to Classify Email into“Speech Acts”. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing .Google Scholar
- Simon Corston-Oliver, Eric Ringger, Michael Gamon, and Richard Campbell. 2004. Task-focused summarization of email. Text Summarization Branches Out (2004).Google Scholar
- Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu. 2007. Boosting for transfer learning. In Proceedings of the 24th International Conference on Machine Learning. 193--200. Google ScholarDigital Library
- Rachele De Felice. 2013. A corpus-based classification of commitments in Business English. In Yearbook of Corpus Linguistics and Pragmatics 2013. 153--171.Google Scholar
- Michael Gamon, Saliha Azzam, Yizheng Cai, Nicholas Caldwell, and Ye-Yi Wang. 2013. Automatic Task Extraction and Calendar Entry. (2013). US Patent App. 13/170,660.Google Scholar
- Yaroslav Ganin and Victor Lempitsky. 2014. Unsupervised domain adaptation by backpropagation. arXiv preprint arXiv:1409.7495 (2014). Google ScholarDigital Library
- Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. Journal of Machine Learning Research, Vol. 17, 1 (2016), 2096--2030. Google ScholarDigital Library
- Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on Machine Learning (ICML-11). 513--520. Google ScholarDigital Library
- Jacek Gwizdka. 2002. Reinventing the inbox: supporting the management of pending tasks in email. In CHI'02 Extended Abstracts on Human Factors in Computing Systems. 550--551. Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780. Google ScholarDigital Library
- Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in NLP. In Proceedings of the 45th annual meeting of the association of computational linguistics. 264--271.Google Scholar
- Anup K Kalia, Hamid R Motahari-Nezhad, Claudio Bartolini, and Munindar P Singh. 2013. Monitoring commitments in people-driven service engagements. In Services Computing (SCC), 2013 IEEE International Conference on. 160--167. Google ScholarDigital Library
- Bryan Klimt and Yiming Yang. 2004. The enron corpus: A new dataset for email classification research. In Proceedings of European Conference on Machine Learning. 217--226. Google ScholarDigital Library
- Wouter M. Kouw, Laurens J.P. van der Maaten, Jesse H. Krijthe, and Marco Loog. 2016. Feature-Level Domain Adaptation. Journal of Machine Learning Research, Vol. 17, 171 (2016), 1--32. Google ScholarDigital Library
- Andrew Lampert, Robert Dale, and Cecile Paris. 2008. Requests and Commitments in Email are More Complex Than You Think: Eight Reasons to be Cautious. In Proceedings of the Australasian Language Technology Association Workshop 2008. 64--72.Google Scholar
- Andrew Lampert, Robert Dale, and Cecile Paris. 2010. Detecting emails containing requests for action. In Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 984--992. Google ScholarDigital Library
- Andrew Lampert, Cécile Paris, Robert Dale, et almbox. 2007. Can requests-for-action and commitments-to-act be reliably identified in email messages. In Proceedings of the 12th Australasian Document Computing Symposium. 48--55.Google Scholar
- Guillaume Lample, Alexis Conneau, Marc'Aurelio Ranzato, Ludovic Denoyer, and Herve Jegou. 2018. Word translation without parallel data. In Proceedings of International Conference on Learning Representations .Google Scholar
- Bin Li, Qiang Yang, and Xiangyang Xue. 2009. Transfer learning for collaborative filtering via a rating-matrix generative model. In Proceedings of the 26th annual International Conference on Machine Learning . 617--624. Google ScholarDigital Library
- Chu-Cheng Lin, Dongyeop Kang, Michael Gamon, Madian Khabsa, Ahmed Hassan Awadallah, and Patrick Pantel. 2017. Actionable Email Intent Modeling with Reparametrized RNNs. arXiv preprint arXiv:1712.09185 (2017).Google Scholar
- Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial Multi-task Learning for Text Classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics . 1--10.Google ScholarCross Ref
- Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013a. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168 (2013).Google Scholar
- Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. 2013b. Exploiting Similarities among Languages for Machine Translation. In Proceedings of International Conference on Learning Representations .Google Scholar
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013c. Distributed representations of words and phrases and their compositionality. In Proceedings of Advances in Neural Information Processing Systems . 3111--3119. Google ScholarDigital Library
- Hamid R Motahari Nezhad, Kalpa Gunaratna, and Juan Cappi. 2017. eassistant: Cognitive assistance for identification and auto-triage of actionable conversations. In Proceedings of the 26th International Conference on World Wide Web Companion. 89--98. Google ScholarDigital Library
- Douglas Oard, William Webber, David Kirsch, and Sergey Golitsynskiy. 2015. Avocado research email collection. Linguistic Data Consortium (2015).Google Scholar
- Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, Vol. 22, 10 (2010), 1345--1359. Google ScholarDigital Library
- Chris Quirk, Pallavi Choudhury, Jianfeng Gao, Hisami Suzuki, Kristina Toutanova, Michael Gamon, Wen-tau Yih, Lucy Vanderwende, and Colin Cherry. 2012. MSR SPLAT, a language analysis toolkit. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Demonstration Session. 21--24. Google ScholarDigital Library
- Rajat Raina, Andrew Y Ng, and Daphne Koller. 2006. Constructing informative priors using transfer learning. In Proceedings of the 23rd International Conference on Machine learning. 713--720. Google ScholarDigital Library
- Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method. The annals of mathematical statistics (1951), 400--407.Google Scholar
- Hidetoshi Shimodaira. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, Vol. 90, 2 (2000), 227--244.Google ScholarCross Ref
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of Advances in neural information processing systems. 3104--3112. Google ScholarDigital Library
- Gokhan Tur, Andreas Stolcke, Lynn Voss, Stanley Peters, Dilek Hakkani-Tur, John Dowding, Benoit Favre, Raquel Fernández, Matthew Frampton, Mike Frandsen, et almbox. 2010. The CALO meeting assistant system. IEEE Transactions on Audio, Speech, and Language Processing, Vol. 18, 6 (2010), 1601--1611.Google Scholar
- Steve Whittaker and Candace Sidner. 1996. Email overload: exploring personal information management of email. In Proceedings of the SIGCHI conference on Human factors in computing systems . 276--283. Google ScholarDigital Library
- Pengcheng Wu and Thomas G Dietterich. 2004. Improving SVM accuracy by training on auxiliary data sources. In Proceedings of the twenty-first International Conference on Machine learning. 110. Google ScholarDigital Library
- Fuzhen Zhuang, Xiaohu Cheng, Ping Luo, Sinno Jialin Pan, and Qing He. 2015. Supervised Representation Learning: Transfer Learning with Deep Autoencoders. In Proceedings of the 24th International Conference on Artificial Intelligence . 4119--4125. Google ScholarDigital Library
- Fuzhen Zhuang, Ping Luo, Hui Xiong, Yuhong Xiong, Qing He, and Zhongzhi Shi. 2010. Cross-domain learning from multiple sources: A consensus regularization perspective. IEEE Transactions on Knowledge and Data Engineering, Vol. 22, 12 (2010), 1664--1678. Google ScholarDigital Library
Index Terms
- Domain Adaptation for Commitment Detection in Email
Recommendations
Domain Adaptation for Face Recognition: Targetize Source Domain Bridged by Common Subspace
In many applications, a face recognition model learned on a source domain but applied to a novel target domain degenerates even significantly due to the mismatch between the two domains. Aiming at learning a better face recognition model for the target ...
Integrating Priors into Domain Adaptation Based on Evidence Theory
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementDomain adaptation aims to build up a learning model for target domain by leveraging transferable knowledge from different but related source domains. Existing domain adaptation methods generally transfer the knowledge from source domain to target domain ...
Zero-Shot Deep Domain Adaptation
Computer Vision – ECCV 2018AbstractDomain adaptation is an important tool to transfer knowledge about a task (e.g. classification) learned in a source domain to a second, or target domain. Current approaches assume that task-relevant target-domain data is available during training. ...
Comments