Jai Gupta
Donald Metzler
ABSTRACT
Spell correction is a must-have feature for any modern search engine in applications such as web or e-commerce search. Typical spell correction solutions used in production systems consist of large indexed lookup tables based on a global model trained across many users over a large scale web corpus or a query log.
For search over personal corpora, such as email, this global solution is not sufficient, as it ignores the user's personal lexicon. Without personalization, global spelling fails to correct tail queries drawn from a user's own, often idiosyncratic, lexicon. Personalization using existing algorithms is difficult due to resource constraints and unavailability of sufficient data to build per-user models.
In this work, we propose a simple and effective personalized spell correction solution that augments existing global solutions for search over private corpora. Our event driven spell correction candidate generation method is specifically designed with personalization as the key construct. Our novel spell correction and query completion algorithms do not require complex model training and is highly efficient. The proposed solution has shown over 30% click-through rate gain on affected queries when evaluated against a range of strong commercial personal search baselines - Google's Gmail, Drive, and Calendar search production systems.
- 2018. I before E except after C. Retrieved 2018-10-30 from https://en.wikipedia.org/wiki/I_before_E_except_after_CGoogle Scholar
- Qingyao Ai, Susan T. Dumais, Nick Craswell, and Dan Liebling. 2017. Characterizing Email Search Using Large-scale Behavioral Logs and Surveys. In WWW. 1511-1520. Google ScholarDigital Library
- Andrei Broder, Peter Ciccolo, Evgeniy Gabrilovich, Vanja Josifovski, Donald Metzler, Lance Riedel, and Jeffrey Yuan. 2009. Online Expansion of Rare Queries for Sponsored Search. In WWW. 511-520. Google ScholarDigital Library
- Fei Cai, Shangsong Liang, and Maarten de Rijke. 2014. Time-sensitive Personalized Query Auto-Completion. In CIKM. 1599-1608. Google ScholarDigital Library
- David Carmel, Guy Halawi, Liane Lewin-Eytan, Yoelle Maarek, and Ariel Raviv. 2015. Rank by Time or by Relevance?: Revisiting Email Search. In CIKM. 283-292. Google ScholarDigital Library
- Surajit Chaudhuri and Raghav Kaushik. 2009. Extending Autocompletion to Tolerate Errors. In ACM SIGMOD. 707-718. Google ScholarDigital Library
- Qing Chen, Mu Li, and Ming Zhou. 2007. Improving Query Spelling Correction Using Web Search Results. In EMNLP-CoNLL. 181-189.Google Scholar
- Silviu Cucerzan and Eric Brill. 2004. Spelling Correction as an Iterative Process that Exploits the Collective Knowledge of Web Users.. In EMNLP. 293-300.Google Scholar
- Dong Deng, Guoliang Li, He Wen, H. V. Jagadish, and Jianhua Feng. 2016. META: An Efficient Matching-based Method for Error-tolerant Autocompletion. In VLDB. 828-839. Google ScholarDigital Library
- Huizhong Duan and Bo-June Hsu. 2011. Online Spelling Correction for Query Completion. In WWW. 117-126. Google ScholarDigital Library
- Susan Dumais, Edward Cutrell, JJ Cadiz, Gavin Jancke, Raman Sarin, and Daniel C. Robbins. 2003. Stuff I'Ve Seen: A System for Personal Information Retrieval and Re-use. In SIGIR. 72-79. Google ScholarDigital Library
- Mohammad Ali Elmi and Martha Evens. 1998. Spelling correction using context. In ACL. 360-364.Google Scholar
- Pravallika Etoori, Manoj Chinnakotla, and Radhika Mamidi. 2018. Automatic Spelling Correction for Resource-Scarce Languages using Deep Learning. In ACL, Student Research Workshop. 146-152.Google Scholar
- Pieter Fivez, Simon Šuster, and Walter Daelemans. 2017. Unsupervised Context-Sensitive Spelling Correction of English and Dutch Clinical Free-Text with Word and Character N-Gram Embeddings. arXiv preprint arXiv:1710.07045(2017).Google Scholar
- Jianfeng Gao, Xiaolong Li, Daniel Micol, Chris Quirk, and Xu Sun. 2010. A Large Scale Ranker-based System for Search Query Spelling Correction. In COLING. 358-366. Google ScholarDigital Library
- Sasa Hasan, Carmen Heger, and Saab Mansour. 2015. Spelling Correction of User Search Queries through Statistical Machine Translation. In EMNLP. 451-460.Google Scholar
- Michael Herscovici, Dan Guez, and Hyung-Jin Kim. 2017 granted. Autocompletion using previously submitted query data. In US9740780B1.Google Scholar
- Maryam Kamvar, Melanie Kellar, Rajan Patel, and Ya Xu. 2009. Computers and Iphones and Mobile Phones, Oh My!: A Logs-based Comparison of Search Users on Different Devices. In WWW. 801-810. Google ScholarDigital Library
- Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady. 707-710.Google Scholar
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS. 3111-3119. Google ScholarDigital Library
- Harshit Pande. 2017. Effective search space reduction for spell correction using character neural embeddings. In EAACL. 170-174.Google Scholar
- Milad Shokouhi. 2013. Learning to Personalize Query Auto-completion. In SIGIR. 103-112. Google ScholarDigital Library
- Xu Sun, Jianfeng Gao, Daniel Micol, and Chris Quirk. 2010. Learning Phrase-based Spelling Error Models from Clickthrough Data. In ACL. 266-274. Google ScholarDigital Library
- Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. 2016. Learning to Rank with Selection Bias in Personal Search. In SIGIR. 115-124. Google ScholarDigital Library
Recommendations
Situational Context for Ranking in Personal Search
WWW '17: Proceedings of the 26th International Conference on World Wide WebModern search engines leverage a variety of sources, beyond the conventional query-document content similarity, to improve their ranking performance. Among them, query context has attracted attention in prior work. Previously, query context was mainly ...
An Architecture of an Academic Search Engine with Personalized Search Result Ranking Mechanism
ICNCC '16: Proceedings of the Fifth International Conference on Network, Communication and ComputingA rapid increasing of information on the Internet and World Wide Web causes information overloaded problem. Thus, search engines become important tools to help WWW users to discover the information they need. With an exponentially increasing of ...
Personalized interactive faceted search
WWW '08: Proceedings of the 17th international conference on World Wide WebFaceted search is becoming a popular method to allow users to interactively search and navigate complex information spaces. A faceted search system presents users with key-value metadata that is used for query refinement. While popular in e-commerce and ...
Comments