poster

CloudSpeller: query spelling correction by using a unified hidden markov model with web-scale resources

Authors:

Yanen Li,

Huizhong Duan,

ChengXiang ZhaiAuthors Info & Claims

WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

Pages 561 - 562

https://doi.org/10.1145/2187980.2188130

Published: 16 April 2012 Publication History

Get Access

Abstract

Query spelling correction is an important component of modern search engines that can help users to express an information need more accurately and thus improve search quality. In this work we proposed and implemented an end-to-end speller correction system, namely CloudSpeller. The CloudSpeller system uses a Hidden Markov Model to effectively model major types of spelling errors in a unified framework, in which we integrate a large-scale lexicon constructed using Wikipedia, an error model trained from high confidence correction pairs, and the Microsoft Web N-gram service. Our system achieves excellent performance on two search query spelling correction datasets, reaching 0.960 and 0.937 F1 scores on the TREC dataset and the MSN dataset respectively.

References

[1]

http://research.microsoft.com/en-us/collaboration/focus/cs/web-ngram.aspx.

Google Scholar

[2]

E. Brill and R. Moore. An improved error model for noisy channel spelling correction. In ACL 2000.

Digital Library

Google Scholar

[3]

Q. Chen, M. Li, and M. Zhou. Improving query spelling correction using web search results. In EMNLP 2007.

Google Scholar

[4]

J. Gao, X. Li, D. Micol, C. Quirk, and X. Sun. A large scale ranker-based system for search query spelling correction. In COLING 2010.

Digital Library

Google Scholar

[5]

S. Cucerzan and E. Brill. Spelling correction as an iterative process that exploits the collective knowledge of web users. In EMNLP, 2004.

Google Scholar

Cited By

View all

Ye DTian BFan JLiu JZhou TChen XLi MMa JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Query Correction Using Pre-train Language Model In Search EnginesProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614930(2999-3008)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614930
Neto ABezerra BToselli A(2020)Towards the Natural Language Processing as Spelling Correction for Offline Handwritten Text Recognition SystemsApplied Sciences10.3390/app1021771110:21(7711)Online publication date: 31-Oct-2020
https://doi.org/10.3390/app10217711
Duan JJi TWu MWang H(2019)Query Error Correction Algorithm Based on Fusion Sequence to Sequence ModelComputational Collective Intelligence10.1007/978-3-030-28374-2_2(13-25)Online publication date: 9-Aug-2019
https://doi.org/10.1007/978-3-030-28374-2_2

Index Terms

CloudSpeller: query spelling correction by using a unified hidden markov model with web-scale resources
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

A Large-Scale Query Spelling Correction Corpus
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

We present a new large-scale collection of 54,772 queries with manually annotated spelling corrections. For 9,170 of the queries (16.74%), spelling variants that are different to the original query are proposed. With its size, our new corpus is an order ...
A generalized hidden Markov model with discriminative training for query spelling correction
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval

Query spelling correction is a crucial component of modern search engines. Existing methods in the literature for search query spelling correction have two major drawbacks. First, they are unable to handle certain important types of spelling errors, ...
Query spelling correction using multi-task learning
WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

This paper explores the use of online multi-task learning for search query spelling correction, by effectively transferring information from different and biased training datasets for improving spelling correction across datasets. Experiments were ...

Comments

Information & Contributors

Information

Published In

WWW '12 Companion: Proceedings of the 21st International Conference on World Wide Web

April 2012

1250 pages

ISBN:9781450312301

DOI:10.1145/2187980

General Chairs:
Alain Mille
Université de Lyon, France
,
Fabien Gandon
INRIA, France
,
Jacques Misselis
HP, France
,
Program Chairs:
Michael Rabinovich
Case Western Reserve University, USA
,
Steffen Staab
University of Koblenz-Landau, Germany

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

WWW 2012

Sponsor:

Univ. de Lyon

WWW 2012: 21st World Wide Web Conference 2012

April 16 - 20, 2012

Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
168
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ye DTian BFan JLiu JZhou TChen XLi MMa JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Query Correction Using Pre-train Language Model In Search EnginesProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614930(2999-3008)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614930
Neto ABezerra BToselli A(2020)Towards the Natural Language Processing as Spelling Correction for Offline Handwritten Text Recognition SystemsApplied Sciences10.3390/app1021771110:21(7711)Online publication date: 31-Oct-2020
https://doi.org/10.3390/app10217711
Duan JJi TWu MWang H(2019)Query Error Correction Algorithm Based on Fusion Sequence to Sequence ModelComputational Collective Intelligence10.1007/978-3-030-28374-2_2(13-25)Online publication date: 9-Aug-2019
https://doi.org/10.1007/978-3-030-28374-2_2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

A Large-Scale Query Spelling Correction Corpus

A generalized hidden Markov model with discriminative training for query spelling correction

Query spelling correction using multi-task learning

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations