research-article

AS-index: a structure for string search using n-grams and algebraic signatures

Authors:

Cédric du Mouza,

Philippe Rigaux,

Thomas SchwarzAuthors Info & Claims

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Pages 295 - 304

https://doi.org/10.1145/1645953.1645993

Published: 02 November 2009 Publication History

Abstract

AS-Index is a new index structure for exact string search in disk resident databases. It uses hashing, unlike known alternatives, whether baesd on trees or tries. It typically indexes every n-gram in the database, though non-dense indexing is also possible. The hash function uses the algebraic signatures of all n-grams. Use of hashing provides for constant index access time for arbitrarily long patterns, unlike other structures whose search cost is at best logarithmic. The storage overhead of AS-Index is basically 500 - 600%, similar to that of alternatives or smaller. We show the index structure, our use of algebraic signatures and the search algorithm. We discuss the design trade-offs and present the theoretical and experimental performance analysis. We compare the AS-Index to main alternatives. We conclude that AS-Index is an attractive structure and we indicate directions for future work.

References

[1]

S. N. A. Andersson. Efficient Implementation of Suffix Trees. Software- Practice and Experience, 25(2):129--141, 1995.

Digital Library

[2]

X. Cao, S. C. Li, and A. K. H. Tung. Indexing dna sequences using q-grams. In Proc. Database Systems for Advanced Applications (DASFAA), pages 4--16, 2005.

Digital Library

[3]

C. Charras, T. Lecroq, and J. D. Pehoushek. A Very Fast String Matching Algorithm for Small Alphabets and Long Patterns. In Proc. Combinatorial Pattern Matching, 1998. LNCS 1448.

Digital Library

[4]

M. Crochemore and M. Lecroq. Pattern Matching and Text Compression ALgorithms. CRC Press Inc, 2004.

[5]

R. Dementiev, J. Kärkkäinen, J. Mehnert, and P. Sanders. Better External Memory Suffix Array Construction. ACM Journal of Experimental Algorithmics, 12:1--24, 2008.

Digital Library

[6]

C. Faloutsos. Signature Files. In Information Retrieval: Data Structures&Algorithms, pages 44--65. 1992.

Digital Library

[7]

P. Ferragina and R. Grossi. The String B-tree: A New Data Structure for String Search in External Memory and Its Applications. J. ACM, 46(2):236--280, 1999.

Digital Library

[8]

J. Gray and B. Fitzgerald. Flash Disk Opportunity for Server-Applications. In Intl. Conf. on Innovative Data Systems research (CIDR), 2007.

[9]

D. Gusfield. Algorithms on Strings, Trees, and Sequences: Computer Science and Computational Biology. Cambridge University Press, 1997.

Digital Library

[10]

J. Kärkkäinen. Suffix Cactus: A Cross between Suffix Tree and Suffix Array. In Proc. Intl.Symp. on Combinatorial Pattern Matching (CPM), pages 191--204, 1995.

[11]

M. S. Kim, K. Whang, J. G. Lee, and M. J. Lee. n-Gram/2L: A Space and Time Efficient Two-level n-Gram Inverted Index Structure. In Proc. Intl. Conf. on Very Large Databases (VLDB), 2005.

Digital Library

[12]

S. Kurtz. Reducing the Space Requirement of Suffix Trees. Software - Practice and Experience, 29(13):1149--1171, 1999.

Digital Library

[13]

W. Litwin and T. Schwarz. Algebraic Signatures for Scalable Distributed Data Structures. In Proc. Intl. Conf. on Data Engineering (ICDE), 2004.

Digital Library

[14]

U. Manber and E. W. Myers. Sufix Arrays: A New Method for On-Line String Searches. SIAM Journal on Computing, 22(5):935--948, 1993.

Digital Library

[15]

G. Margaritis and S. V. Anastasiadis. Unleashing the Power of Full-text Search on File Systems. In Usenix Conf. on File and Storage Technology, 2007.

Digital Library

[16]

E. Miller, D. Shen, J. Liu, and C. Nicholas. Performance and Scalability of a Large-Scale n-gram Based Information Retrieval System. Journal of Digital Information, 1(5), 2000.

[17]

J. C. Na and K. Park. Simple Implementation of String B-tree. In Proc. String Processing and Information Retrieval (SPIRE), pages 214--215, 2004.

[18]

B. Phoophakdee and M. J. Zaki. Genome-scale disk-based suffix tree indexing. In Proc. Intl. Conf. on Management of Data (SIGMOD), pages 833--844, 2007.

Digital Library

[19]

Z. Tan, X. Cao, B. C. Ooi, and A. K. H. Tung. The ed-tree: An Index for Large DNA Sequence Databases. In Proc. Intl. Conf. on Scientific and Statistical Databases (SSDBM), 2003.

Digital Library

[20]

S. Tata, R. Hankins, and J. Patel. Practical Suffix Tree Construction. In Proc. Intl. Conf. on Very Large Databases (VLDB), pages 36--48, Toronto, Canada, 2004.

Digital Library

[21]

H. Williams and J. Zobel. Indexing and Retrieval for Genomic Databases. IEEE Transactions on Knowledge and Data Engineering, 14(1):63--78, 2002.

Digital Library

[22]

I. Witten, A. Moffat, and T. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan-Kaufmann, 1999.

Digital Library

[23]

J. Zobel, A. Moffat, and K. Ramamohanarao. Inverted Files Versus Signature Files for Text Indexing. ACM Trans. on Database Systems, 23(4):453--490, 1998

Digital Library

Cited By

Constantin Cdu Mouza CLitwin WRigaux PSchwarz T(2016)AS-Index: A Structure for String Search Using n-Grams and Algebraic SignaturesJournal of Computer Science and Technology10.1007/s11390-016-1618-631:1(147-166)Online publication date: 8-Jan-2016
https://doi.org/10.1007/s11390-016-1618-6
Wandelt SLeser U(2013)QGramProjectorProceedings of the 17th East European Conference on Advances in Databases and Information Systems - Volume 813310.1007/978-3-642-40683-6_20(260-273)Online publication date: 1-Sep-2013
https://dl.acm.org/doi/10.1007/978-3-642-40683-6_20
Atanassov I(2012)An approach for indexing file names in a directoryProceedings of the 13th International Conference on Computer Systems and Technologies10.1145/2383276.2383315(259-266)Online publication date: 22-Jun-2012
https://dl.acm.org/doi/10.1145/2383276.2383315
Show More Cited By

Index Terms

AS-index: a structure for string search using n-grams and algebraic signatures

Recommendations

A linear size index for approximate pattern matching

This paper revisits the problem of indexing a text S[1..n] for pattern matching with up to k errors. A naive solution either has a worst-case matching time complexity of @W(m^k) or requires @W(n^k) space, where m is the length of the pattern. Devising a ...
Comparison between the zeroth-order Randić index and the sum-connectivity index

The zeroth-order Randić index and the sum-connectivity index are very popular topological indices in mathematical chemistry. These two indices are based on vertex degrees of graphs and attracted a lot of attention in recent years. Recently Li and Li (...
Szeged index, edge Szeged index, and semi-star trees

A semi-star tree is a star tree whose some edges may be replaced by paths of length more than one. In this paper we present some increasing and decreasing transformations for Szeged index of the semi-star trees. Then we introduce palm semi-star tree and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

November 2009

2162 pages

ISBN:9781605585123

DOI:10.1145/1645953

General Chairs:
David Cheung
University of Hong Kong, Hong Kong
,
Il-Yeol Song
Drexel University, USA
,
Program Chairs:
Wesley Chu
UCLA, USA
,
Xiaohua Hu
Drexel University, USA
,
Jimmy Lin
University of Maryland, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '09

Sponsor:

CIKM '09: Conference on Information and Knowledge Management

November 2 - 6, 2009

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 1,466 of 6,316 submissions, 23%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
270
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Constantin Cdu Mouza CLitwin WRigaux PSchwarz T(2016)AS-Index: A Structure for String Search Using n-Grams and Algebraic SignaturesJournal of Computer Science and Technology10.1007/s11390-016-1618-631:1(147-166)Online publication date: 8-Jan-2016
https://doi.org/10.1007/s11390-016-1618-6
Wandelt SLeser U(2013)QGramProjectorProceedings of the 17th East European Conference on Advances in Databases and Information Systems - Volume 813310.1007/978-3-642-40683-6_20(260-273)Online publication date: 1-Sep-2013
https://dl.acm.org/doi/10.1007/978-3-642-40683-6_20
Atanassov I(2012)An approach for indexing file names in a directoryProceedings of the 13th International Conference on Computer Systems and Technologies10.1145/2383276.2383315(259-266)Online publication date: 22-Jun-2012
https://dl.acm.org/doi/10.1145/2383276.2383315
Fontoura MGurevich MJosifovski VVassilvitskii S(2011)Efficiently encoding term co-occurrences in inverted indexesProceedings of the 20th ACM international conference on Information and knowledge management10.1145/2063576.2063626(307-316)Online publication date: 24-Oct-2011
https://dl.acm.org/doi/10.1145/2063576.2063626

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten