skip to main content
10.1145/2786006.2786007acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

LSM-Based Storage and Indexing: An Old Idea with Timely Benefits

Published: 31 May 2015 Publication History

Abstract

With the social-media data explosion, near real-time queries, particularly those of a spatio-temporal nature, can be challenging. In this paper, we show how to efficiently answer queries that target recent data within very large data sets. We describe a solution that exploits a natural partitioning property that LSM-based indexes have for components, allowing us to filter out many components when answering queries. Our solution is generalizable to any LSM-based index structure, and can be applied not just on temporal fields (e.g., based on recency), but on any "time-correlated fields" such as Universally Unique Identifiers (UUIDs), user-provided integer ids, etc. We have implemented and experimentally evaluated the solution in the context of the AsterixDB system.

References

[1]
S. Alsubaiee. Spatial Indexing in the Era of Social Media. Ph.D. thesis, UC Irvine, 2014.
[2]
S. Alsubaiee et al. AsterixDB: A scalable, open source BDMS. VLDB, 2014.
[3]
S. Alsubaiee et al. Storage management in AsterixDB. VLDB, 2014.
[4]
R. Grover and M. J. Carey. Data ingestion in AsterixDB. EDBT, 2015.
[5]
C. Jermaine, E. Omiecinski, and W. G. Yee. The partitioned exponential file for database storage management. The VLDB Journal., 16(4), 2007.
[6]
P. Muth et al. Design, implementation, and performance of the LHAM log-structured history data access method. VLDB, 1998.
[7]
P. O'Neil, E. Cheng, D. Gawlick, and E. O'Neil. The log-structured merge-tree (LSM-tree). Acta Inf., 33(4), 1996.
[8]
R. Sears and R. Ramakrishnan. bLSM: a general purpose log structured merge tree. SIGMOD, 2012.

Cited By

View all
  • (2024)TrieKV: A High-Performance Key-Value Store Design With Memory as Its First-Class CitizenIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.347301335:12(2479-2496)Online publication date: Dec-2024
  • (2024)A survey of LSM-Tree based Indexes, Data Systems and KV-stores2024 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS)10.1109/SCEECS61402.2024.10482249(1-6)Online publication date: 24-Feb-2024
  • (2024)An update-intensive LSM-based R-tree indexThe VLDB Journal10.1007/s00778-024-00876-734:1Online publication date: 10-Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GeoRich'15: Second International ACM Workshop on Managing and Mining Enriched Geo-Spatial Data
May 2015
44 pages
ISBN:9781450336680
DOI:10.1145/2786006
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2015

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SIGMOD/PODS'15
Sponsor:
SIGMOD/PODS'15: International Conference on Management of Data
May 31 - June 4, 2015
VIC, Melbourne, Australia

Acceptance Rates

GeoRich'15 Paper Acceptance Rate 5 of 13 submissions, 38%;
Overall Acceptance Rate 25 of 50 submissions, 50%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)18
  • Downloads (Last 6 weeks)2
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)TrieKV: A High-Performance Key-Value Store Design With Memory as Its First-Class CitizenIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.347301335:12(2479-2496)Online publication date: Dec-2024
  • (2024)A survey of LSM-Tree based Indexes, Data Systems and KV-stores2024 IEEE International Students' Conference on Electrical, Electronics and Computer Science (SCEECS)10.1109/SCEECS61402.2024.10482249(1-6)Online publication date: 24-Feb-2024
  • (2024)An update-intensive LSM-based R-tree indexThe VLDB Journal10.1007/s00778-024-00876-734:1Online publication date: 10-Dec-2024
  • (2023)Perseid: A Secondary Indexing Mechanism for LSM-Based Storage SystemsACM Transactions on Storage10.1145/363328520:2(1-28)Online publication date: 17-Nov-2023
  • (2022)Accelerating range queries of primary and secondary indices for key-value separationProceedings of the 13th Symposium on Cloud Computing10.1145/3542929.3563479(226-239)Online publication date: 7-Nov-2022
  • (2022)Proteus: A Self-Designing Range FilterProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526167(1670-1684)Online publication date: 10-Jun-2022
  • (2022)RepKV: A Replicated Key-Value Store to Boost Multiple Indices for Key-Value Separation2022 IEEE 40th International Conference on Computer Design (ICCD)10.1109/ICCD56317.2022.00036(187-194)Online publication date: Oct-2022
  • (2022)Subscribing to big data at scaleDistributed and Parallel Databases10.1007/s10619-022-07406-w40:2-3(475-520)Online publication date: 7-Apr-2022
  • (2021)The LSM RUM-Tree: A Log Structured Merge R-Tree for Update-intensive Spatial Workloads2021 IEEE 37th International Conference on Data Engineering (ICDE)10.1109/ICDE51399.2021.00238(2285-2290)Online publication date: Apr-2021
  • (2020)An LSM-based tuple compaction framework for Apache AsterixDBProceedings of the VLDB Endowment10.14778/3397230.339723613:9(1388-1400)Online publication date: 26-Jun-2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media