Article

FlashDB: dynamic self-tuning database for NAND flash

Authors:

Aman KansalAuthors Info & Claims

IPSN '07: Proceedings of the 6th international conference on Information processing in sensor networks

Pages 410 - 419

https://doi.org/10.1145/1236360.1236412

Published: 25 April 2007 Publication History

Abstract

FlashDB is a self-tuning database optimized for sensor networks using NAND flash storage. In practical systems flash is used in different packages such as on-board flash chips, compact flash cards, secure digital cards and related formats. Our experiments reveal non-trivial differences in their access costs. Furthermore, databases may be subject to different types of workloads. We show that existing databases for flash are not optimized for all types of flash devices or for all workloads and their performance is thus suboptimal in many practical systems. FlashDB uses a novel self-tuning index that dynamically adapts its storage structure to workload and underlying storage device. We formalize the self-tuning nature of an index as a two-state task system and propose a 3-competitive online algorithm that achieves the theoretical optimum. We also provide a framework to determine the optimal size of an index node that minimizes energy and latency for a given device. Finally, we propose optimizations to further improve the performance of our index. We prototype and compare different indexing schemes on multiple flash devices and workloads, and show that our indexing scheme outperforms existing schemes under all workloads and flash devices we consider.

References

[1]

AZAR, Y., FEIGE, U., AND NATH, S. On the work function algorithm for two state task systems. Tech. Rep. MSR-TR-2007-20, Microsoft Corporation, February 2007.

[2]

BLACK, D. L., AND SLEATOR, D. D. Competitive algorithms for replication and migration problems. Tech. Rep. CMU-CS-89-201, Carnegie Mellon University, 1989.

[3]

BORODIN, A., LINIAL, N., AND SAKS, M. An optimal online algorithm for metrical task systems. In ACM STOC (1987).

Digital Library

[4]

BURKE, J., ESTRIN, D., HANSEN, M., PARKER, A., RAMANATHAN, N., REDDY, S., AND SRIVASTAVA, M. B. Participatory sensing. In ACM Sensys Workshop on World-Sensor-Web (2006).

[5]

DAI, H., NEUFELD, M., AND HAN, R. ELF: an efficient log-structured flash file system for micro sensor nodes. In ACM SenSys (2004).

Digital Library

[6]

DESNOYERS, P., GANESAN, D., AND SHENOY, P. TSAR: A two tier sensor storage architecture using interval skip graphs. In ACM Sensys (2005).

Digital Library

[7]

DIAO, Y., GANESAN, D., MATHUR, G., AND SHENOY, P. Re-thinking data management for storage-centric sensor networks. In Third Biennial Conference on Innovative Data Systems Research (CIDR), Asilomar (January 2007).

[8]

GIROD, L., LUKAC, M., TRIFA, V., AND ESTRIN, D. The design and implementation of a self-calibrating distributed acoustic sensing platform. In ACM SenSys (2006).

Digital Library

[9]

GRAY, J., AND GRAEFE, G. The five-minute rule ten years later, and other computer storage rules of thumb. SIGMOD Rec. 26, 4 (1997), 63--68.

Digital Library

[10]

HULL, B., BYCHKOVSKY, V., ZHANG, Y., CHEN, K., GORACZKO, M., MIU, A., SHIH, E., BALAKRISHNAN, H., AND MADDEN, S. CarTel: A Distributed Mobile Sensor Computing System. In 4th ACM SenSys (2006).

Digital Library

[11]

INTEL. Intel mote 2. http://www.intel.com/research/downloads/imote overview.pdf.

[12]

KANSAL, A., XIAO, L., AND ZHAO, F. Relevance metrics for coverage extension using community collected cell-phone camera imagery. In ACM Sensys Workshop on World-Sensor-Web: Mobile Device Centric Sensor Networks and Applications (October 2006), pp. 12--16.

[13]

LYMBEROPOULOS, D., AND SAVVIDES, A. Xyz: A motion-enabled, power aware sensor node platform for distributed sensor network applications. In IPSN SPOTS (April 2005).

Digital Library

[14]

MATHUR, G., DESNOYERS, P., GANESAN, D., AND SHENOY, P. Capsule: An energy-optimized object storage system for memory-constrained sensor devices. In ACM SenSys (2006).

Digital Library

[15]

MICROSOFT. "sql server 2005 everywhere edition". http://www.microsoft.com/sql/ctp sqlserver2005everywhereedition.mspx.

[16]

NATH, S., AND KANSAL, A. Flashdb: Dynamic self-tuning database for nand flash. Tech. Rep. MSR-TR-2006-168, Microsoft Corporation, 2006.

[17]

PON, R., BATALIN, M., GORDON, J., KANSAL, A., LIU, D., SHIRACHI, L., KAISER, W., SUKHATME, G., AND SRIVASTAVA, M. Networked infomechanical systems: A mobile wireless sensor network platform. In IEEE/ACM IPSN-SPOTS (April 2005).

Digital Library

[18]

ROSENBLUM, M., AND OUSTERHOUT, J. K. The design and implementation of a log-structured file system. ACM Transactions on Computer Systems 10, 1 (1992).

Digital Library

[19]

SAMSUNG. Samsung K9K1G08R0B 128M x 8 bit NAND Flash Memeory.

[20]

SILBERSCHATZ, A., KORTH, H. F., AND SUDARSHAN, S. Database Systems Concepts. McGraw Hill, 2002.

Digital Library

[21]

WU, C.-H., CHANG, L.-P., AND KUO, T.-W. An efficient b-tree layer for flash-memory storage systems. In RTCSA (2003).

[22]

ZEINALIPOUR-YAZTI, D., LIN, S., KALOGERAKI, V., GUNOPULOS, D., AND NAJJAR, V. MicroHash: An efficient index structure for flash-based sensor devices. In USENIX FAST (2005).

Digital Library

[23]

ZHANG, P., SADLER, C. M., LYON, S. A., AND MARTONOSI, M. Hardware design experiences in zebranet. In ACM SenSys (2004).

Digital Library

Cited By

Wei YWang HJin P(2024)Optimizing the B+tree Index with Hotness Awareness and AdaptivityAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5581-3_29(356-367)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-981-97-5581-3_29
Lee CChae SMoon SKim KPark SNam B(2023)Improving Compaction in LSM Trees through ZNS Simple Copy: (Extended Abstract)2023 IEEE 12th Non-Volatile Memory Systems and Applications Symposium (NVMSA)10.1109/NVMSA58981.2023.00013(64-65)Online publication date: Aug-2023
https://doi.org/10.1109/NVMSA58981.2023.00013
Papon TAthanassoulis M(2023)ACEing the Bufferpool Management Paradigm for Modern Storage Devices2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00106(1326-1339)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00106
Show More Cited By

Index Terms

FlashDB: dynamic self-tuning database for NAND flash
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
  2. Information retrieval
    1. Document representation
    2. Search engine architectures and scalability
      1. Search engine indexing
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

μ-tree: an ordered index structure for NAND flash memory
EMSOFT '07: Proceedings of the 7th ACM & IEEE international conference on Embedded software

As NAND flash memory becomes increasingly popular as data storage for embedded systems, many file systems and database management systems are being built on it. They require an efficient index structure to locate a particular item quickly from a huge ...
Flash-optimized B+-tree

With the rapid increasing capacity of flash memory, flash-aware indexing techniques are highly desirable for flash devices. The unique features of flash memory, such as the erase-before-write constraint and the asymmetric read/write cost, severely ...
Tide-tree: A self-tuning indexing scheme for hybrid storage system

Main memory index is built with the assumption that the RAM is sufficiently large to hold data. Due to the volatility and high unit price of main memory, indices under secondary memory such as SSD and HDD are widely used. However, the I/O operation with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IPSN '07: Proceedings of the 6th international conference on Information processing in sensor networks

April 2007

592 pages

ISBN:9781595936387

DOI:10.1145/1236360

General Chair:
Tarek Abdelzaher
University of Illinois at Urbana-Champaign
,
Program Chairs:
Leo Guibas
Stanford University
,
Matt Welsh
Harvard University

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IPSN07

Sponsor:

IPSN07: The 6th International Conference on Information Processing in Sensor Networks

April 25 - 27, 2007

Massachusetts, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 143 of 593 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

141
Total Citations
View Citations
1,157
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wei YWang HJin P(2024)Optimizing the B+tree Index with Hotness Awareness and AdaptivityAdvanced Intelligent Computing Technology and Applications10.1007/978-981-97-5581-3_29(356-367)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-981-97-5581-3_29
Lee CChae SMoon SKim KPark SNam B(2023)Improving Compaction in LSM Trees through ZNS Simple Copy: (Extended Abstract)2023 IEEE 12th Non-Volatile Memory Systems and Applications Symposium (NVMSA)10.1109/NVMSA58981.2023.00013(64-65)Online publication date: Aug-2023
https://doi.org/10.1109/NVMSA58981.2023.00013
Papon TAthanassoulis M(2023)ACEing the Bufferpool Management Paradigm for Modern Storage Devices2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00106(1326-1339)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00106
Han H(2022)Design and Implementation of an Efficient Key-Value Storage Engine for Mobile Edge ComputingJournal of Digital Contents Society10.9728/dcs.2022.23.5.92123:5(921-927)Online publication date: 31-May-2022
https://doi.org/10.9728/dcs.2022.23.5.921
Hu DChen ZChe WSun JChen HIves ZBonifati AEl Abbadi A(2022)Halo: A Hybrid PMem-DRAM Persistent Hash Index with Fast RecoveryProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517884(1049-1063)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517884
Shedge SSharma NAgarwal AAbouzour MAluc G(2022)An Extended SSD-Based Cache for Efficient Object Store Access in SAP IQ2022 IEEE 38th International Conference on Data Engineering (ICDE)10.1109/ICDE53745.2022.00185(1861-1873)Online publication date: May-2022
https://doi.org/10.1109/ICDE53745.2022.00185
Papon TAthanassoulis M(2021)A Parametric I/O Model for Modern Storage DevicesProceedings of the 17th International Workshop on Data Management on New Hardware10.1145/3465998.3466003(1-11)Online publication date: 20-Jun-2021
https://dl.acm.org/doi/10.1145/3465998.3466003
Li CZhuang HWang QWang CZhou X(2021)LKSM: Light Weight Key-Value Store for Efficient Application Services on Local Distributed Mobile DevicesIEEE Transactions on Services Computing10.1109/TSC.2018.285268314:4(1026-1039)Online publication date: 1-Jul-2021
https://doi.org/10.1109/TSC.2018.2852683
Mustafa FShaikh M(2021)Aspects of Security for Accelerating Artificial Intelligence inside Internet of Things Centric Distributed Storage Network2021 6th International Multi-Topic ICT Conference (IMTIC)10.1109/IMTIC53841.2021.9719866(1-9)Online publication date: 10-Nov-2021
https://doi.org/10.1109/IMTIC53841.2021.9719866
Bonnet PBouganim LKoltsidas IViglas S(2020)System co-design and data management for flash devicesProceedings of the VLDB Endowment10.14778/3402755.34028074:12(1504-1505)Online publication date: 3-Jun-2020
https://dl.acm.org/doi/10.14778/3402755.3402807
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten