Abstract
Recently, with the widespread use of large-scale sensor network, time-series data is vastly generated and requires to be processed. Those traditional databases, however, show their limitations in storage when handling such a large stream data. Besides, the actual dependability of databases are also difficult to be guaranteed. In this paper, we present FluteDB, an efficient and dependable time-series database storage engine, which is composed of multiple time-series enhanced sub-modules. The validations of all sub-modules have demonstrated that our improved strategies significantly outperform the existing methods in real time-series environment. Meanwhile, the complete FluteDB utilizes various measures to guarantee its dependability and achieves a higher overall storage efficiency than the state-of-the-art time-series databases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Write rate is a quantitative measurement of the storage ability.
- 2.
Though the interval of data sampling is not specified, it will be limited in a fixed range according its own streaming and large-scale characteristics.
- 3.
The size of slice is determined by the average size of delta of the deltas.
- 4.
Our method, which is a partial sequential operation strategy, is different from LSM Tree [8].
- 5.
An ability to respond the possible node or Information Data Center (IDC) failure.
References
TimescaleDB: SQL made scalable for time-series data. http://www.timescale.com/papers/timescaledb.pdf
Storage Engine of InfluxData. https://docs.influxdata.com/influxdb/v1.2/concepts/storage_engine/
The world’s most popular open source database. https://www.mysql.com/
The world’s most advanced open source database. https://www.postgresql.org/
An open source in-memory data structure store. https://redis.io/
Pelkonen, T., Franklin, S., Teller, J., Huang, Q., Cavallaro, P., Meza, J., Veeraraghavan, K.: Gorilla: a fast, scalable, in-memory time series database. PVLDB 8(12), 1816–1827 (2015)
Rhea, S., Wang, E., Wong, E., Atkins, E., Storer, N.: LittleTable: a time-series database and its uses. In: Proceedings of SIGMOD, pp. 125–138. ACM Press, Chicago (2017)
Sears, R., Ramakrishnan, R.: bLSM: a general purpose log structured merge tree. In: Proceedings of SIGMOD, pp. 217–228. ACM Press, Scottsdale (2012)
Cai, Y., Tong, H., Fan, W., Ji, P., He, Q.: Facets: fast comprehensive mining of coevolving high-order time series. In: Proceedings of the 21th SIGKDD, pp. 79–88. ACM Press, Sydney (2015)
Papadopoulos, S., Datta, k., Madden, S., Mattson, T.: The TileDB array data storage manager. In: Proceedings of VLDB, pp. 349–360. Springer Press (2017)
Jermaine, C., Omiecinski, E., Yee, W.G.: The partitioned exponential file for database storage management. The VLDB J. 16(4), 417–437 (2007)
Eamonn, J.K., Kaushik, C., Sharad, M., Michael, J.P.: Locally adaptive dimensionality reduction for indexing large time series databases. In: Proceedings of SIGMOD, pp. 151–162. ACM Press, California (2001)
Bassiouni, M.A.: Data compression in scientific and statistical databases. IEEE Trans. Softw. Eng. SE–11(10), 1047–1058 (2006)
Podlipnig, S., Böszörmenyi, L.: A survey of web cache replacement strategies. ACM Comput. Surv. 35(4), 374–398 (2003)
Acknowledgments
This work is supported by NSFC program (No. 61472022, 61421003), SKLSDE-2016ZX-11, and the Beijing Advanced Innovation Center for Big Data and Brain Computing.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Li, C., Li, J., Si, J., Zhang, Y. (2017). FluteDB: An Efficient and Dependable Time-Series Database Storage Engine. In: Wang, G., Atiquzzaman, M., Yan, Z., Choo, KK. (eds) Security, Privacy, and Anonymity in Computation, Communication, and Storage. SpaCCS 2017. Lecture Notes in Computer Science(), vol 10658. Springer, Cham. https://doi.org/10.1007/978-3-319-72395-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-72395-2_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-72394-5
Online ISBN: 978-3-319-72395-2
eBook Packages: Computer ScienceComputer Science (R0)