Sean Rhea
ABSTRACT
We present LittleTable, a relational database that Cisco Meraki has used since 2008 to store usage statistics, event logs, and other time-series data from our customers' devices.
LittleTable optimizes for time-series data by clustering tables in two dimensions. By partitioning rows by timestamp, it allows quick retrieval of recent measurements without imposing any penalty for retaining older history. By further sorting within each partition by a hierarchically-delineated key, LittleTable allows developers to optimize each table for the specific patterns with which they intend to access it.
LittleTable further optimizes for time-series data by capitalizing on the reduced consistency and durability needs of our applications, three of which we present here. In particular, our applications are single-writer and append-only. At most one process inserts a given type of data collected from a given device, and applications never update rows written in the past, simplifying both lock management and crash recovery. Our most recently written data is also recoverable, as it can generally be re-read from the devices themselves, allowing LittleTable to safely lose some amount of recently-written data in the event of a crash.
As a result of these optimizations, LittleTable is fast and efficient, even on a single processor and spinning disk. Querying an uncached table of 128-byte rows, it returns the first matching row in 31 ms, and it returns 500,000 rows/second thereafter, approximately 50% of the throughput of the disk itself. Today Meraki stores 320 TB of data across several hundred LittleTable servers system-wide.
- LevelDB. http://leveldb.org/.Google Scholar
- LZO real-time data compression library. http://www.oberhumer.com/opensource/lzo/.Google Scholar
- Round robin database tool. http://oss.oetiker.ch/rrdtool/.Google Scholar
- The virtual table mechanism of SQLite. https://sqlite.org/vtab.html.Google Scholar
- Welcome to Apache HBase. https://hbase.apache.org/.Google Scholar
- J. Baker, C. Bond, J. C. Corbett, J. Furman, A. Khorlin, J. Larson, J.-M. Leon, Y. Li, A. Lloyd, and V. Yushprakh. Megastore: Providing scalable, highly available storage for interactive services. In Proceedings of CIDR, 2011.Google Scholar
- N. Bronson, Z. Amsden, G. Cabrera, P. Chakka, P. Dimov, H. Ding, J. Ferris, A. Giardullo, S. Kulkarni, H. Li, M. Marchukov, D. Petrov, L. Puzar, Y. J. Song, and V. Venkataramani. TAO: Facebook rights distributed data store for the social graph. In Proceedings of USENIX ATC, 2013. Google ScholarDigital Library
- F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A distributed storage system for structured data. In Proceedings of OSDI, 2006. Google ScholarDigital Library
- J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, W. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig, Y. Saito, M. Szymaniak, C. Taylor, R. Wang, and D. Woodford. Spanner: Google's globally-distributed database. In Proceedings of OSDI, 2012. Google ScholarDigital Library
- P. Flajolet, É. Fusy, O. Gandouet, and F. Meunier. HyperLogLog: The analysis of a near-optimal cardinality estimation algorithm. In Proceedings of the International Conference on Analysis of Algorithms, 2007.Google ScholarCross Ref
- B. Hegerfors. Date-tiered compaction in Apache Cassandra. https://labs.spotify.com/2014/12/18/date-tiered-compaction/, Dec. 2014.Google Scholar
- C. Jermaine, E. Omiecinski, and W. G. Yee. The partitioned exponential file for database storage management. The VLDB Journal, 16(4):417--437, Oct. 2007. Google ScholarDigital Library
- C. Kolovson and M. Stonebraker. Indexing techniques for historical databases. In Proceedings of ICDE, 1989. Google ScholarDigital Library
- A. Lakshman and P. Malik. Cassandra: A decentralized structured storage system. SIGOPS Oper. Syst. Rev., 44(2):35--40, 2010. Google ScholarDigital Library
- D. Lomet and B. Salzberg. Access methods for multiversion data. In Proceedings of SIGMOD, 1989. Google ScholarDigital Library
- D. Lomet and B. Salzberg. The performance of a multiversion access method. In Proceedings of SIGMOD, 1990. Google ScholarDigital Library
- Y. Matsunobu. MyRocks: A space- and write-optimized MySQL database. https://code.facebook.com/posts/190251048047090/myrocks-a-space-and-write-optimized-mysql-database/, Aug. 2016.Google Scholar
- P. Muth, P. O'Neil, A. Pick, and G. Weikum. The LHAM log-structured history data access method. The VLDB Journal, 8(3--4):199--221, 2000. Google ScholarDigital Library
- P. O'Neil, E. Cheng, D. Gawlick, and E. O'Neil. The log-structured merge-tree (LSM-tree). Acta Inf., 33(4):351--385, 1996. Google ScholarDigital Library
- S. Papadopoulos, K. Datta, S. Madden, and T. Mattson. The TileDB array data storage manager. In Proceedings of VLDB, 2017.Google Scholar
- M. Rosenblum and J. K. Ousterhout. The design and implementation of a log-structured file system. ACM Trans. Comput. Syst., 10(1):26--52, 1992. Google ScholarDigital Library
- R. Sears and R. Ramakrishnan. bLSM: A general purpose log structured merge tree. In Proceedings of SIGMOD, 2012. Google ScholarDigital Library
- M. Seltzer, K. A. Smith, H. Balakrishnan, J. Chang, S. McMains, and V. Padmanabhan. File system logging versus clustering: A performance comparison. In Proceedings of the USENIX Technical Conference, 1995. Google ScholarDigital Library
- W. Tan, S. Tata, Y. Tang, and L. Fong. Diff-index: Differentiated index in distributed log-structured data stores. In Proceedings of EDBT, 2014.Google Scholar
- H. T. Vo, S. Wang, D. Agrawal, G. Chen, and B. C. Ooi. LogBase: A scalable log-structured database system in the cloud. Proceedings of the VLDB Endowment, 5(10):1004--1015, 2012. Google ScholarDigital Library
- T. Wolpe. MongoDB CTO: How our new WiredTiger storage engine will earn its stripes. http://www.zdnet.com/article/ mongodb-cto-how-our-new-wiredtiger-storage-engine-will-earn-its-stripes/, Nov. 2014.Google Scholar
Index Terms
- LittleTable: A Time-Series Database and Its Uses
Recommendations
Modular data storage with Anvil
SOSP '09: Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principlesDatabases have achieved orders-of-magnitude performance improvements by changing the layout of stored data -- for instance, by arranging data in columns or compressing it before storage. These improvements have been implemented in monolithic new engines,...
An Efficient NoSQL-Based Storage Schema for Large-Scale Time Series Data
In IoT (internet of things), most data from the connected devices change with time and have sampling intervals, which are called time-series data. It is challenging to design a time series storage model that can write massive time-series data in a short ...
Optimization of RocksDB for Redis on Flash
ICCDA '17: Proceedings of the International Conference on Compute and Data AnalysisRocksDB is a popular key-value store, optimized for fast storage. With Solid-State Drives (SSDs) becoming prevalent, RocksDB gained widespread adoption and is now common in production settings. Specifically, various software stacks embed RocksDB as a ...
Comments