Kian-Lee Tan
Skip Abstract Section
Abstract
The increase in the capacity of main memory coupled with the decrease in cost has fueled research in and development of in-memory databases. In recent years, the emergence of new hardware has further given rise to new challenges which have attracted a lot of attention from the research community. In particular, it is widely accepted that hardware solutions can provide promising alternatives for realizing the full potential of in-memory systems. Here, we argue that naive adoption of hardware solutions does not guarantee superior performance over software solutions, and identify problems in such hardware solutions that limit their performance. We also highlight the primary challenges faced by in-memory databases, and summarize their potential solutions, from both software and hardware perspectives
- A. Belay, G. Prekas, A. Klimovic, S. Grossman, C. Kozyrakis, and E. Bugnion. Ix: A protected dataplane operating system for high throughput and low latency. In OSDI '14, pages 49--65, 2014. Google Scholar
Digital Library
- Q. Cai, H. Zhang, G. Chen, B. C. Ooi, and K.-L. Tan. Memepic: Towards a database system architecture without system calls. Technical report, NUS, 2015.Google Scholar
- C. Cascaval, C. Blundell, M. Michael, H. W. Cain, P. Wu, S. Chiras, and S. Chatterjee. Software transactional memory: Why is it only a research toy? Queue, 6(5):40:46--40:58, Sept. 2008. Google ScholarDigital Library
- Chelsio. Roce at a crossroads. Technical report, Chelsio Communications Inc., 2014.Google Scholar
- S. Damaraju, V. George, S. Jahagirdar, T. Khondker, R. Milstrey, S. Sarkar, S. Siers, I. Stolero, and A. Subbiah. A 22nm ia multi-cpu and gpu system-on-chip. In ISSCC '12, pages 56--57, 2012.Google ScholarCross Ref
- J. DeBrabant, A. Joy, A. Pavlo, M. Stonebraker, S. Zdonik, and S. R. Dulloor. A prolegomenon on oltp database systems for non-volatile memory. In ADMS '14, pages 57--63, 2014.Google Scholar
- Z. Feng, E. Lo, B. Kao, and W. Xu. Byteslice: Pushing the envelop of main memory data processing with a new storage layout. In SIGMOD '15, 2015. Google ScholarDigital Library
- S. Harizopoulos, D. J. Abadi, S. Madden, and M. Stonebraker. Oltp through the looking glass, and what we found there. In SIGMOD '08, pages 981--992, 2008. Google ScholarDigital Library
- S. Jha, B. He, M. Lu, X. Cheng, and H. P. Huynh. Improving main memory hash joins on intel xeon phi processors: An experimental approach. In PVLDB '15, pages 642--653, 2015. Google ScholarDigital Library
- E. P. C. Jones, D. J. Abadi, and S. Madden. Low overhead concurrency control for partitioned main memory databases. In SIGMOD '10, pages 603--614, 2010. Google ScholarDigital Library
- A. Kalia, M. Kaminsky, and D. G. Andersen. Using rdma efficiently for key-value services. In SIGCOMM '14, pages 295--306, 2014. Google ScholarDigital Library
- A. Kemper and T. Neumann. Hyper: A hybrid oltp&olap main memory database system based on virtual memory snapshots. In ICDE '11, pages 195--206, 2011. Google ScholarDigital Library
- P.-A. Larson, S. Blanas, C. Diaconu, C. Freedman, J. M. Patel, and M. Zwilling. High-performanceconcurrency control mechanisms for main-memory databases. In PVLDB '11, pages 298--309, 2011. Google ScholarDigital Library
- J. Lee, Y. S. Kwon, F. Farber, M. Muehle, C. Lee, C. Bensberg, J. Y. Lee, A. H. Lee, and W. Lehner. Sap hana distributed in-memory database system: Transaction, session, and metadata management. In ICDE '13, pages 1165--1173, 2013. Google ScholarDigital Library
- S. Lee, M. Kim, G. Do, S. Kim, H. Lee, J. Sim, N. Park, S. Hong, Y. Jeon, K. Choi, et al. Programming disturbance and cell scaling in phase change memory: For up to 16nm based 4F2 cell. In VLSIT '10, pages 199--200, 2010.Google ScholarCross Ref
- V. Leis, A. Kemper, and T. Neumann. Exploiting hardware transactional memory in main-memory databases. In ICDE '14, pages 580--591, 2014.Google ScholarCross Ref
- F. Li, B. C. Ooi, M. T. Özsu, and S. Wu. Distributed data management using mapreduce. ACM Computing Surveys, 46(3):31:1--31:42, Jan. 2014. Google ScholarDigital Library
- H. Li, X. Wang, Z.-L. Ong, W.-F. Wong, Y. Zhang, P. Wang, and Y. Chen. Performance, power, and reliability tradeoffs of STT-RAM cell subject to architecture-level requirement. IEEE Transactions on Magnetics, 47(10):2356--2359, Oct. 2011.Google ScholarCross Ref
- D. Loghin, B. M. Tudor, H. Zhang, B. C. Ooi, and Y. M. Teo. A performance study of big data on small nodes. In PVLDB '15, 2015. Google ScholarDigital Library
- L. M. Maas, T. Kissinger, D. Habich, and W. Lehner. Buzzard: A numa-aware in-memory indexing system. In SIGMOD '13, pages 1285--1286, 2013. Google ScholarDigital Library
- N. Malviya, A. Weisberg, S. Madden, and M. Stonebraker. Rethinking main memory oltp recovery. In ICDE '14, pages 604--615, 2014.Google ScholarCross Ref
- C. Mitchell, Y. Geng, and J. Li. Using one-sided rdma reads to build a fast, cpu-efficient key-value store. In USENIX ATC '13, pages 103--114, 2013. Google ScholarDigital Library
- T. Neumann, T. Mühlbauer, and A. Kemper. Fast serializable multi-version concurrency control formain-memory database systems. In SIGMOD '15, 2015. Google ScholarDigital Library
- A. Pavlo, C. Curino, and S. Zdonik. Skew-aware automatic database partitioning in shared-nothing,parallel oltp systems. In SIGMOD '12, pages 61--72, 2012. Google ScholarDigital Library
- K. Ren, A. Thomson, and D. J. Abadi. Lightweight locking for main memory database systems. In PVLDB '13, pages 145--156, 2013. Google ScholarDigital Library
- L. Rizzo. Netmap: A novel framework for fast packet i/o. In USENIX ATC '12, pages 101--112, 2012. Google ScholarDigital Library
- S. M. Rumble, A. Kejriwal, and J. Ousterhout. Log-structured memory for dram-based storage. In FAST '14, pages 1--16, 2014. Google ScholarDigital Library
- S. Sanfilippo and P. Noordhuis. Redis. http://redis.io, 2009.Google Scholar
- S. Tu, W. Zheng, E. Kohler, B. Liskov, and S. Madden. Speedy transactions in multicore in-memory databases. In SOSP '13, pages 18--32, 2013. Google ScholarDigital Library
- T. Willhalm, N. Popovici, Y. Boshmaf, H. Plattner, A. Zeier, and J. Schaffner. Simd-scan: Ultra fast in-memory table scan using on-chip vector processing units. In PVLDB '09, pages 385--394, 2009. Google ScholarDigital Library
- C. Yao, D. Agrawal, P. Chang, G. Chen, B. C. Ooi, W.-F. Wong, and M. Zhang. Dgcc: A new dependency graph based concurrency control protocol for multicore database systems. ArXiv e-prints, Mar. 2015.Google Scholar
- X. Yu, G. Bezerra, A. Pavlo, S. Devadas, and M. Stonebraker. Staring into the abyss: An evaluation of concurrency control with one thousand cores. In PVLDB '15, pages 209--220, 2014. Google ScholarDigital Library
- H. Zhang, G. Chen, B. C. Ooi, K.-L. Tan, and M. Zhang. In-memory big data management and processing: A survey. TKDE, 27(7):1920--1947, July 2015.Google ScholarDigital Library
- H. Zhang, G. Chen, W.-F. Wong, B. C. Ooi, S. Wu, and Y. Xia. Anti-caching-based elastic data management for big data. In ICDE '15, pages 592--603, 2014.Google Scholar
- H. Zhang, B. M. Tudor, G. Chen, and B. C. Ooi. Efficient in-memory data management: An analysis. In PVLDB '14, pages 833--836, 2014. Google ScholarDigital Library
Index Terms
- In-memory Databases: Challenges and Opportunities From Software and Hardware Perspectives
Index terms have been assigned to the content through auto-classification.
Recommendations
Integrating reliable memory in databases
Recent results in the Rio project at the University of Michigan show that it is possible to create an area of main memory that is as safe as disk from operating system crashes. This paper explores how to integrate the reliable memory provided by the Rio ...
Write-once-memory-code phase change memory
DATE '14: Proceedings of the conference on Design, Automation & Test in EuropeThis paper describes a write-once-memory-code phase change memory (WOM-code PCM) architecture for next-generation non-volatile memory applications. Specifically, we address the long latency of the write operation in PCM --- attributed to PCM SET --- by ...
Enabling Hybrid PCM Memory System with Inherent Memory Management
RACS '16: Proceedings of the International Conference on Research in Adaptive and Convergent SystemsReplacing the traditional volatile main memory, e.g., DRAM, with a non-volatile phase change memory (PCM) has become a possible solution to reduce the energy consumption of computing systems. To further reduce the bit cost of PCM, the development trend ...
Comments