Skip to main content
Springer Nature Link
Log in

Efficient Multi-version Storage Engine for Main Memory Data Store

  • Conference paper
  • First Online:
Web Information Systems Engineering – WISE 2017 (WISE 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10570))

Included in the following conference series:

  • 1632 Accesses

Abstract

Multi-version storage engine is the fundamental component of modern main memory data store using the popular multiple version concurrency control (MVCC). The straightforward implementation of storage engine is to use a linked list to store multiple versions of an object. A read operation has to traverse the list for the specified version, which incurs pointer chasing. An optimization method implemented in HyPer is to store the current version in the object header, which is friendly to read the latest snapshot of data. However, a read operation still needs one extra pointer chasing in memory when accessing an object being updated. In this paper, we propose an efficient multi-version storage (EMS), a new storage engine for main memory data store. EMS embeds two latest versions in each object header, so that it can avoid the overhead of traversal of version list, especially in the update-intensive scenario. We present an implementation mechanism of widely used snapshot isolation level over EMS. The experimental results demonstrate that EMS outperforms the exiting multi-version storage engine of well-known main memory data stores in terms of throughput without excessive memory consumption.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Similar content being viewed by others

References

  1. MemSQL (2017). http://www.memsql.com

  2. NuoDB (2017). http://www.nuodb.com

  3. PostgreSQL (2017). http://www.memsql.com

  4. Berenson, H., Bernstein, P., Gray, J., Melton, J., O’Neil, E., O’Neil, P.: A critique of ANSI SQL isolation levels. SIGMOD Rec. 24(2), 1–10 (1995)

    Article  Google Scholar 

  5. Bernstein, P.A., Das, S., Ding, B., et al.: Optimizing optimistic concurrency control for tree-structured, log-structured databases. In: SIGMOD, pp. 1295–1309 (2015)

    Google Scholar 

  6. Diaconu, C., Freedman, C., Ismert, E., et al.: Hekaton: SQL server’s memory-optimized OLTP engine. In: SIGMOD, pp. 1243–1254 (2013)

    Google Scholar 

  7. Grund, M., Krüger, J., Plattner, H., et al.: HYRISE: a main memory hybrid storage engine. Proc. VLDB Endow. 4(2), 105–116 (2010)

    Article  Google Scholar 

  8. Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. TODS 6(2), 213–226 (1981)

    Article  Google Scholar 

  9. Larson, P.-A., Blanas, S., Diaconu, C., et al.: High-performance concurrency control mechanisms for main-memory databases. Proc. VLDB Endow. 5(4), 298–309 (2011)

    Article  Google Scholar 

  10. Lee, J., Shin, H., Park, C.G., et al.: Hybrid garbage collection for multi-version concurrency control in SAP HANA. In: SIGMOD, pp. 1307–1318 (2016)

    Google Scholar 

  11. Lomet, D., Fekete, A., Wang, R., Ward, P.: Multi-version concurrency via timestamp range conflict management. In: ICDE, pp. 714–725 (2012)

    Google Scholar 

  12. Neumann, T., Mühlbauer, T., Kemper, A.: Fast serializable multi-version concurrency control for main-memory database systems. In: SIGMOD, pp. 677–689 (2015)

    Google Scholar 

  13. Pavlo, A., Aslett, M.: What’s really new with NewSQL? SIGMOD Rec. 45(2), 45–55 (2016)

    Article  Google Scholar 

  14. Ports, D.R.K., Grittner, K.: Serializable snapshot isolation in PostgreSQL. Proc. VLDB Endow. 5(12), 1850–1861 (2012)

    Article  Google Scholar 

  15. Sikka, V., Färber, F., Lehner, W., et al.: Efficient transaction processing in SAP HANA database: the end of a column store myth. In: SIGMOD, pp. 731–742 (2012)

    Google Scholar 

  16. Tu, S., Zheng, W., Kohler, E., Liskov, B., Madden, S.: Speedy transactions in multicore in-memory databases. In: SOSP, pp. 18–32 (2013)

    Google Scholar 

  17. Weisz, G., Melber, J., Wang, Y., et al.: A study of pointer-chasing performance on shared-memory processor-FPGA systems. In: FPGA, pp. 264–273 (2016)

    Google Scholar 

  18. Wu, Y., Arulraj, J., Lin, J., et al.: An empirical evaluation of in-memory multi-version concurrency control. Proc. VLDB Endow. 10(7), 781–792 (2017)

    Article  Google Scholar 

  19. Zheng, W., Tu, S., Kohler, E., Liskov, B.: Fast databases with fast durability and recovery through multicore parallelism. In: OSDI, pp. 465–477 (2014)

    Google Scholar 

Download references

Acknowledgments

This work is partially supported by National High-tech R&D Program (863 Program) under grant number 2015AA015307, National Science Foundation of China under grant numbers 61432006 and 61672232, and Guangxi Key Laboratory of Trusted Software (kx201602).

Author information

Authors and Affiliations

  1. Institute for Data Science and Engineering, East China Normal University, Shanghai, 200062, People’s Republic of China

    Jinwei Guo, Bing Xiao, Peng Cai, Weining Qian & Aoying Zhou

  2. Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin, 541004, People’s Republic of China

    Peng Cai

Authors
  1. Jinwei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weining Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aoying Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Cai .

Editor information

Editors and Affiliations

  1. University of Sydney, Darlington, NSW, Australia

    Athman Bouguettaya

  2. Zhejiang University, Hangzhou, China

    Yunjun Gao

  3. Institute of Computing for Physics and Technology, Protvino, Russia

    Andrey Klimenko

  4. Nanyang Technological University, Singapore, Singapore

    Lu Chen

  5. King Abdullah University of Science and Technology, Thuwal, Saudi Arabia

    Xiangliang Zhang

  6. Institute of Computing for Physics and Technology, Protvino, Russia

    Fedor Dzerzhinskiy

  7. Shanghai Jiao Tong University, Minhang Qu, China

    Weijia Jia

  8. Institute of Computing for Physics and Technology, Protvino, Russia

    Stanislav V. Klimenko

  9. City University of Hong Kong, Kowloon, Hong Kong

    Qing Li

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Guo, J., Xiao, B., Cai, P., Qian, W., Zhou, A. (2017). Efficient Multi-version Storage Engine for Main Memory Data Store. In: Bouguettaya, A., et al. Web Information Systems Engineering – WISE 2017. WISE 2017. Lecture Notes in Computer Science(), vol 10570. Springer, Cham. https://doi.org/10.1007/978-3-319-68786-5_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68786-5_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68785-8

  • Online ISBN: 978-3-319-68786-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics