Skip to main content
SpringerLink
Log in

Conceptual Modeling on Tencent’s Distributed Database Systems

  • Conference paper
  • First Online:
Conceptual Modeling (ER 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11157))

Included in the following conference series:

Abstract

Tencent is the largest Internet service provider in China. Typical services include WeChat, games, payment, cloud storage and computing. Tencent serves billions of users and millions of enterprises, and some services like WeChat, are required to have the ability to handle more than 208,000 TPS requests at peak time. To do this, Tencent has built an elastic and scalable database service system, namely TDSQL, which can efficiently support their ever-growing service requests. TDSQL is deployed and runs on top of more than ten thousands of compute nodes. In this paper, we present the main challenges that we have encountered, and give our practice of conceptual modeling on TDSQL. First, failures of compute nodes often occur in an X86-based large-scale distributed system architecture. To address this issue, we introduce a fault tolerance model to guarantee the high availability of the services. Second, Tencent serves a huge number of requests, while different types of requests require different storage and compute resources. To improve the resource utilization, we propose a resource scheduling model that enables TDSQL to serve the requests elastically. Third, TDSQL provides a hybrid data modeling to support various data models, and develops DBaaS services to serve 100,000 + DB instances. Finally, we present how to fast develop applications in terms of conceptual modeling on top of TDSQL.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.tpc.org/tpcc.

  2. 2.

    https://github.com/akopytov/sysbench.

References

  1. 3PC. https://en.wikipedia.org/wiki/Three-phase_commit_protocol

  2. Tencent Distributed Database System(TDSQL). http://tdsql.org

  3. Aken, D.V., Pavlo, A., Gordon, G.J., Zhang, B.: Automatic database management system tuning through large-scale machine learning. In: SIGMOD Conference, pp. 1009–1024. ACM (2017)

    Google Scholar 

  4. Batra, R.K., Rusinkiewicz, M., Georgakopoulos, D.: A decentralized deadlock-free concurrency control method for multidatabase transactions. In: ICDCS, pp. 72–79. IEEE Computer Society (1992)

    Google Scholar 

  5. Gray, J.N.: Notes on data base operating systems. In: Bayer, R., Graham, R.M., Seegmüller, G. (eds.) Operating Systems. LNCS, vol. 60, pp. 393–481. Springer, Heidelberg (1978). https://doi.org/10.1007/3-540-08755-9_9

    Chapter  Google Scholar 

  6. Gray, J., Lamport, L.: Consensus on transaction commit. ACM Trans. Database Syst. 31(1), 133–160 (2006)

    Article  Google Scholar 

  7. Li, H., Yi Feng, P.F.: The Art of Database Transaction Processiong: Transaction Management and Concurrency Control. China Machine Press, Beijing (2017)

    Google Scholar 

  8. Haller, K., Schuldt, H.: Towards a decentralized implementation of transaction management. In: Grundlagen von Datenbanken, pp. 57–61. Fakultät für Informatik, Universität Magdeburg (2003)

    Google Scholar 

  9. Hwang, B., Son, S.H.: Decentralized transaction management in multidatabase systems. In: COMPSAC, pp. 192–198. IEEE Computer Society (1996)

    Google Scholar 

  10. Kang, I.E., Keefe, T.F.: Supporting reliable and atomic transaction management in multidatabase systems. In: ICDCS, pp. 457–464. IEEE Computer Society (1993)

    Google Scholar 

  11. Kulkarni, S.S., Demirbas, M., Madappa, D., Avva, B., Leone, M.: Logical physical clocks. In: Aguilera, M.K., Querzoni, L., Shapiro, M. (eds.) OPODIS 2014. LNCS, vol. 8878, pp. 17–32. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-14472-6_2

    Chapter  Google Scholar 

  12. Lampson, B., Sturgis, H.E.: Crash recovery in a distributed data storage system (1979)

    Google Scholar 

  13. Lomet, D.B., Fekete, A., Wang, R., Ward, P.: Multi-version concurrency via timestamp range conflict management. In: ICDE, pp. 714–725. IEEE Computer Society (2012)

    Google Scholar 

  14. Pang, G.: Scalable Transactions for Scalable Distributed Database Systems. Ph.D. thesis, University of California, Berkeley, USA (2015)

    Google Scholar 

  15. Skeen, D., Stonebraker, M.: A formal model of crash recovery in a distributed system. IEEE Trans. Softw. Eng. SE-9(3), 219–228 (1983)

    Article  Google Scholar 

  16. Skeen, D., Stonebraker, M.: A formal model of crash recovery in a distributed system. IEEE Trans. Softw. Eng. 9(3), 219–228 (1983)

    Article  Google Scholar 

  17. Veijalainen, J., Wolski, A.: Prepare and commit certification for decentralized transaction management in rigorous heterogeneous multidatabases. In: ICDE, pp. 470–479. IEEE Computer Society (1992)

    Google Scholar 

  18. Wang, W., Zhang, M., Chen, G., Jagadish, H.V., Ooi, B.C., Tan, K.: Database meets deep learning: challenges and opportunities. SIGMOD Record 45(2), 17–22 (2016)

    Article  Google Scholar 

  19. Yu, X., Pavlo, A., Sánchez, D., Devadas, S.: Tictoc: time traveling optimistic concurrency control. In: SIGMOD Conference, pp. 1629–1642. ACM (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tencent Inc., Shenzhen, China

    Anqun Pan, Xiaoyu Wang & Haixiang Li

Authors
  1. Anqun Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haixiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anqun Pan .

Editor information

Editors and Affiliations

  1. Lucentia, University of Alicante, Alicante, Spain

    Juan C. Trujillo

  2. Miami University, Oxford, OH, USA

    Karen C. Davis

  3. Renmin University of China, Beijing, China

    Xiaoyong Du

  4. Northwestern Polytechnical University, Xian, China

    Zhanhuai Li

  5. Department of Computer Science, National University of Singapore, Singapore, Singapore

    Tok Wang Ling

  6. Department of Computer Science and Technology, Tsinghua University, Beijing, Beijing, China

    Guoliang Li

  7. National University of Singapore, Singapore, Singapore

    Mong Li Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pan, A., Wang, X., Li, H. (2018). Conceptual Modeling on Tencent’s Distributed Database Systems. In: Trujillo, J., et al. Conceptual Modeling. ER 2018. Lecture Notes in Computer Science(), vol 11157. Springer, Cham. https://doi.org/10.1007/978-3-030-00847-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00847-5_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00846-8

  • Online ISBN: 978-3-030-00847-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation