Abstract
The Transaction Processing Performance Council (TPC) has a very successful history of disseminating objective and verifiable performance data to the industry. However, it lacks the ability to create new benchmarks in a timely fashion. In its initial years, the TPC defined benchmarks in about two years on average while recently this number increased to about eight years.
Although TPC benchmarks measure fundamental performance characteristics of transaction systems, which do not change quickly over time, new feature development, e.g. SQL language cannot be ignored.
This paper analyses the reasons for this increased development time and proposes ideas on how to improve benchmark development time by using a domain independent benchmark evolution model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Shanley, K.: Chief Operating Officer, Transaction Processing Performance Council: “History and Overview of the TPC”. http://www.tpc.org/information/about/history.asp
Poess, M., Floyd, C.: New TPC benchmarks for decision support and web commerce. ACM SIGMOD RECORD 29(4), 64–71 (2000)
Transaction Processing Performance Council Policies Version 5.17.http://www.tpc.org/information/about/documentation/spec/TPC_Policies_v5.17.pdf
TPC General Council Minutes. http://www.tpc.org/Members_Only/GenMtgMinutes/GenMtgMin.html
Stephens, J.M., Poess, M.: “MUDD: a multi-dimensional data generator”. In: WOSP, pp. 104–109 (2004)
Poess, M., Stephens, J.M.: Generating thousand benchmark queries in seconds”. In: VLDB, pp. 1045–1053 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Poess, M. (2021). Analysis of Benchmark Development Times in the Transaction Processing Performance Council and Ideas on How to Reduce It with a Domain Independent Benchmark Evolution Model. In: Nambiar, R., Poess, M. (eds) Performance Evaluation and Benchmarking. TPCTC 2020. Lecture Notes in Computer Science(), vol 12752. Springer, Cham. https://doi.org/10.1007/978-3-030-84924-5_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-84924-5_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84923-8
Online ISBN: 978-3-030-84924-5
eBook Packages: Computer ScienceComputer Science (R0)