Abstract
In this paper, we present the Wise toolkit for microservice-based system performance experiments. Wise comprises a microservice-based application benchmark with controllable workload generation; milliScope, a set of system resource and event monitoring tools; and WED-Make, a workflow language and code generation tool for the construction and execution of system experiments with automatic provenance collection. We also show a running example reproducing the experimental verification of the millibottleneck theory of performance bugs to illustrate how we have used Wise for the performance study of microservice-based benchmark applications in the cloud.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Apache Thrift. https://thrift.apache.org/. Accessed 05 Mar 2020
CloudLab Scientific Cloud Infrastructure. https://cloudlab.us/. Accessed 05 Mar 2020
Collectl Performance Monitoring Tool. http://collectl.sourceforge.net/. Accessed 05 Mar 2020
Davidson, S.B., Freire, J.: Provenance and scientific workflows: challenges and opportunities. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1345–1350 (2008)
Dean, J.: The tail at scale. Commun. ACM 56(2), 74–80 (2013)
DeCandia, G., et al.: Dynamo: amazon’s highly available key-value store. ACM SIGOPS Operating Syst. Rev. 41(6), 205–220 (2007)
Ferreira, J.E., et al.: Transactional recovery support for robust exception handling in business process services. In: 2012 IEEE 19th International Conference on Web Services, pp. 303–310. IEEE (2012)
Flask. https://palletsprojects.com/p/flask/. Accessed 05 Mar 2020
Gan, Y., et al.: An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 3–18 (2019)
Gan, Y., et al.: Seer: leveraging big data to navigate the complexity of performance debugging in cloud microservices. In: Proceedings of the Twenty- Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 19–33 (2019)
Gregg, B.: Systems Performance: Enterprise and the Cloud. Pearson Education (2013)
Jimenez, I., et al.: Popper: making reproducible systems performance evaluation practical. In: UC Santa Cruz School of Engineering, Technical report UCSC-SOE-16-10 (2016)
Jung, G., Pu, C., Swint, G.: Mulini: an automated staging framework for QoS of distributed multi-tier applications. In: Proceedings of the 2007 Workshop on Automating Service Quality: Held at the International Conference on Automated Software Engineering (ASE), pp. 10–15. ACM (2007)
Kohavi, R., Henne, R.M., Sommerfield, D.: Practical guide to controlled experiments on the web: listen to your customers not to the hippo. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 959–967 (2007)
Kohavi, R.: Online experiments: lessons learned. Computer 40(9), 103–105 (2007)
Lai, C.A., et al.: milliScope: a fine-grained monitoring framework for performance debugging of n-tier Web services. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 92–102. IEEE (2017)
Lima, R.A., Kimball, J., Ferreira, J.E., Pu, C.: Systematic construction, execution, and reproduction of complex performance benchmarks. In: Da Silva, D., Wang, Q., Zhang, L.-J. (eds.) CLOUD 2019. LNCS, vol. 11513, pp. 26–37. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23502-4_3
Padilha, B., Roberto, R.L., Schwerz, A.L., Pu, C., Ferreira, J.E.: WED-SQL: an intermediate declarative language for PAIS execution. In: Jin, H., Wang, Q., Zhang, L.-J. (eds.) ICWS 2018. LNCS, vol. 10966, pp. 407–421. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94289-6_26
PostgreSQL. https://www.postgresql.org/. Accessed 05 Mar 2020
Pu, C., et al.: The millibottleneck theory of performance bugs, and its experimental verification. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 1919–1926. IEEE (2017)
RUBBoS Benchmark. http://jmob.ow2.org/rubbos.html. Accessed 05 Mar 2020
Shan, H., Wang, Q., Pu, C.: Tail attacks on web applications. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1725–1739 (2017)
Sigelman, B.H., et al.: Dapper, a large-scale distributed systems tracing infrastructure (2010)
sysstat: Performance Monitoring Tools for Linux. https://github.com/ sysstat/sysstat. Accessed 05 Mar 2020
The Apache HTTP Server Project. https://httpd.apache.org/. Accessed 05 Mar 2020
Wang, Q., et al.: Lightning in the cloud: A study of very short bottlenecks on n-tier web application performance. In: Proceedings of USENIX Conference on Timely Results in Operating Systems (2014)
Zhang, S., et al.: Tail amplification in n-tier systems: a study of transient cross-resource contention attacks. In: 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 1527–1538. IEEE.(2019)
Zhou, X., et al.: Poster: benchmarking microservice systems for software engineering research. In: 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion), pp. 323–324. IEEE (2018)
Acknowledgements
This research has been partially funded by National Science Foundation by CISEís SAVI/RCN (1402266, 1550379), CNS (1421561), CRISP (1541074), SaTC (1564097) programs, an REU supplement (1545173), and gifts, grants, or contracts from Fujitsu, HP, Intel, and Georgia Tech Foundation through the John P. Imlay, Jr. Chair endowment. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or other funding agencies and companies mentioned above.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lima, R.A., Kimball, J., Ferreira, J.E., Pu, C. (2020). Wise Toolkit: Enabling Microservice-Based System Performance Experiments. In: Zhang, Q., Wang, Y., Zhang, LJ. (eds) Cloud Computing – CLOUD 2020. CLOUD 2020. Lecture Notes in Computer Science(), vol 12403. Springer, Cham. https://doi.org/10.1007/978-3-030-59635-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-59635-4_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59634-7
Online ISBN: 978-3-030-59635-4
eBook Packages: Computer ScienceComputer Science (R0)