Abstract
The assessment of the dependability properties of a system (dependability benchmarking) is a critical step when choosing among similar components/products. This paper presents a proposal for the benchmarking of the dependability properties of web-servers. Our benchmark is composed of the three key components: measures, workload, and faultload. We use the SPECWeb99 benchmark as starting point, adopting the workload and performance measures from this performance benchmark, and we added the faultload and new measures related to dependability. We illustrate the use of the proposed benchmark through a case-study involving two widely used web servers (Apache and Abyss) running on top of three different operating systems. The faultloads used encompass software faults, hardware faults and network faults. We show that by using the proposed dependability benchmark it is possible to observe clear differences regarding dependability properties of the web-servers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Trivedi, K.S., Haverkort, B., Rindos, A., Mainkar, V.: Methods and Tools for Reliability and Performability: Problems and Perspectives. In: Haring, G., Kotsis, G. (eds.) TOOLS 1994. LNCS, vol. 794. Springer, Heidelberg (1994)
Jenn, E., Arlat, J., Rimén, M., Ohlsson, J., Karlsson, J.: Fault Injection into VHDL Models: The MEFISTO Tool. In: Randell, B., Laprie, J.-C., Kopetz, H., Littlewood, B. (eds.) Predictably Dependable Computing Systems. Springer, Berlin (1995)
Gray, J. (ed.): The Benchmark Handbook for Database and Transaction Processing Systems, p. 592. Morgan Kaufmann Publishers, San Francisco (1993)
Arlat, J., Costes, A., Crouzet, Y., Laprie, J.-C., Powell, D.: Fault Injection and Dependability Evaluation of Fault-Tolerant Systems. IEEE Trans. on Comp. 42(8) (1993)
Koopman, P., De Vale, J.: Comparing the Robustness of POSIX Operating Systems. In: Proc. 29th Int. Symp. Fault-Tolerant Computing (FTCS 29), Madison, USA (1999)
Koopman, P., Madeira, H.: Dependability Benchmarking & Prediction: A Grand Challenge Technology Problem. In: 1st IEEE Int. Workshop on Real-Time Mission-Critical Systems: Grand Challenge Problems; Phoenix, Arizona, USA, November 30 (1999)
Vieira, M., Madeira, H.: A Dependability Benchmark for OLTP Application Environments. In: 29th Int. Conf. on Very Large Data Bases (VLDB 2003), Berlin, Germany (2003)
Buchacker, K., Tschaeche, O.: TPC Benchmark-c version 5.2 Dependability Benchmark Extensions (2003), http://www.faumachine.org/papers/tpcc-depend.pdf
Kalakech, A., Kanoun, K., Crouzet, Y., Arlat, A.: Benchmarking the Dependability of Windows NT, 2000 and XP. In: Proc. Int. Conf. on Dependable Systems and Networks (DSN 2004), Florence, Italy. IEEE CS Press, Los Alamitos (2004)
Durães, J., Madeira, H.: Characterization of Operating Systems Behaviour in the Presence of Faulty Drivers Through Software Fault Emulation. In: Proc. 2002 Pacific Rim Int. Symp. on Dependable Computing (PRDC 2002), Tsukuba, Japan (2002)
Brown, L., Chung, W., Kakes, C., Ling, D.A.: Dependability Benchmarking of Human-Assisted Recovery Processes. In: Dependable Computing and Communications, DSN 2004, Florence, Italy (June 2004)
Wilson, D., Murphy, B., Spainhower, L.: Progress on Deining Standardized Classes of Computing the Dependability of Computer Systems. In: Proc. DSN 2002 Workshop on Dependability Benchmarking, Washington, D.C., USA, p. F1-5 (2002)
Zhu, J., Mauro, J., Pramanick, I.: R3 - A Framwork for Availability Benchmarking. In: Proc. Int. Conf. on Dependable Systems and Networks (DSN 2003), San Francisco (2003)
Ji, J., Zhu, J.: Robustness Benchmarking for Hardware Maintenance Events. In: Proc. Int. Conf. on Dependable Systems and Networks (DSN 2003), San Francisco, CA, USA, pp. 115–122. IEEE CS Press, Los Alamitos (2003)
Mauro, J., Zhu, J., Pramanick, I.: The System Recovery Benchmark. In: Proc. 2004 Pacific Rim Int. Symp. on Dependable Computing, Papeete, Polynesia. IEEE CS Press, Los Alamitos (2004)
Lightstone, S., Hellerstein, J., Tetzlaff, W., Janson, P., Lassettre, E., Norton, C., Rajaraman, B., Spainhower, L.: Towards Benchmarking Autonomic Computing Maturity. In: 1st IEEE Conf. on Industrial Automatics (INDIN-2003), Banff, Canada (August 2003)
SPEC - Standard Performance Evaluation Corporation, SPECweb99 Release 1.02 (Design Document) (July 2000), http://www.spec.org/web99/
Durães, J., Madeira, H.: Definition of Software Fault Emulation Operators: a Field Data Study. In: Proc. of the International Conference on Dependable Systems and Networks, DSN 2003, San Francisco, CA, June 22-25 (2003)
Durães, J., Madeira, H.: Generic Faultloads Based on Software Faults for Dependability Benchmarking. In: Accepted for publication at the Int. Conf. on Dependable Systems and Networks, Dependable Computing and Comm., DSN 2004, Florence, Italy (June 2004)
Christmansson, J., Chillarege, R.: Generation of an Error Set that Emulates Software Faults. In: Proc. of the 26th IEEE Fault Tolerant Computing Symposium, FTCS-26, Sendai, Japan, June 1996, pp. 304–313 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Durães, J., Vieira, M., Madeira, H. (2004). Dependability Benchmarking of Web-Servers. In: Heisel, M., Liggesmeyer, P., Wittmann, S. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2004. Lecture Notes in Computer Science, vol 3219. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30138-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-540-30138-7_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23176-9
Online ISBN: 978-3-540-30138-7
eBook Packages: Springer Book Archive