Skip to main content
SpringerLink
Log in

Analysis and characterization of large‐scale Web server access patterns and performance

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

In this paper we develop a general methodology for characterizing the access patterns of Web server requests based on a time‐series analysis of finite collections of observed data from real systems. Our approach is used together with the access logs from the IBM Web site for the Olympic Games to demonstrate some of its advantages over previous methods and to construct a particular class of benchmarks for large‐scale heavily‐accessed Web server environments. We then apply an instance of this class of benchmarks to analyze aspects of large‐scale Web server performance, demonstrating some additional problems with methods commonly used to evaluate Web server performance at different request traffic intensities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Almeida, J. and P. Cao (1998), "Wisconsin Proxy Benchmark 1.0," http://www.cs.wisc.edu/”cao/wpb1.0.html.

  • Arlitt, M.F. and C.L. Williamson (1996), "Web Server Workload Characterization: The Search for Invariants," In Proceedings of the ACM Sigmetrics Conference on Measurement and Modeling of Computer Systems, pp. 126–137.

  • Arlitt, M.F. and C.L.Williamson (1997), "Internet Web Servers: Workload Characterization and Performance Implications," IEEE/ACM Transactions on Networking 5, 5, 631–645.

    Article  Google Scholar 

  • Asmussen, S. (1997), "Phase-Type Distributions and Related Point Processes: Fitting and Recent Advances," In Matrix-Analytic Methods in Stochastic Models, <nt>eds.</nt> S.R. Chakravarthy and A.S. Alfa, Marcel Dekker, pp. 137–149.

  • Banga, G., F. Douglis, and M. Rabinovich (1997), "Optimistic Deltas for WWW Latency Reduction," In Proceedings of the USENIX 1997 Technical Conference.

  • Banga, G. and P. Druschel (1997), "Measuring the Capacity of a Web Server," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 61–72.

  • Barford, P. and M.E. Crovella (1998), "Generating Representative Web Workloads for Network and Server Performance Evaluation," In Proceedings of the ACM Sigmetrics Conference on Measurement and Modeling of Computer Systems, pp. 151–160.

  • Basseville, M. and I. Nikiforov (1992), Detection of Abrupt Changes: Theory and Application, Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Bestavros, A. (1995), "Using Speculation to Reduce Server Load and Service Time on the WWW," In Proceedings of the 4th International Conference on Information and Knowledge Management.

  • Bolot, J. and P. Hoschka (1997), "Performance Engineering of the World Wide Web: Application to Dimensioning and Cache Design," World Wide Web Journal, 185–195.

  • Bowerman, B.L. and R.T. O'Connell (1987), Time Series Forecasting: Unified Concepts and Computer Implementation, Duxbury Press, Boston.

    Google Scholar 

  • Brockwell, P.J. and R.A. Davis (1987), Time Series: Theory and Methods, Springer, Berlin.

    Google Scholar 

  • Caceres, R., F. Douglis, A. Feldmann, G. Glass, and M. Rabinovich (1998), "Web Proxy Caching: The Devil is in the Details," In Proceedings of the Internet Server Performance Workshop, pp. 111–118.

  • Cao, P. and S. Irani (1997), "Cost-Aware WWW Proxy Caching Algorithms," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 193–206.

  • Challenger, J., P. Dantzig, and A. Iyengar (1998), "A Scalable and Highly Available System for Serving Dynamic Data at Frequently Accessed Web Sites," In Proceedings of SC '98.

  • Challenger, J., A. Iyengar, and P. Dantzig (1999), "A Scalable System for Consistently Caching Dynamic Web Data," In Proceedings of INFOCOM '99.

  • Chankhunthod, A., P.B. Danzig, C. Neerdaels, M.F. Schwartz, and K.J. Worrell (1996), "A Hierarchical Internet Object Cache," In Proceedings of the 1996 USENIX Technical Conference, pp. 153–163.

  • Courage, M. and S. Manley (1998), "An Evaluation of CGI Traffic and Its Effect on WWW Latency," http://www.eecs. harvard.edu/”vino/web/ hbench-web.

  • Crovella, M.E. and A. Bestavros (1997), "Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes," IEEE/ACM Transactions on Networking 5, 6, 835–845.

    Article  Google Scholar 

  • Cunha, C.R. and C.F.B. Jaccoud (1997), "Determining WWW User's Next Access and Its Application to Prefetching," In Proceedings of the International Symposium on Computers and Communications.

  • Dias, D., W. Kish, R. Mukherjee, and R. Tewari (1996), "A Scalable and Highly Available Web Server," In Proceedings of the 1996 IEEE Computer Conference (COMPCON).

  • Douglis, F., A. Feldmann, B. Krishnamurthy, and J. Mogul (1997a), "Rate of Change and other Metrics: a Live Study of the World Wide Web," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 147–158.

  • Douglis, F., A. Haro, and M. Rabinovich (1997b), "HPP: HTML Macro-Preprocessing to Support Dynamic Document Caching," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 83–94.

  • Duska, B.M., D. Marwood, and M.J. Feeley (1997), "The Measured Access Characteristics of World-Wide-Web Client Proxy Caches," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 23–36.

  • Feldmann, A. and W. Whitt (1998), "Fitting Mixtures of Exponentials to Long-Tail Distributions to Analyze Network Performance Models," Performance Evaluation 31, 245–279.

    Google Scholar 

  • Gadde, S., J. Chase, and M. Rabinovich (1998), "A Taste of Crispy Squid," In Proceedings of the Internet Server Performance Workshop, pp. 129–136.

  • Granger, C. and P. Newbold (1986), Forecasting Economic Time Series, Academic Press, New York.

    MATH  Google Scholar 

  • Gribble, S.D. and E.A. Brewer (1997), "System Design Issues for Internet Middleware Services: Deductions from a Large Client Trace," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 207–218.

  • Gwertzman, J. and M. Seltzer (1996), "World-Wide Web Cache Consistency," In Proceedings of the 1996 USENIX Technical Conference, pp. 141–151.

  • Hu, J., I. Pyarali, and D. Schmidt (1997), "Measuring the Impact of Event Dispatching and Concurrency Models on Web Server Performance Over High-speed Networks," In Proceedings of GLOBECOM '97, pp. 1924–1931.

  • Hunt, G., G. Goldszmidt, R. King, and R. Mukherjee (1998), "Network Dispatcher: A Connection Router for Scalable Internet Services," In Proceedings of the 7th International World Wide Web Conference.

  • Iyengar, A. and J. Challenger (1997), "Improving Web Server Performance by Caching Dynamic Data," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 49–60.

  • Iyengar, A., E. MacNair, and T. Nguyen (1997), "An Analysis of Web Server Performance," In Proceedings of GLOBECOM '97, pp. 1943–1947.

  • Iyengar, A.K., E.A. MacNair, M.S. Squillante, and L. Zhang (1998a), "A General Methodology for Characterizing Access Patterns and Analyzing Web Server Performance," In Proceedings of the International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 167–174.

  • Iyengar, A.K., M.S. Squillante, and L. Zhang (1998b), "Analysis and Characterization of Large-Scale Web Server Access Patterns and Performance," Technical Report RC 21328, IBM Research Division.

  • Jagerman, D.L. and B. Melamed (1992), "The Transition and Autocorrelation Structure of TES Processes Part I: General Theory," Stochastic Models 8, 2, 193–219.

    MATH  MathSciNet  Google Scholar 

  • Kendall, M. and J. Ord (1990), Time Series, Oxford University Press, Oxford.

    Google Scholar 

  • Kimbrel, T. and A.R. Karlin (1996), "Near-optimal parallel prefetching and caching," In Proceedings of the IEEE Symposium on Foundations of Computer Science.

  • Krishnamurthy, B. and C.E. Wills (1997), "Study of Piggyback Cache Validation for Proxy Caches in the World Wide Web," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 1–12.

  • Kroeger, T.M., D.D.E. Long, and J.C. Mogul (1997), "Exploring the Bounds of Web Latency Reduction from Caching and Prefetching," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 13–22.

  • Liu, Y.H., P. Dantzig, C.E.Wu, J. Challenger, and L.M. Ni (1996), "A Distributed Web Server and its Performance Analysis on Multiple Platforms," In Proceedings of the International Conference for Distributed Computing Systems.

  • Loon, T.S. and V. Bharghavan (1997), "Alleviating the Latency and Bandwidth Problems in WWW Browsing," In Proceedings of the Usenix Symposium on Internet Technologies and Systems, pp. 219–230.

  • Mogul, J., F. Douglis, A. Feldmann, and B. Krishnamurthy (1997), "Potential Benefits of Delta-Encoding and Data Compression for HTTP," In Proceedings of ACM SIGCOMM '97, pp. 181–194.

  • Mosberger, D. and T. Jin (1998), "httperf - A Tool for Measuring Web Server Performance," In Proceedings of the Internet Server Performance Workshop, pp. 59–67.

  • Mosedale, D., W. Foss, and R. McCool (1997), "Lessons Learned Administering Netscape's Internet Site," IEEE Internet Computing 1, 2, 28–35.

    Article  Google Scholar 

  • Nabeshima, M. (1997), "The Japan Cache Project: An Experiment on Domain Cache," In Sixth International World Wide Web Conference Proceedings.

  • Niclausse, N., Z. Liu, and P. Nain (1998), "A New Efficient Caching Policy for the World Wide Web," In Proceedings of the Internet Server Performance Workshop, pp. 119–128.

  • Padmanabhan, V.N. and J.C. Mogul (1996), "Using Predictive Prefetching to Improve World Wide Web Latency," Computer Communications Review 26, 22–36.

    Google Scholar 

  • Scheuermann, P., J. Shim, and R. Vingralek (1997), "A Case for Delay-Conscious Caching of Web Documents," In Sixth International World Wide Web Conference Proceedings.

  • SPECweb (1996), SPECweb96 Benchmark, System Performance Evaluation Cooperative (SPEC).

  • Squillante, M.S. (1998), "A Matrix-Analytic Approach to a General Class of G/G/cQueues," Technical report, IBM Research Division.

  • Squillante, M.S., D.D. Yao, and L. Zhang (1998), "Web Traffic Modeling and Web Server Performance Analysis," Technical report, IBM Research Division.

  • WebStone (1996), World Wide Web Server Benchmarking, Silicon Graphics, Inc.

  • Williams, S., M. Abrams, C.R. Standridge, G. Abdulla, and E.A. Fox (1996), "Removal Policies in Network Caches for World-Wide Web Documents," In Proceedings of SIGCOMM '96, pp. 293–305.

  • Wooster, R.P. and M. Abrams (1997), "Proxy Caching That Estimates Page Load Delays," In Sixth International World Wide Web Conference Proceedings.

  • Yeager, N.J. and R.E. McGrath (1996), Web Server Technology, Morgan Kaufmann Publishers, Inc., San Francisco, CA.

    MATH  Google Scholar 

  • Yoshida, A. (1997), "MOWS: Distributed Web and Cache Server in Java," In 6th International World Wide Web Conference Proceedings.

Download references

Author information

Authors and Affiliations

  1. IBM Research Division, Thomas J. Watson Research Center, Yorktown Heights, NY, 10598, USA

    Arun K. Iyengar, Mark S. Squillante & Li Zhang

Authors
  1. Arun K. Iyengar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark S. Squillante
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Iyengar, A.K., Squillante, M.S. & Zhang, L. Analysis and characterization of large‐scale Web server access patterns and performance. World Wide Web 2, 85–100 (1999). https://doi.org/10.1023/A:1019244621570

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1019244621570

Keywords

Search

Navigation