Abstract
Replication of information across a server cluster provides a promising way to support popular Web sites. However, a Web‐server cluster requires some mechanism for the scheduling of requests to the most available server. One common approach is to use the cluster Domain Name System (DNS) as a centralized dispatcher. The main problem is that WWW address caching mechanisms (although reducing network traffic) only let this DNS dispatcher control a very small fraction of the requests reaching the Web‐server cluster. The non‐uniformity of the load from different client domains, and the high variability of real Web workload introduce additional degrees of complexity to the load balancing issue. These characteristics make existing scheduling algorithms for traditional distributed systems not applicable to control the load of Web‐server clusters and motivate the research on entirely new DNS policies that require some system state information. We analyze various DNS dispatching policies under realistic situations where state information needs to be estimated with low computation and communication overhead so as to be applicable to a Web cluster architecture. In a model of realistic scenarios for the Web cluster, a large set of simulation experiments shows that, by incorporating the proposed state estimators into the dispatching policies, the effectiveness of the DNS scheduling algorithms can improve substantially, in particular if compared to the results of DNS algorithms not using adequate state information.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, E., D. Patterson, and E. Brewer (1996), “The Magicrouter, An Application of Fast Packet Interposing,” http://www.cs.berkeley.edu/_eanders/projects/ magicrouter/osdi96-mr-submission.ps.
Andresen, D., T. Yang, V. Holmedahl, and O.H. Ibarra (1996), “SWEB: Toward a Scalable World Wide Web Server on Multicomputers,” In Proceedings of the 10th International Symposium on Parallel Processing (IPPS'96), Honolulu, HI, pp. 850–856.
Arlitt, M.F. and C.L. Williamson (1997), "Web Server Workload Characterization: The Search for Invariants, IEEE/ACM Transactions on Networking 5, 5, 631–645.
Barford, P. and M. Crovella (1998), “Generating Representative Web Workloads for Network and Server Performance Evaluation,” In Proceedings of Performance '98/ACM Sigmetrics '98, ACM Press, Madison, WI, pp. 151–160.
Bestavros, A. (1997), “WWW Traffic Reduction and Load Balancing Through Server-Based Caching,” IEEE Concurrency 5, 1, 56–67.
Box, G.E.P., G.M. Jenkins, G.C. Reinsel, and G. Jenkins (1994), Time Series Analysis: Forecasting and Control, Third Edition, Prentice-Hall, Englewood Cliffs, NJ.
Brau, H.W. and K.C. Claffy (1995), "Web Traffic Characterization: An Assessment of the Impact of Caching Documents from NCSA's Web Server, Computer Networks and ISDN Systems 28, 37–51.
Cisco (1997a), “Cisco's DistributedDirector,” http://www.cisco.com/warp/public/751/distdir/.
Cisco (1997b), “Cisco's LocalDirector,” http://www.cisco.com/warp/public/751/lodir/.
Colajanni, M., P.S. Yu, and V. Cardellini (1998a), “Dynamic Load Balancing in Geographically Distributed Heterogeneous Web-Servers,” In Proceedings of the IEEE 18th International Conference on Distributed Computing Systems (ICDCS'98), IEEE Computer Society Press, Amsterdam, The Netherlands, pp. 295–302.
Colajanni, M., P.S. Yu, and D.M. Dias (1998b), “Analysis of Task Assignment Policies in Scalable Distributed Web-Server System,” IEEE Transaction on Parallel and Distributed Systems 9, 6, 585–600.
Crovella, M. and A. Bestavros (1997), “Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes,” IEEE/ACM Transactions on Networking 5, 6, 835–846.
Cuhna, C., A. Bestavros, and M. Crovella (1995), “Characteristics of WWW Client-Based Traces,” Technical Report BUCS-95-010, Department of Computer Science, Boston University, Boston, MA.
Deng, S. (1996), “Empirical Model of WWW Document Arrivals at Access Link,” In Proceedings of the IEEE International Conference on Communication (ICC'96), IEEE Computer Society Press, Dallas, TX, pp. 1797–1802.
Dias, D. M., W. Kish, R. Mukherjee, and R. Tewari (1996), “A Scalable and Highly Available Web Server,” In Proceedings of the 41st IEEE Computer Society International Conference (COMPCON'96), IEEE Computer Society Press, pp. 85–92.
Kristol, D.M. and L. Montulli (1997), “HTTP State Management Mechanism (Rev1),” ftp://ds.internic.net/internet-drafts/ draft-ietf-http-state-man-mec-04.ps.
Kwan, T.T., R.E. McGrath, and D.A. Reed (1995), “NCSA's World Wide Web Server: Design and Performance,” IEEE Computer 28, 68–74.
Law, A.M. and W.D. Kelton (1991), Simulation Modeling and Analysis, Mc-Graw Hill.
Mah, B.A. (1997), "An Empirical Model of HTTP Network Traffic, In Proceedings of the IEEE International Conference on Computer Communication (INFOCOM'97), IEEE Computer Society Press, Kobe, Japan.
Mosedale, D., W. Foss, and R. McCool (1997), “Lesson Learned Administering Netscape's Internet Site,” IEEE Internet Computing 1, 2, 28–35.
Pirolli, P., J. Pitkow, and R. Rao (1996), "Silk from a Sow's Ear: Extracting Usable Structures from the Web, In Proceedings of 1996 International Conference on Human Factors in Computing Systems (CHI'96), ACM Press, Vancouver, Canada, pp. 118–125.
Pitkow, J. (1997), “In Search of Reliable Usage Data on the WWW,” In Proceedings of 6th International World Wide Web Conference (WWW6), Santa Clara, CA.
W3C (1995), “Common Logfile Format,” http://www.w3.org/Daemon/User/Config/Logging. html.
Zipf, G.K. (1949), Human Behaviour and the Principles of Least Effort, Addison-Wesley, Cambridge, MA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Cardellini, V., Colajanni, M. & Yu, P.S. DNS dispatching algorithms with state estimators for scalable Web‐server clusters. World Wide Web 2, 101–113 (1999). https://doi.org/10.1023/A:1019296605640
Issue Date:
DOI: https://doi.org/10.1023/A:1019296605640