Elsevier

Computer Communications

Volume 25, Issue 3, 15 February 2002, Pages 329-340
Computer Communications

A novel cache distribution heuristic algorithm for a mesh of caches and its performance evaluation

https://doi.org/10.1016/S0140-3664(01)00364-4Get rights and content

Abstract

The widespread use of the Internet has created two problems: document retrieval latency and network traffic. Caching of documents ‘close’ to users has helped alleviate both problems. Different caching policies have been proposed/implemented to make best use of limited available cache at each caching server. A mesh of caching servers, aided by different data diffusion algorithms and the natural hierarchical structure of the Internet topology, has increased ‘virtual’ size of cache. Yet the size of available cache is small compared to the total size of all documents served, and remains a major resource constraint. In this work, we looked at how to improve document download time, by distributing a fixed amount of total storage in a network or mesh of caches. The intuition behind our cache distribution approach is to give more storage to the caching nodes in the network, which experience more traffic, in the hope that this will reduce the average latency of document retrieval in the network. A heuristic was developed to estimate traffic at each cache of a network. From this traffic estimation, each cache then receives a corresponding percentage of the total storage capacity of the network. Through extensive simulation it is found that the proposed cache distribution algorithm can reduce latency up to 80% over prior work that includes both Harvest-type and demand-driven data diffusion algorithms. Furthermore, the best improvement was achieved in a cache range that corresponds to practical, real world cache ranges.

Introduction

As the (World Wide) Web connects information sources or origin servers around the world, users from everywhere attempt to access these sources. Due to the ease and low cost of access, both the number of users and documents are growing dramatically [21]; per user network traffic is also increasing. When a user accesses a faraway document, the time it takes to download (latency) could be unacceptably high. Moreover, when a user or a group of users connected to a network-neighbourhood repeatedly access a faraway document, the traffic on the network, and hence the demand for bandwidth is increased.

An effective method for reducing both latency and network traffic is to use shared caches [1], [10], [18], [19], [25]. A mesh is a set of cooperative caches connected as a network, and they can directly or indirectly share documents stored in them. Some of these caches provide direct access to the clients; these are called proxy caches. Others are called caching servers and provide direct or indirect access to other caches (see Section 4.2 for more details about network model). As a mesh, caches provide the effect of a ‘large virtual’ cache with a higher hit rate, improved access to the origin servers, reduction in document downloading time, and reduction in network traffic.

The two most commonly employed mesh caching methods are (a) hierarchical and (b) directory-based methods [23], [25]. In the directory-based method, when a request for a document arrives at a caching server, if the caching server has the document in its local cache, it serves it from there. Otherwise, the caching server queries a mapping service to locate the document in the caching mesh. In the case of a hit, the document is fetched from the peer-caching server. In the case of a miss, the document is fetched from the origin server.

In hierarchical caching, the caching servers in a mesh are (logically) organized in a hierarchy. National caches connected to the backbone network are on the top of the hierarchy [2], [8], [9]. Regional caches are connected to national caches and occupy the next lower level in the hierarchy. Subregional caches are connected at the next lower level to regional caches, and institutional caches serve as the ‘gateways’ to subregional caches. This hierarchy naturally fits with the standard topology of the Internet. Because of this natural hierarchical topology, the directory based caching method could be implemented at one or two levels of the hierarchy, and not at all levels of the hierarchy.

To exploit the hierarchical topology of the Internet, the Harvest project originated a data diffusion scheme (will be referred to as Scheme 1 in this paper) that has led to popular commercial products (such as NetCache) and public-domain products (such as Squid) [12]. The Harvest [14], or Harvest-derived data diffusion schemes replicate documents on the mesh hierarchy using a simple strategy: whenever a document is retrieved from an origin or a caching server, it is replicated on every cache along the path to the client. Although this data diffusion scheme greatly reduces the latency and network traffic, it can be improved many ways. One would get significant improvement, by not replicating relatively ‘unpopular’ documents, if it would replace a ‘popular’ document [21].

Demand driven data diffusion schemes replicate documents only if they are relatively popular at that cache. There are two approaches to achieve this replication process. In a top-down protocol, a document is replicated at a cache's child if it appears to be very popular and may create a hot-spot. This protocol is aimed at avoiding hot-spots, and will work well in conjunction with an appropriate hashing technique [4], [5], [13], [15]. To implement this technique, significant changes will be necessary in currently used caching strategies.

The bottom-up approach replicates a document at a cache only when a relatively large number of requests for a document is observed [21]. In Section 2.2, we present this demand driven data diffusion strategy and we refer to this as Scheme 2.

Even if we had the best/optimal data diffusion scheme, another practical problem needs to be addressed: the size of available storage or cache. We use cache size and storage capacity interchangeably. Analysis of access logs from Ref. [22] shows that the storage capacity at a server (unless specified, will refer to a proxy or a caching server) is less than three percent of all documents served by it [16]. Thus, one could think that storage capacity is a limited resource. In our study, we assume that total amount of available storage on all the servers in a mesh of caches is constant. Moreover, it is assumed that available storage can be distributed among the servers as desired. Under these assumptions, a natural question is to ask: how to distribute total available storage to the servers for minimizing latency in document downloading, and also reduce network traffic?

In this work, we propose and evaluate a storage distribution heuristic. The intuition behind our storage distribution heuristic is that a server with more expected traffic should get a bigger share of the available storage. A heuristic was developed to estimate traffic at each cache of a network. From this traffic estimation, each cache then receives a corresponding percentage of the total storage capacity of the network. The proposed storage distribution heuristic is presented in Section 3.

Simulation has been the most common method for evaluation of caching algorithms for a proxy cache server [11]. Two possible sources of input data for simulation are traces of requests from a cache server, or synthetic/artificially generated requests. Trace data now available for a single caching server can be used for realistic evaluation of different caching algorithms running at a proxy server. However, simulation study of any data diffusion scheme requires trace data of an entire mesh of caches, but these are not readily available. Hence, for our study we use synthetic data derived from actual server logs. Our simulation model is presented in Section 4. Our observations from the simulation study are presented and discussed in Section 5. Results show that our cache distribution heuristic achieves up to 80% latency improvement in Harvest-type and demand-driven data diffusion algorithms (when compared to equal size cache in all servers). Furthermore, the best improvement was achieved in a cache range that corresponds to practical, real world cache ranges. Section 6 discusses the findings and possible future directions of study.

Section snippets

Hierarchical caching

In a hierarchical cache organization, each caching node has a parent, siblings, and children. The term ‘neighbor’ is used to refer to parent or siblings that are one cache-hop away. The protocol that caches in a caching hierarchy use is the Internet Cache Protocol (ICP) [24]. The request resolution protocol in hierarchical caching is as follows: When a cache receives a request, it checks whether it has the requested object. If it has the object, it satisfies the request by serving the object

A cache distribution algorithm

The motivation behind distributing the cache is to maximize the benefits of caching. Given a limited amount of cache in the network, placing more cache in the nodes which experience more traffic, while keeping the average cache per node over the whole network the same, may improve the overall latency of the network. Since the cache size of the whole network is fixed, placing more cache at one node means reducing cache at another node. The cache distribution algorithm determines how much cache

Simulation model

To study the effects of the cache distribution using the novel algorithm proposed here, the operation of a mesh of caches is simulated. Both the data diffusion algorithms described in Section 2 were implemented in the simulated environment. The simulator is a C++ program, which makes extensive use of LEDA [17] object libraries, and is an extended version of the simulator used in the study reported in Ref. [21]. The extended version is discussed in Section 4.1. The inputs to the simulator make

Performance comparison

In this section, we present the simulation results of the two caching schemes (see Section 2) with and without cache distribution. When the cache is not distributed, all the nodes, i.e. proxy and caching servers, have the same cache size. The simulations were run on the two network topologies presented in Section 4.4. To keep the simulation time reasonable, a total of 400 objects were served by the origin servers and three different request rates — moderate to very high — were used. The

Conclusions

In this work, we looked at a specific facet of caching as it applies to the Web. Specifically, we looked at how to improve document download time, by distributing a fixed amount of total storage in a mesh of caching servers. We studied the effect of our cache distribution by simulating two data diffusion schemes for hierarchical caches The intuition behind this approach is to give more storage to the caching nodes in the network which experience more traffic, in the hopes that this will reduce

References (25)

  • M. Abrams, C. Standridge, G. Abdulla, S. Williams, E.A. Fox, Caching proxies: limitations and potentials, Proceedings...
  • R. Alonso, M. Blaze, Dynamic hierarchical caching for large-scale distributed file systems, Proceedings of the Twelfth...
  • M.F. Arlitt, C.L. Williamson, Web server workload characterization: the search for invariants, 1996 ACM SIGMETRICS...
  • A. Bestavros, Demand-based document dissemination to reduce traffic and balance load in distributed information...
  • A. Bestavros, R.L. Carter, M.E. Crovella, Application-Level Document Caching in the Internet, Technical Report...
  • L. Breslau, P. Cao, L. Fan, G. Phillips, S. Shenker, Web caching and zipf-like distributions: evidence and...
  • C.M. Bowman, P.B. Danzig, D.R. Hardy, U. Manber, M.F. Schwartz, Harvest: A scalable, customizable discovery and access...
  • A. Chankhunthod, P.B. Danzig, C. Neerdaels, M.F. Schwartz, K. Worrel, A hierarchical internet object cache, Proceedings...
  • P.B. Danzig, M.F. Schwartz, R.S. Hall, A case for caching file objects inside internetworks, Proceedings ACM SIGCOMM 92...
  • B.D. Davidson, A survey of proxy cache evaluation techniques, Fourth International Caching Workshop, San Jose, CA,...
  • S. Gadde, J. Chase, M. Rabinovich, A taste of crispy squid, Proceedings of the Workshop on Internet Service Performance...
  • J. Gwertzman, M. Seltzer, The case for geographical push-caching, Proceedings of the 1995 Workshop on Hot Operating...
  • Cited by (0)

    View full text