Elsevier

Computer Networks

Volume 41, Issue 2, 5 February 2003, Pages 177-192
Computer Networks

Distributed distance measurement for large-scale networks

https://doi.org/10.1016/S1389-1286(02)00373-0Get rights and content

Abstract

There is an increasing trend in the Internet that a set of replicated providers are qualified for a service or resource request from a client. In this case, it is advantageous to select the best provider considering some distance measures, such as hop count or path latency. In this paper, we present a group-based distance measurement service (GDMS), which estimates and disseminates distance information of node-pairs in large-scale wide area networks. GDMS is fully distributed and does not rely on any centralized servers; thus is particularly suitable for the rapidly popularized peer-to-peer applications. The key concept in GDMS is measurement groups (MGroups). Nodes are self-organized into MGroups to form a hierarchical structure. A set of algorithms are proposed to handle network dynamics and optimize the group organization to reduce system costs as well as improve estimation accuracy. Moreover, a novel multicast-based algorithm is used for both intra- and inter-group performance measurements. Performance evaluation over different network topologies shows that GDMS is scalable and provides effective distance information to upper-layer applications at a relatively low cost.

Introduction

In current networks, given a service or resource request from a client, there are usually a set of qualified providers, for example, mirrors of a FTP server or a cluster of Web servers [4], [21], [22]. Thus, a key issue in this case is how to efficiently discover and deliver a required resource with a specific quality-of-services (QoS). This requires two basic services: (1) a resource discovery service to locate the candidate providers; and (2) a distance measurement service to measure the network performance between node-pairs, so that the best provider can be selected based on some distance measures, such as path latency. Even though the network distance may not be the dominating consideration in some scenarios, it is still useful to include the distance to each candidate provider as a factor in making a selection [4], [21].

For large-scale wide area networks, distance measurement using individual probing for each access is clearly not an efficient method. Many existing studies suggest that several measurement servers can be deployed over the global Internet; these servers perform distance measurement on behalf of its local clients, and a client can obtain distance estimations by querying its measurement server. These systems are based on the extensively-studied client/server model. However, it is known that the client/server model has some typical drawbacks, such as expensive (to deploy servers for specific purposes) and vulnerable (since there is a single point of failure) in large scale networks.

Recently, a new communication model, peer-to-peer communication has emerged at the forefront of Internet computing [1]. The rapid and widespread deployment of peer-to-peer applications, such as Napster [2] and Gnutella [3], suggests that there are several advantages of this model over the traditional client/server model. The most important one is its decentralized or distributed nature. That is, resources such as multimedia files are stored in end users’ machines (hosts, or peers in this paper) rather than in a central server and, as opposed to the client/server model, resources are transferred directly between peers. Therefore, with this model, resources stored or replicated in the global Internet can be fully utilized. Moreover, the potential bandwidth or processing bottlenecks at the server’s end can be alleviated, and the hazard caused by the failure of a server is reduced.

Several pure decentralized resource discovery services have been proposed for peer-to-peer applications, such as Chord [15] and CAN [16]. However, to our knowledge, there are few decentralized network distance measurement services that are specifically designed for peer-to-peer applications in a wide area network environment.

In this paper, we propose a decentralized peer-to-peer distance measurement service for large-scale wide area networks, Group-based Distance Measurement Service (GDMS). This service does not rely on any centralized server, but uses a self-organizing infrastructure. The key concept in GDMS is measurement groups (MGroups).1 We distinguish the peer-pair performance information into intra- and inter-group measures to achieve a scalable and efficient solution. Peers are self-organized into MGroups and a group leader is dynamically elected which acts as a representative for inter-group distance measurement. We devise a set of distributed group forming algorithms to handle network dynamics, balance the workload of different peers, and minimize the overall measurement cost. Moreover, a novel multicast-based measurement algorithm is used for both intra- and inter-group measurements. The algorithm is highly scalable and incurs much lower overheads compared to traditional unicast-based measurement algorithms.

We envision GDMS as an underlying measurement service that provides peer-pair distance performance information to upper-layer applications. The performance of GDMS has been evaluated under various network topologies. The results show that GDMS can indeed provide useful distance information for QoS-aware peer-to-peer applications at a reasonable cost.

The rest of the paper is organized as follows. In Section 2, we present some related work and discuss our design objectives. Section 3 describes the system model of GDMS and makes some basic assumptions. Section 4 presents the group forming algorithm for GDMS and its optimizations. An efficient and scalable multicast-based algorithm for both intra- and inter-group measurements is described in Section 5. The cost and scalability of GDMS is analyzed in Section 6, and its performance is evaluated in Section 7 through simulations. Finally, Section 8 concludes the paper.

Section snippets

Related work and our design objectives

There has been extensively work on distance measurement for wide area networks, specifically, the Internet. There is also much work on neighbor discovery or locating the best service provider given some QoS measures [4], [19], [21], [22], [23]. The proposals generally assume there is an underlying distance measurement service or can best be supported by such a service. To date, many of the distance measurement services are designed for the client/server based applications. An example is the

System model and assumptions

We consider a peer-to-peer communication system which consists of a set of nodes (peers) belonging to a wide area network. There is no centralized server in the system. A node can communicate to any other node at will. In addition, a node can join and leave the system or move to another location of the network with the support of Mobile IP [18] at any time.

To facilitate further discussions, we define the following notations:

  • N: The number of nodes (peers) in the system;

  • k: The number of groups

Formation and optimization of groups

At the bootstrapping stage of GDMS, a group forming algorithm is used to form MGroups. When a node joins or leaves the system, or moves to another location, the algorithm is also executed to adjust the group organization. In this section, we first present a basic group forming algorithm, and then devise a set of heuristics to reduce its estimation errors as well as to improve its efficiency.

Intra- and inter-group measurements and sharing

In this section, we first propose a novel multicast-based measurement algorithm for both intra- and inter-group measurements. We then discuss the dissemination and utilization of the measurement information.

Analysis of cost and scalability

In GDMS, there is a tradeoff between the measurement cost and accuracy. In general, GDMS achieves high precision when there are enough number of groups. Specifically, if there are N groups in the system, i.e., each group has only one member (the leader), the accuracy is 100% if the error of probing is not taken into account. Another extreme is the use of only one group. In both cases, GDMS is reduced to a non-hierarchical system whose cost could be very high.

In this section, we formally analyze

Performance evaluation

In this section, we evaluate the performance of GDMS through extensive simulations. Our main objective in the performance evaluation is to investigate whether GDMS provides effective measurement results to upper layer services. Note that the fundamental error of GDMS is the use of the MGroup-based approximation. Therefore, we first simulate different group forming algorithms on a variety of network topologies to investigate the estimation errors caused by grouping. We then investigate the

Conclusions

In this paper, we have proposed a decentralized network distance measurement service for large-scale wide area networks. This Group-based Distance Measurement Service (GDMS) provides a two-level hierarchical measurement framework by using self-organized measurement groups. We have devised a set of distributed group forming algorithms to handle network dynamics, balance the workload of different peers, and minimize the overall measurement cost. Moreover, a novel multicast-based measurement

Acknowledgements

Bo Li’s research is supported in part by grants from Research Grant Council (RGC) under contracts AoE/E-01/99 and HKUST6196/02E. J. Liu’s work is partially supported by a Microsoft fellowship. Part of this work was done when J. Liu and X. Zhang are interns at the Wireless and Networking Group, Microsoft Research, Asia.

Jiangchuan Liu received the B.S. degree (cum laude) in Computer Science from Tsinghua University, Beijing, PR China, in 1999. Currently, he is working towards the Ph.D degree in Computer Science at the Hong Kong University of Science and Technology. In 2000 and 2001, he had Internships with Microsoft Research, Asia, working in video multicast over the Internet and QoS-aware resource discovery, respectively. He is a recipient of a Microsoft research fellowship. His current research interests

References (23)

  • Napster....
  • Gnutella....
  • Z. Fei, S. Bhattacharjee, E.W. Zegura, M.H. Ammar, A novel server selection technique for improving the response time...
  • K. Moore, J. Cox, S. Green, Sonar––a Network Proximity Service, Internet Draft, Available at...
  • P. Francis, A Call for an Internet-wide Host Proximity Service (HOPS), White Paper, March 1997. Available at...
  • V. Paxson et al.

    An architecture for large-scale Internet measurement

    IEEE Communications Magazine

    (1998)
  • P. Francis et al.

    ID Maps: A global Internet host distance estimation service

    IEEE/ACM Transactions on Networking

    (2001)
  • J. Nonnenmacher, E.W. Biersack, Optimal multicast feedback, in: Proceedings of IEEE INFOCOM’98, March...
  • B.M. Waxman

    Routing of multipoint connections

    IEEE Journal on Selected Areas in Communications

    (1988)
  • E.W. Zegura, K. Calvert, S. Bhattacharjee, How to model an internetwork, in: Proceedings of IEEE Infocom 96, April...
  • Cited by (0)

    Jiangchuan Liu received the B.S. degree (cum laude) in Computer Science from Tsinghua University, Beijing, PR China, in 1999. Currently, he is working towards the Ph.D degree in Computer Science at the Hong Kong University of Science and Technology. In 2000 and 2001, he had Internships with Microsoft Research, Asia, working in video multicast over the Internet and QoS-aware resource discovery, respectively. He is a recipient of a Microsoft research fellowship. His current research interests include video transmission over wired and wireless networks.

    Xinyan Zhang received B.S. degree in Computer Science from Tsinghua University in 2001. He is currently a M.Phil student in Department of Information Engineering of The Chinese University of Hong Kong. His research interests include modeling, analysis and measurement on Internet, especially peer-to-peer network.

    Bo Li received the B.S. and M.S. degrees in the Computer Science from Tsinghua University, Beijing in 1987 and 1989, respectively, and the Ph.D. degree in the Computer Engineering from University of Massachusetts at Amherst in 1993. Between 1994 and 1996, he worked on high performance routers and ATM switches in IBM Networking System Division, Research Triangle Park, North Carolina. Since then, he has been with the Computer Science Department, the Hong Kong University of Science and Technology.

    His research over the years has been focusing on WDM based optical networks, capacity and resource management in wireless cellular networks, web proxy replication and video multicasting. He has published about 110 papers (50 in various journals) in related areas.

    He has served as an editor or a guest editor for over 10 journals including IEEE Transactions on Wireless Communications, Transactions on Vehicular Technology, Journal of Selected Areas in Communications (JSAC), Communications Magazine, ACM WINET, MC2R and Performance Evaluation. He has been involved in organizing over 30 conferences, esp. IEEE Infocom since 1996.

    Qian Zhang received the B.S., M.S., and Ph.D. degrees from Wuhan University, China, in 1994, 1996, and 1999, respectively, all in computer science.

    She joined Microsoft Research China in July 1999 as Associate Researcher in Internet Media Group and now is Researcher of Wireless and Networking Group. Dr. Zhang has published about 40 refereed papers. She is the inventor of several pending patents. Her current research interest includes multimedia delivery over wireless, Internet, next generation wireless networks, P2P network/ad hoc network. Currently, she is actively participating in TCP/IP header compression in ROHC WG in IETF. She is the principle contributor of the IETF ROHC-TCP WG draft.

    Wenwu Zhu joined Microsoft Research China in November 1999 as Researcher in Internet Media Group and now is Research Manager of Wireless and Networking Group. Prior to his current post, he worked at Bell Labs, Lucent technologies as Member of Technical Staff from July 1996 to October 1999. While he was at Bell Labs, he performed research and development in the area of video conferencing, Internet video, and video streaming over IP networks. Dr. Zhu has published over 100 refereed papers. He is the inventor of more than a dozen of pending patents. His current research interest is in the area of wireless/Internet multimedia communication and networking.

    Dr. Zhu is member of Eta Kappa Nu. He is Guest Editor for the Special Issue on Wireless Video in IEEE Trans. On Circuits and Systems for Video Technology. He served as Guest Editor for the Special Issue on Streaming Video in IEEE Trans. On Circuits and Systems for Video Technology. Dr. Zhu are members of Technical Committees of Video Signal Processing and Communications, and Multimedia System in IEEE Circuits and System Society. He also serves member of Multimedia Communication committee in IEEE Communications Society. He has contributed to IETF ROHC WG draft on robust TCP/IP header compression over wireless links. He received the Best Paper Award in IEEE Trans. on Circuits and System for Video Technology in 2000.

    Dr. Zhu received the B.E. and M.E. degrees from National University of Science and Technology, China, in 1985 and 1988, respectively. He received the M.S. degree from Illinois Institute of Technology, Chicago, IL, and the Ph.D. degree from Polytechnic University, Brooklyn, NY, in 1993 and 1996, respectively, all in Electrical Engineering. From August 1988 to December 1990, he was with Graduate School, University of Science and Technology of China (USTC), and Institute of Electronics, Chinese Academy of Sciences, Beijing.

    View full text