Yibo Pi
Skip Abstract Section
Abstract
Load balancers choose among load-balanced paths to distribute traffic as if it makes no difference using one path or another. This work shows that the latency difference between load-balanced paths (called latency imbalance ), previously deemed insignificant, is now prevalent from the perspective of the cloud and affects various latency-sensitive applications. In this work, we present the first large-scale measurement study of latency imbalance from a cloud-centric view. Using public cloud around the globe, we measure latency imbalance both between data centers (DCs) in the cloud and from the cloud to the public Internet. Our key findings include that 1) Amazon's and Alibaba's clouds together have latency difference between load-balanced paths larger than 20ms to 21% of public IPv4 addresses; 2) Google's secret in having lower latency imbalance than other clouds is to use its own well-balanced private WANs to transit traffic close to the destinations and that 3) latency imbalance is also prevalent between DCs in the cloud, where 8 pairs of DCs are found to have load-balanced paths with latency difference larger than 40ms. We further evaluate the impact of latency imbalance on three applications (i.e., NTP, delay-based geolocation and VoIP) and propose potential solutions to improve application performance. Our experiments show that all three applications can benefit from considering latency imbalance, where the accuracy of delay-based geolocation can be greatly improved by simply changing how \textttping measures the minimum path latency.
- Brice Augustin, Timur Friedman, and Renata Teixeira. Measuring load-balanced paths in the Internet. In IMC, 2007.Google Scholar
Digital Library
- Sandeep Kumar Singh, Tamal Das, and Admela Jukan. A survey on Internet multipath routing and provisioning. IEEE Communications Surveys & Tutorials, 17(4):2157--2175, 2015.Google ScholarDigital Library
- Brice Augustin, Xavier Cuvellier, Benjamin Orgogozo, Fabien Viger, Timur Friedman, Matthieu Latapy, Clémence Magnien, and Renata Teixeira. Avoiding traceroute anomalies with Paris Traceroute. In IMC, 2006.Google ScholarDigital Library
- Kevin Vermeulen, Stephen D. Strowes, Olivier Fourmaux, and Timur Friedman. Multilevel MDA-Lite Paris Traceroute. In IMC, 2018.Google ScholarDigital Library
- Imtiaz Parvez, Ali Rahmati, Ismail Guvenc, Arif I. Sarwat, and Huaiyu Dai. A survey on low latency towards 5G: RAN, core network and caching solutions. IEEE Communications Surveys & Tutorials, 20(4):3098--3130, 2018.Google ScholarDigital Library
- Rupa Krishnan, Harsha V. Madhyastha, Sridhar Srinivasan, Sushant Jain, Arvind Krishnamurthy, Thomas Anderson, and Jie Gao. Moving beyond end-to-end path information to optimize CDN performance. In IMC, pages 190--201, 2009.Google ScholarDigital Library
- Zachary Weinberg, Shinyoung Cho, Nicolas Christin, Vyas Sekar, and Phillipa Gill. How to catch when proxies lie: Verifying the physical locations of network proxies with active geolocation. In IMC, pages 203--217, 2018.Google ScholarDigital Library
- Frank Dabek, Russ Cox, Frans Kaashoek, and Robert Morris. Vivaldi: A decentralized network coordinate system. In SIGCOMM CCR, pages 15--26, 2004.Google Scholar
- Sathiya Kumaran Mani, Ramakrishnan Durairajan, Paul Barford, and Joel Sommers. MNTP: Enhancing time synchronization for mobile devices. In IMC, 2016. Proc. ACM Meas. Anal. Comput. Syst., Vol. 4, No. 2, Article 32. Publication date: June 2020. 32:24 Yibo Pi et al.Google Scholar
- Cristina D. Murta, Pedro R. Torres Jr, and Prasant Mohapatra. Qrpp1--4: Characterizing quality of time and topology in a time synchronization network. In Globecom, pages 1--5, 2006.Google ScholarCross Ref
- Cristel Pelsser, Luca Cittadini, Stefano Vissicchio, and Randy Bush. From Paris to Tokyo: On the suitability of ping to measure latency. In IMC, 2013.Google ScholarDigital Library
- Mark Gondree and Zachary NJ Peterson. Geolocation of data in the cloud. In Proceedings of the third ACM conference on Data and application security and privacy, pages 25--36, 2013.Google ScholarDigital Library
- Junchen Jiang et al. Via: Improving Internet telephony call quality using predictive relay selection. In SIGCOMM, 2016.Google Scholar
- Our tool and dataset. https://github.com/yibopi/latency-imbalance.Google Scholar
- Yuval Shavitt and Noa Zilberman. A geolocation databases study. IEEE Journal on Selected Areas in Communications, 29(10):2044--2056, 2011.Google ScholarCross Ref
- David L. Mills. Simple network time protocol (SNTP) version 4 for IPv4, IPv6 and OSI. https://tools.ietf.org/html/rfc4330, 2006.Google Scholar
- Jonathan Ledlie, Paul Gardner, and Margo I. Seltzer. Network coordinates in the wild. In NSDI, volume 7, pages 299--311, 2007.Google ScholarDigital Library
- Zhiruo Cao, Zheng Wang, and Ellen Zegura. Performance of hashing-based schemes for Internet load balancing. In INFOCOM, pages 332--341, 2000.Google Scholar
- Darryl Veitch, Brice Augustin, Renata Teixeira, and Timur Friedman. Failure control in multipath route tracing. In INFOCOM, 2009.Google ScholarCross Ref
- Rafael Almeida, Osvaldo Fonseca, Elverton Fazzion, Dorgival Guedes, Wagner Meira, and Ítalo Cunha. A characterization of load balancing on the IPv6 Internet. In PAM, 2017.Google ScholarCross Ref
- Yves Vanaubel, Pascal Mérindol, Jean-Jacques Pansiot, and Benoit Donnet. MPLS under the microscope: Revealing actual transit path diversity. In IMC, 2015.Google ScholarDigital Library
- Eric Elena, Jean-Louis Rougier, and Stefano Secci. Characterisation of AS-level path deviations and multipath in Internet routing. In 6th EURO-NGI Conference on Next Generation Internet, 2010.Google ScholarCross Ref
- Matthew Luckie, Young Hyun, and Bradley Huffaker. Traceroute probe method and forward IP path inference. In IMC, 2008.Google ScholarDigital Library
- Zhe Wu, Curtis Yu, and Harsha V. Madhyastha. CosTLO: Cost-effective redundancy for lower latency variance on cloud storage services. In NSDI, 2015.Google ScholarDigital Library
- Bamba Gueye, Artur Ziviani, Mark Crovella, and Serge Fdida. Constraint-based geolocation of Internet hosts. TON, 14(6):1219--1232, 2006.Google Scholar
- Fangfei Chen, Ramesh K. Sitaraman, and Marcelo Torres. End-user mapping: Next generation request routing for content delivery. In CCR, 2015.Google Scholar
- Yibo Pi, Sugih Jamin, Peter Danzig, and Jacob Shaha. AP-Atoms: A high-accuracy data-driven client aggregation for global load balancing. IEEE/ACM Transactions on Networking, 26(6):2748--2761, December 2018.Google ScholarDigital Library
- Mohammed S. Elbamby, Cristina Perfecto, Mehdi Bennis, and Klaus Doppler. Toward low-latency and ultra-reliable virtual reality. IEEE Network, 32(2):78--84, 2018.Google ScholarCross Ref
- Roy D. Yates, Mehrnaz Tavan, Yi Hu, and Dipankar Raychaudhuri. Timely cloud gaming. In INFOCOM, 2017.Google ScholarCross Ref
- Zhiruo Cao, Zheng Wang, and Ellen Zegura. Performance of hashing-based schemes for Internet load balancing. In INFOCOM, 2000.Google Scholar
- Yilong Geng, Shiyu Liu, Zi Yin, Ashish Naik, Balaji Prabhakar, Mendel Rosenblum, and Amin Vahdat. Exploiting a natural network effect for scalable, fine-grained clock synchronization. In NSDI, pages 81--94, 2018.Google Scholar
- Ethan Katz-Bassett, Harsha V. Madhyastha, Vijay Kumar Adhikari, Colin Scott, Justine Sherry, Peter Van Wesep, Thomas E. Anderson, and Arvind Krishnamurthy. Reverse traceroute. In NSDI, 2010.Google ScholarDigital Library
- MDA-Lite Gitlab repository. https://gitlab.planet-lab.eu/cartography/multilevel-mda-lite, Sep 2018.Google Scholar
- Abhinav Pathak, Himabindu Pucha, Ying Zhang, Y. Charlie Hu, and Z. Morley Mao. A measurement study of Internet delay asymmetry. In PAM, 2008.Google ScholarCross Ref
- Robert Beverly. Yarrp'ing the Internet: Randomized high-speed active topology discovery. In IMC, pages 413--420, 2016.Google ScholarDigital Library
- Perry R. Hinton. Statistics explained. Routledge, 2014.Google ScholarCross Ref
- Christophe Leys, Christophe Ley, Olivier Klein, Philippe Bernard, and Laurent Licata. Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4):764--766, 2013.Google ScholarCross Ref
- Youndo Lee and Neil Spring. Identifying and aggregating homogeneous IPv4 /24 blocks with Hobbit. In IMC, pages 151--165, 2016.Google ScholarDigital Library
- Ki Suh Lee, Han Wang, Vishal Shrivastav, and Hakim Weatherspoon. Globally synchronized time via datacenter networks. In SIGCOMM, 2016.Google ScholarDigital Library
- IP latency statistics. https://enterprise.verizon.com/terms/latency/, July 2019. Proc. ACM Meas. Anal. Comput. Syst., Vol. 4, No. 2, Article 32. Publication date: June 2020. Latency Imbalance Among Internet Load-Balanced Paths: A Cloud-Centric View 32:25Google Scholar
- Sadjad Fouladi, John Emmons, Emre Orbay, Catherine Wu, Riad S. Wahby, and Keith Winstein. Salsify: low-latency network video through tighter integration between a video codec and a transport protocol. In NSDI, 2018.Google Scholar
- CAIDA. Routeviews prefix to AS mappings dataset for IPv4 and IPv6. https://www.caida.org/data/routing/routeviewsprefix2as. xml, Sep 2018.Google Scholar
- Timothy Wood, Prashant J. Shenoy, Alexandre Gerber, Jacobus E. van der Merwe, and Kadangode K. Ramakrishnan. The case for enterprise-ready virtual private clouds. In HotCloud, 2009.Google Scholar
- Sathiya Kumaran Mani, Ramakrishnan Durairajan, Paul Barford, and Joel Sommers. An architecture for IoT clock synchronization. In Proceedings of the 8th International Conference on the Internet of Things, 2018.Google ScholarDigital Library
- Alexander Marder, Matthew Luckie, Amogh Dhamdhere, Bradley Huffaker, kc claffy, and Jonathan M. Smith. Pushing the boundaries with bdrmapit: Mapping router ownership at Internet scale. In IMC, 2018.Google ScholarDigital Library
- CAIDA. The CAIDA Internet Topology Data Kit. https://www.caida.org/data/internet-topology-data-kit, August 2018.Google Scholar
- AFRINIC Extended Allocation and Assignment Reports. ftp://ftp.afrinic.net/pub/stats/afrinic/, 2019.Google Scholar
- RIPE Extended Allocation and Assignment Reports. ftp://ftp.afrinic.net/pub/stats/ripencc/, 2019.Google Scholar
- LACNIC Extended Allocation and Assignment Reports. ftp://ftp.afrinic.net/pub/stats/lacnic/, 2019.Google Scholar
- APNIC Extended Allocation and Assignment Reports. ftp://ftp.afrinic.net/pub/stats/apnic/, 2019.Google Scholar
- ARIN Extended Allocation and Assignment Reports. ftp://ftp.afrinic.net/pub/stats/arin/, 2019.Google Scholar
- Euro-IX IXP Directory. https://www.euro-ix.net/tools/ixp-directory, 2019.Google Scholar
- PeeringDB. https://peeringdb.com/api, 2019.Google Scholar
- Packet Clearing House: Internet Exchange Directory. https://www.pch.net/applications/ixpdir/menu_download.php, 2019.Google Scholar
- CAIDA. The CAIDA AS Relationships Dataset. https://www.caida.org/data/as-relationships/, Feb 2019.Google Scholar
- Google cloud network service tiers. https://cloud.google.com/network-tiers#tab1.Google Scholar
- Matthew Luckie, Amogh Dhamdhere, kc Claffy, and David Murrell. Measured impact of crooked traceroute. In CCR, pages 14--21, 2011.Google ScholarDigital Library
- Ramesh Govindan and Vern Paxson. Estimating router ICMP generation delays. In PAM, 2002.Google Scholar
- Yves Vanaubel, Pascal Mérindol, Jean-Jacques Pansiot, and Benoit Donnet. Through the wormhole: Tracking invisible MPLS tunnels. In IMC, 2017.Google ScholarDigital Library
- Joel Sommers, Paul Barford, and Brian Eriksson. On the prevalence and characteristics of MPLS deployments in the open Internet. In IMC, 2011.Google ScholarDigital Library
- Benoit Donnet, Matthew Luckie, Pascal Mérindol, and Jean-Jacques Pansiot. Revealing MPLS tunnels obscured from traceroute. In SIGCOMM CCR, 2012.Google ScholarDigital Library
- Abhinav Pathak, Ming Zhang, Y. Charlie Hu, Ratul Mahajan, and Dave Maltz. Latency inflation with MPLS-based traffic engineering. In IMC, 2011.Google ScholarDigital Library
- Chi-Yao Hong, Srikanth Kandula, Ratul Mahajan, Ming Zhang, Mohan Nanduri, and Roger Wattenhofer. Achieving high utilization with software-driven WAN. In SIGCOMM CCR, 2013.Google ScholarDigital Library
- Sushant Jain et al. B4: Experience with a globally-deployed software defined WAN. In SIGCOMM, pages 3--14, 2013.Google Scholar
- Ramakrishnan Durairajan, Sathiya Kumaran Mani, Joel Sommers, and Paul Barford. Time's Forgotten: Using NTP to understand Internet latency. In HotNets, 2015.Google ScholarDigital Library
- Amazon AWS IoT. https://aws.amazon.com/iot/.Google Scholar
- Karyn Benson, Rafael Dowsley, and Hovav Shacham. Do you know where your cloud files are? In Proceedings of the 3rd ACM workshop on Cloud computing security workshop, pages 73--82, 2011.Google ScholarDigital Library
- Yang Xu, Chenguang Yu, Jingjiang Li, and Yong Liu. Video telephony for end-consumers: Measurement study of Google+, iChat, and Skype. In IMC, pages 371--384, 2012.Google ScholarDigital Library
- Osama Haq, Mamoon Raja, and Fahad R. Dogar. Measuring and improving the reliability of wide-area cloud paths. In WWW, 2017.Google ScholarDigital Library
- Venkata N. Padmanabhan and Lakshminarayanan Subramanian. An investigation of geographic mapping techniques for Internet hosts. In SIGCOMM, 2001.Google Scholar
- Planetlab locations. https://www.planet-lab.org/db/pub/sites.php, April, 2019.Google Scholar
- iPlane project dataset. http://web.eecs.umich.edu/~harshavm/iplane/iplane_logs/data/.Google Scholar
- G.107. The E-Model, a computational model for user in transmission planning, 2017. https://www.itu.int/rec/T-REC-G.107.Google Scholar
- Haiyong Xie and Yang Richard Yang. A measurement-based study of the skype peer-to-peer VoIP performance. In IPTPS, 2012.Google Scholar
Index Terms
- Latency Imbalance Among Internet Load-Balanced Paths: A Cloud-Centric View
Recommendations
Latency Imbalance Among Internet Load-Balanced Paths: A Cloud-Centric View
SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsLoad balancers choose among load-balanced paths to distribute traffic as if it makes no difference using one path or another. This work shows that the latency difference between load-balanced paths (called latency imbalance), previously deemed ...
Latency Imbalance Among Internet Load-Balanced Paths: A Cloud-Centric View
Load balancers choose among load-balanced paths to distribute traffic as if it makes no difference using one path or another. This work shows that the latency difference between load-balanced paths (called latency imbalance), previously deemed ...
Load-balanced three-stage switch
A load-balanced two-stage switch is scalable and can provide close to 100% throughput. Its major problem is that packets can be mis-sequenced when they arrive at outputs. In a recent work, the packet mis-sequencing problem is elegantly solved by a ...
Comments