Joel Sommers
ABSTRACT
Empirical research in the Internet is fraught with challenges. Among these is the possibility that local environmental conditions (e.g., CPU load or network load) introduce unexpected bias or artifacts in measurements that lead to erroneous conclusions. In this paper, we describe a framework for local environment monitoring that is designed to be used during Internet measurement experiments. The goals of our work are to provide a critical, expanded perspective on measurement results and to improve the opportunity for reproducibility of results. We instantiate our framework in a tool we call SoMeta, which monitors the local environment during active probe-based measurement experiments. We evaluate the runtime costs of SoMeta and conduct a series of experiments in which we intentionally perturb different aspects of the local environment during active probe-based measurements. Our experiments show how simple local monitoring can readily expose conditions that bias active probe-based measurement results. We conclude with a discussion of how our framework can be expanded to provide metadata for a broad range of Internet measurement experiments.
- M Allman. 2013. On changing the culture of empirical Internet assessment. ACM SIGCOMM Computer Communication Review 43, 3 (2013), 78--83. Google Scholar
Digital Library
- M. Allman and V. Paxson. 2007. Issues and etiquette concerning use of shared measurement data. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 135--140. Google ScholarDigital Library
- B. Augustin, X. Cuvellier, B. Orgogozo, F. Viger, T. Friedman, M. Latapy, C. Magnien, and R. Teixeira. 2006. Avoiding traceroute anomalies with Paris traceroute. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 153--158. Google ScholarDigital Library
- F. Baccelli, S. Machiraju, D. Veitch, and J. C. Bolot. 2007. On optimal probing for delay and loss measurement. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 291--302. Google ScholarDigital Library
- V. Bajpai, A. W. Berger, P. Eardley, J. Ott, and J. Schönwälder. 2016. Global Measurements: Practice and Experience (Report on Dagstuhl Seminar# 16012). ACM SIGCOMM Computer Communication Review 46, 1 (2016), 32--39. Google ScholarDigital Library
- V. Bajpai, S. Eravuchira, and J. Schönwälder. 2015. Lessons learned from using the RIPE Atlas platform for measurement research. ACM SIGCOMM Computer Communication Review 45, 3 (2015), 35--42. Google ScholarDigital Library
- R. Beverly. 2016. Yarrp'ing the Internet: Randomized High-Speed Active Topology Discovery. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. Google ScholarDigital Library
- BGPmon {n. d.}. BGPmon Deployment. http://bgpmon.netsec.colostate.edu/about.html. ({n. d.}). Accessed May 2017.Google Scholar
- CAIDA {n. d.}. Ark Monitor Site Information. http://www.caida.org/projects/ark/siteinfo.xml. ({n. d.}). Accessed May 2017.Google Scholar
- CAIDA {n. d.}. CAIDA Ark project. http://www.caida.org/projects/ark/. ({n. d.}). Accessed May 2017.Google Scholar
- Center for Applied Internet Data Analysis (CAIDA). {n. d.}. DatCat: Internet Measurement Data Catalog. http://www.datcat.org. ({n. d.}). Accessed May 2017.Google Scholar
- Center for Applied Internet Data Analysis (CAIDA). 2009. How to Document a Data Collection. http://www.caida.org/data/how-to/how-to_document_data.xml. (March 2009). Accessed May 2017.Google Scholar
- C. Dovrolis, K. Gummadi, A. Kuzmanovic, and S. D. Meinrath. 2010. Measurement lab: Overview and an invitation to the research community. ACM SIGCOMM Computer Communication Review 40, 3 (2010), 53--56. Google ScholarDigital Library
- J. Heidemann, Y. Pradkin, R. Govindan, C. Papadopoulos, G. Bartlett, and J. Bannister. 2008. Census and Survey of the Visible Internet (extended). ISI-TR-2008-649 (2008).Google Scholar
- T. Holterbach, C. Pelsser, R. Bush, and L. Vanbever. 2015. Quantifying interference between measurements on the RIPE Atlas platform. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 437--443. Google ScholarDigital Library
- J. D. Hunter. 2007. Matplotlib: A 2D graphics environment. Computing In Science & Engineering 9, 3 (2007), 90--95. Google ScholarDigital Library
- D. Kotz and T. Henderson. 2005. CRAWDAD: A community resource for archiving wireless data at Dartmouth. IEEE Pervasive Computing 4, 4 (2005), 12--14. Google ScholarDigital Library
- D. Kotz, T. Henderson, I. Abyzov, and J. Yeo. {n. d.}. CRAWDAD: A Community Resource for Archiving Wireless Data At Dartmouth. https://crawdad.cs.dartmouth.edu. ({n. d.}). Accessed May 2017.Google Scholar
- B. Krishnamurthy, W. Willinger, P. Gill, and M. Arlitt. 2011. A Socratic method for validation of measurement-based networking research. Computer Communications 34, 1 (2011), 43--53. Google ScholarDigital Library
- M. Luckie. 2010. Scamper: a scalable and extensible packet prober for active measurement of the Internet. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 239--245. Google ScholarDigital Library
- M-Lab {n. d.}. M-Lab. https://www.measurementlab.net. ({n. d.}). Accessed May 2017.Google Scholar
- R. K. P. Mok, W. Li, and R. K. C. Chang. 2015. Improving the packet send-time accuracy in embedded devices. In Passive and Active Network Measurement Conference. 332--344.Google ScholarCross Ref
- Nagios {n. d.}. Nagios: The Industry Standard in IT Infrastructure Monitoring. https://www.nagios.org. ({n. d.}). Accessed May 2017.Google Scholar
- C. Partridge and M. Allman. 2016. Ethical considerations in network measurement papers. Commun. ACM 59, 10 (2016), 58--64. Google ScholarDigital Library
- V. Paxson. 2004. Strategies for Sound Internet Measurement. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 263--271. Google ScholarDigital Library
- C. Pelsser, L. Cittadini, S. Vissicchio, and R. Bush. 2013. From Paris to Tokyo: On the suitability of ping to measure latency. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 427--432. Google ScholarDigital Library
- RIPE {n.d.}. RIPE Atlas - RIPE Network Coordination Centre. https://atlas.ripe.net. ({n. d.}). Accessed May 2017.Google Scholar
- C. Shannon, D. Moore, K. Keys, M. Fomenkov, B. Huffaker, et al. 2005. The Internet Measurement Data Catalog. ACM SIGCOMM Computer Communication Review 35, 5 (2005), 97--100. Google ScholarDigital Library
- J. Sommers. 2015. Lowering the Barrier to Systems-level Networking Projects. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE). 651--656. Google ScholarDigital Library
- J. Sommers and P. Barford. 2007. An active measurement system for shared environments. In Proceedings of the ACM SIGCOMM Internet Measurement Conference. 303--314. Google ScholarDigital Library
Index Terms
- Automatic metadata generation for active measurement
Recommendations
A network measurement framework for named data networks
ICN '17: Proceedings of the 4th ACM Conference on Information-Centric NetworkingIn this poster, we propose a network measurement framework for NDN. We define the goals of network measurement and discuss how these goals can be achieved by identifying the necessary measurement operations that must be built on top of NDN's primitives. ...
Using metadata to improve experiment reliability in shared environments
TMA'12: Proceedings of the 4th international conference on Traffic Monitoring and AnalysisExperimental network research is subject to challenges since the experiment outcomes can be influenced by undesired effects from other activities in the network. In shared experiment networks, control over resources is often limited and QoS guarantees ...
Active/passive combination-type performance measurement method using change-of-measure framework
We propose a new method of performance measurement called the Change-of-Measure Based Passive/Active Monitoring (CoMPACT Monitor), in which estimates of the actual performance seen by users are obtained based on both active and passive measurement data. ...
Comments