Paul Tune
Matthew Roughan
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Traffic matrices describe the volume of traffic between a set of sources and destinations within a network. These matrices are used in a variety of tasks in network planning and traffic engineering, such as the design of network topologies. Traffic matrices naturally possess complex spatiotemporal characteristics, but their proprietary nature means that little data about them is available publicly, and this situation is unlikely to change.
Our goal is to develop techniques to synthesize traffic matrices for researchers who wish to test new network applications or protocols. The paucity of available data, and the desire to build a general framework for synthesis that could work in various settings requires a new look at this problem. We show how the principle of maximum entropy can be used to generate a wide variety of traffic matrices constrained by the needs of a particular task, and the available information, but otherwise avoiding hidden assumptions about the data. We demonstrate how the framework encompasses existing models and measurements, and we apply it in a simple case study to illustrate the value.
Supplemental Material
- T. Benson, A. Akella, and D. A. Maltz. Network traffic characteristics of data centers in the wild. In Proc. ACM SIGCOMM Internet Measurement Conference (IMC), pages 267--280, November 2010. Google Scholar
Digital Library
- T. Benson, A. Anand, A. Akella, and M. Zhang. MicroTE: Fine grained traffic engineering for data centers. In Proc. ACM SIGCOMM CoNEXT, pages 1--12, December 2011. Google ScholarDigital Library
- V. Bharti, P. Kankar, L. Setia, G. Gürsun, A. Lakhina, and M. Crovella. Inferring invisible traffic. In Proc. ACM SIGCOMM CoNEXT, pages 22:1--22:12, 2010. Google ScholarDigital Library
- R. Bowden. COLD: PoP-level topology generator. https://github.com/rhysbowden/COLD/, 2014.Google Scholar
- R. Bowden, M. Roughan, and N. Bean. COLD: PoP-level topology synthesis. In Proc. ACM SIGCOMM CoNEXT, pages 173--184, December 2014. Google ScholarDigital Library
- R. S. Cahn. Wide Area Network Design. Morgan Kaufmann, 1998.Google Scholar
- A. Calaprice. The Ultimate Quotable Einstein. Princeton University Press, 2010.Google Scholar
- E. Candes and B. Recht. Exact matrix completion via convex optimization. Found. of Comput. Math., 9:717--772, 2008. Google ScholarDigital Library
- J. Chabarek, J. Sommers, P. Barford, C. Estan, D. Tsiang, and S. Wright. Power awareness in network design and routing. In Proc. IEEE INFOCOM, April 2008.Google ScholarCross Ref
- T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley and Sons, Inc., 2nd edition, 2006. Google ScholarDigital Library
- N. Cressie and C. K. Wikle. Statistics for Spatiotemporal Data. Wiley, 2011.Google Scholar
- L. Devroye. Non-Uniform Random Variate Generation. Springer, 1986.Google ScholarCross Ref
- P. Erdös and A. Rényi. On random graphs. Publicationes Mathematicae, 6:290--297, 1959.Google ScholarCross Ref
- V. Erramilli, M. Crovella, and N. Taft. An independent-connection model for traffic matrices. In Proc. ACM SIGCOMM Internet Measurement Conference (IMC), pages 251--256, October 2006. Google ScholarDigital Library
- E. T. Jaynes. Information theory and statistical mechanics. Phys. Rev., 106:620--630, May 1957.Google ScholarCross Ref
- S. Kandula, S. Sengupta, A. Greenberg, P. Patel, and R. Chalken. The nature of datacenter traffic: Measurements and analysis. In Proc. ACM SIGCOMM Internet Measurement Conference (IMC), pages 202--208, November 2009. Google ScholarDigital Library
- S. Knight, H. Nguyen, N. Falkner, R. Bowden, and M. Roughan. The Internet Topology Zoo. IEEE J. Select. Areas Commun., 29(9):1765--1775, October 2011.Google ScholarCross Ref
- T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455--500, September 2009. Google ScholarDigital Library
- A. Lakhina, M. Crovella, and C. Diot. Diagnosing network-wide traffic anomalies. In Proc. ACM SIGCOMM, pages 219--230, September 2004. Google ScholarDigital Library
- A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk, and N. Taft. Structural analysis of network traffic flows. SIGMETRICS Perform. Eval. Rev., 32(1):61--72, June 2004. Google ScholarDigital Library
- L. Li, D. Alderson, W. Willinger, and J. Doyle. A first-principles approach to understanding the Internet's router-level topology. In Proc. ACM SIGCOMM, pages 3--14, August 2004. Google ScholarDigital Library
- A. Medina, N. Taft, K. Salmatian, S. Bhattacharyya, and C. Diot. Traffic matrix estimation: Existing techniques and new directions. In Proc. ACM SIGCOMM, 2002. Google ScholarDigital Library
- NLANR. Abilene Trace Data. http://pma.nlanr.net/Special/ipls3.html.Google Scholar
- A. Nucci, A. Sridharan, and N. Taft. The problem of synthetically generating IP traffic matrices: Initial recommendations. ACM Comput. Commun. Rev., 35:19--32, July 2005. Google ScholarDigital Library
- K. N. Oikonomou. Analytic forms for most likely matrices derived from incomplete information. Int. J. Systems Science, 43(3):443--458, September 2010.Google ScholarCross Ref
- A. Pakman and L. Paninski. Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians. J. Comput. Graph. Stat., 23(2):518--542, April 2014.Google ScholarCross Ref
- S. Y. Park and A. K. Bera. Maximum entropy autoregressive conditional heteroskedasticity model. J. Econometrics, 150:219--230, 2009.Google ScholarCross Ref
- R. B. Potts and R. M. Oliver. Flows in Transportation Networks. Academic Press, 1972.Google Scholar
- N. M. Radford. MCMC using Hamiltonian dynamics. In S. Brooks, A. Gelman, G. L. Jones, and X.-L. Meng, editors, Handbook of Markov Chain Monte Carlo, volume 31--32, pages 113--162. CRC Press, 2010.Google Scholar
- B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum--rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471--501, 2010. Google ScholarDigital Library
- M. Roughan. Simplifying the synthesis of Internet traffic matrices. ACM Comput. Commun. Rev., 35(5):93--96, 2005. Google ScholarDigital Library
- M. Roughan. Robust network planning. In C. R. Kalmanek, S. Misra, and R. Yang, editors, The Guide to Reliable Internet Services and Applications, chapter 5, pages 137--177. Springer, 2010.Google ScholarCross Ref
- M. Roughan, A. Greenberg, C. Kalmanek, M. Rumsewicz, J. Yates, and Y. Zhang. Experience in measuring backbone traffic variability: Models, metrics, measurements and meaning. In Proc. ACM SIGCOMM Internet Measurement Conference (IMC), 2002. Google ScholarDigital Library
- J. E. Shore and R. W. Johnson. Axiomatic derivation of the principle maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Info. Theory, 26(1):26--37, January 1980. Google ScholarDigital Library
- A. Soule, A. Lakhina, N. Taft, K. Papagiannaki, K. Salmatian, A. Nucci, M. Crovella, and C. Diot. Traffic matrix: Balancing measurements, inference and modeling. In Proc. ACM SIGMETRICS, pages 362--373, June 2005. Google ScholarDigital Library
- A. Soule, A. Nucci, R. Cruz, E. Leonardi, and N. Taft. How to identify and estimate the largest traffic matrix elements in a dynamic environment. SIGMETRICS Perform. Eval. Rev., 32(1):73--84, June 2004. Google ScholarDigital Library
- S. M. Stigler. The history of statistics: The measurement of uncertainty before 1900. Belknap Press of Harvard University Press, 1986.Google Scholar
- P. Tune and M. Roughan. Internet traffic matrices: A primer. In H. Haddadi and O. Bonaventure, editors, Recent Advances in Networking, Vol. 1. ACM SIGCOMM, August 2013.Google Scholar
- P. Tune and M. Roughan. Maximum entropy traffic matrix synthesis. In ACM SIGCOMM MAMA, June 2014. http://tinyurl.com/kk7757k.Google ScholarDigital Library
- P. Tune and M. Roughan. Network design sensitivity analysis. In Proc. ACM SIGMETRICS, pages 449--461, June 2014. Google ScholarDigital Library
- S. Uhlig, B. Quoitin, J. Lepropre, and S. Balon. Providing public intradomain traffic matrices to the research community. ACM Comput. Commun. Rev., 36(1):83--86, January 2006. Google ScholarDigital Library
- Y. Vardi. Network Tomography: Estimating source-destination traffic intensities from link data. J. Am. Statist. Assoc., 91:365--377, 1996.Google ScholarCross Ref
- A. Zellner and R. A. Highfield. Calculation of maximum entropy distributions and approximation of marginal posterior distributions. J. Econom., 37:195--209, 1988.Google ScholarCross Ref
- Y. Zhang, Z. Ge, A. Greenberg, and M. Roughan. Network anomography. In Proc. ACM SIGCOMM Internet Measurement Conference (IMC), Berkeley, California, USA, October 2005. Google ScholarDigital Library
- Y. Zhang, M. Roughan, N. Duffield, and A. Greenberg. Fast accurate computation of large-scale IP traffic matrices from link loads. In Proc. ACM SIGMETRICS, pages 206--217, 2003. Google ScholarDigital Library
- Y. Zhang, M. Roughan, C. Lund, and D. Donoho. An information-theoretic approach to traffic matrix estimation. In Proc. ACM SIGCOMM, pages 301--312, 2003. Google ScholarDigital Library
- Y. Zhang, M. Roughan, C. Lund, and D. Donoho. Estimating Point-to-Point and Point-to-Multipoint traffic matrices: An information-theoretic approach. IEEE/ACM Trans. Netw., 13(5):947--960, October 2005. Google ScholarDigital Library
- Y. Zhang, M. Roughan, W. Willinger, and L. Qiu. Spatio-Temporal compressive sensing and Internet traffic matrices. In Proc. ACM SIGCOMM, pages 267--278, August 2009. Google ScholarDigital Library
- R. Zhang-Shen and M. McKeown. Designing a predictable Internet backbone with Valiant load-balancing. In Proceedings of the 13th International Conference on Quality of Service (IWQoS), pages 178--192, 2005. Google ScholarDigital Library
Index Terms
- Spatiotemporal Traffic Matrix Synthesis
Recommendations
Spatiotemporal Traffic Matrix Synthesis
SIGCOMM '15: Proceedings of the 2015 ACM Conference on Special Interest Group on Data CommunicationTraffic matrices describe the volume of traffic between a set of sources and destinations within a network. These matrices are used in a variety of tasks in network planning and traffic engineering, such as the design of network topologies. Traffic ...
Invited A new traffic engineering manager for DiffServ/MPLS networks: design and implementation on an IP QoS Testbed
In a multi-service network, different applications have varying QoS requirements. The IETF has proposed the DiffServ architecture as a scalable solution to provide Quality of Service (QoS) in IP Networks. In order to provide quantitative guarantees and ...
Bandwidth Management for Supporting Differentiated Service Aware Traffic Engineering
This paper presents a bandwidth management framework for the support of the differentiated service aware traffic engineering (DS-TE) in the multiprotocol label switching (MPLS) networks. Our bandwidth management framework contains both bandwidth ...
Comments