Skip to main content
SpringerLink
Log in

Scalable datacenter multicast using in-packet bitmaps

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

Scalability is a key issue in datacenter multicast as it needs to support a large number of groups in commodity switches with limited fast memory. Previous in-packet Bloom filter-based datacenter multicast schemes have been proposed to address the scalability issue. They encode a multicast tree in a Bloom filter carried in each packet. However, these schemes induce high bandwidth overhead due to the false positives inherent in Bloom filters, and cannot scale well to the increasing variety of group sizes. In this paper, we propose an in-packet bitmap-based approach towards scalable datacenter multicast, improving the bandwidth efficiency. We use a bitmap to encode switch ports of a multicast tree in the packet header, eliminating false positive forwarding. In addition, we use a clustered Golomb coding method to compress in-packet bitmaps for further reducing the bandwidth overhead. Experimental results on simulations and a Click-based switch prototype demonstrate that our scheme achieves up to several orders of magnitude reductions in bandwidth overhead compared to previous schemes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Vigfusson, Y., Abu-Libdeh, H., Balakrishnan, M., Burgess, R., Birman, K., Chockler, G., Li, H., Tock, Y.: Dr. multicast: Rx for data center communication scalability. In: EuroSys (2010)

  2. Li, D., Yu, J., Yu, J., Wu, J.: Exploring efficient and scalable multicast routing in future data center networks. In: INFOCOM (2011)

  3. Cao, J., Guo, C., Lu, G., Xiong, Y., Zheng, Y., Zhang, Y., Zhu, Y., Chen, C.: Datacast: a scalable and efficient reliable group data delivery service for data centers. In: CoNEXT (2012)

  4. Li, X., Freedman, M.: Scaling IP multicast on datacenter topologies. In: CoNEXT (2013)

  5. Chu, Y., Rao, S., Seshan, S., Zhang, H.: A case of end system multicast. IEEE J. Sel. Areas Commun. 20(8), 1456–1471 (2002)

    Article  Google Scholar 

  6. Castro, M., Druschel, P., Kermarrec, A., Nandi, A., Rowstron, A., Singh, A.: SplitStream: high-bandwidth multicast in cooperative environments. In: SOSP (2003)

  7. Al-Fares, M., Loukissas, A., Vahdat, A.: A scalable, commodity data center network architecture. In: SIGCOMM (2008)

  8. Mysore, R., Pamboris, A., Farrington, N., Huang, N., Miri, P., Radhakrishnan, S., Subramanya, V., Vahdat, A.: PortLand: a scalable fault-tolerant layer 2 data center network fabric. In: SIGCOMM (2009)

  9. Greenberg, A., Jain, N., Kandula, S., Kim, C., Lahiri, P., Maltz, D., Patel, P., Sengupta, S.: VL2: a scalable and flexible data center network. In: SIGCOMM (2009)

  10. Guo, G., Lu, G., Li, D., Wu, H., Zhang, X., Shi, Y., Tian, C., Zhang, Y., Lu, S.: BCube: a high performance, server-centric network architecture for modular data centers. In: SIGCOMM (2009)

  11. Singla, A., Hong, C., Popa, L., Godfrey, P.: Jellyfish: networking data centers randomly. In: NSDI (2012)

  12. Ratnasamy, S., Ermolinskiy, A., Shenker, S.: Revisiting IP multicast. In: SIGCOMM (2006)

  13. Jokela, P., Zahemszky, A., Rothenberg, C., Arianfar, S., Nikander, P.: LIPSIN: line speed publish/subscribe inter-networking. In: SIGCOMM (2009)

  14. Sarela, M., Rothenberg, C., Aura, T., Zahemszky, A., Nikander, P., Ott, J.: Forwarding anomalies in Bloom filter-based multicast. In: INFOCOM (2011)

  15. Tian, X., Cheng, Y.: Loop mitigation in Bloom filter based multicast: a destination-oriented approach. In: INFOCOM (2012)

  16. Li, D., Cui, H., Hu, Y., Xia, Y., Wang, X.: Scalable data center multicast using multi-class Bloom filter. In: ICNP (2011)

  17. Yu, M., Fabrikant, A., Rexford, J.: BUFFALO: Bloom filter forwarding architecture for large organizations. In: CoNEXT (2009)

  18. Hu, S., Chen, K., Wu, H., Bai, W., Lan, C., Wang, H., Zhao, H., Guo, C.: Explicit path control in commodity data centers: design and applications. In: NSDI (2015)

  19. Liu, V., Halperin, D., Krishnamurthy, A., Anderson, T.: F10: a fault-tolerant engineered network. In: NSDI (2013)

  20. Sayood, K.: Introduction to data compression, 3rd edn. The Morgan Kaufmann Press, Burlington (2005)

    MATH  Google Scholar 

  21. Estrin, D., Farinacci, D., Helmy, A., Thaler, D., Deering, S., Handley, M., Jacobson, V., Liu, C., Sharma, P., Wei, L.: Protocol independent multicast-sparse mode (PIM-SM): protocol specification. In: IETF RFC, p. 2362 (1998)

  22. Yang, T., Xie, G., Li, Y., Fu, Q., Liu, A., Li, Q., Mathy, L.: Guarantee IP lookup performance with FIB explosion. In: SIGCOMM (2014)

  23. Yang, T., Zhang, H., Wang, H., Shahzad, M., Liu, X., Xin, Q., Li, X.: FID-sketch: an accurate sketch to store frequencies in data streams. World Wide Web (2018). https://doi.org/10.1007/s11280-018-0546-5

    Google Scholar 

  24. Yang, T., Liu, A., Shahzad, M., Zhong, Y., Fu, Q., Li, Z., Xie, G., Li, X.: A shifting Bloom filter framework for set queries. In: VLDB Endowment (2016)

  25. Yang, T., Liu, A., Shahzad, M., Zhong, Y., Fu, Q., Li, Z., Xie, G., Li, X.: A shifting framework for set queries. IEEE/ACM Trans. Netw. 25(5), 3116–3131 (2017)

    Article  Google Scholar 

  26. Jia, W.: A scalable multicast source routing architecture for data center networks. IEEE J. Sel. Areas Commun. 32(1), 116–123 (2014)

    Article  Google Scholar 

  27. Jia, W., Liu, G., Chen, Y.: A scalable source multipath routing architecture for datacenter networks. In: CNSM (2015)

  28. Fan, F., Hu, B., Yeung, K.: Distributed and dynamic multicast scheduling in FatTree data center networks. In: ICC (2016)

  29. Koponen, T., Casado, M., Gude, N., Stribling, J., Poutevski, L., Zhu, M., Ramanathan, R., Iwata, Y., Inoue, H., Hama, T., Shenker, S.: Onix: a distributed control platform for large-scale production networks. In: OSDI (2010)

  30. Zhou, D., Fan, B., Lim, H., Kaminsky, M., Andersen, D.: Scalable, high performance Ethernet forwarding with cuckooswitch. In: CoNEXT (2013)

  31. Kohler, E., Morris, R., Chen, B., Jannotti, J., Kaashoek, M.: The click modular router. ACM Trans. Comput. Syst. 18(3), 263–297 (2000)

    Article  Google Scholar 

  32. Broder, A., Mitzenmacher, M.: Network applications of Bloom filters: a survey. Internet Math. 1(4), 485–509 (2003)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to thank Wei Zhao and Jie Zhang for their great helps in the experiments. This work was supported in part by the National Science and Technology Major Project of China under Grant No. 2017ZX03001013-002, the Science and Technology Projects of Hunan Province China under Grant No. 2016JC2074, and the Research Foundation of Education Bureau of Hunan Province China under No. 16B085.

Author information

Authors and Affiliations

  1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Kun Huang

  2. Key Laboratory of Network Investigational Technology, Hunan Police Academy, Changsha, China

    Xin Su

Authors
  1. Kun Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kun Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, K., Su, X. Scalable datacenter multicast using in-packet bitmaps. Distrib Parallel Databases 36, 445–460 (2018). https://doi.org/10.1007/s10619-018-7224-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10619-018-7224-6

Keywords

Search

Navigation