Abstract
Software-Defined Networking (SDN) offers a flexible and programmable model to manage and schedule network resources. With the separation of control plane and data plane in SDN, the centralized controller frequently updates the flow tables in data plane to control network, and the data plane also communicates its requests with the control plane for forwarding actions. Due to the asynchronous control nature of SDN, a SDN network may suffer from inconsistency issues caused by concurrency violations, which may lead to network misbehaviors, such as communication interruption, policy violation, and performance degradation. To make SDN correct and stable, consistency is an essential feature that SDN should possess. In this paper, we make a survey about the control consistency in SDN from three aspects: (1) a consistent network view and control decisions made by the control plane, (2) consistency and efficiency during data plane update, (3) maintaining consistent flow tables in the data plane. We summarize the fundamental causes of consistency issues, the potential inconsistency problems and the state-of-art solutions, and then discuss the challenges to guarantee the consistency for SDN.





Similar content being viewed by others
References
Al-Shaer, E., Marrero, W., El-Atawy, A., Elbadawi, K.: Network configuration in a box: towards end-to-end verification of network reachability and security. In: IEEE International Conference on Network Protocols (ICNP), pp. 123–132 (2009)
Al-Shaer, E., Al-Haj, S.: Flowchecker: Configuration analysis and verification of federated openflow infrastructures. In: Proceedings of the 3rd ACM Workshop on Assurable and Usable Security Configuration, pp. 37–44 (2010)
Alimi, R., Wang, Y., Yang, Y.R.: Shadow configuration as a network management primitive. ACM SIGCOMM Comput. Commun. Rev. 38(4), 111–122 (2008)
Aslan, M., Matrawy, A.: Adaptive consistency for distributed SDN controllers. In: 17th International Telecommunications Network Strategy and Planning Symposium (Networks) (2016), pp. 150–157
Aslan, M., Matrawy, A.: Maintaining an up-to-date global network view in SDN. arXiv:1612.04944 (2016)
AuYoung, A., Ma, Y., Banerjee, S., Lee, J., Sharma, P., Turner, Y., Liang, C., Mogul, J.C.: Democratic resolution of resource conflicts between SDN control programs. In: ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) (2014)
Bairley, A., Xie, G.G.: Orchestrating network control functions via comprehensive trade-off exploration. In: NFV-SDN, pp. 114–120 (2016)
Ball, T., Bjørner, N., Gember, A., Itzhaky, S., Karbyshev, A., Sagiv, M., Schapira, M., Valadarsky, A.: Vericon: Towards verifying controller programs in software-defined networks. ACM SIGPLAN Notices 49(6), 282–293 (2014)
Benmessaoud, N., Tulloch, M., Williams, C., Mudigonda, U.M.: Microsoft System Center-Network Virtualization and Cloud Computing. Pearson Education, London (2014)
Berde, P., Gerola, M., Hart, J., Higuchi, Y., Kobayashi, M., Koide, T., Lantz, B., O’Connor, B., Radoslavov, P., Snow, W., et al.: Onos: towards an open, distributed SDN os. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2014)
Big Switch Networks: Project floodlight (2019). http://www.projectfloodlight.org. Accessed 10 July 2019
Bjorklund, M.: Yang-a data modeling language for the network configuration protocol, RFC6020. IETF (2010)
Brewer, E.: Cap twelve years later: How the” rules” have changed. Computer 45(2), 23–29 (2012)
Canini, M., Venzano, D., Peresini, P., Kostic, D., Rexford, J.: A nice way to test openflow applications. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI) (2012)
Canini, M., Kuznetsov, P., Levin, D., Schmid, S.: Software transactional networking: concurrent and consistent policy composition. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2013)
Chen, H., Benson, T.: The case for making tight control plane latency guarantees in SDN switches. In: ACM Symposium on SDN Research (2017)
Curtis, A.R., Mogul, J.C., Tourrilhes, J., Yalagandula, P., Sharma, P., Banerjee, S.: Devoflow: scaling flow management for high-performance networks. In: ACM SIGCOMM Conference (SIGCOMM) (2011)
Davie, B., Koponen, T., Pettit, J., Pfaff, B., Casado, M., Gude, N., Padmanabhan, A., Petty, T., Duda, K., Chanda, A.: A database approach to SDN control plane design. ACM SIGCOMM Comput. Commun. Rev. 47(1), 15–26 (2017)
Dixit, A., Hao, F., Mukherjee, S., Lakshman, T., Kompella, R.: Towards an elastic distributed SDN controller. ACM SIGCOMM Comput. Commun. Rev. 43(4), 7–12 (2013)
Dong, Q., Banerjee, S., Wang, J., Agrawal, D., Shukla, A.: Packet classifiers in ternary cams can be smaller. ACM SIGMETRICS Perform. Eval. Rev. 34(1), 311–322 (2006)
Dong, M., Li, H., Ota, K., Xiao, J.: Rule caching in SDN-enabled mobile access networks. IEEE Netw. 29(4), 40–45 (2015)
El-Hassany, A., Miserez, J., Bielik, P., Vanbever, L., Vechev, M.: Sdnracer: concurrency analysis for software-defined networks. ACM SIGPLAN Notices 51(6), 402–415 (2016)
Fayaz, S.K., Sharma, T., Fogel, A., Mahajan, R., Millstein, T.D., Sekar, V., Varghese, G.: Efficient network reachability analysis using a succinct control plane representation. In: USENIX Symposium on Operating Systems Design and Implementations (OSDI), pp. 217–232 (2016)
Ferguson, A.D., Guha, A., Liang, C., Fonseca, R., Krishnamurthi, S.: Hierarchical policies for software defined networks. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2012)
Ferguson, A.D., Guha, A., Liang, C., Fonseca, R., Krishnamurthi, S.: Participatory networking: an api for application control of SDNs. In: ACM SIGCOMM Conference (SIGCOMM) (2013)
Foerster, K.-T., Schmid, S., Vissicchio, S.: Survey of consistent network updates. arXiv:1609.02305 (2016)
Fogel, A., Fung, S., Pedrosa, L., Walraed-Sullivan, M., Govindan, R., Mahajan, R., Millstein, T.D.: A general approach to network configuration analysis. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp. 469–483 (2015)
Foster, N., Harrison, R., Freedman, M.J., Monsanto, C., Rexford, J., Story, A., Walker, D.: Frenetic: a network programming language. In: ACM SIGPLAN International Conference on Functional Programming (ICFP) (2011)
Förster, K.-T., Mahajan, R., Wattenhofer, R.: Consistent updates in software defined networks: on dependencies, loop freedom, and blackholes. In: IFIP International Conferences on Networking (Networking) (2016)
Google: A software defined wan architecture (2012). http://opennetsummit.org/archives/apr12/vahdat-wed-sdnstack.pdf. Accessed 30 Sept 2017
Hinrichs, T.L., Gude, N.S., Casado, M., Mitchell, J.C., Shenker, S.: Practical declarative network management. In: ACM SIGCOMM Workshop on Research on Enterprise Networking (WREN) (2009)
Hong, C.-Y., Kandula, S., Mahajan, R., Zhang, M., Gill, V., Nanduri, M., Wattenhofer, R.: Achieving high utilization with software-driven wan. In: ACM SIGCOMM Conference (SIGCOMM) (2013)
Jain, S., Kumar, A., Mandal, S., Ong, J., Poutievski, L., Singh, A., Venkata, S., Wanderer, J., Zhou, J., Zhu, M., et al.: B4: experience with a globally-deployed software defined wan. In: ACM SIGCOMM Conference (SIGCOMM) (2013)
Jin, X., Liu, H.H., Gandhi, R., Kandula, S., Mahajan, R., Zhang, M., Rexford, J., Wattenhofer, R.: Dynamic scheduling of network updates. In: ACM SIGCOMM Conference (SIGCOMM) (2014)
Jin, X., Gossels, J., Rexford, J., Walker, D.: Covisor: A compositional hypervisor for software-defined networks. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI) (2015)
Kang, N., Liu, Z., Rexford, J., Walker, D.: Optimizing the one big switch abstraction in software-defined networks. In: Proceedings of the ninth ACM conference on Emerging networking experiments and technologies, pp. 13–24 (2013)
Katta, N.P., Rexford, J., Walker, D.: Incremental consistent updates. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2013)
Katta, N., Alipourfard, O., Rexford, J., Walker, D.: Cacheflow: dependency-aware rule-caching for software-defined networks. In: ACM Symposium on SDN Research (2016)
Kazemian, P., Chan, M., Zeng, H., Varghese, G., McKeown, N., Whyte, S.: Real time network policy checking using header space analysis. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp. 99–111 (2013)
Kazemian, P., Varghese, G., McKeown, N.: Header space analysis: static checking for networks. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI), vol. 12, pp. 113–126 (2012)
Khurshid, A., Zhou, W., Caesar, M., Godfrey, P.: Veriflow: Verifying network-wide invariants in real time. ACM SIGCOMM Comput. Commun. Rev. 42(4), 467–472 (2012)
Kim, C., Caesar, M., Gerber, A., Rexford, J.: Revisiting route caching: The world should be flat. In: Passive and Active Measurement Conference (PAM), vol. 9, pp. 3–12 (2009)
Kim, H., Reich, J., Gupta, A., Shahbaz, M., Feamster, N., Clark, R.: Kinetic: Verifiable dynamic network control. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI) (2015)
Koley, B.: Software defined networking at scale (2014). https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42948.pdf. Accessed 30 Sept 2017
Koponen, T., Casado, M., Gude, N., Stribling, J., Poutievski, L., Zhu, M., Ramanathan, R., Iwata, Y., Inoue, H., Hama, T., et al.: Onix: a distributed control platform for large-scale production networks. In: USENIX Symposium on Operating Systems Design and Implementations (OSDI) (2010)
Kuźniar, M., Perešíni, P., Kostić, D.: What you need to know about SDN flow tables. In: International Conference on Passive and Active Network Measurement, pp. 347–359 (2015)
Levin, D., Wundsam, A., Heller, B., Handigol, N., Feldmann, A.: Logically centralized?: state distribution trade-offs in software defined networks. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN), pp. 1–6 (2012)
Liu, A.X., Meiners, C.R., Torng, E.: Tcam razor: a systematic approach towards minimizing packet classifiers in tcams. IEEE/ACM Trans. Netw. 18(2), 490–500 (2010)
Liu, H.H., Wu, X., Zhang, M., Yuan, L., Wattenhofer, R., Maltz, D.: zUpdate: updating data center networks with zero loss. In: ACM SIGCOMM Conference (SIGCOMM) (2013)
Loo, B.T., Condie, T., Hellerstein, J.M., Maniatis, P., Roscoe, T., Stoica, I.: Implementing declarative overlays. ACM SIGOPS Oper. Syst. Rev. 39(5), 75–90 (2005)
Loo, B.T., Condie, T., Garofalakis, M., Gay, D.E., Hellerstein, J.M., Maniatis, P., Ramakrishnan, R., Roscoe, T., Stoica, I.: Declarative networking: language, execution and optimization. In: ACM Conference on Management of Data (SIGMOD) (2006)
Lopes, N.P., Bjørner, N., Godefroid, P., Jayaraman, K., Varghese, G.: Checking beliefs in dynamic networks. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI), pp. 499–512 (2015)
Ludwig, A., Rost, M., Foucard, D., Schmid, S.: Good network updates for bad packets: waypoint enforcement beyond destination-based routing policies. In: ACM Workshop on Hot Topics in Networks (HotNets) (2014)
Luo, S., Yu, H., Li, L.: Practical flow table aggregation in SDN. Comput. Netw. 92, 72–88 (2015)
Mahajan, R., Wattenhofer, R.: On consistent updates in software defined networks. In: ACM Workshop on Hot Topics in Networks (HotNets) (2013)
Mai, H., Khurshid, A., Agarwal, R., Caesar, M., Godfrey, P., King, S.T.: Debugging the data plane with anteater. ACM SIGCOMM Comput. Commun. Rev. 41(4), 290–301 (2011)
Marsico, A., Doriguzzi-Corin, R., Siracusa, D.: An effective swapping mechanism to overcome the memory limitation of SDN devices. In: IFIP/IEEE Symposium on Integrated Network and Service Management (IM) (2017)
McClurg, J., Hojjat, H., Foster, N.: Efficient synthesis of network updates. In: ACM SIGPLAN Symposium on Programming Language Design & Implementation (PLDI) (2015)
McGeer, R.: A safe, efficient update protocol for openflow networks. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2012)
Medved, J., Varga, R., Tkacik, A., Gray, K.: Opendaylight: towards a model-driven SDN controller architecture. In: IEEE International Symposium on A World of Wireless (WoWMoM) (2014)
Meiners, C.R., Liu, A.X., Torng, E.: Bit weaving: a non-prefix approach to compressing packet classifiers in tcams. IEEE/ACM Trans. Netw. 20(2), 488–500 (2012)
Microsoft: Sdn in windows server overview (2018). https://technet.microsoft.com/en-us/windows-server-docs/networking/sdn/software-defined-networking. Accessed 30 Sept 2017
Mijumbi, R., Serrat, J., Gorricho, J.-L., Bouten, N., De Turck, F., Boutaba, R.: Network function virtualization: state-of-the-art and research challenges. IEEE Commun. Surv. Tutor. 18(1), 236–262 (2016)
Mizrahi, T., Moses, Y.: Time-based updates in software defined networks. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN), pp. 163–164 (2013)
Mizrahi, T., Moses, Y.: Software defined networks: it’s about time. In: IEEE Conference on Computer Communications (INFOCOM) (2016)
Mizrahi, T., Rottenstreich, O., Moses, Y.: Timeflip: scheduling network updates with timestamp-based tcam ranges. In: IEEE Conference on Computer Communications (INFOCOM) (2015)
Mizrahi, T., Saat, E., Moses, Y.: Timed consistent network updates in software-defined networks. IEEE/ACM Trans. Netw. 24(6), 3412–3425 (2016)
Mogul, J.C., AuYoung, A., Banerjee, S., Popa, L., Lee, J., Mudigonda, J., Sharma, P., Turner, Y.: Corybantic: towards the modular composition of SDN control programs. In: ACM Workshop on Hot Topics in Networks (HotNets) (2013)
Nelson, T., Ferguson, A.D., Scheer, M.J., Krishnamurthi, S.: Tierless programming and reasoning for software-defined networks. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI), vol. 14, pp. 519–531 (2014)
Nicira: Nox (2014). https://github.com/noxrepo/. Accessed 10 July 2019
ON.Lab: Onos (2019). https://onosproject.org. Accessed 10 July 2019
OpenDaylight: Opendaylight Project (2018). http://www.opendaylight.org/. Accessed 10 July 2019
Panda, A., Zheng, W., Hu, X., Krishnamurthy, A., Shenker, S.: Scl: simplifying distributed SDN control planes. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI) (2017)
Porras, P., Shin, S., Yegneswaran, V., Fong, M., Tyson, M., Gu, G.: A security enforcement kernel for openflow networks. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2012)
Prakash, C., Lee, J., Turner, Y., Kang, J.-M., Akella, A., Banerjee, S., Clark, C., Ma, Y., Sharma, P., Zhang, Y.: Pga: Using graphs to express and automatically reconcile network policies. In: ACM SIGCOMM Conference (SIGCOMM) (2015)
Reitblatt, M., Foster, N., Rexford, J., Walker, D.: Consistent updates for software-defined networks: change you can believe in! In: ACM Workshop on Hot Topics in Networks (HotNets) (2011)
Reitblatt, M., Foster, N., Rexford, J., Schlesinger, C., Walker, D.: Abstractions for network update. In: ACM SIGCOMM Conference (SIGCOMM) (2012)
NTT: Ryu (2017). https://osrg.github.io/ryu/. Accessed 10 July 2019
Sarrar, N., Uhlig, S., Feldmann, A., Sherwood, R., Huang, X.: Leveraging zipf’s law for traffic offloading. ACM SIGCOMM Comput. Commun. Rev. 42(1), 16–22 (2012)
Shah, D., Gupta, P.: Fast updating algorithms for tcam. IEEE Micro 21(1), 36–47 (2001)
Shelly, N., Jackson, E.J., Koponen, T., McKeown, N., Rajahalme, J.: Flow caching for high entropy packet fields. ACM SIGCOMM Comput. Commun. Rev. 44(4), 151–156 (2014)
Soulé, R., Basu, S., Marandi, P.J., Pedone, F., Kleinberg, R., Sirer, E.G., Foster, N.: Merlin: a language for provisioning network resources. In: ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) (2014)
Sun, P., Mahajan, R., Rexford, J., Yuan, L., Zhang, M., Arefin, A.: A network-state management service. In: ACM SIGCOMM Conference (SIGCOMM) (2014)
Tootoonchian, A., Ganjali, Y.: Hyperflow: A distributed control plane for openflow. In: Proceedings of the 2010 internet network management conference on Research on enterprise networking, pp. 3–3 (2010)
NEC: Trema (2018). https://trema.github.io/trema/. Accessed 10 July 2019
Vamanan, B., Vijaykumar, T.: Treecam: decoupling updates and lookups in packet classification. In: ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) (2011)
Vanian, J.: Facebook shows the promise of sdn with new networking tech (2014). https://gigaom.com/2014/11/14/facebook-shows-the-promise-of-sdn-with-new-networking-tech/. Accessed 30 Sept 2017
Vishnoi, A., Poddar, R., Mann, V., Bhattacharya, S.: Effective switch memory management in openflow networks. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, ACM, pp. 177–188 (2014)
Vissicchio, S., Cittadini, L.: Flip the (flow) table: fast lightweight policy-preserving SDN updates. In: IEEE Conference on Computer Communications (INFOCOM) (2016)
Voellmy, A., Wang, J., Yang, Y.R., Ford, B., Hudak, P.: Maple: Simplifying SDN programming using algorithmic policies. ACM SIGCOMM Comput. Commun. Rev. 43(4), 87–98 (2013)
Voellmy, A., Agarwal, A., Hudak, P.: Nettle: functional reactive programming for openflow networks. In: International Symposium on Practical Aspects of Declarative Languages (PADL) (2011)
Volpano, D.M., Sun, X., Xie, G.G.: Towards systematic detection and resolution of network control conflicts. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN) (2014)
Wang, Z., Che, H., Kumar, M., Das, S.K.: Coptua: consistent policy table update algorithm for tcam without locking. IEEE Trans. Comput. 53(12), 1602–1614 (2004)
Wang, W., He, W., Su, J.: Redactor: reconcile network control with declarative control programs in SDN. In: IEEE International Conference on Network Protocols (ICNP) (2016a)
Wang, W., He, W., Su, J., Chen, Y.: Cupid: congestion-free consistent data plane update in software defined networks. In: IEEE Conference on Computer Communications (INFOCOM) (2016b)
Wang, C., Mei, X., Croft, J., Caesar, M., Godfrey, B.: Ravel: A database-defined network. In: ACM Symposium on SDN Research (2016c)
Wen, X., Diao, C., Zhao, X., Chen, Y., Li, L.E., Yang, B., Bu, K.: Compiling minimum incremental update for modular SDN languages. In: ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN), pp. 193–198 (2014)
Wen, X., Yang, B., Chen, Y., Li, L.E., Bu, K., Zheng, P., Yang, Y., Hu, C.: Ruletris: minimizing rule update latency for tcam-based SDN switches. In: IEEE International Conference on Distributed Computing Systems (ICDCS) (2016)
Yang, H., Lam, S.S.: Real-time verification of network properties using atomic predicates. IEEE/ACM Trans. Netw. 24(2), 887–900 (2016)
Yeganeh, S.H., Tootoonchian, A., Ganjali, Y.: On scalability of software-defined networking. IEEE Commun. Mag. 51(2), 136–141 (2013)
Yu, Y., Qian, C., Wu, W., Zhang, Y.: Netcp: Consistent, non-interruptive and efficient checkpointing and rollback of SDN. In: IEEE/ACM International Symposium on Quality of Service (IWQoS) (2018)
Yuan, B., Zou, D., Yu, S., Jin, H., Qiang, W., Shen, J.: Defending against flow table overloading attack in software-defined networks. IEEE Trans. Serv. Comput. (2016)
Zeng, H., Zhang, S., Ye, F., Jeyakumar, V., Ju, M., Liu, J., McKeown, N., Vahdat, A.: Libra: divide and conquer to verify forwarding tables in huge networks. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI), vol. 14, pp. 87–99 (2014)
Zheng, J., Chen, G., Schmid, S., Dai, H., Wu, J.: Chronus: consistent data plane updates in timed SDNs. In: IEEE International Conference on Distributed Computing Systems (ICDCS) (2017)
Zhou, W., Jin, D., Croft, J., Caesar, M., Godfrey, P.B.: Enforcing customizable consistency properties in software-defined networks. In: USENIX Symposium on Networked Systems Design and Implementation (NSDI) (2015)
Acknowledgements
This work is partially supported by the National Key R&D Program of China (Grant no. 2017YFB0802300) and the National Natural Science Foundation of China (Grant nos. 61602503, 61601483).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Su, J., Wang, W. & Liu, C. A survey of control consistency in Software-Defined Networking. CCF Trans. Netw. 2, 137–152 (2019). https://doi.org/10.1007/s42045-019-00022-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42045-019-00022-w