Software defined cloud: Survey, system and evaluation

https://doi.org/10.1016/j.future.2015.10.015Get rights and content

Highlights

  • A comprehensive survey for Software Defined Systems.

  • Proposing a novel framework for Software Defined Cloud.

  • Integrating different Software Defined Systems components in one platform.

  • Validate applicability of the proposed model using simulation.

Abstract

Next generation cloud systems will require a paradigm shift in how they are constructed and managed. Conventional control and management platforms are facing considerable challenges regarding flexibility, dependability and security that next generation systems must handle. The cloud computing technology has already contributed in alleviating a number of the problems associated with resource allocation, utilization and management. However, many of the elements of a well-designed cloud environment remain “stiff” and hard to modify and adapt in an integrated fashion. This includes the underlying networking topologies, many aspects of the user control over IaaS, PaaS or SaaS layers, construction of XaaS services, provenance and meta-data collection, to mention but few. In many situations the problem may be due to inadequacy of service abstraction. Software Defined Systems (SDSys) is a concept that help abstract the actual hardware at different layers with software components; one classical example of this abstractions are hypervisors. Such abstraction provides an opportunity for system administrators to construct and manage their systems, more easily, through flexible software layers. SDSys is an umbrella for different software defined subsystems including Software Defined Networking (SDN), Software Defined Storage (SDStorage), Software Defined Servers (Virtualization), Software Defined Data Centers (SDDC), Software Defined Security (SDSec) etc. and ultimately Software Defined Clouds (SDCloud). Individual solutions and seamless integration of these different abstractions remains in many respects a challenge. In this paper, the authors introduce Software Defined Cloud (SDCloud), a novel software defined cloud management framework that integrates different software defined cloud components to handle complexities associated with cloud computing systems. The first part of paper presents, for the first time, an extensive state of the art critical review of different components of software defined systems, constructing the proposed SDCloud. The second part of the paper proposes the novel concept of SDCloud, which is implemented and evaluated for its feasibility, flexibility and potential superiority.

Introduction

With the rapid shift to the cloud computing paradigm, one of the most critical issues faced by system administrators is the construction and management of systems in a manner that eliminates or hides their complexity from the end users, and, at the same time, maintain their control, flexibility, dependability, and security. To achieve these goals, Software Defined Systems (SDSys) has emerged recently to address these control and management challenges that exist in traditional platforms. The cost of management and administration operations are very high compared to other system operations in today’s computing systems  [1]. Thus, adopting the SDSys paradigm to cut the management and administration cost has become very appealing.

The cloud computing technology has already contributed in alleviating a number of the problems associated with resource allocation, utilization and management. However, many of the elements of a well-designed cloud environment remain “stiff” and hard to modify and adapt in an integrated fashion. This includes the underlying networking topologies, many aspects of the user control over IaaS, PaaS or SaaS layers, construction of XaaS services, provenance and meta-data collection, to mention but few. SDSys comes to supplement the cloud computing technologies by addressing the way to efficiently relax this stiffness and integrate the elements that are still hard to modify or adapt in cloud computing. SDSys is a concept that help abstract the actual hardware at different layers with software components. Such abstraction provides system administrators with the ability to construct and handle all control and management decisions by a central decision maker, through flexible software layers. For example, one of the most important features of cloud computing is providing “on-demand” services; however, this is challenging to achieve in a decentralized control unit, where every component manages itself and has no information about other components. The request for adding a new decision or make some enhancements on the system requires knowing information about the other components of the system. Such information will have to be collected and analyzed in a single unit. Consequently, there will be some delay in response, which contrasts with the need for timely on-demand response. The problem of combining the achievement of multiple objectives with fast responses required by on-demand services in a single system is what SDSys is set to achieve  [2], [3].

SDSys is an umbrella for different software defined subsystems including Software Defined Network (SDN), Software Defined Security (SDSec), Software Defined Storage (SDStorage), Software Defined Data Center (SDDC)  [4], Software Defined Infrastructure (SDI), Software Defined Management (SDM), Software Defined Compute (SDCompute), Software Defined Server (SDSer), Software Defined Internet of Things (SDIoT)  [5], Software Defined Radio (SDR), and Software Defined Enterprise (SDEn). The software layer can control different types of hardware devices in various contexts giving rise to the term Software Defined everything or anything (SDx).1 Many advantages can be driven from SDSys such as increasing the performance, scalability, and security of the system to facilitate resource management.

SDSys provide the ability to control a wide range of computing resources in a work-flow driven and dynamic fashion by separating the control layer from the data work flow layer, i.e., isolating the control from hardware devices and setting it in a software layer. The main idea behind the SDSys is to build an orchestrated system, controller, to handle the control for all independent devices by using standard and general protocols.2 This is achieved by means of virtualization. As virtualization is considered to be a key concept of the cloud computing technology, it also plays an essential role in SDSys. Virtualization creates a virtual platform for different devices or system components (like network, OS and storage devices) that emulates the characteristics of the real devices. The transformation from hardware control to software centric control is, therefore, achieved through virtualization, in which the functionalities of a single or multiple systems can be abstracted allowing the integration of the benefits of these systems into a multiple purpose function. As an illustration, consider the Software Defined Data Center (SDDC) example, which integrates a set of servers, storage devices, and networks into a single comprehensive system or resource pools.3

There are many reasons why the world is focusing so much on virtualization. The most important ones are as follows.

  • 1.

    Resource sharing: If we have idle resources that exceed our need then we can increase the resource utilization by splitting them among several virtual machines. The resources can be anything like storage, disks or even links.

  • 2.

    User isolation: In certain situations, there is a need to keep a level of confidentiality between the users. Using virtualization, each user may have its own VM, which is separate from other users’ VMs.

  • 3.

    Resources aggregation: If there is a need to perform a task that requires resources beyond what we have, then we can use the virtualization to build suitable and useful virtual resources large enough to complete the task.

  • 4.

    Dynamic management and control: The users requirements are changing frequently which makes the response for these changes in virtualization easier than when dealing directly with physical resources.

  • 5.

    Simplify the management: Manage devices through a software layer is easier than manage them in a physical layer.

It is the software defined concept and other related concepts such as “software deployed”. In the former, the APIs and software are used to control and mange resources and devices. On the other hand, the software deployed concept means that the functionality of the service is deployed in a computer hardware object. Using the software to manage and control the resources is not a new concept. The essential difference, which was brought by the software defined concept is the ability of the control layer to control all the underlying resources regardless of their vendor variations by physically isolating them from the hardware resources in the data layer  [6]. The concept of abstraction in SDSys is similar to the idea of Object Oriented (OO), where the implementation is separated from the interface representing the data layer and the control layer respectively in SDSys. The reason behind this separation is to simplify the modification process, since any change in the implementation will not affect the interface and vice versa  [6].

To recall, implementing SDSys solutions for cloud computing is currently very fragmented and is still a growing research and development project, despite its proven advantages and wider acceptance amongst specialists from academia and industry. The aim to achieve a fully integrated SDSys cloud computing, combining all aspects of software defined systems, is still far from being realized. This ultimate system, is referred to as Software Defined Cloud (SDCloud); a concept which was first introduced in the Harmony system  [3]. Harmony proposed to integrate and manage the main components of any computing system such as the computing resources, storage resources and networks into one platform that represents the SDCloud concept. To the best of the authors’ knowledge, the main attempt in literature to address a practical implementation of SDCloud is reported in  [2] who presented an architecture for Software Defined Clouds for data centers. They consider different cloud applications and services, focusing on mobile cloud applications. However, the proposed architecture focuses mainly on SDN, Software-Defined Middleboxes Networking and Network Virtualization.

The proposed system in this paper is an attempt to further expand the SDSys capabilities in order to implement the ultimate aim of SDCloud, by integrating several aspects of SDSys, working cohesively together. In this paper, we focus mainly on the integration of Software Defined Networking (SDN), Software Defined Compute (SDCompute), Software Defined Storage (SDStorage), and software defined security (SDSec), which makes it, the most complex implementation of SDCloud, yet to be reported. SDCompute is considered as a core components of any SDCloud  [7], [8], [9], [10], [11], [12]. SDCompute is the first step towards realizing a complete SDSys which represent the virtualization of the physical server (nodes) into a set of virtual resources using one of virtualization technologies like Xen. The proposed system is implemented and evaluated by introducing major extensions to the Mininet simulation tool which is an open source tool introduced for SDN  [13]. Since research in SDSys is still a growing and very fragmented area, it is worthwhile presenting a comprehensive review of the selected software defined subsystems, more specifically, SDN, SDStorage, and SDSec. This review is also a current gap in literature; it combines efforts from both academia and industry. Therefore, presenting this critical review in the paper is another worthwhile contribution. Due to the lack of academic published research in SDSys, and since there are more developments in SDSys by industry, the review included references to industrial white papers/pages and experts blogs, to ensure coverage.

The rest of this paper is structured as follows. Sections  2 Software Defined Network (SDN), 3 Software Defined Storage (SDStorage), 4 Software Defined Security (SDSec) provide state of the art work on the selected software defined subsystems, SDN, SDStorage, and SDSec respectively. Section  5 presents and discusses the proposed SDCloud architecture design and prototype integrating the aforementioned software defined subsystems. The proposed system prototype and experimental evaluation is the presented in Section  6. Finally, we conclude and present our future plans in Section  7.

Section snippets

Software Defined Network (SDN)

Software Defined Network (SDN), is considered to be the most popular form of SDSys, and it is much more researched than other forms of SDSys. As with any SDSys, SDN is also based on virtualization where the network resources are virtualized  [14]. Virtualization of network resources is not a new concept. For example, in different types of multiple access schemes, channels are “virtualized” to allow multiple users to share the same physical channel. The same thing applies to the virtual local

Software Defined Storage (SDStorage)

The main goal for any software defined system is to hide all the complexities of the management and control functionality of the system resources from the end users. These resources may refer to any system component like the components and devices that store and process the data. SDSys provides a new concept called Software Defined Storage (SDStorage) to facilitate and simplify such complexity, and at the same time, maintain an acceptable level of QoS  [48]. SDStorage comes to handle all the

Software Defined Security (SDSec)

Virtualization is increasingly being adopted considering the reduced running costs and increase system flexibility. However, security remains one of the major issues. with the emergence of SDSys, the challenge of security has been highlighted even more  [65]. It is not practical to follow traditional security mechanisms in SDN and SDStorage systems. The new architecture in software defined systems requires alternative solutions to be able to deal with such differences. SDN central  [66]

SDCloud: System architecture and design

The design principles of SDSys served as a motivation to propose a comprehensive system architecture framework as an indication of how the system architecture will be designed in the future. This framework in composed of the software defined subsystems previously discussed in this paper: SDStorage, SDN and SDSec. Many ideas are abstracted, integrated and extended inside our model. Such integration guarantees provisioning an entirely software-based architecture for any system. The motivation

Experimental results and evaluation

In this section, we discuss the implementation details of the proposed framework. We also discuss the experiments conducted on it and the results obtained from these experiments. We start with how the framework is implemented by expanding an existing SDN simulator.

Conclusion

This paper argues that the emergence of Software-Defined Systems was an inevitable result of the paradigm shift from traditional computing models to utility-based Cloud computing. Cloud computing providers typically rely on virtualization (an abstraction of computing resources through software) to effectively and efficiently manage their underlying hardware. Virtualization provides the ability to logically divide physical resources, which allows secure, efficient, multi-tenancy upon single

Acknowledgments

The authors would like to thank the Deanship of Research at the Jordan University of Science and Technology for funding this work, grant number 20150050. Also, they would like to thank IBM and IBM Cloud Academy for their support.

Yaser Jararweh received his Ph.D. in Computer Engineering from University of Arizona in 2010. He is currently an assistant professor of Computer Science at Jordan University of Science and Technology, Jordan. He has co-authored about seventy technical papers in established journals and conferences in fields related to cloud computing, HPC, SDN and Big Data. He was one of the TPC Co-Chair, IEEE Globecom 2013 International Workshop on Cloud Computing Systems, and Networks, and Applications

References (82)

  • Y. Jararweh et al.

    Cloudexp: a comprehensive cloud computing experimental framework

    Simul. Model. Practice Theory

    (2014)
  • S. Hariri et al.

    The autonomic computing paradigm

    Cluster Comput.

    (2006)
  • R. Buyya, R.N. Calheiros, J. Son, A.V. Dastjerdi, Y. Yoon, Software-defined cloud computing: Architectural elements and...
  • R. Grandl et al.

    Harmony: Coordinating network, compute, and storage in software-defined clouds

  • M.A.-A. AlaDarabseh, Y. Jararweh, E. Benkhelifa, M. Vouk, A. Rindos, Sddc: A software defined datacenter experimental...
  • Y. Jararweh et al.

    Sdiot: a software defined based internet of things framework

    J. Ambient Intell. Humanized Comput.

    (2015)
  • L. MacVittie, Stop conflating software-defined with software-deployed, 2014,...
  • P. Barham et al.

    Xen and the art of virtualization

    ACM SIGOPS Oper. Syst. Rev.

    (2003)
  • E. Bugnion et al.

    Bringing virtualization to the x86 architecture with the original vmware workstation

    ACM Trans. Comput. Syst.

    (2012)
  • G. Kandiraju et al.

    Software defined infrastructures

    IBM J. Res. Dev.

    (2014)
  • C. Li et al.

    Software defined environments: An introduction

    IBM J. Res. Dev.

    (2014)
  • M. Al-Ayyoub et al.

    Multi-agent based dynamic resource provisioning and monitoring for cloud computing systems infrastructure

    Cluster Comput.

    (2015)
  • R. de Oliveira, A. Shinoda, C. Schweitzer, L. Rodrigues Prete, Using mininet for emulation and prototyping...
  • R. Jain et al.

    Network virtualization and software defined networking for cloud computing: a survey

    IEEE Commun. Mag.

    (2013)
  • B. Nunes et al.

    A survey of software-defined networking: Past, present, and future of programmable networks

    IEEE Commun. Surv. Tutor.

    (2014)
  • F. Hu et al.

    A survey on software defined networking (sdn) and openflow: From concept to implementation

    IEEE Commun. Surv. Tutor.

    (2014)
  • A. Tootoonchian et al.

    Hyperflow: A distributed control plane for openflow

  • K. Phemius et al.

    Disco: distributed sdn controllers in a multi-domain environment

  • D.A. Drutskoy

    Software-defined network virtualization with flown

    (2012)
  • B. Sonkoly, A. Gulyas, F. Nemeth, J. Czentye, K. Kurucz, B. Novak, G. Vaszkun, Openflow virtualization framework with...
  • J. Matias, E. Jacob, D. Sanchez, Y. Demchenko, An openflow based network virtualization framework for the cloud, in:...
  • J. Matias, B. Tornero, A. Mendiola, E. Jacob, N. Toledo, Implementing layer 2 network virtualization using openflow:...
  • M. El-azzab, I. Bedhiaf, Y. Lemieux, O. Cherkaoui, Slices isolator for a virtualized openflow node, in: Network Cloud...
  • D. Turull, M. Hidell, P. Sjodin, libnetvirt: The network virtualization library, in: Communications, ICC, 2012 IEEE...
  • C. Dixon et al.

    Software defined networking to support the software defined environment

    IBM J. Res. Dev.

    (2014)
  • Openstack, http://www.openstack.org/  [Online; accessed...
  • Y. Jarraya et al.

    A survey and a layered taxonomy of software-defined networking

    IEEE Commun. Surv. Tutor.

    (2014)
  • A. Doria et al.

    Forwarding and control element separation (forces) protocol specification, Internet standards track document

    Internet Eng. Task Force

    (2010)
  • N. Foster et al.

    Languages for software-defined networks

    IEEE Commun. Mag.

    (2013)
  • D. Kotani, K. Suzuki, H. Shimonishi, A design and implementation of openflow controller handling ip multicast with fast...
  • Z. Bozakov, P. Papadimitriou, Autoslice: Automated and scalable slicing for software-defined networks, in: Proceedings...
  • A. Tootoonchian, S. Gorbunov, Y. Ganjali, M. Casado, R. Sherwood, On controller performance in software-defined...
  • T. Luo, H.-P. Tan, P. Quan, Y.W. Law, J. Jin, Enhancing responsiveness and scalability for openflow networks via...
  • Y. Kanizo, D. Hay, I. Keslassy, Palette: Distributing tables in software-defined networks, in: INFOCOM, 2013...
  • Y. Hu, W. Wang, X. Gong, X. Que, S. Cheng, Balanceflow: Controller load balancing for openflow networks, in: Cloud...
  • H. Egilmez, B. Gorkemli, A. Tekalp, S. Civanlar, Scalable video streaming over openflow networks: An optimization...
  • H. Egilmez, S. Dane, K. Bagci, A. Tekalp, Openqos: An openflow controller design for multimedia delivery with...
  • B.-Y. Ke, P.-L. Tien, Y.-L. Hsiao, Parallel prioritized flow scheduling for software defined data center network, in:...
  • M. Canini et al.

    A nice way to test openflow applications

  • S. Shin et al.

    Fresco: Modular composable security services for software-defined networks

  • L. Liu et al.

    Field trial of an openflow-based unified control plane for multilayer multigranularity optical switching networks

    J. Lightwave Technol.

    (2013)
  • Cited by (100)

    • QoS Performance Enhancement Policy through Combining Fog and SDN

      2021, Simulation Modelling Practice and Theory
      Citation Excerpt :

      Scaling can usually be achieved horizontally & vertically while vertical scaling is preferable to horizontal scaling. A number of QoS issues are still unresolved that finds in our literature survey (reported in [27-56]), and others have potential to improve. By collaborating SDN with FC, these QoS issues can be solved and can be enhanced where appropriate.

    • Deep learning architectures in emerging cloud computing architectures: Recent development, challenges and next research trend

      2020, Applied Soft Computing Journal
      Citation Excerpt :

      In SDN, the control plane and the data plane are disjointed making the control plane centralized while the data panel remains distributed. With centralized control plane, decision making becomes much faster [64]. It also allows modification of the control panel features on many devices to suit any design objective [61].

    View all citing articles on Scopus

    Yaser Jararweh received his Ph.D. in Computer Engineering from University of Arizona in 2010. He is currently an assistant professor of Computer Science at Jordan University of Science and Technology, Jordan. He has co-authored about seventy technical papers in established journals and conferences in fields related to cloud computing, HPC, SDN and Big Data. He was one of the TPC Co-Chair, IEEE Globecom 2013 International Workshop on Cloud Computing Systems, and Networks, and Applications (CCSNA). He is a steering committee member for CCSNA 2014 and CCSNA 2015 with ICC. He is the General Co-Chair in IEEE International Workshop on Software Defined Systems SDS-2014 and SDS 2015. He is also chairing many IEEE events such as ICICS, SNAMS, BDSN, IoTSMS and many others. Dr. Jararweh served as a guest editor for many special issues in different established journals. Also, he is the steering committee chair of the IBM Cloud Academy Conference.

    Mahmoud Al-Ayyoub Received his Ph.D. in Computer Science from Stony Brook University in 2010. He is currently an assistant professor of Computer Science at Jordan University of Science and Technology (JUST). His research interests include as cloud computing, high performance computing, machine learning and AI. He is the co-director of the High Performance and Cloud Computing research lab at JUST.

    Ala’a Drabseh is a Master student of Computer Science at Jordan University of Science and Technology (JUST). Her research interests include cloud computing, IoT and SDS.

    Elhadj Benkhelifa Elhadj is an Associate Professor (Reader), at Staffordshire University, UK, with extensive experience in working with industry on real world business problems. Elhadj is the Faculty Director of the Mobile Fusion Applied Research Centre (45 Ph.D. students and 15+ Staff). During his academic career Elhadj has built a rich portfolio of successful national and international collaborations. Over the past 3 years, Elhadj successfully secured external funding in excess of $1.5 million USD. Elhadj is the Founding Head of the Cloud Computing and Applications Research Group, leading a team of 10 Ph.D. Students and Research Staff. Elhadj has a strong research publication and dissemination track record and a co-founding chair of several conferences/workshops IEEE CCSNA, IEEE BDSNA, IEEE SNAMS, IEEE SDS, IEEE IOTSMS, to mention but few. Elhadj is a Senior R&D Advisor to a number of companies in the UK and a member of several scientific and industrial panels and committees within the UK and internationally.

    Mladen Vouk ​received Ph.D. from the King’s College, University of London, U.K. He is Department Head and Professor of Computer Science, and Associate Vice Provost for Information Technology at N.C. State University, Raleigh, N.C., U.S.A. Dr. Vouk has extensive experience in both commercial software production and academic computing. He is the author/co-author of over 300 publications. His research and development interests include software engineering, scientific computing and analytics, information technology (IT) assisted education, and high-performance computing and clouds. Dr. Vouk has extensive professional visibility through organization of professional meetings, membership on professional journal editorial boards, and professional consulting. Dr. Vouk is a member of the IFIP Working Group 2.5 on Numerical Software, and a recipient of the IFIP Silver Core award. He is an IEEE Fellow, and a recipient of the IEEE Distinguished Service and Gold Core Awards. He is a member of several IEEE societies, and of ASEE, ASQ (Senior Member), ACM, and Sigma Xi.

    Andy Rindos is the head of the IBM Research Triangle Park Center for Advanced Studies (RTP CAS). He supports and coordinates university relations for the large IBM development and services. He coordinates the efforts to launch a campus research and education cloud computing using the open source Apache Virtual Computing Lab (VCL) solution. He was the Program Co-Chairs and Steering Committee member for the 1st and the 2nd International IBM Cloud Academy Conference 2012 and 2014.

    View full text