1 Introduction

Resources are spread across datacenters, enterprise clusters and Internet of Things (IoT) devices. Cloud computing virtualize datacenters and provide massive computing resource but at a relatively high cost. Edge computing reorganized and made sufficient use of computing resource in enterprise clusters, personal laptop and IoT devices among the edge, avoiding waste of edge resource and cut costs on using computing resource. Having their own pros and cons, computation offloading is an acceptable solution to reach high performance and utilize of resource statically. However, conditions such as computation overload, internet bandwidth and connection latency may change over time. Applications may have better performance running on the edge now and even better in the cloud second later. Thus, real-time re-scheduling between cloud and edge or redirecting among the edge is needed.

Migration should retain integrity and consistency of the application. Considering the cost of migration and a variety of problems such as dependency loss and mutable situations we may encounter during migrating, a lightweight virtualization solution other than VM is needed [14]. Containerization is currently discussed as a frequently used lightweight virtualization solution [16, 17]. Containers provide more interoperable, standardized and lightweight application packing in the cloud which reduces the overhead of application migration, for example, less time-consuming [13]. Standardized with container technology, application and resource are simpler to manage across the cloud. Containerized application could be distributed with restrictions on resource usage set in advance to scheduled compute node. Furthermore, containers are enough reliable and private to make the isolation of various applications easy [15]. Most orchestration strategy on containers in the edge cloud [3] orchestrates the applications with a scheduler in the data center ignoring different demands of various edge clouds. Strategies may be difficult to formulate due to the ever-changing circumstance of different edge clouds.

Regarding for the challenge of minimizing computing overhead and effectively utilize resource on the edge, we come up with a new research on improving the performance of dynamic re-scheduling between cloud and edge and among edge. Our proposal is to build a modular system architecture, named Boundless Application an Resource Orchestrator (BRO), aims at provisioning application to suitable edge node or to the cloud automatically and dynamically to reach either best performance or best utilization of resource based on container technology. The orchestration depends on multiple parameters such as computation overload, internet bandwidth and the cost of resources. The presented architecture considers single container as the minimum unit of scheduling and a basic resource unit for service distribution. Containerized application is available in a public registry maintained in the core cloud. Distribution and deployment could be done in one click and are transparent to the end users. We implement a master-slave paradigm instead of traditional centralized scheduling setting associate schedulers in each edge cloud running different strategies according to the circumstance of specified edge cloud. Schedulers will make an overall consideration combining the parameters set in advance and real-time conditions on switching containerized application between cloud and edge or across the edge to ensure the performance of the containerized application meet the need of users in minimum cost. Edge devices are able to share idle resources by acting as a pseudo node of the architecture.

Ultimately, the paper put forward an attempt to make resource and application boundless, referring that all resource across the cloud and the edge is managed and could be efficiently used. Applications are somewhere to gain best performance with least cost automatically and dynamically under the proposed architecture. The remainder of this paper is organized as follows. Section 2 lists the related works. Section 3 shows the modular system architecture. Section 4 reveals the details on the implementation of the dedicated system architecture and algorithms included. In Sect. 5, we evaluate the impact of using our architecture on top of different use cases with various aspects taken into consideration. Section 6 concludes the paper and looks into the future work.

2 Related Works

2.1 Edge Computing

Edge Computing is an emerging computational architecture that shares an idea on moving the computation closer to the edge devices in order to avoid limitations such as bandwidth, latency and high cost in traditional cloud computing. Edge computing act as an extension of traditional cloud computing, coming up with the multi-cloud paradigm in line with the trend of decentralization and IoT. In constant to traditional cloud computing, edge computing orchestrates resource and computation on the edge as well as provide all services in a distributed way. Research on edge computing [8, 11, 12] has proved that edge computing meets the demand for computing architecture through now to the future and edge computing is currently widely accepted as an optional infrastructure of IoT [10]. In [9], a strategy for automatic task sharing and switching between cloud and edge computing has been put forward and a smart IoT gateway is designed based on machine learning and cognitive analytics to make an orchestration for applications between the cloud and the edge. However, orchestration on applications in edge computing may be difficult due to the diversity of edge devices and mutable circumstance.

2.2 Container in Edge Computing

Container technology is a widely accepted lightweight virtualization solution which provides isolation, standardization and flexibility with least overhead in cloud computing. Researches shows that the characteristic and advantages of container technology perfectly match the requirements of the edge computing technology on virtualization. In [5], a performance evaluation is done on container technology as the virtualization method in IoT edge architecture. The research shows that container-virtualization technology produces negligible impact in terms of performance when compared to native executions but can enhance IoT security. In [1, 7], other technical evaluation and experiment on specified fundamental edge computing using case is made and claim Docker container as a viable candidate for edge computing architecture while still having rooms for improvement. In general, container technology, providing more security, isolation and higher performance, is the optimal choice rather than VM in edge computing. Researches on the orchestration of containers in edge computing [2,3,4, 6] are in progress as well. Common orchestration strategy is making scheduling decisions by a scheduler in the core cloud and delivers containers to separate edge nodes considering dynamic circumstance, appearing to be cumbersome and inefficient in contrast to native orchestration on edge computing.

3 Architecture Design

Cloud computing is moving from centralized to the decentralized pattern to avoid network bottleneck and efficiently utilize edge computing resource. Boundless application and resource system is an architecture attempting high performance though scheduling applications between the cloud and the edge or among the edge based on container technology (Fig. 1).

Fig. 1.
figure 1

Architecture of BRO

Full size image

BRO has the following features:

  • Based on container: considering container as the basic unit of application and resource management and scheduling.

  • High adaptability: supports various edge devices from enterprise server to personal laptop. Any edge devices that support container can play a role in the architecture.

  • High-performance dynamic scheduling strategy: taking real-time condition into consideration, applies suitable mechanism such as BPLC mechanism and tagging mechanism corresponding to different scenarios to achieve the best performance at minimum cost dynamically.

  • Sufficient utilize of edge resource through sharing rule: leveraging container technology, it is able to share idle resources of your server or even your laptop.

  • Fault tolerant: improve the stability and reliability of the architecture through a complete set of recovery mechanism.

3.1 Containers

Compare to cloud computing, edge computing could be relatively unstable and unreliable. Due to the complex network and heterogeneous structure of edge cloud, the environment of edge computing may change time to time. Widely discussed as an outstanding lightweight virtualization solution, container technology provide isolation for applications among changeable environment with much lower overhead than VMs. While the performance of the application is affected by the real-time condition, re-scheduling is common in edge computing to achieve higher performance. Containerizing application makes scheduling simple. Container technology could provide standardize, isolation, less overhead and high fault tolerant for edge computing, leaving dependency problems behind. Managing distributed containerized application could be implemented as same as managing container clusters in the cloud. Thus, containers are even suitable for the circumstance of edge computing and cloud computing.

3.2 Pseudo Node

BRO aims at making application and resource boundless implying making remote resource reachable and local resource sharable. The most straightforward method is attaching your devices to the cloud, afterward, cloud computing resource is next to your hand. However, joining the cloud is too cumbersome for edge devices and may lead to many problems. As the name suggests, pseudo node initializes the edge device as an incomplete or fake minion with strong restrictions. If the edge device has been initialized to a pseudo node, we can participate in BRO, gain cloud computing resource and share local resources among the cloud portable and safe.

3.3 Regional Autonomy

BRO applies the concept of regional autonomy in edge computing in order to deal with the ever-changing circumstance of the edge cloud. According to the concept of regional autonomy, the architecture divides the edge into different portions according to various properties and characteristic such as logical or physical location, computation overhead and stability. Scheduling will be first done within current portion of edge cloud and then across the edge if lack of resource. Regional autonomy could improve the performance of scheduling efficiency and accuracy as well as be easy to adapt to diverse infrastructures.

4 Implementation

See Fig. 2.

Fig. 2.
figure 2

Lifecycle of edge computing under BRO

Full size image

4.1 Application and Resource Unit Management

BRO takes container as the basic unit of application and resource management. All application running under this architecture should be containerized. Thus, we can manage application in a similar way with containers.

Application migration could be simply implemented by transferring container images, which is so small and the state of the application will be stored in the container image. Leveraging orientated scheduling of container cluster is another solution for containerized application migration. The application manager records the behaviors of the containerized application and generates a dockerfile, responsible for lifecycle management of the containerized application. Application rollback could be implemented by going through the dockerfile.

Container preconfigured a threshold of resource usage referring the maximum resource the application could occupy using cgroup and namespace [20]. Thus, a container can be a unit to measure resource. A special container with the same configuration can be distributed to the objective node to test the resource before the containerized application is scheduled.

4.2 Monitor Container

Monitor container is a special container created during initializing pseudo node. Monitor container consists of various data collectors and several tools for testing, pressure test and heartbeat detection, for example. A pseudo node is considered unavailable when the monitor container is down.

The monitor container supervises the pseudo node and containerized application running upon and is responsible for collecting real-time condition data such as CPU usage, memory usage and I/O usage. Monitor container uploads real-time condition data to the scheduler as a critical counter for the producing of scheduling strategy (Tables 1 and 2).

Table 1. Sets used in this article
Full size table
Table 2. Common condition data referenced by the scheduler.
Full size table

Monitor container will provide the performance of containerized applications scheduled on current pseudo node to the learning module of the scheduler to optimize the algorithm. If low performance or congestion is detected, the monitor container will inform the scheduler to reschedule the containerized applications in time and send feedback for better performance.

Monitor container plays an important role in the tag system. It tries to quantify data collected and add various tags to the pseudo node such as “stable” for pseudo nodes with sustained connectivity and “massive” for pseudo nodes with massive computing resources. Besides the default data collector, self-defined collectors are available for special cases. Before scheduling, a monitor container with a self-defined collector will be delivered to suitable nodes for addition dedicate condition data. Self-defined tags could be added by self-defined collectors for further sharing and scheduling.

4.3 Master-Slave Paradigm

BRO implements regional autonomy through the master-slave paradigm. In contrast to the traditional centralized paradigm, master-slave paradigm divides the edge into parts according to the characteristic and property of the certain range of edge. Each edge cloud owns a dedicate associate scheduler apart from the master scheduler in the data center. The associate scheduler has a set of independent scheduling mechanisms depending on the property and characteristic of the edge cloud. Containerized application could gain best performance scheduling among the edge cloud under the dedicate scheduling mechanism. The associate scheduler runs under the monitor of the master scheduler in the data center. While current edge cloud cannot meet the requirement of the job, the associate scheduler will call for the master scheduler to make an orchestration across different edge clouds or upload to the data center. The master scheduler is also reasonable of instructing associate scheduler and adjusting edge scheduling mechanism analyzing feedback uploaded to the data center. Through master-slave paradigm, BRO implemented regional autonomy and retains the benefits of decentralization. Equation (1) shows default clustering mechanism to divide the edge. \( L^{i} \) and \( L^{\text{l }} \) respectively represent self-defined thresholds for corresponding dimensions.

$$ \begin{array}{*{20}c} {\left\{ { e | T_{e}^{i} < L^{i} ,\,\,l_{ue} < L^{\text{l}} ,\,\,{\text{u}} \in {\text{S}},\,\,i = 1,2,3 } \right\}} \\ \end{array} $$
(1)

4.4 Scheduling Strategy

Conditions of the cloud may change in time. An application may have fine performance running on the edge for some time and have even better performance running in the cloud for some other time. Thus, scheduling is required to achieve the best performance at all time. Scheduling under BRO consists of scheduling between the cloud and the edge and among the edge, while cloud resource is commonly known as stable, massive but expensive and edge resource usually unstable, unreliable but low-cost.

The scheduler is responsible for generating scheduling strategies and initiating a scheduling procedure through commanding the built-in scheduler of the container cluster to schedule the target application. Scheduling strategy under BRO is usually divided into two categories: static strategy and dynamic strategy. As the platform is initialized statically and relatively stable, applications are initially scheduled under the static strategy. Once real-time conditions such as computing overhead, internet bandwidth or connectivity changes, the dynamic strategy should be used.

A traditional solution is categorizing the containerized applications manually into few types such as high traffic or heavy computation, which brings lack of accuracy. BRO guides the scheduling process according to the Best Performance at Least Cost (BPLC) algorithm with the assistance of a tagging mechanism. BPLC aims at gaining best performance at least cost through effective scheduling of containerized applications. The performance could be measured through the average job finishing time. Average job finishing time is made up of the average job waiting time implying scheduling overhead and average job execution time that depends on resource conditions. Scheduling overhead depends on latency \( {\text{l}}_{\text{uv}} \) and data transmission overhead \( {\text{O}}_{\text{a}} \). We express average job execution time using resource conditions of the edge. Equation (2) shows the performance of the static scheduling strategy and Eq. (3) shows the performance of the dynamic scheduling strategy. \( \upalpha \), \( \beta \) and \( \gamma \) are tuning parameters for the weight of the separate parts.

$$ \begin{array}{*{20}c} {\upalpha\mathop \sum \limits_{i = 1,2,3} \gamma_{i} \mathop \sum \limits_{e \in S} T_{e}^{i} - \beta \left( {\mathop \sum \limits_{e \in S} \mathop {\hbox{min} }\limits_{c \in C} l_{ce} + \mathop \sum \limits_{a \in A} O_{a} } \right)} \\ \end{array} $$
(2)
$$ \begin{array}{*{20}c} {\upalpha \mathop \sum \limits_{i = 1,2,3} \gamma_{i} \mathop \sum \limits_{e \in S} T_{e}^{i} - \beta \left( {\mathop \sum \limits_{{e_{1} \in S}} \mathop {\hbox{min} }\limits_{{e_{2} \in S}} l_{{e_{1} e_{2} }} + \mathop \sum \limits_{a \in A} O_{a} } \right)} \\ \end{array} $$
(3)

Except gaining best performance, BPLC attempts to reduce the cost of running applications in the edge cloud. We assume the cost on the device in the edge mainly depend on the resource it provides. Cost of running applications is measured as Eq. (4). \( {\text{S}}_{\text{a}} \) represents edge nodes occupied running application a and \( {\text{R}}_{a}^{\text{i}} \) respectively represent resources required by application a.

$$ \begin{array}{*{20}c} {\mathop \sum \limits_{i} \varepsilon_{i} \mathop \sum \limits_{{e \in S_{a} }} T_{e}^{i} } \\ \end{array} $$
(4)

Considering both performance and the cost, object function of BPLC is shown as Eq. (5). Equation (6)–(7) reveal part of the constraints such as bandwidth conservation and computational resource conservation. \( {\text{a}}_{\text{uv}} \) represents application scheduling through node u and node v.

$$ \begin{array}{*{20}c} {\mathop {\hbox{max} }\limits_{{e \in S_{a} }} \alpha \mathop \sum \limits_{i = 1,2,3} \gamma_{i} \mathop \sum \limits_{{e \in S_{a} }} T_{e}^{i} - \beta \left( {\mathop \sum \limits_{{e_{1} \in S_{a} }} \mathop {\hbox{min} }\limits_{{e_{2} \in S, e_{2} \notin S_{a} }} l_{ce} + \mathop \sum \limits_{a \in A} O_{a} } \right) - \theta \mathop \sum \limits_{i} \varepsilon_{i} \mathop \sum \limits_{{e \in S_{a} }} T_{e}^{i} } \\ \end{array} $$
(5)
$$ \begin{array}{*{20}c} {\mathop \sum \limits_{{a_{{e_{1} e_{2} }} \in A, }} {\text{O}}_{{{\text{a}}_{{{\text{e}}_{1} e_{2} }} }} \le\updelta_{{{\text{e}}_{1} e_{2} }} \,\,\forall e_{1} ,e_{2} \in S_{a} } \\ \end{array} $$
(6)
$$ \begin{array}{*{20}c} {\mathop \sum \limits_{{e \in S_{a} }} T_{e}^{i} > R_{a}^{i} \forall i \in I} \\ \end{array} $$
(7)

Tagging mechanism act as a supplement to the BPLC algorithm. According to the tagging mechanism, pseudo nodes on the edge is tagged during initialization by the monitor container according to the instant condition of the devices such as computing resource available, internet bandwidth and connection persistency and stability. Application managers add tags to the containerized application according to the demand of the application, massive or long living, for example. Though comparing the tags, combing the overhead of migration and suitability of the pseudo node, it is easy to determine a target node for scheduling.

However, common tags cannot cover the massive properties on demand. We come up with customized tags added by monitor containers with self-defined data collectors enriching the properties covered. With increasing amount of tags, fuzzy matching of tags is effective but not efficient enough. The problem is that it is hard to match tags on application to various customized tags. We put forward a new tagging mechanism based on machine learning. Before scheduling, we run a simulation container that acts as the same as the target containerized application on a sample node with a comprehensive monitor container. The dedicate monitor container is trained with a machine learning algorithm, classifying all customize tags and extract features. After a period, the dedicate monitor container will analyze the behavior of the specified testing container and add a group of customized tags with the corresponding features. The whole tagging procedure is done automatically and will be more accurate with time. Feedback such as error report and operational situation sent by monitor containers across the edge can further improve the performance of the new tagging mechanism. Nevertheless, additional overhead is required for the first time the containerized application is scheduled.

5 Experiment and Use Case

We present two case study respectively to examine master-slave paradigm and regional autonomy as an impressive orchestration pattern for edge computing architecture based on containers and estimate the strategy mentioned in scheduling containerized applications.

5.1 Examining Effects of Master-Slave Paradigm

We setup a lab environment as showed at Fig. 3 with a datacenter and four groups of edge devices ranges from dedicate servers to personal laptop. We use Docker [18] as an implement of container technology and deploy a container cluster with Kubernetes in the datacenter as the basic core cloud. Kubernetes [19] is a production grade container orchestration tool and is the best solution to act as the native container cluster manager in our architecture. Node manager, application manager and scheduler in our architecture will implement upon the native container cluster manager we choose, Kubernetes during this experiment, for example.

Fig. 3.
figure 3

Topology of lab environment

Full size image

Edge devices are initialized to pseudo nodes, as a fake minion of the container cluster. Each edge devices will have different properties: group a has least computing overload, group b has low network latency, group c has maximum internet bandwidth and group d is balanced on all properties. All edge devices are reachable by another and accessible from the container cluster.

In this experiment, we created a group of containers running machine learning and another group of containers running web crawlers program through the dockerfile and randomly deploy them among the edge devices. In reality, containerized applications should be pushed to the public registry in the cloud and deployed from the datacenter to the edge cluster. We deploy them directly among the edge devices for convenience to observe the scheduling procedure.

Four set of experiments is taken following the same procedure mentioned before under various infrastructure. We take VM, native environment and centralized paradigm as parallel experiments to the master-slave paradigm. We respectively record job average waiting time and job average execution time running different amount of jobs. Job average waiting time implies the time cost of scheduling and job average execution time indicates the performance of the architecture. We use the number of jobs to simulate the scale of the computation and observe the performance of four infrastructure under different circumstance.

The outcome of the experiment proves container technology as an effective solution providing isolation, reducing scheduling overhead and fastening the scheduling procedure. Containerized application could be a choice under edge computing scenario. According to the record in Figs. 4 and 5, we can temporarily draw some conclusions:

Fig. 4.
figure 4

Job average waiting time in different scale of orchestration

Full size image
Fig. 5.
figure 5

Job average execution time in different scale of orchestration

Full size image

  • Container technology cuts the overhead of scheduling and is a better choice compared traditional VMs. However, container technology makes negligible improvement in terms of performance compared to native executions.

  • Master-slave paradigm acts similar to the centralized paradigm in small scale but gains better performance with the increase in the number of jobs.

  • Master-slave paradigm accelerates scheduling and owns high accuracy comparing to centralized paradigm dealing with massive jobs. Regional autonomy contributes to this feature while dealing each edge cloud with more suitable strategy.

5.2 Estimating BPLC Associated with Tagging Mechanism

In order to estimate the effectiveness of tag mechanism under the circumstance mentioned in the paper, we present a case study to compare the tag mechanism with the native container scheduling method and IoT gateway [9] without containers.

We setup a lab environment with a datacenter and four groups of edge devices. The datacenter is initialized as same as the former experiment and the edge devices are given random combinations of properties such as high internet bandwidth with high computing overload or low computing overload with less network latency.

After initializing the lab environment, we create a group of containers running different types of applications ranging from high traffic to high computing overload. We deploy them from the datacenter to get a clear view of the scheduling procedure between cloud and edge and among edge.

We will repeat the experiment procedure three times respectively with native container orchestration mechanism, BPLC mechanism and IoT gateway and record a set of data. The statistic of the result is revealed in Figs. 6 and 7.

Fig. 6.
figure 6

Job average waiting time under different orchestration strategy

Full size image
Fig. 7.
figure 7

Job average execution time under different orchestration strategy

Full size image

According to the result recorded, Job avg. waiting time reveals the total time spent scheduling the jobs to suitable nodes. The BPLC mechanism takes a little more time in small scale but shows better performance than native container orchestration mechanism along with the growth of jobs. IoT gateway owns least waiting time cause of none overhead. Job avg. execution time shows the performance of the jobs. Statistic reveals that BPLC mechanism performs much better than the others especially with massive jobs, claiming the accuracy of the orchestration and high resource utilization of edge resource. The outcome of the experiment proves that the BPLC mechanism proposed in the paper is effective in scheduling containerized application among the edge cloud.

5.3 Use Case

BRO owns strong adaptability and could be widely adopted by companies, public organizations and even personal laptops. Use cases of BRO vary from applying as an enhanced implementation of IoT scenarios such as Smart Cities, E-health and smart homes to acting as an invisible frame making resource gaining and application running automatic and convenient for the public.

A common use case is acting as an invisible frame contributing to accelerating the automatic process of application orchestrating and resource utilize optimization. For example, training a model for a deep learning job could be finished within one-step setting parameters with a template. The specified job will be initialized, containerized and orchestrated to suitable nodes for best performance without the intervention of operators. Other use cases include but not limited to idle resource liquidation, traffic shaving and fast deployment.

6 Conclusion and Future Work

This paper has put forward an architecture based on container technology aims at combing cloud computing and edge computing. Appling container technology in edge computing effectively increases stability and performance through optimizing the orchestration procedure, lower the cost of scheduling. BRO attempts to deliver applications to suitable edge node or to the cloud automatically and dynamically to reach either best performance or best utilize of resource based on container technology. Rather than centralized dispatching from the data center, the master-slave paradigm is much more elastic, reliable and effective while regional autonomy can be better adapted to the ever-changing environment of edge computing. Furthermore, orchestration strategy BPLC with tagging mechanism is designed for edge computing under the container-based architecture and improves the performance of cloud computing and edge computing.

Currently, there are still limits accessing resources in the cloud or among the edge due to plenty of problems such as security problems. The emergence of BRO and BPLC solves these problems slightly but not completely. Future work place emphasis on improving the performance of container in edge computing and optimizing dedicate orchestration strategy under various circumstances. Hopefully, resource and application will be standardized and reachable anywhere in the visible future.