ROAR: A QoS-oriented modeling framework for automated cloud resource allocation and optimization

Highlights

• ROAR transparently converts QoS goals to a set of optimized cloud resources.

• ROAR implements a fully automatic endtoend resource optimization process.

• ROAR supports generic web applications including multitier distributed architecture.

• ROAR supports multiple cloud platforms such as AWS and GCE.

• ROAR generates complete resource deployment template for the specific cloud platform.

Abstract

Cloud computing offers a fast, easy and cost-effective way to configure and allocate computing resources for web applications, such as consoles for smart grid applications, medical records systems, and security management platforms. Although a diverse collection of cloud resources (e.g., servers) is available, choosing the most optimized and cost-effective set of cloud resources for a given web application and set of quality of service (QoS) goals is not a straightforward task. Optimizing cloud resource allocation is a critical task for offering web applications using a software as a service model in the cloud, where minimizing operational cost while ensuring QoS goals are met is critical to meeting customer demands and maximizing profit. Manual load testing with different sets of cloud resources, followed by comparison of test results to QoS goals is tedious and inaccurate due to the limitations of the load testing tools, challenges characterizing resource utilization, significant manual test orchestration effort, and challenges identifying resource bottlenecks.

This paper introduces our work using a modeling framework – ROAR (Resource Optimization, Allocation and Recommendation System) to simplify, optimize, and automate cloud resource allocation decisions to meet QoS goals for web applications, including complex multi-tier application distributed in different server groups. ROAR uses a domain-specific language to describe the configuration of the web application, the APIs to benchmark and the expected QoS requirements (e.g., throughput and latency), and the resource optimization engine uses model-based analysis and code generation to automatically deploy and load test the application in multiple resource configurations in order to derive a cost-optimal resource configuration that meets the QoS goals.

Introduction

Cloud computing shifts computing from local dedicated resources to distributed, virtual, elastic, and multi-tenant resources. This paradigm provides end-users with on-demand access to computing, storage, and software services (OGRAPH and MORGENS, 2008). A number of cloud computing providers, such as Amazon Web Services (AWS) (Amazon Web Services (ELB), 2015) and Google Compute Engine (GCE) (Google Compute Engine, 2015), offer cloud computing platforms that provide custom applications with high availability and scalability. Users can allocate, execute, and terminate the instances (i.e., cloud servers) as needed, and pay for the cost of time and storage that active instances use based on a utility cost model of Rappa (2004).

To satisfy the computing resource needs of a wide variety of application types, cloud providers offer a menu of server types with different configurations of CPU capacity, memory, network capacity, disk I/O performance, and disk storage size. Table 1 shows a subset of the server configurations provided by AWS as of September 2014. For example, the m3.medium server with 3 ECU (i.e., EC2 Compute Unit – this is the relative measure of the CPU processing power of an Amazon EC2 cloud server) and 3.75GB memory costs $0.07/h, while the more powerful m3.2xlarge server costs $0.56/h. By comparison, the GCE cloud provider offers competitive options, as shown in Table 2. Although the server types are named differently (e.g., n1-standard-1 stands for the standard general-purpose server type with 1 virtual CPU using the n1-series of machines), the resource configurations at each pricing range vary only slightly. Cloud computing users must use this information to determine the appropriate subset of these resource configurations that will run an application cost-effectively, yet still meet its QoS goals, such as response time.

A common use case of the cloud is to offer existing software products, particularly web-based applications, through a software as a service (SaaS) model. In an SaaS model, the SaaS provider runs the web application in the cloud and customers remotely access the software platform, while the provider manages and maintains the software. SaaS providers typically provide service level agreements (SLAs) (Shu and Meina, 2010) to their clients, which dictate the number of users they will support, the availability of the service, the response time, and other parameters. For example, a provider of a SaaS electronic medical records system may offer an SLA that ensures a certain number of employees in a hospital can simultaneously access the system and that it will provide response times under 1 s.

An important consideration of SaaS providers is minimizing their operational costs while ensuring that the QoS requirements specified in their SLAs are met. For example, in the medical records system outlined above, the SaaS provider would like to minimize the cloud resources allocated to it to reduce operational cost, while ensuring that the chosen cloud resources can support the number of simultaneous clients and response times agreed to in the client SLAs. Moreover, as new clients are added and QoS requirements become more stringent (particularly in terms of the number of supported clients), the SaaS provider would like to know how adding resources on-demand will affect the application’s performance. Blindly allocating resources to the application to meet increasing load is not cost effective. Auto-scaling should instead be guided by a firm understanding of how resource allocations impact QoS goals.

Open Problem. While conventional cloud providers support simple and relatively quick resource allocation for applications, it is not an easy or straightforward task to decide an optimized and cost-effective resource configuration to run a specific application based on its QoS requirements. For instance, if a custom web application is expected to support 1000 simultaneous users with a throughput of 1000 requests/min, it is hard to decide the type and minimum number of servers and cloud providers needed by simply looking at the hardware configurations. Although this type of challenge existed in the traditional computing environments such as self-hosted data centers and server rooms, it has been augmented in the context of cloud computing due to the increasing number of cloud providers to choose and the virtualization-based nature of cloud computing.

Cloud providers and users have traditionally performed complex experimentation and load testing with applications on a wide variety of resource configurations. The most common practice is to deploy the application and perform a load stress test on each type of resource configuration, followed by analysis of the test results and selection of a resource configuration (Menascé, 2002). A number of load testing tools (e.g., jMeter, Apache JMeter, Halili, 2008, ApacheBench, Apache HTTP Server Benchmarking Tool, 2015, and HP LoadRunner, HP LoadRunner, 2015) are available to trigger a large amount of test requests automatically and collect the performance data. The situation gets even harder when an application includes a complex distributed architecture with multiple tiers, which requires the allocation of different groups of resources for the multiple tiers, followed by linking and connecting them properly.

Despite the importance of selecting a cost optimized resource allocation to meet QoS goals, many organizations do not have the time, resources, or experience to derive and perform a myriad of load testing experiments on a wide variety of resource types. Instead, developers typically employ a trial and error approach where they guess at the appropriate resource allocation, load test the application, and then accept it if the performance is at or above QoS goals. Optimization is usually only performed months or years into the application’s life in the cloud, when insight into the affect of resource allocations on QoS goals is better understood. Even then, the optimization is often not systematic.

The primary challenges that impede early resource allocation optimization stem from limitations with load testing tools and a number of manual procedures required in the cloud resource optimization process. It is often hard to specify the customized load tests and correlate load test configuration with the expected QoS goals. Moreover, it is tedious and error-prone to manually perform load testing with different cloud resources to derive an optimized resource configuration, particularly when the application has a complex multi-tier architecture in a heterogeneous cloud platform.

Even when an optimized resource configuration is finally obtained, allocating and deploying all of the resources also requires significant orchestration and complexity. Prior research has addressed some challenges separately (e.g., modeling realistic user test behavior to produce customized load test, Draheim et al., 2006, monitoring target test server performance metrics for capacity planning,Custom Plugins for Apache JMeter, 2015). However, a comprehensive, fully automated approach designed specifically for benchmarking, deriving, and implementing optimized cloud resource allocations has not been developed, particularly for multi-tier applications.

Solution approach ⇒ QoS-oriented modeling framework for Resource Optimization, Allocation and Recommendation System (ROAR). To address these challenges, this paper presents a model-based system called “ROAR” that raises the level of abstraction when performing load testing and automates cloud resource optimization and allocation to transparently convert users’ application-specific QoS goals to a set of optimized resources running in the cloud. A textual domain-specific language (DSL) called the Generic Resource Optimization for Web applications Language (GROWL) is defined to specify the high-level and customizable load testing plan and QoS requirements without low-level configuration details, such as the number of threads to use, the concurrent clients, and the duration of keeping opened connections.

The model built from the GROWL DSL can generate a test specification that is compatible with our extended version of the jMeter load testing tool (Apache JMeter, Halili, 2008). Likewise, ROAR automates the process of deploying the application to the given cloud platform, executing the load test, collecting and aligning performance metrics, analyzing the test performance model, and controlling the next test iteration. ROAR can derive the appropriate cloud resource configurations to test, as well as automatically orchestrate the tests against each resource allocation before using the results to recommend a cost-optimized resource allocation to meet the QoS goals. When the optimal configuration is decided, ROAR can also generate the resource templates to automate the final deployment process.

The work presented in this paper extends the initial prototype of ROAR (Sun et al., 2014), with the following new contributions and extensions:

• Multi-tier distributed web applications that require deployment in different groups of cloud servers can be supported by ROAR. Multi-tier configuration can be specified in the GROWL DSL. Likewise, the ROAR controller can automatically allocate the right resources to deploy and link different tiers.

• The average latency of the deployment target has been added as another key QoS goal, in addition to the target throughput supported in our earlier work. The ROAR resource optimization engine aligns the average latency metrics together with the throughput and resource utilization metrics, and filters out resource configurations that support the target throughput but fail to meet an average latency goal.

• Rather than focusing on a single cloud platform (such as AWS), the ROAR deployment manager has been extended to support deploying multi-tier applications to multiple cloud providers, which broadens its scope. In particular, we have included GCE as another platform supported by ROAR, which offers a wider range of optimization choices.

• The motivating example has been changed from a simple single-tier web application to a canonical three-tier web application with a database. Additional experiments and evaluation have also been included to showcase and quantify the performance, cost, and benchmarking effort for the resource configurations generated by ROAR.

The remainder of this paper is organized as follows: Section 2 summarizes a concrete motivating example in the context of a multi-tier web application, followed by describing the key challenges of cloud resource allocation and optimization in Section 3. Section 4 analyzes our solution by explaining each key component in the ROAR framework. Section 5 discusses the validation of this framework by presenting the generated resources for the motivating example. Section 6 compares our work on ROAR with the related research and Section 7 presents concluding remarks.