Elasticity management of Streaming Data Analytics Flows on clouds

https://doi.org/10.1016/j.jcss.2016.11.002Get rights and content
Under an Elsevier user license
open archive

Highlights

  • Optimal resource share analysis across the data analytics flow layers.

  • Multi-layered dynamic resource allocation for cloud-hosted data analytics flows.

  • Reducing the deviation from desired utilization of the resources by up to 48%.

Abstract

In this paper, we present a framework for resource management of Streaming Data Analytics Flows (SDAF). Using advanced techniques in control and optimization theory, we design an adaptive control system tailored to the data ingestion, analytics, and storage layers of the SDAF that is able to continuously detect and self-adapt to workload changes for meeting the users' service level objectives. Our experiments based on a real-world SDAF show that, the proposed control scheme is able to reduce the deviation from desired utilization of resources by up to 48% compared to existing techniques.

Keywords

Data analytics flow
Control theory
Data-intensive workloads
Resource management
Public clouds
Multi-objective optimization

Cited by (0)