Skip to main content
SpringerLink
Log in

AuraEN: Autonomous Resource Allocation for Cloud-Hosted Data Processing Pipelines

  • Conference paper
  • First Online:
Service-Oriented Computing – ICSOC 2020 Workshops (ICSOC 2020)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12632))

Included in the following conference series:

  • 1975 Accesses

Abstract

Ensuring cost-effective end-to-end QoS in an IoT data processing pipeline (DPP) is a non-trivial task. A key factor that affects the overall performance is the amount of computing resources allocated to each service in the pipeline. In this demo paper, we present AuraEN, an Autonomous resource allocation ENgine that can proactively scale the resources of each individual service in the pipeline in response to predicted workload variations so as to ensure end-to-end QoS while optimizing the associated costs. We briefly describe the AuraEN system architecture and its implementation and demonstrate how it can be used to manage the resources of a DPP hosted on the Amazon EC2 cloud.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/samantsunil/AuraEN.

  2. 2.

    https://github.com/samantsunil/data-generator.

  3. 3.

    https://github.com/samantsunil/data-processor-app.

References

  1. Samant, S.S., Baruwal Chhetri, M., Vo, Q.B., Kowalczyk, R., Nepal, S.: Towards end-to-end QoS and cost-aware resource scaling in cloud-based IoT data processing pipelines. In: 2018 IEEE International Conference on Services Computing (SCC), pp. 287–290. IEEE (2018)

    Google Scholar 

  2. Samant, S.S., Baruwal Chhetri, M., Vo, Q.B., Nepal, S., Kowalczyk, R.: Benchmarking for end-to-end QoS Sustainability in Cloud-hosted Data Processing Pipelines. In: 2019 IEEE 5th International Conference on Collaboration and Internet Computing (CIC), pp. 39–48. IEEE (2019)

    Google Scholar 

Download references

Acknowledgement

This research is supported by a PhD Scholarship from CSIRO Data61, an Australian federal government agency responsible for scientific research.

Author information

Authors and Affiliations

  1. Swinburne University of Technology, Melbourne, Australia

    Sunil Singh Samant, Mohan Baruwal Chhetri, Quoc Bao Vo & Ryszard Kowalczyk

  2. CSIRO Data61, Sydney, Australia

    Mohan Baruwal Chhetri & Surya Nepal

  3. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Ryszard Kowalczyk

Authors
  1. Sunil Singh Samant
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohan Baruwal Chhetri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Quoc Bao Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ryszard Kowalczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Surya Nepal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sunil Singh Samant .

Editor information

Editors and Affiliations

  1. Zayed University, Dubai, United Arab Emirates

    Hakim Hacid

  2. Zayed University, Dubai, United Arab Emirates

    Fatma Outay

  3. University of New South Wales, Sydney, NSW, Australia

    Hye-young Paik

  4. Huawei Paris Research Center, Paris, France

    Amira Alloum

  5. National Research Council C.N.R., Pisa, Italy

    Marinella Petrocchi

  6. Deakin University, Waurn Ponds, VIC, Australia

    Mohamed Reda Bouadjenek

  7. Macquarie University, Sydney, NSW, Australia

    Amin Beheshti

  8. Rochester Institute of Technology, Rochester, NY, USA

    Xumin Liu

  9. Université de Paris, Paris, France

    Abderrahmane Maaradji

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Samant, S.S., Chhetri, M.B., Vo, Q.B., Kowalczyk, R., Nepal, S. (2021). AuraEN: Autonomous Resource Allocation for Cloud-Hosted Data Processing Pipelines. In: Hacid, H., et al. Service-Oriented Computing – ICSOC 2020 Workshops. ICSOC 2020. Lecture Notes in Computer Science(), vol 12632. Springer, Cham. https://doi.org/10.1007/978-3-030-76352-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-76352-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76351-0

  • Online ISBN: 978-3-030-76352-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation