Skip to main content
Springer Nature Link
Log in

Towards application-level elasticity on shared cluster: an actor-based approach

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

In current cluster computing, several distributed frameworks are designed to support elasticity for business services adapting to environment fluctuation. However, most existing works support elasticity mainly at the resource level, leaving application level elasticity support problem to domain-specific frameworks and applications. This paper proposes an actor-based general approach to support application-level elasticity for multiple cluster computing frameworks. The actor model offers scalability and decouples language-level concurrency from the runtime environment. By extending actors, a new middle layer called Unisupervisor is designed to “sit” between the resource management layer and application framework layer. Actors in Unisupervisor can automatically distribute and execute tasks over clusters and dynamically scale in/out. Based on Unisupervisor, high-level profiles (MasterSlave, MapReduce, Streaming, Graph, and Pipeline) for diverse cluster computing requirements can be supported. The entire approach is implemented in a prototype system called UniAS. In the evaluation, both benchmarks and real applications are tested and analyzed in a small scale cluster. Results show that UniAS is expressive and efficiently elastic.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lu X C, Wang H M, Wang J, Xu J, Li D S. Internet-based virtual computing environment: beyond the data center as a computer. Future Generation Computer Systems, 2013, 29(1): 309–322

    Article  Google Scholar 

  2. Toshniwal A, Taneja S, Shukla A, Ramasamy K, Patel J M, Kulkarni S, Jackson J, Gade K, Fu M S, Donham J, Bhagat N, Mittal S, Ryaboy D. Storm@twitter. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. 2014, 147–156

    Google Scholar 

  3. Hindman B, Konwinski A, Zaharia M, Ghodsi A, Joseph A D, Katz R, Shenker S, Stoica I. Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation. 2011, 295–308

    Google Scholar 

  4. Vavilapalli V K, Murthy A C, Douglas C, Agarwal S, Konar M, Evans R, Graves T, Lowe J, Shah H, Seth S, Saha B, Curino C, O’Malley O, Radia S, Reed B, Baldeschwieler E. Apache Hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th ACM Symposium on Cloud Computing. 2013

    Google Scholar 

  5. Cole M. Algorithmic Skeletons: Structured Management of Parallel Computation. Cambridge, Massachusetts: MIT Press, 1991

    MATH  Google Scholar 

  6. Bahga A, Madisetti V K. Rapid prototyping of multitier cloud-based services and systems. Computer, 2013, 46(11): 76–83

    Article  Google Scholar 

  7. Kächele S, Hauck F J. Component-based scalability for cloud applications. In: Proceedings of the 3rd International Workshop on Cloud Data and Platforms. 2013, 19–24

    Chapter  Google Scholar 

  8. Caromel D, Leyton M. Fine tuning algorithmic skeletons. Lecture Notes in Computer Science, vol 4641. Berlin: Springer-Verlag, 2007, 72–81

    Article  Google Scholar 

  9. Agha G A. Actors: a model of concurrent computation in distributed systems. Dissertation for the doctoral Degree. Cambridge: Massachusetts Institute of Technology, 1985

    Google Scholar 

  10. Zhan H L, Kang L H, Cao D G. DETS: a dynamic and elastic task scheduler supporting multiple parallel schemes. In: Proceedings of the 8th IEEE International Symposium on Service Oriented System Engineering. 2014, 278–283

    Google Scholar 

  11. Matei Zharia, Das T, Li H Y, Hunter T, Shenker S, Stoica I. Discretized Streams: Fault-tolerant streaming computation at scale. In: Proceedings of the 24th ACM Symposium on Operating Systems Principles (OSDI). 2013, 423–438

    Google Scholar 

  12. Lin J, Zha L, Xu Z W. Consolidated cluster systems for data centers in the cloud age: a survey and analysis. Frontiers of Computer Science, 2013, 7(1): 1–19

    Article  MathSciNet  Google Scholar 

  13. Liu L. Computing infrastructure for big data processing. Frontiers of Computer Science, 2013, 7(2): 165–170

    Article  MathSciNet  Google Scholar 

  14. Zacharia F, Govindaraju M. Delma: dynamically elastic mapreduce framework for CPU-intensive applications. In: Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2011, 454–463

    Google Scholar 

  15. Gordon A W, Lu P. Elastic phoenix: malleable mapreduce for sharedmemory systems. Network and Parallel Computing, Vol 6985. Berlin: Springer, 2011

  16. Shi W, Hong B. Clotho: an elastic MapReduce workload/runtime codesign. In: Proceedings of the 12th International Workshop on Adaptive and Reflective Middleware. 2013

    Google Scholar 

  17. Waldemar H, Satzger B, Dustdar S. Elastic stream processing in the Cloud. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2013, 3(5): 333–345

    Google Scholar 

  18. Qian Z, He Y, Su C Z, Wu Z J, Zhu H Y, Zhang T Z, Zhou L D, Yu Y, Zhang Z. Timestream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14

    Google Scholar 

  19. Bugra G, Schneider S, Hirzel M, Wu K L. Elastic scaling for data stream processing. IEEE Transcations on Parallel and Distributed Systems, 2014, 25(6): 1447–1463

    Article  Google Scholar 

  20. Huang C, Zheng G B, Kale L, Kumar S. Performance evaluation of adaptive MPI. In: Proceedings of the 11th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2006, 12–21

    Google Scholar 

  21. Wei Y, Sukumar K, Vecchiola C, Karunamoorthy D, Buyya R. Aneka cloud application platform and its integration with Windows Azure. 2011, arXiv preprint arXiv: 1103.2590

    Book  Google Scholar 

  22. Bykov S, Geller A, Kliot G, Larus J R, Pandya R, Thelin J. Orleans: cloud computing for everyone. In: Proceedings of the 2nd ACM Symposium on Cloud Computing. 2011

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National High-Tech Research and Development Plan of China (2015AA01A202), National Basic Research Program of China (973) (2011CB302604), and the National Natural Science Foundation of China (Grant Nos. 61272154 and 61421091).

Author information

Authors and Affiliations

  1. Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, Beijing, 100871, China

    Donggang Cao, Lianghuan Kang, Hanglong Zhan & Hong Mei

  2. Beida (Binhai) Information Research, Tianjin, 300450, China

    Donggang Cao, Lianghuan Kang & Hanglong Zhan

Authors
  1. Donggang Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lianghuan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanglong Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Donggang Cao.

Additional information

Donggang Cao received the PhD in computer science from Peking University, China in 2004. He is currently an associate professor in the school of Electronics Engineering and Computer Science at Peking University. His research interests include parallel and distributed processing, system software, network technologies, etc.

Lianghuan Kang received the PhD in computer science from the school of Electronics Engineering and Computer Science at Peking University, China in 2015. His research interests include parallel and distributed processing, programming languages, etc.

Hanglong Zhan is a PhD candidate in computer science in the school of Electronics Engineering and Computer Science at Peking University, China. His research interests include big data processing, graph processing, etc.

Hong Mei received the PhD degree in computer science from Shanghai Jiaotong University, China in 1992. He is currently a professor in the school of Electronics Engineering and Computer Science at Peking University, China. His research interests include software engineering, system software, etc.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, D., Kang, L., Zhan, H. et al. Towards application-level elasticity on shared cluster: an actor-based approach. Front. Comput. Sci. 11, 803–820 (2017). https://doi.org/10.1007/s11704-016-5386-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-016-5386-9

Keywords

Search

Navigation