Abstract
With the rise of online applications such as machine learning, stream processing, and interactive data-intensive applications in shared clusters, container cluster scheduling in data centers is facing new challenges. In order to solve the problem that application performance and economic cost cannot be balanced in a container cluster deploying a hybrid application, this paper proposes a container cluster scheduling strategy based on delay decision under multi-dimensional constraints. Formal language-based application placement constraints were introduced, and a task reorder model was established based on delayed decision-making. The experiments show that this strategy improves application performance and cluster utilization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vavilapalli, V.K., Murthy, A.C., Douglas, C.: Apache Hadoop YARN: yet another resource negotiator. In: Symposium on Cloud Computing, pp. 1–16. ACM (2013)
MartÃn, A.: TensorFlow: learning functions at scale. In: ACM Sigplan International Conference on Functional Programming, p. 1. ACM (2016)
Verma, A., Pedrosa, L., Korupolu, M.: Large-scale cluster management at Google with Borg. In: Tenth European Conference on Computer Systems, pp. 1–17. ACM (2015)
Xingcan, C., Xiaohui, Y., Yang, L.: Overview of distributed stream processing technology. Comput. Res. Develop. 52(2), 318–332 (2015)
Abadi, M., Barham, P., Chen, J.: TensorFlow: a system for large-scale machine learning. In: Usenix Conference on Operating Systems Design and Implementation, pp. 265–283. USENIX Association (2016)
Zaharia, M., Chowdhury, M., Franklin, M.J.: Spark: cluster computing with working sets. In: Usenix Conference on Hot Topics in Cloud Computing, p. 10. USENIX Association (2010)
Apache HBase[EB/OL] (2018). http://hbase.apache.org
Jyothi, S.A., Curino, C., Menache, I.: Morpheus: towards automated SLOs for enterprise clusters. In: Usenix Conference on Operating Systems Design and Implementation, pp. 117–134. USENIX Association (2016)
Rajan, K., Kakadia, D., Curino, C.: PerfOrator: eloquent performance models for Resource Optimization. In: ACM Symposium on Cloud Computing, pp. 415–427. ACM (2016)
Xu, G., Xu, C.-Z.: Prometheus: online estimation of optimal memory demands for workers in in-memory distributed computation. In: ACM Symposium on Cloud Computing, pp. 655–667. ACM (2017)
Alibaba trace [DB/OL] (2018). https://github.com/alibaba/clusterdata
Nathuji, R., Kansal, A., Ghaffarkhah, A.: Q-clouds: managing performance interference effects for QoS-aware clouds. In: European Conference on Computer Systems, Proceedings of the, European Conference on Computer Systems, EUROSYS 2010, Paris, France, April, pp. 237–250. DBLP (2010)
Avadi, B., Abawajy, J., Buyya, R.: Failure-aware resource provisioning for hybrid Cloud infrastructure. J. Parallel Distrib. Comput. 72(10), 1318–1331 (2012)
Tumanov, A., Zhu, T., Park, J.W.: TetriSched: global rescheduling with adaptive plan-ahead in dynamic heterogeneous clusters. In: Eleventh European Conference on Computer Systems, pp. 35–36. ACM (2016)
Hindman, B., Konwinski, A., Zaharia, M.: Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, pp. 429–483. USENIX Association (2010)
Karanasos, K., Suresh, A., Douglas, C.: Advancements in YARN resource manager 43(3), 51–60 (2018)
Verma, A., Pedrosa, L., Korupolu, M.: Large-scale cluster management at Google with Borg. In: Tenth European Conference on Computer Systems, pp. 1–17. ACM (2015)
Ananthanarayanan, G., Kandula, S., Greenberg, A.: Reining in the outliers in map-reduce clusters using Mantri. In: Usenix Conference on Operating Systems Design and Implementation, pp. 265–278. USENIX Association (2010)
Ferguson, A.D., Bodik, P., Kandula, S.: Jockey: guaranteed job latency in data parallel clusters. In: European Conference on Computer Systems, EUROSYS, pp. 99–112 (2012)
Zaharia, M., Konwinski, A., Joseph, A.D.: Improving MapReduce performance in heterogeneous environments. In: Usenix Conference on Operating Systems Design and Implementation, pp. 29–42. USENIX Association (2008)
Boutin, E., Ekanayake, J., Lin, W.: Apollo: scalable and coordinated scheduling for cloud-scale computing. In: Usenix Conference on Operating Systems Design and Implementation, pp. 285–300. USENIX Association (2014)
Ousterhout, K., Wendell, P., Zaharia, M.: Sparrow: distributed, low latency scheduling. In: Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 69–84 (2013)
Ghodsi, A., Zaharia, M., Shenker, S.: Choosy: max-min fair sharing for datacenter jobs with constraints. In: ACM European Conference on Computer Systems, pp. 365–378 (2013)
Isard, M., Prabhakaran, V., Currey, J.: Quincy: fair scheduling for distributed computing clusters. In: IEEE International Conference on Recent Trends in Information Systems, pp. 261–276 (2009)
Acknowledgment
This work is supported by the Natural Science Foundation of China (No. 61762008), the Guangxi Natural Science Foundation Project (No. 2017GXNSFAA198141), and the National Key Research and Development Project of China (No. 2018YFB1404404).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xue, Y., Chen, N., Xie, Y. (2020). Container Cluster Scheduling Strategy Based on Delay Decision Under Multidimensional Constraints. In: Zeng, J., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2020. Communications in Computer and Information Science, vol 1257. Springer, Singapore. https://doi.org/10.1007/978-981-15-7981-3_51
Download citation
DOI: https://doi.org/10.1007/978-981-15-7981-3_51
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7980-6
Online ISBN: 978-981-15-7981-3
eBook Packages: Computer ScienceComputer Science (R0)