Abstract
Typical Integrated Circuit (IC) design projects use Electronic Design Automation (EDA) tool flows to launch thousands of regressions every day on shared compute grids to complete the IC design verification process. These regressions in turn launch compute jobs with varied resource requirements and inter-job dependency constraints. Traditional grid schedulers, such as the Univa Grid Engine (UGE) [12] prioritize fairness over performance to maximize the number of jobs run with equal distribution of resources at any time. A constant challenge in day-to-day operations is to schedule these jobs for minimum overall job completion time so that developers can expect predictable regression turn-around time (TAT).
We propose a resource-aware scheduling mechanism that balances performance and fairness for real-word EDA-centric workloads. We present an analysis of historical profile information from a set of regressions with complex inter-job dependencies and highly variable resource requirements to show that many of these regression jobs are well suited for efficient packing on grid machines.
We formulate the regression scheduling problem as a variant of the bin packing problem, where the size of bins and balls may vary according to job-resource requirements and differing server configurations on the grid. We propose using two analytic techniques – namely k-means clustering [8] and adaptive binning [10], to solve this problem. We then evaluate the performance of our proposed solution using real workloads from daily regressions on an enterprise compute grid.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gen, M., Cheng, R.: Genetic Algorithms and Engineering Optimization, vol. 7. Wiley, Hoboken (2000)
Grandl, R., Ananthanarayanan, G., Kandula, S., Rao, S., Akella, A.: Multi-resource packing for cluster schedulers. ACM SIGCOMM 44(4), 455–466 (2015)
Hao, X., Lin, L., Gen, M., Ohno, K.: Effective estimation of distribution algorithm for stochastic job shop scheduling problem. Procedia Comput. Sci. 20, 102–107 (2013)
Harchol-Balter, M., Schroeder, B., Agrawal, M., Bansal, N.: Size-based scheduling to improve web performance. ACM Trans. Comput. Syst. 21, 207–233 (2003)
Holte, R.C.: Very simple classification rules perform well on most commonly used datasets. Mach. Learn. 11, 63–91 (1993). https://doi.org/10.1023/A:1022631118932
Kamath, V., Giri, R., Muralidhar, R.: Experiences with a private enterprise cloud: providing fault tolerance and high availability for interactive EDA applications. In: Sixth IEEE International Conference on Cloud Computing (CLOUD), pp. 770–777, June 2013. https://doi.org/10.1109/CLOUD.2013.72
Lawler, E.L., Lenstra, J.K., Kan, A.H.R., Shmoys, D.B.: Sequencing and scheduling: algorithms and complexity. Handb. Oper. Res. Manag. Sci. 4, 445–522 (1993)
Macqueen, J.: Some methods for classification and analysis of multivariate observations. In: 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Poli, R., Woodward, J., Burke, E.K.: A histogram-matching approach to the evolution of bin-packing strategies. In: 2007 IEEE Congress on Evolutionary Computation, pp. 3500–3507. IEEE (2007)
Tibshirani, R., Walther, G., Hastie, T.: Estimating the number of clusters in a dataset via the gap statistic. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 63, 411–423 (2000)
Univa Corporation: Univa® grid engine®. http://www.univa.com/. http://www.univa.com/products
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Nanda, S., Parthasarathy, G., Choudhary, P., Venkatachar, A. (2020). Resource Aware Scheduling for EDA Regression Jobs. In: Schwardmann, U., et al. Euro-Par 2019: Parallel Processing Workshops. Euro-Par 2019. Lecture Notes in Computer Science(), vol 11997. Springer, Cham. https://doi.org/10.1007/978-3-030-48340-1_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-48340-1_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-48339-5
Online ISBN: 978-3-030-48340-1
eBook Packages: Computer ScienceComputer Science (R0)