Abstract
The performance of Grid applications may be very unstable, especially when using workflows for job distribution. This is mainly due to the Grid overheads, like scheduling and queuing, introduced before the job is executed on a worker node. Optimization problems using Genetic Algorithms (GAs) can be easily and efficiently implemented on Grids using Grid workflows. Due to the file dependencies introduced in the Grid workflows for GAs, mainly for genetic material interchange, these overheads are cumulative and thus very noticeable. This problem is also very evident when the jobs are short compared to the Grid overheads, i.e. the job spends more time waiting in a queue to be executed than the execution itself.
In this paper we introduce a framework that enables users to easily utilize the Grid infrastructure for their optimization using GAs. It allows a user to preallocate certain number of pilot jobs, and also to dynamically manage their number for optimal availability of resources during the optimization process. In this way, once an application starts to execute the workloads, it will have at least one available pilot for execution of pooled tasks. This introduces better utilization of the Grid resources, as well boost the confidence in the infrastructure from users point of view.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alt, M., Hoheisel, A., Pohl, H.W., Gorlatch, S.: Using high level Petri-nets for describing and analysing hierarchical grid workflows. In: Proceedings of the CoreGRID Integration Workshop, pp. 267–276, Pisa, Italy, Nov 2005
Herrera, J., Huedo, E., Montero, R.S., Llorente, I.M.: A grid-oriented genetic algorithm. In: Sloot, P.M.A., Hoekstra, A.G., Priol, T., Reinefeld, A., Bubak, M. (eds.) EGC 2005. LNCS, vol. 3470, pp. 315–322. Springer, Heidelberg (2005)
Iosup, A., Epema, D.: Grid computing workloads: bags of tasks workflows, pilots, and others. IEEE Internet Comput. Mag. 15(2), 19–26 (2011)
JDL (Job Description Language). http://www-numi.fnal.gov/computing/minossoft/releases/R2.0/GridTools/docs/jobs_jdl.html. Accessed 20 Apr 2013
Jakimovski, B., Sahpaski, D., Velinov, G.: Performance improvement of genetic algorithms by adaptive grid workflows. In: SYNACS ’09, pp. 221–228, Timisoara, Romania, Sept 2009
Khalili, O., He, J., Olschanowsky, C., Snavely, A., Casanova, H.: Measuring the performance and reliability of production computational grids. In: Proceedings of the 7th IEEE/ACM International Conference on Grid Computing, pp. 293–300 (2006)
Meffert, K.: JGAP - Java Genetic Algorithms and Genetic Programming Package. http://jgap.sf.net
Pellegrini, S., Giacomini, F.: Design of a Petri net-based workflow engine. In: Proceedings of the 3rd International Conference on Grid and Pervasive Computing Workshops 2008, pp. 81–86, Kunming, May 2008
South Eastern European GRid-enabled eInfrastructure Development (SEE-GRID). http://www.see-grid.org/. Accessed 20 Apr 2013
Velinov, G., Jakimovski, B., Cerepnalkoski, D., Kon-Popovska, M.: Improvement of data warehouse optimization process by workflow gridification. In: Atzeni, P., Caplinskas, A., Jaakkola, H. (eds.) ADBIS 2008. LNCS, vol. 5207, pp. 295–304. Springer, Heidelberg (2008)
Luckow, A., et al.: Towards a common model for pilot-jobs. In: Proceedings of the 21st International Symposium on High-Performance Parallel and Distributed Computing, HPDC12, pp. 123–124, Delft, The Netherlands, June 2012
Tsaregorodtsev, A., et al.: DIRAC3 - the new generation of the LHCb grid software. J. Phys. Conf. Ser. 219(6), 062029 (2009)
Sarrut, D., Guigues, L.: Region-oriented CT image representation for reducing computing time of Monte Carlo simulations. Med. Phys. 35(4), 1452–1463 (2008)
Sfiligoi, I.: glideInWMS: a generic pilot-based workload management system. J. Phys. Conf. Ser. 119(6), 062044 (2008)
Silberstein, M., et al.: Gridbot: execution of bags of tasks in multiple grids. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, pp. 1–12 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jakimovski, B., Ilijoski, B., Velinov, G., Sahpaski, D. (2014). Framework for Genetic Algorithms Using Pilot Jobs in Adaptive Grid Workflows. In: Lirkov, I., Margenov, S., Waśniewski, J. (eds) Large-Scale Scientific Computing. LSSC 2013. Lecture Notes in Computer Science(), vol 8353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43880-0_59
Download citation
DOI: https://doi.org/10.1007/978-3-662-43880-0_59
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43879-4
Online ISBN: 978-3-662-43880-0
eBook Packages: Computer ScienceComputer Science (R0)