Skip to main content
SpringerLink
Log in

Customized Plug-in Modules in Metascheduler CSF4 for Life Sciences Applications

  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

As more and more life science researchers start to take advantages of grid technologies in their work, the demand increases for a robust yet easy to use metascheduler or resource broker. In this paper, we have extended the metascheduler CSF4 by providing a Virtual Job Model (VJM) to synchronize the resource co-allocation for cross-domain parallel jobs. The VJM eliminates dead-locks and improves resource usage for multi-cluster parallel applications compiled with MPICH-G2. Taking advantage of the extensible scheduler plug-in model of CSF4, one may develop customized metascheduling policies for life sciences applications. As an example, an array-job scheduler plug-in is developed for pleasantly parallel applications such as AutoDock and Blast. The performance of the VJM is evaluated through experiments with mpiBLAST-g2 using a Gfarm data grid testbed. Furthermore, a CSF4 portlet has been released to provide a graphical user interface for transparent grid access, with the use of Gfarm for data staging and simplified data management. The platform is open source at sourceforge.net/projects/gcsf/ and has been deployed in life science gateways by projects such as My WorkSphere, and PRAGMA Biosciences Portal. The VJM enables the development of support for more sophisticated workflows and metascheduling policies in the near future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Karonis, N.T., Toonen, B. and Foster, I., “A MPICH-G2: A Grid-enabled implementation of the Message Passing Interface,” Journal of Parallel and Distributed Computing, 2003.

  2. Abramson, D., Giddy, J. and Kotler, L., “High performance parametric modeling with Nimrod/G: Killer application for the global grid,” IPDPS, 2000.

  3. Wei, X., Ding, Z. and Yuan, S., “CSF4: A WSRF Compliant Meta-Scheduler,” Int. Conf. 06’ ’ on Grid Computing and Applications, June 26-29, 2006.

  4. Goodsell, D.S., Morris, G.M. and Olson, A.J., “Automated docking of flexible ligands: applications of AutoDock,” J. Mol Recognit, vol. 9, pp. 1-5, 1996.

    Article  Google Scholar 

  5. Altschul, S.F., Madden, T.L., Schaffer, A.A., Zhang, J., Zhang, Z., Miller, W. and Lipman, D.J., “Gapped BLAST and PSI-BLAST: a new generation of protein database search programs,” Nucleic Acids Res, vol. 25, pp. 3389-402, 1997.

    Article  Google Scholar 

  6. Li, W.W., Krishnan, S., Mueller, K., Ichikawa, K., Date, S., Dallakyan, S., Sanner, M., Misleh, C., Ding, Z., Wei, X., Tatebe, O. and Arzberger, P.W., “Building cyberinfrastructure for bioinformatics using service oriented architecture,” Sixth IEEE Int. Symp. on Cluster Computing and the Grid Workshops Singapore, 2006.

  7. Uthayopas, P., Maneesilp, J. and Ingongnam, P., “SCMS: An Integrated Cluster Management Tool for Beowulf Cluster System,” Proc. of the Int. Conf. on Parallel and Distributed Proceeding Techniques and Applications, 2000.

  8. Foster, I., “Globus Toolkit Version 4: Software for Service-Oriented Systems,” LNCS, 3779, pp. 2-13, 2006.

    Google Scholar 

  9. Globus Alliance “Globus” http://www.globus.org, 2004.

  10. Li, W.W., Baker, N.A. Baldridge, K., McCammon, J.A., Ellisman, M.H. Gupta, A., Holst, M.J., McCulloch, A.D., Michailova, A., Papadopoulos, P., Olson, A. Sanner, M. and Arzberger, P.W., “National Biomedical Computation Resource (NBCR): Developing End-to-End Cyberinfrastructure for Multiscale Modeling in Biomedical Research,” CTWatch Quarterly, 2, pp. 6-17, 2006.

    Google Scholar 

  11. Tatebe, O., Morita, Y. and Matsuoka, S., “Grid Datafarm Architecture for Petascale Data Intensive Computing,” Proc. of the 2nd IEEE/ACM Int. Symp. on Cluster Computing and the Grid, pp.102-110, 2002.

  12. Geist, A., Beguelin, A., Dongarra, J., Jiang, W., Manchek, B. and Sunderam, V., PVM:Parallel Virtual Machine-A User’s Guide and Tutorial for Network Parallel Computing, MIT Press, Cambridge, MA, 1994.

  13. Message Passing Interface Forum, “MPI:A message-passing interface standard,” Int. J. Supercomput. Appl. 8(3/4), 165-414, 1994.

  14. Czajkowski, K., Foster, I., Kesselman, C., Sander, V. and Tuecke, S., “SNAP: A Protocol for negotiating service level agreements and coordinating resource management in distributed systems,” LNCS, 2537, pp. 153-183, 2002.

    Google Scholar 

  15. Distributed Architecture Group from Universidad Complutense, “Gridway” http://www.gridway.org/ , 2006.

  16. Cluster Resources, Inc. “SILVER Design Specification,” http://www.supercluster.org/projects/silver/, Jan, 2004.

  17. Frey, J., Tannenbaum, T., Livny, M., Foster, I. and Tuecke, S., “Condor-G: A Computation Management Agent for Multi-Institutional Grids,” Journal of Cluster Computing, 5, 3 July, 2002.

  18. ROCKS Cluster Distribution, 2005, http://www.rocksclusters.org.

  19. National Biomedical Computation Resource, 2005, http://nbcr.net.

  20. Massie, M.L., Chun, B.N. and Culler, D.E., “The ganglia distributed monitoring system: design, implementation, and experience,” Parallel Computing, 30, pp. 817-840, 2004.

    Article  Google Scholar 

  21. Foster, I., Kesselman, C., Lee, C., Lindell, R, Nahrstedt, K., and Roy, A., “A Distributed Resource Management Architecture that Supports Advance Reservations and Co-Allocation,” Int. Workshop on Quality of Service, 1999.

  22. “Gridsphere,” http://www.gridsphere.org, 2004.

  23. Bhatia, K., Chandra, S., and Mueller, K., “GAMA: Grid Account Management Architecture,” 1st IEEE Int. Conf. on e-Science and Grid Computing, 2006.

  24. Krishnan, S., Stearn, B., Bhatia, K., Baldridge, K. and Li, W.W., et al. “Opal: Simple Web Service Wrappers for Scientific Applications,” Int. Conf. for Web Services, 2006.

  25. von Laszewski, G., Foster, I., Gawor, J. and Lane, P., “A Java Commodity Grid Kit,” Concurrency and Computation: Practice and Experience, 13, pp. 643-662, 2001.

    Article  Google Scholar 

  26. Ding, Z., Luo, Y., Wei, X., Misleh, C., Li, W.W., Arzberger, P.W. and Tatebe, O, “My WorkSphere: Integrated and Transparent Acess to Gfarm Computational Data Grid through GridSphere Portal with Metascheduler CSF4,” 3rd Int. Life Sciences Grid Workshop, 2006.

  27. Allcock, W., Bester, J., Bresnahan, J., Chervenak, A., Liming, L. and Tuecke, S., “GridFTP: Protocol Extension to FTP for the Grid, Grid Forum Internet-Draft,” 2001.

  28. Novotny, J., Tuecke, S. and Welch, V., “An Online Credential Repository for the Grid: MyProxy,” High Performance Distributed Computing, 2001

  29. “Pacific Rim Applications and Grid Middleware Assembly,” 2004, http://www.pragma-grid.net/.

  30. Darling, A., “The Design, Implementation, and Evaluation of mpiBLAST,” ClusterWorld, 2003.

  31. “Community Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis,” 2006, http://camera.calit2.net/.

  32. “TeraGrid,” 2004, http://www.teragrid.org.

  33. “Open Science Grid,” 2006, http://www.opensciencegrid.org/.

Download references

Author information

Authors and Affiliations

  1. College of Computer Science & Technology, Jilin University, Changchun, Jilin, 130012, China

    Zhaohui Ding, Xiaohui Wei, Yuan Luo & Da ma

  2. Center for Research in Biological Systems, University of California, San Diego, La Jolla, CA, 92093, USA

    Peter W. Arzberger

  3. San Diego Supercomputer Center, University of California, San Diego, La Jolla, CA, 92093, USA

    Wilfred W. Li

Authors
  1. Zhaohui Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohui Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Da ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter W. Arzberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wilfred W. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhaohui Ding.

About this article

Cite this article

Ding, Z., Wei, X., Luo, Y. et al. Customized Plug-in Modules in Metascheduler CSF4 for Life Sciences Applications. New Gener. Comput. 25, 373–394 (2007). https://doi.org/10.1007/s00354-007-0024-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-007-0024-6

Keywords

Search

Navigation