skip to main content
10.1145/2753524.2753529acmconferencesArticle/Chapter ViewAbstractPublication PageshpdcConference Proceedingsconference-collections
research-article

Apache Airavata as a Laboratory: Architecture and Case Study for Component-Based Gateway Middleware

Published: 16 June 2015 Publication History

Abstract

Science gateways are more than user interfaces to computational grids and clouds. Gateways are middleware in their own right, providing flexible, lightweight federations of heterogenous collections of computing resources (such as campus clusters, supercomputers, computational clouds), all of which remain challenges for many alternative middleware approaches. Gateways also are notable for providing science application-centric interfaces to computing resources rather than resource-centric views. An important challenge for science gateway research is to generalize specific science gateway strategies, defining a reference architecture that emcompasses major gateway capabilities while enabling implementation flexibility. Such a reference architecture should also enable "platform as a service" approaches that provide hosted versions of common gateway capabilities. In this paper, we summarize the Apache Airavata software system as a candidate reference architecture for science gateways. We propose the use of a component-based architecture to encompass major gateway capabilities (such as metadata management, meta-scheduling, execution management, and messaging). We examine the messaging system component in this abstract architecture in detail and describe its re-implementation and validation using third party messaging system software to replace a custom-built messaging system. Besides the operational validation of this specific component, we infer a preliminary validation of the overall architecture. The flexibility of component implementations within an overall architecture is essential as it allows gateway middleware to be the subject of distributed computing research for its own sake while also ensuring that we don't get locked into less than optimal implementations for gateway operations.

References

[1]
D. Blankenberg, G. V. Kuster, N. Coraor, G. Ananda, R. Lazarus, M. Mangan, A. Nekrutenko, and J. Taylor. Galaxy: a web-based genome analysis tool for experimentalists. Current protocols in molecular biology, pages 19--10, 2010.
[2]
W. Chathuri. Airavata rabbitmq integration. https://issues.apache.org/jira/browse/AIRAVATA-1442, 2014.
[3]
M. Christie and S. Marru. The lead portal: a teragrid gateway and application service architecture. Concurrency and Computation: Practice and Experience, 19(6):767--781, 2007.
[4]
B. Demeler. UltraScan: a comprehensive data analysis software package for analytical ultracentrifugation experiments. Analytical Ultracentrifugation: Techniques And Methods, pages 210--229, 2005.
[5]
R. Dooley, K. Milfeld, C. Guiang, S. Pamidighantam, and G. Allen. From proposal to production: Lessons learned developing the computational chemistry grid cyberinfrastructure. Journal of Grid Computing, 4(2):195--208, 2006.
[6]
R. Dooley and J. Stubbs. Dynamically provisioning portable gateway infrastructure using docker and agave. In Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment, page 55. ACM, 2014.
[7]
B. Erickson, R. Singh, A. E. Evrard, M. R. Becker, M. T. Busha, A. V. Kravtsov, S. Marru, M. Pierce, and R. H. Wechsler. Enabling dark energy survey science analysis with simulations on xsede resources. In Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to Discovery, page 16. ACM, 2013.
[8]
J. Ghosh, N. Singh, Y. Fan, S. Marru, K. Vanomesslaeghe, and S. Pamidighantam. Molecular Parameter Optimization Gateway (ParamChem). In Proceedings of the 2011 TeraGrid Conference. ACM, 2011.
[9]
C. Herath, F. Liu, S. Marru, L. Gunathilake, M. Sosonkina, J. P. Vary, P. Maris, and M. Pierce. Web service andworkflow abstractions to large scale nuclear physics calculations. In Services Computing (SCC), 2012 IEEE Ninth International Conference on, pages 703--710. IEEE, 2012.
[10]
Y. Huang and D. Gannon. A flexible and efficient approach to reconcile different web services-based event notification specifications. In Web Services, 2006. ICWS'06. International Conference on, pages 735--742. IEEE, 2006.
[11]
Y. Huang, A. Slominski, C. Herath, and D. Gannon. Ws-messenger: A web services-based messaging system for service-oriented grid computing. In Cluster Computing and the Grid, 2006. CCGRID 06. Sixth IEEE International Symposium on, volume 1, pages 8--pp. IEEE, 2006.
[12]
P. Kacsuk, Z. Farkas, M. Kozlovszky, G. Hermann, A. Balasko, K. Karoczkai, and I. Marton. Ws-pgrade/guse generic dci gateway framework for a large variety of user communities. Journal of Grid Computing, 10(4):601--630, 2012.
[13]
Y. W. Kim, H.-Y. Kim, S. Bae, H.-Y. Kim, Y.-C. Woo, S.-J. Park, and W. Choi. Design of maha supercomputing system for human genome analysis. Korea Information Processing Society Transactions on Software and Data Engineering, 2(2):2, 2013.
[14]
G. Klimeck, M. McLennan, S. P. Brophy, G. B. Adams III, and M. S. Lundstrom. nanohub. org: Advancing education and research in nanotechnology. Computing in Science & Engineering, 10(5):17--23, 2008.
[15]
J. Kreps, N. Narkhede, J. Rao, et al. Kafka: A distributed messaging system for log processing. In Proceedings of 6th International Workshop on Networking Meets Databases (NetDB), Athens, Greece, 2011.
[16]
J. Krüger, R. Grunzke, S. Gesing, S. Breuers, A. Brinkmann, L. de la Garza, O. Kohlbacher, M. Kruse, W. E. Nagel, L. Packschies, et al. The mosgrid science gateway--a complete solution for molecular simulations. Journal of Chemical Theory and Computation, 10(6):2232--2245, 2014.
[17]
H. Lee et al. Biovlab-mmia: A cloud environment for microrna and mrna integrated analysis (mmia) on amazon ec2. NanoBioscience, IEEE Transactions on, 2012.
[18]
S. Marru et al. Apache airavata: a framework for distributed applications and computational workflows. In Proceedings of the 2011 ACM workshop on Gateway computing environments. ACM, 2011.
[19]
F. Martin. Microservices. http://martinfowler.com/articles/microservices.html, 2014.
[20]
M. McLennan and R. Kennell. Hubzero: a platform for dissemination and collaboration in computational science and engineering. Computing in Science & Engineering, 12(2):48--53, 2010.
[21]
M. A. Miller, W. Pfeiffer, and T. Schwartz. Creating the cipres science gateway for inference of large phylogenetic trees. In Gateway Computing Environments Workshop (GCE), 2010, pages 1--8. IEEE, 2010.
[22]
S. Perera, S. Marru, and C. Herath. Workflow infrastructure for multi-scale science gateways. In TeraGrid Conference, 2008.
[23]
M. Pierce, S. Marru, B. Demeler, R. Singh, and G. Gorbet. The apache airavata application programming interface: overview and evaluation with the ultrascan science gateway. In Proceedings of the 9th Gateway Computing Environments Workshop, pages 25--29. IEEE Press, 2014.
[24]
M. Pierce, S. Marru, L. Gunathilake, T. A. Kanewala, R. Singh, S. Wijeratne, C. Wimalasena, C. Herath, E. Chinthaka, C. Mattmann, et al. Apache airavata: Design and directions of a science gateway framework. In Science Gateways (IWSG), 2014 6th International Workshop on, pages 48--54. IEEE, 2014.
[25]
M. Pierce, S. Marru, R. Singh, A. Kulshrestha, and K. Muthuraman. Open grid computing environments: advanced gateway support activities. In Proceedings of the 2010 TeraGrid Conference, page 16. ACM, 2010.
[26]
A. Qpid. Open source amqp messaging. URL: http://qpid.apache.org), 2013.
[27]
J. Towns. Evolving from teragrid to xsede. Bulletin of the American Physical Society, 2011.
[28]
K. Vanommeslaeghe, E. P. Raman, and A. D. MacKerell Jr. Automation of the charmm general force field (cgenff) ii: assignment of bonded parameters and partial atomic charges. Journal of chemical information and modeling, 52(12):3155--3168, 2012.
[29]
A. Videla and J. J. Williams. RabbitMQ in action: distributed messaging for everyone. Manning, 2012.
[30]
S. Vinoski. Advanced message queuing protocol. IEEE Internet Computing, pages 87--89, 2006.
[31]
N. Wilkins-Diehr, D. Gannon, G. Klimeck, S. Oster, and S. Pamidighantam. Teragrid science gateways and their impact on science. Computer, 2008.
[32]
N. Wilkins-Diehr, S. Gesing, and T. Kiss. Science gateway workshops 2013 special issue conference publications. Concurrency and Computation: Practice and Experience, 2014.

Cited By

View all
  • (2024)A Comparative Analysis of Apache Cloud Projects for Data Storage2024 7th International Conference on Advanced Communication Technologies and Networking (CommNet)10.1109/CommNet63022.2024.10793258(1-9)Online publication date: 4-Dec-2024
  • (2023)An Innovative Metric-based Clustering Approach for Increased Scalability and Dependency Elimination in Monolithic Legacy SystemsEngineering, Technology & Applied Science Research10.48084/etasr.604813:4(11375-113876)Online publication date: 9-Aug-2023
  • (2019)Improving Science Gateways usage reporting for XSEDEPractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3338096(1-4)Online publication date: 28-Jul-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SCREAM '15: Proceedings of the 1st Workshop on The Science of Cyberinfrastructure: Research, Experience, Applications and Models
June 2015
82 pages
ISBN:9781450335669
DOI:10.1145/2753524
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 June 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. apache airavata
  2. distributed systems
  3. evolutionary architecture
  4. microservice architecture
  5. science gateways

Qualifiers

  • Research-article

Funding Sources

Conference

HPDC'15
Sponsor:

Acceptance Rates

SCREAM '15 Paper Acceptance Rate 8 of 12 submissions, 67%;
Overall Acceptance Rate 8 of 12 submissions, 67%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Comparative Analysis of Apache Cloud Projects for Data Storage2024 7th International Conference on Advanced Communication Technologies and Networking (CommNet)10.1109/CommNet63022.2024.10793258(1-9)Online publication date: 4-Dec-2024
  • (2023)An Innovative Metric-based Clustering Approach for Increased Scalability and Dependency Elimination in Monolithic Legacy SystemsEngineering, Technology & Applied Science Research10.48084/etasr.604813:4(11375-113876)Online publication date: 9-Aug-2023
  • (2019)Improving Science Gateways usage reporting for XSEDEPractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3338096(1-4)Online publication date: 28-Jul-2019
  • (2019)Experiences from scaling scale Science Gateway operationsPractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3333159(1-4)Online publication date: 28-Jul-2019
  • (2019)Implementing a Flexible, Fault Tolerant Job Management System for Science GatewaysPractice and Experience in Advanced Research Computing 2019: Rise of the Machines (learning)10.1145/3332186.3332233(1-8)Online publication date: 28-Jul-2019
  • (2019)Straddling the crevasse: A review of microservice software architecture foundations and recent advancementsSoftware: Practice and Experience10.1002/spe.272949:10(1448-1484)Online publication date: 19-Jul-2019
  • (2018)Simplifying Access to Campus Resources at Southern Illinois University with a Science GatewayProceedings of the Practice and Experience on Advanced Research Computing: Seamless Creativity10.1145/3219104.3229252(1-4)Online publication date: 22-Jul-2018
  • (2017)Guidelines for adopting frontend architectures and patterns in microservices-based systemsProceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering10.1145/3106237.3117775(902-907)Online publication date: 21-Aug-2017
  • (2016)Scientific WorkflowsACM Computing Surveys10.1145/301242949:4(1-39)Online publication date: 12-Dec-2016
  • (2016)Anatomy of the SEAGrid Science GatewayProceedings of the XSEDE16 Conference on Diversity, Big Data, and Science at Scale10.1145/2949550.2949591(1-8)Online publication date: 17-Jul-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media