Abstract
Many Physics experiments today generate large volumes of data. That data is then processed in many ways in order to achieve the understanding of fundamental physical phenomena. Virtual Data is a concept that unifies the view of the data whether it is raw or derived. It provides a new degree of transparency in how data-handling and processing capabilities are integrated to deliver data products to end-users or applications, so that requests for such products are easily mapped into computation and/or data access at multiple locations. GriPhyN (Grid Physics Network) is a NSF-funded project, which aims to realize the concepts of Virtual Data. Among the physics applications participating in the project is the Laser Interferometer Gravitational-wave Observatory (LIGO), which is being built to observe the gravitational waves predicted by general relativity. LIGO will produce large amounts of data, which are expected to reach hundreds of petabytes over the next decade. Large communities of scientists, distributed around the world, need to access parts of these datasets and perform efficient analysis on them. It is expected that the raw and processed data will be distributed among various national centers, university computing centers, and individual workstations. In this paper we describe some of the challenges associated with building Virtual Data Grids for experiments such as LIGO.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. ”data management and transfer in high-performance computational grid environments.”. Parallel Computing, 2001.
W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. ”secure, efficient data transport and replica management for high-performance data-intensive computing.”. In IEEE Mass Storage Conference, 2001.
W. Allcock, I. Foster, V. Nefedova, A. Chervenak, E. Deelman, C. Kesselman, A. Sim, A. Shoshani, B. Drach, and D. Williams. ”high-performance remote access to climate simulation data: A challenge problem for data grid technologies”. 2001. SC’2001.
B. Allen, E. Deelman, C. Kesselman, A. Lazzarini, T. Prince, J. Romano, and R. Williams. ”ligo’s virtual data requirements.”. Technical Report 6, GriPhyN, 2001.
O. Babaoglu and K. Marzullo. Consistent global states of distributed systems: Fundamental concepts and mechanisms. Technical Report UBLCS-93-1, University of Bologna, 1983.
K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, and S. Tuecke. A resource management architecture for metacomputing systems. In The 4th Workshop on Job Scheduling Strategies for Parallel Processing, pages 62–82, 1998.
Karl Czajkowski, Steven Fitzgerald, Ian Foster, and Carl Kesselman. Grid information services for distributed resource sharing. In Proc. 10th IEEE Symp. on High Performance Distributed Computing, 2001.
E. Deelman, C. Kesselman, and G. Mehta. ”transformation catalog design for griphyn”. Technical Report 17, GriPhyN, 2001.
E. Deelman, C. Kesselman, R. Williams, A. Lazzarini, T. A. Prince, J. Romano, and B. Allen. ”a virtual data grid for ligo”. volume 2110 of Lecture Notes in Computer Science, pages 3–12, 2001.
I. Foster and C. Kesselman. Globus: A metacomputing infrastructure toolkit. Intl. Journal of Supercomputing Applications, 11(2):115–128, 1997.
I. Foster and C. Kesselman. The Globus project: A status report. In Proceedings of the Heterogeneous Computing Workshop, pages 4–18. IEEE Computer Society Press, 1998.
I. Foster and C. Kesselman, editors. The Grid: Blueprint for a Future Computing Infrastructure. 1999.
I. Foster, C. Kesselman, G. Tsudik, and S. Tuecke. A security architecture for computational grids. In ACM Conference on Computers and Security, pages 83–91. ACM Press, 1998.
I. Foster, C. Kesselman, and S. Tuecke. ”the anatomy of the grid: Enabling scalable virtual organizations.”. Intl. J. Supercomputer Applications, 15(3), 2001.
J. Frey, T. Tannenbaum, I. Foster, M. Livny, and S. Tuecke. ”condor-g: A computation management agent for multi-institutional grids”. In Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), 2001.
”MetaNEOS Project”. http://www-unix.mcs.anl.gov/metaneos/nug30/pr.html, 2001.
R. Wolski. Forecasting network performance to support dynamic scheduling using the network weather service. In Proc. 6th IEEE Symp. on High Performance Distributed Computing, Portland, Oregon, 1997. IEEE Press.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Deelman, E., Kesselman, C., Williams, R., Blackburn, K., Lazzarini, A., Koranda, S. (2002). Applications of Virtual Data in the LIGO Experiment. In: Wyrzykowski, R., Dongarra, J., Paprzycki, M., Waśniewski, J. (eds) Parallel Processing and Applied Mathematics. PPAM 2001. Lecture Notes in Computer Science, vol 2328. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48086-2_3
Download citation
DOI: https://doi.org/10.1007/3-540-48086-2_3
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43792-5
Online ISBN: 978-3-540-48086-0
eBook Packages: Springer Book Archive