Abstract
In this short paper we outline the data vault, a database-attached external file repository. It provides a true symbiosis between a DBMS and existing file-based repositories. Data is kept in its original format while scalable processing functionality is provided through the DBMS facilities. In particular, it provides transparent access to all data kept in the repository through an (array-based) query language using the file-type specific scientific libraries.
The design space for data vaults is characterized by requirements coming from various fields. We present a reference architecture for their realization in (commercial) DBMSs and a concrete implementation in MonetDB for remote sensing data geared at content-based image retrieval.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alagiannis, I., Borovica, R., Branco, M., Idreos, S., Ailamaki, A.: NoDB: Efficient Query Execution on Raw Data Files. In: SIGMOD (2012)
Baumann, P.: Large-Scale Earth Science Services: A Case for Databases. In: ER (Workshops), pp. 75–84 (2006)
Baumann, P., et al.: The multidimensional database system RasDaMan. SIGMOD Rec. 27(2), 575–577 (1998)
Cerra, D., Datcu, M.: Image Retrieval using Compression-based Techniques. In: International ITG Conference on Source and Channel Coding (2010)
Dumitru, C.O., Molina, D.E., et al.: TELEIOS WP3: KDD concepts and methods proposal: report and design recommendations, http://www.earthobservatory.eu/deliverables/FP7-257662-TELEIOS-D3.1.pdf
FITS. Flexible Image Transport System, http://heasarc.nasa.gov/docs/heasarc/fits.html
GeoTIFF, http://trac.osgeo.org/geotiff/
Hey, T., Tansley, S., Tolle, K.: The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research (2009)
Ivanova, M., Kersten, M., Nes, N., Gonçalves, R.: An Architecture for Recycling Intermediates in a Column-store. ACM Trans. Database Syst. 35(4), 24 (2010)
Kunchithapadam, K., Zhang, W., et al.: Oracle Database Filesystem. In: SIGMOD, pp. 1149–1160 (2011)
MonetDB (2012), http://www.monetdb.org/
Oracle. Oracle Spatial GeoRaster Developer’s Guide, 11g Release 2 (11.2)
PostGIS, http://www.postgis.org/
SEED. Standard for the exchange of earthquake data (May 2010), http://www.iris.edu/manuals/SEEDManual_V2.4.pdf
SQL/MED. ISO/IEC 9075-9:2008 Information technology - Database languages - SQL - Part 9: Management of External Data (SQL/MED)
Stolte, E., von Praun, C., Alonso, G., Gross, T.R.: Scientific data repositories: Designing for a moving target. In: SIGMOD Conference, pp. 349–360 (2003)
Zhang, Y., Kersten, M., Ivanova, M., Nes, N.: SciQL: Bridging the Gap between Science and Relational DBMS. In: IDEAS, pp. 124–133 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ivanova, M., Kersten, M., Manegold, S. (2012). Data Vaults: A Symbiosis between Database Technology and Scientific File Repositories. In: Ailamaki, A., Bowers, S. (eds) Scientific and Statistical Database Management. SSDBM 2012. Lecture Notes in Computer Science, vol 7338. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31235-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-31235-9_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31234-2
Online ISBN: 978-3-642-31235-9
eBook Packages: Computer ScienceComputer Science (R0)