Gary Planthaber
Michael Stonebraker
ABSTRACT
Earth scientists are increasingly experiencing difficulties with analyzing rapidly growing volumes of complex data. Those who must perform analysis directly on low-level National Aeronautics and Space Administration (NASA) Moderate Resolution Imaging Spectroradiometer (MODIS) Level 1B calibrated and geolocated data, for example, encounter an arcane, high-volume data set that is burdensome to make use of. Instead, Earth scientists typically opt to use higher-level "canned" products provided by NASA. However, when these higher-level products fail to meet the requirements of a particular project, a cruel dilemma arises: cope with data products that don't exactly meet the project's needs or spend an enormous amount of resources extracting what is needed from the unadulterated low-level data. In this paper, we present EarthDB, a system that eliminates this dilemma by offering the following contributions:
1. Enabling painless importing of MODIS Level 1B data into SciDB, a highly scalable science-oriented database platform that abstracts away the complexity of distributed storage and analysis of complex multi-dimensional data,
2. Defining a schema that unifies storage and representation of MODIS Level 1B data, regardless of its source file,
3. Supporting fast filtering and analysis of MODIS data through the use of an intuitive, high-level query language rather than complex procedural programming and,
4. Providing the ability to easily define and reconfigure entire analysis pipelines within the SciDB database, allowing for rapid ad-hoc analysis. To demonstrate this ability, we provide sample benchmarks for the construction of true-color (RGB) and Normalized Difference Vegetative Index (NDVI) images from raw MODIS Level 1B data using relatively simple queries with scalable performance.
- Paul Brown et al., "Overview of SciDB: Large Scale Array Storage, Processing and Analysis," in Proceedings of the 2010 International Conference on Management of Data (SIGMOD 2010), 2010, pp. 963--968. Google Scholar
Digital Library
- Philippe Cudre-Mauroux et al. (2009) SS-DB: A Standard Science DBMS Benchmark.Google Scholar
- Philippe Cudre-Mauroux et al., "A Demonstration of SciDB: A Science-Oriented DBMS," in Proceedings of the 2009 VLDB Endowment (August 2009), Lyon, France, 2009, pp. 1534--1537. Google ScholarDigital Library
- HDF Group. HDF-EOS Tools and Information Center. {Online}. http://hdfeos.org/Google Scholar
- Kantor et. al. Designing for peta-scale in the LSST database. Astronomical Data Analysis Software and Systems XVI, 376:3--11.Google Scholar
- NASA. Aqua Satellite. {Online}. http://aqua.nasa.gov/Google Scholar
- NASA. MODIS. {Online}. http://modis.gsfc.nasa.gov/Google Scholar
- NASA. MODIS Brochure. {Online}. http://modis.gsfc.nasa.gov/about/media/modis_brochure.pdfGoogle Scholar
- NASA. MODIS Data Products. {Online}. http://modis.gsfc.nasa.gov/data/dataprod/index.phpGoogle Scholar
- NASA. (1997, August) MODIS Level 1A Earth Location: Algorithm Theoretical Basis Document Version 3.0. {Online}. http://modis.gsfc.nasa.gov/data/atbd/atbd_mod28_v3.pdfGoogle Scholar
- NASA. (2006, August) MODIS Level 1B Product User's Guide. {Online}. http://mcst.gsfc.nasa.gov/sites/mcst.gsfc/files/file_attachments/M1054.pdfGoogle Scholar
- NASA. (2006, April) MODIS Vegetation Index Algorithm Theoretical Basis Document Version 3.0. {Online}. http://modis.gsfc.nasa.gov/data/atbd/atbd_mod13.pdfGoogle Scholar
- NASA. Terra Satellite. {Online}. http://terra.nasa.gov/sdfGoogle Scholar
- Paradigm4. (2012) SciDB User's Guide.Google Scholar
- Michael Stonebraker et al., "Requirements for Science Data Bases and SciDB," in Proceedings of the 2009 Conference on Innovative Data Systems Research (CIDR 2009), 2009.Google Scholar
- Michael Stonebraker, Paul Brown, Alex Poliakov, and Suchi Raman, "The Architecture of SciDB," in Proceedings of the 23rd International Conference on Scientific and Statistical Database Management (SSDBM 2011), 2011, pp. 1--16. Google ScholarDigital Library
- Daniel Liwei Want et al., A Basic Loader of HDF Files into SciDB. {Online}. https://github.com/wangd/SciDB-HDF5Google Scholar
- Michael Way. (2011, August) Challenges for LSST scale data sets. {Online}. http://arxiv.org/abs/1108.5124v1Google Scholar
- Wenli Yang and Liping Di, "An Accurate and Automated Approach to Georectification of HDF-EOS Swath Data," Photogrammetric Engineering and Remote Sensing, vol. 70, no. 4, pp. 397--404, April 2004.Google ScholarCross Ref
- XLDB. About the XLDB Organization. {Online}. http://www.xldb.org/about/Google Scholar
Index Terms
- EarthDB: scalable analysis of MODIS data using SciDB
Recommendations
Validation of synthetic daily Landsat NDVI time series data generated by the improved spatial and temporal data fusion approach
Daily 30m data were generated by combining MODIS and Landsat data.A temporal validation method for synthetic daily 30m data was proposed and tested.Vegetation phenology was monitored using synthetic daily Landsat NDVI data.Phenology was monitored at 30m ...
Visualizing and quantifying the movement of vegetative drought using remote-sensing data and GIS
Remote-sensing-based drought monitoring methods provide fast and useful information for a sustainable management strategy of drought impact over a region. Common pixel-based monitoring methods are limited in the analysis of the dynamics of this impact ...
Application of EOS/MODIS-NDVI at Different Time Sequences on Monitoring Winter Wheat Acreage in Henan Province
ESIAT '09: Proceedings of the 2009 International Conference on Environmental Science and Information Application Technology - Volume 02Calculation of winter wheat acreage in Henan Province using EOS/MODIS-NDVI data at different time sequences. [Method] After process of EOS/MODIS images, geographical adjustment, wave band combination, normal difference vegetation index (NDVI) was ...
Comments