research-article

Big earth observation data analytics: matching requirements to system architectures

Authors:

Gilberto Camara,

Luiz Fernando Assis,

Gilberto Ribeiro,

Karine Reis Ferreira,

Lubia VinhasAuthors Info & Claims

BigSpatial '16: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data

Pages 1 - 6

https://doi.org/10.1145/3006386.3006393

Published: 31 October 2016 Publication History

Abstract

Earth observation satellites produce petabytes of geospatial data. To manage large data sets, researchers need stable and efficient solutions that support their analytical tasks. Since the technology for big data handling is evolving rapidly, researchers find it hard to keep up with the new developments. To lower this burden, we argue that researchers should not have to convert their algorithms to specialised environments. Imposing a new API to researchers is counterproductive and slows down progress on big data analytics. This paper assesses the cost of research-friendliness, in a case where the researcher has developed an algorithm in the R language and wants to use the same code for big data analytics. We take an algorithm for remote sensing time series analysis on compare it use on map/reduce and on array database architectures. While the performance of the algorithm for big data sets is similar, organising image data for processing in Hadoop is more complicated and time-consuming than handling images in SciDB. Therefore, the combination of the array database SciDB and the R language offers an adequate support for researchers working on big Earth observation data analytics.

References

[1]

A. Aji, F. Wang, H. Vo, R. Lee, Q. Liu, X. Zhang, and J. Saltz. Hadoop GIS: A high performance spatial data warehousing system over mapreduce. Proc. VLDB Endow., 6(11):1009--1020, 2013.

Digital Library

[2]

P. Baumann, A. Dehmel, P. Furtado, R. Ritsch, and N. Widmann. The multidimensional database system RasDaMan. ACM SIGMOD Record, 27(2):575--577, 1998.

Digital Library

[3]

M. Broich, M. C. Hansen, P. Potapov, B. Adusei, E. Lindquist, and S. V. Stehman. Time-series analysis of multi-resolution optical imagery for quantifying forest cover loss in Sumatra and Kalimantan, Indonesia. International Journal of Applied Earth Observation and Geoinformation, 13(2):277--291, 2011.

[4]

P. G. Brown. Overview of SciDB: Large scale array storage, processing and analysis. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pages 963--968, New York, NY, USA, 2010. ACM.

Digital Library

[5]

J. Dean and S. Ghemawat. MapReduce: a flexible data processing tool. Communications of the ACM, 53(1):72--77, 2010.

Digital Library

[6]

A. Eldawy and M. F. Mokbel. SpatialHadoop: A MapReduce framework for spatial data. In IEEE 31st International Conference on Data Engineering (ICDE 2015), volume 1, pages 1352--1363, 2015.

[7]

P. Esling and C. Agon. Time-series data mining. ACM Computing Surveys, 45(1):12:1--12:34, 2012.

Digital Library

[8]

M. A. Friedl, D. Sulla-Menashe, B. Tan, A. Schneider, N. Ramankutty, A. Sibley, and X. Huang. MODIS collection 5 global land cover: Algorithm refinements and characterization of new datasets. Remote Sensing of Environment, 114(1):168 -- 182, 2010.

[9]

G. L. Galford, J. F. Mustard, J. Melillo, A. Gendrin, C. C. Cerri, and C. E. Cerri. Wavelet analysis of MODIS time series to detect expansion and intensification of row-crop agriculture in brazil. Remote Sensing of Environment, 112(2):576--587, 2008.

[10]

N. Gorelick. Google Earth Engine. In AGU Fall Meeting Abstracts, volume 1, page 04, 2012.

[11]

J. Gray, D. T. Liu, M. Nieto-Santisteban, A. Szalay, D. J. DeWitt, and G. Heber. Scientific data management in the coming decade. SIGMOD Rec., 34(4):34--41, 2005.

Digital Library

[12]

P. Griffiths, S. van der Linden, T. Kuemmerle, and P. Hostert. A pixel-based Landsat compositing algorithm for large area land cover mapping. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(5):2088--2101, 2013.

[13]

M. Hansen, P. Potapov, R. Moore, M. Hancher, S. Turubanova, A. Tyukavina, D. Thau, S. Stehman, S. Goetz, T. Loveland, A. Kommareddy, A. Egorov, L. Chini, C. Justice, and J. Townshend. High-resolution global maps of 21st-century forest cover change. Science (New York, N.Y.), 342(2013):850--3, 2013.

[14]

P. Jönsson and L. Eklundh. Timesat---a program for analyzing time-series of satellite sensor data. Computers & Geosciences, 30(8):833 -- 845, 2004.

[15]

A. Kaptué Tchuenté, J. Roujean, and S. De Jong. Comparison and relative quality assessment of the GLC2000, GLOBCOVER, MODIS and ECOCLIMAP land cover data sets at the african continental scale. International Journal of Applied Earth Observation and Geoinformation, 13(2):207--219, 2011.

[16]

R. E. Kennedy, Z. Yang, and W. B. Cohen. Detecting trends in forest disturbance and recovery using yearly Landsat time series. Remote Sensing of Environment, 114(12):2897--2910, 2010.

[17]

E. Keogh and C. A. Ratanamahatana. Exact indexing of dynamic time warping. Knowledge Information Systems, 7(3):358--386, 2005.

Digital Library

[18]

L. Krcal and S. Ho. A SciDB-based framework for efficient satellite data storage and query based on dynamic atmospheric event trajectory. In 4th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, Seattle, WA, USA, 2015.

Digital Library

[19]

V. Maus, G. Câmara, R. Cartaxo, A. Sanchez, F. M. Ramos, and G. R. de Queiroz. A time-weighted dynamic time warping method for land-use and land-cover mapping. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(8):3729 -- 3739, 2016.

[20]

I. McCallum, M. Obersteiner, S. Nilsson, and A. Shvidenko. A spatial comparison of four satellite derived 1km global land cover datasets. International Journal of Applied Earth Observation and Geoinformation, 8(4):246--255, 2006.

[21]

S. Nativi, P. Mazzetti, M. Santoro, F. Papeschi, M. Craglia, and O. Ochiai. Big data challenges in building the Global Earth Observation System of Systems. Environmental Modelling & Software, 68:1--26, 2015.

Digital Library

[22]

F. Petitjean, J. Inglada, and P. Gancarski. Satellite image time series analysis under time warping. IEEE Transactions on Geoscience and Remote Sensing, 50(8):3081--3095, 2012.

[23]

G. Planthaber, M. Stonebraker, and J. Frew. EarthDB: scalable analysis of MODIS data using SciDB. Proceedings of the 1st ACM SIGSPATIAL Workshop on Analytics for Big Geospatial Data - BigSpatial '12, pages 11--19, 2012.

Digital Library

[24]

R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2015.

[25]

G. J. Roerink, M. Menenti, and W. Verhoef. Reconstructing cloudfree NDVI composites using Fourier analysis of time series. International Journal of Remote Sensing, 21(9):1911--1917, 2000.

[26]

M. Stonebraker, P. Brown, D. Zhang, and J. Becla. Scidb: A database management system for applications with complex analytics. Computing in Science & Engineering, 15(3):54--62, 2013.

Digital Library

[27]

R. R. Vatsavai, A. Ganguly, V. Chandola, A. Stefanidis, S. Klasky, and S. Shekhar. Spatiotemporal data mining in the era of big spatial data: Algorithms and applications. In Proceedings of the 1st ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, pages 1--10. ACM, 2012.

Digital Library

[28]

J. Verbesselt, R. Hyndman, G. Newnham, and D. Culvenor. Detecting trend and seasonal changes in satellite image time series. Remote Sensing of Environment, 114(1):106--115, 2010.

[29]

X. Zhang, M. A. Friedl, C. B. Schaaf, A. H. Strahler, J. C. Hodges, F. Gao, B. C. Reed, and A. Huete. Monitoring vegetation phenology using modis. Remote Sensing of Environment, 84(3):471 -- 475, 2003.

Cited By

N SMurthy M S N(2025)Deep spatio-textural feature driven multi-constraints pool-based active learning model for resource-efficient big earth observationsInternational Journal of Remote Sensing10.1080/01431161.2025.245712846:6(2611-2645)Online publication date: 10-Feb-2025
https://doi.org/10.1080/01431161.2025.2457128
Fotakidis VRoustanis TPanayiotou KChrysafis IFitoka EMallinis G(2024)The EL-BIOS Earth Observation Data Cube for Supporting Biodiversity Monitoring in GreeceRemote Sensing10.3390/rs1620377116:20(3771)Online publication date: 11-Oct-2024
https://doi.org/10.3390/rs16203771
Miller LPelletier CWebb G(2024)Deep Learning for Satellite Image Time-Series Analysis: A reviewIEEE Geoscience and Remote Sensing Magazine10.1109/MGRS.2024.339301012:3(81-124)Online publication date: Sep-2024
https://doi.org/10.1109/MGRS.2024.3393010
Show More Cited By

Index Terms

Big earth observation data analytics: matching requirements to system architectures
1. Information systems
  1. Information systems applications

Recommendations

Responsible Big Data Analytics for E-Business Services
ICBDR '21: Proceedings of the 5th International Conference on Big Data Research

This paper examines responsible big data analytics for e-business services and looks at how to use responsible big data analytics to obtain responsible e-business services. It addresses why responsibility matters to big data analytics and e-business ...
Big Data Analytics
Multimedia Big Data Analytics: A Survey

With the proliferation of online services and mobile technologies, the world has stepped into a multimedia big data era. A vast amount of research work has been done in the multimedia area, targeting different aspects of big data analytics, such as the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

BigSpatial '16: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data

October 2016

65 pages

ISBN:9781450345811

DOI:10.1145/3006386

Editors:
Varun Chandola
State University of New York at Buffalo, NY
,
Ranga Raju Vatsavai
North Carolina State University, NC

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Fundação de Amparo à Pesquisa do Estado de São Paulo

Conference

SIGSPATIAL'16

SIGSPATIAL'16: 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems

October 31, 2016

California, Burlingame

Acceptance Rates

BigSpatial '16 Paper Acceptance Rate 8 of 14 submissions, 57%;

Overall Acceptance Rate 32 of 58 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
456
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

N SMurthy M S N(2025)Deep spatio-textural feature driven multi-constraints pool-based active learning model for resource-efficient big earth observationsInternational Journal of Remote Sensing10.1080/01431161.2025.245712846:6(2611-2645)Online publication date: 10-Feb-2025
https://doi.org/10.1080/01431161.2025.2457128
Fotakidis VRoustanis TPanayiotou KChrysafis IFitoka EMallinis G(2024)The EL-BIOS Earth Observation Data Cube for Supporting Biodiversity Monitoring in GreeceRemote Sensing10.3390/rs1620377116:20(3771)Online publication date: 11-Oct-2024
https://doi.org/10.3390/rs16203771
Miller LPelletier CWebb G(2024)Deep Learning for Satellite Image Time-Series Analysis: A reviewIEEE Geoscience and Remote Sensing Magazine10.1109/MGRS.2024.339301012:3(81-124)Online publication date: Sep-2024
https://doi.org/10.1109/MGRS.2024.3393010
Gomes VQueiroz GFerreira KPebesma EBarbosa C(2024)Brazil Data Cube Workflow Engine: a tool for big Earth observation data processingInternational Journal of Digital Earth10.1080/17538947.2024.231309917:1Online publication date: 9-Feb-2024
https://doi.org/10.1080/17538947.2024.2313099
Atesoglu AAyyildiz EKarakaya IBulut FSerengil Y(2024)Land cover and drought risk assessment in Türkiye’s mountain regions using neutrosophic decision support systemEnvironmental Monitoring and Assessment10.1007/s10661-024-13155-3196:11Online publication date: 12-Oct-2024
https://doi.org/10.1007/s10661-024-13155-3
Kothari SShah JVerma JMankad SGarg S(2024)Raster Big Data Processing Using Spark with GeoTrellisComputing, Communication and Learning10.1007/978-3-031-56998-2_22(260-271)Online publication date: 31-Mar-2024
https://doi.org/10.1007/978-3-031-56998-2_22
Radampola ADawalagala HGowsigan PDassanayake SJayawardena CChaminda S(2023)Google Earth Engine Based Assessment of Seasonal Spatial Distribution of Water Hyacinth in North Bolgoda Lake, Sri Lanka2023 Moratuwa Engineering Research Conference (MERCon)10.1109/MERCon60487.2023.10355426(385-390)Online publication date: 9-Nov-2023
https://doi.org/10.1109/MERCon60487.2023.10355426
Horry MChakraborty SPradhan BShulka NAlmazroui M(2023)Two-Speed Deep-Learning Ensemble for Classification of Incremental Land-Cover Satellite Image PatchesEarth Systems and Environment10.1007/s41748-023-00343-37:2(525-540)Online publication date: 24-Mar-2023
https://doi.org/10.1007/s41748-023-00343-3
Chakhar AHernández-López DZitouna-Chebbi RMahjoub IBallesteros RMoreno M(2022)Optimized Software Tools to Generate Large Spatio-Temporal Data Using the Datacubes Concept: Application to Crop Classification in Cap Bon, TunisiaRemote Sensing10.3390/rs1419501314:19(5013)Online publication date: 8-Oct-2022
https://doi.org/10.3390/rs14195013
Kuchler PSimões MFerraz RArvor Dde Almeida Machado PRosa MGaetano RBégué A(2022)Monitoring Complex Integrated Crop–Livestock Systems at Regional Scale in Brazil: A Big Earth Observation Data ApproachRemote Sensing10.3390/rs1407164814:7(1648)Online publication date: 30-Mar-2022
https://doi.org/10.3390/rs14071648
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten