Skip to main content
SpringerLink
Log in

A scalable cyberinfrastructure solution to support big data management and multivariate visualization of time-series sensor observation data

  • Research Article
  • Published:
Earth Science Informatics Aims and scope Submit manuscript

Abstract

This paper reports our research in developing a cyberinfrastructure platform to support multivariate visualization of data collected from distributed sensor network. Three new techniques were introduced in this platform: (1) a hybrid data caching strategy that takes advantages of a scalable and distributed time series database, OpenTSDB, to realize efficient data retrieval; (2) a hyper-dimensional data cube is established to map and translate multivariate and heterogeneous sensor data into a common data structure to support location-aware visual analysis; and (3) a data-driven visualization module is implemented to support interactive and dynamic visualization on a simulated virtual globe. A series of experiments were conducted to demonstrate the good runtime performance of the proposed system. We expect this work to make a major contribution to both the visualization building block development in cyberinfrastructure research and the advancement of visual presentation and analysis of sensor data in domain sciences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Acker, J. G., Leptoukh, G., 2007. Online analysis enhances use of NASA earth science data. Eos T. Am Geophys Union, 88(2), 14–17.

    Article  Google Scholar 

  • Baerwald TJ (2010) Prospects for geography as an interdisciplinary discipline. Ann Assoc Am Geogr 100(3):493–501

    Article  Google Scholar 

  • Bell, D. G., Kuehnel, F., Maxwell, C., Kim, R., Kasraie, K., Gaskins, T., Hogan., P., Coughlan, J., (2007) NASA World Wind: Opensource GIS for mission operations. In 2007 I.E. Aerospace Conference, IEEE, pp. 1–9

  • Borgman, C. L., Bowker, G. C., Finholt, T. A., & Wallis, J. C., (2009) Towards a virtual organization for data cyberinfrastructure. In Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries, pp. 353–356

  • Botts, M., Percivall, G., Reed, C., Davidson, J., (2008) OGC® sensor web enablement: Overview and high level architecture. In GeoSensor networks. Springer Berlin Heidelberg, pp. 175–190

  • Bryan BA (2003) Physical environmental modeling, visualization and query for supporting landscape planning decisions. Landsc. Urban Plan. 65(4):237–259

    Article  Google Scholar 

  • Chamoni P, Stock S (1999) Temporal structures in data warehousing. In International Conference on Data Warehousing and Knowledge Discovery, Springer Berlin Heidelberg, pp 353–358

  • Chan WWY (2006) A survey on multivariate data visualization. Department of Computer Science and Engineering. Hong Kong Univ. Sci. Technol. 8(6):1–29

    Google Scholar 

  • Chaturvedi K (2014) Web based 3D analysis and visualization using HTML5 and WebGL. http://www.iirs.gov.in/iirs/sites/default/files/StudentThesis/MSc_Thesis_KanishkChaturvedi.pdf. Accessed 8 July 2016

  • Chen A, Leptoukh G, Kempler S, Lynnes C, Savtchenko A, Nadeau D, Farley J (2009) Visualization of A-train vertical profiles using Google earth. Comput Geosci 35(2):419–427

    Article  Google Scholar 

  • Chourasia A, Cutchin S, Aagaard B (2008) Visualizing the ground motions of the 1906 San Francisco earthquake. Comput Geosci 34(12):1798–1805

    Article  Google Scholar 

  • Congote, J., Segura, A., Kabongo, L., Moreno, A., Posada, J., Ruiz, O., 2011. Interactive visualization of volumetric data with webgl in real-time. In Proceedings of the 16th International Conference on 3D Web Technology, ACM, pp. 137–146

  • Craglia M, de Bie K, Jackson D, Pesaresi M, Remetey-Fülöpp G, Wang C, Woodgate P (2012) Digital Earth 2020: towards the vision for the next decade. Int. J. Digital Earth 5(1):4–21

    Article  Google Scholar 

  • De Oliveira MCF, Levkowitz H (2003) From visual data exploration to visual data mining: A survey. IEEE T. Vis. Comput. Gr. 9(3):378–394

    Article  Google Scholar 

  • Demir I, Krajewski WF (2013) Towards an integrated Flood Information System: Centralized data access, analysis, and visualization. Environ. Modell. Softw. 50:77–84

    Article  Google Scholar 

  • Deri, L., Mainardi, S., & Fusco, F. (2012) tsdb: A compressed database for time series (pp. 143–156). Springer Berlin Heidelberg

  • DiBiase D, Reeves C, MacEachren AM, Von Wyss M, Krygier JB, Sloan JL, Detweiler MC (1994) Multivariate display of geographic data: Applications in earth system science. Visual. Mod. Cartogr.:287–312

  • Engel-Cox JA, Holloman CH, Coutant BW, Hoff RM (2004) Qualitative and quantitative evaluation of MODIS satellite sensor data for regional and urban scale air quality. Atmos Environ 38(16):2495–2509

    Article  Google Scholar 

  • Goodchild MF, Guo H, Annoni A, Bian L, de Bie K, Campbell F, Woodgate P (2012) Next-generation digital earth. Proc Natl Acad Sci 109(28):11088–11094

    Article  Google Scholar 

  • Gore, A. (1998) The digital earth: understanding our planet in the twenty-first century. Australian surveyor 43(2), 89–91

  • Grundy E, Jones MW, Laramee RS, Wilson RP, Shepard EL (2009) Visualisation of sensor data from animal movement. Comput. Graph. Forum. 28(3):815–822

    Article  Google Scholar 

  • Guo D (2009) Flow mapping and multivariate visualization of large spatial interaction data. IEEE T. Vis. Comput. Gr. 15(6):1041–1048

    Article  Google Scholar 

  • Guo D, Gahegan M, MacEachren AM, Zhou B (2005) Multivariate analysis and geovisualization with an integrated geographic knowledge discovery approach. Cartogr. Geogr. Inform 32(2):113–132

    Article  Google Scholar 

  • Guo D, Chen J, MacEachren AM, Liao K (2006) A visualization system for space-time and multivariate patterns (vis-stamp). IEEE T. Vis. Comput. Gr. 12(6):1461–1474

    Article  Google Scholar 

  • ISO 19156:2011 Geographic information – Observations and measurements. (2011) URL: http://www.iso.org/iso/catalogue_detail.htm?csnumber=32574. Retrieved 2011–12–20

  • Jain A (2014) Data visualization with the D3.JS JavaScript library. J. Comput. Sci. Colleges 30(2):139–141

    Google Scholar 

  • Jern, M., Franzen, J., (2006) “GeoAnalytics”-Exploring spatio-temporal and multivariate data. IEEE Tenth International Conference on Information Visualization, pp. 25–31

  • Kehrer J, Hauser H (2013) Visualization and visual analysis of multifaceted scientific data: A survey. IEEE T. Vis. Comput. Gr. 19(3):495–513

    Article  Google Scholar 

  • Kemp Z, Meaden G (2002) Visualization for fisheries management from a spatiotemporal perspective. ICES J Mar Sci 59(1):190–202

    Article  Google Scholar 

  • Kitsiou D, de Madron XD, Arnau PA (2001) Development of a data visualization and analysis tool to study the particle dynamics in the coastal zone. Mar Pollut Bull 43(7):262–269

    Article  Google Scholar 

  • Kohonen T (1990) The self-organizing map. P. IEEE 78(9):1464–1480

    Article  Google Scholar 

  • Kubicek P, Kozel J, Stampach R, Lukas V (2013) Prototyping the visualization of geographic and sensor data for agriculture. Comput. Electron. Agr. 97:83–91

    Article  Google Scholar 

  • Li, W., Yang, C., Raskin, R., (2008a) A Semantic Enhanced Search for Spatial Web Portals. In AAAI Spring Symposium: Semantic Scientific Knowledge Integration, pp. 47–5

  • Li, W., Yang, P., Zhou, B., (2008b) Internet-based spatial information retrieval. In Encyclopedia of GIS, Springer US, pp. 596–599

  • Li W, Yang C, Nebert D, Raskin R, Houser P, Wu H, Li Z (2011) Semantic-based web service discovery and chaining for building an Arctic spatial data infrastructure. Comput Geosci 37(11):1752–1762

  • Li W, Goodchild MF, Raskin R (2012) Towards geospatial semantic search: exploiting latent semantic relations in geospatial data. Int. J. Digital Earth 7(1):17–37

    Article  Google Scholar 

  • Li, X., Li, W., Anselin, L., Rey, S., Koschinsky, J., (2014) A MapReduce algorithm to create contiguity weights for spatial analysis of big data. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, ACM, pp. 50–53

  • Li W, Zhou B, Song M, Cao K, Gao S (2015a) Performance improvement techniques for geospatial web services in a cyberinfrastructure environment. – A case study with a disaster management portal. Comput. Environ. Urban. doi:10.1016/j.compenvurbsys.2015.04.003

    Google Scholar 

  • Li, W, Goodchild, M.F., Anselin, L., Weber, K., (2015b) A Service-Oriented Smart CyberGIS Framework for Data-Intensive Geospatial Problems, In S. Wang and M. Goodchild (Eds.), CyberGIS: Fostering a New Wave of Geospatial Discovery and Innovation Berlin, Springer-Verlag (in press)

  • MacEachren, A. M., Taylor, D.R.F., (1994) Visualization in modern cartography, 2. Oxford, Pergamon Press

  • MacEachren AM, Wachowicz M, Edsall R, Haug D, Masters R (1999) Constructing knowledge from multivariate spatiotemporal data: integrating geographical visualization with knowledge discovery in database methods. Int J Geogr Inf Sci 13(4):311–334

    Article  Google Scholar 

  • MacEachren AM, Gahegan M, Pike W, Brewer I, Cai G, Lengerich E, Hardistry F (2004) Geovisualization for knowledge construction and decision support. IEEE Comput. Graph. 24(1):13–17

    Article  Google Scholar 

  • McCormick, E., De Volder, K., (2004) JQuery: finding your way through tangled code. In Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, ACM, pp. 9–10

  • NSF (2006) NSF’s cyberinfrastructure vision for twenty-first century discovery. http://www.nsf.gov.ezproxy1.lib.asu.edu/pubs/2007/nsf0728/index.jsp. Accessed 8 Jul 2016

  • Oreizy, P., Medvidovic, N., Taylor, R.N., (1998) Architecture-based runtime software evolution. In Proceedings of the 20th international conference on Software engineering, IEEE Computer Society, pp. 177–186

  • Sheppard SR, Cizek P (2009) The ethics of Google earth: crossing thresholds from spatial data to landscape visualisation. J Environ Manag 90(6):2102–2117

    Article  Google Scholar 

  • Siirtola H, Mäkinen E (2005) Constructing and reconstructing the reorderable matrix. Inf. Visual. 4(1):32–48

    Article  Google Scholar 

  • Sips M, Schneidewind J, Keim DA (2007) Highlighting space–time patterns: effective visual encodings for interactive decision-making. Int J Geogr Inf Sci 21(8):879–893

    Article  Google Scholar 

  • Spirkovska L, Lodha SK (2002) AWE: aviation weather data visualization environment. Comput Graph 26(1):169–191

    Article  Google Scholar 

  • Stolte C, Tang D, Hanrahan P (2002) Polaris: a system for query, Analysis, and visualization of multidimensional relational databases. IEEE Trans. Vis. Comput. Graph. 8(1):52–65

    Article  Google Scholar 

  • Stolte C, Tang D, Hanrahan P (2003) Multiscale visualization using data cubes. IEEE Trans. Vis. Comput. Graph 9(2):176–187

    Article  Google Scholar 

  • Thomas SM (2014) PostgreSQL 9 High Availability Cookbook. Packt Publishing Ltd., UK

    Google Scholar 

  • van Wijk, J. J., van Liere, R., (1993) HyperSlice: visualization of scalar functions of many variables. In Proceedings of the 4th conference on Visualization'93, IEEE Computer Society, pp. 119–125

  • Wlodarczyk, T. W. (2012) Overview of time series storage and processing in a cloud environment. In Cloud Computing Technology and Science (CloudCom), 2012 I.E. 4th International Conference on (pp. 625–628). IEEE

  • Wong, P.C., Bergeron, R.D., (1994) 30 Years of Multidimensional Multivariate Visualization. In Proceeding Scientific Visualization, Overviews, Methodologies, and Techniques IEEE Computer Society, pp. 3–33

  • Yang C, Li W, Xie J, Zhou B (2008) Distributed geospatial information processing: sharing distributed geospatial resources to support Digital Earth. Int J. Digit Earth 1(3):259–278

    Article  Google Scholar 

  • Yang C, Wu H, Huang Q, Li Z, Li J, Li W, Miao L, Sun M (2011) WebGIS performance issues and solutions. In: Li S, Suzana D, Bert V (eds) Advances in Web-based GIS, Mapping Services and Applications, Taylor and Francis: New York, NY, USA, pp. 121–138

  • Zhang X, Pazner M (2004) The icon image map technique for multivariate geospatial data visualization: approach and software system. Cartogr Geogr Inf Sci 31(1):29–41

    Article  Google Scholar 

Download references

Acknowledgments

This paper is in part supported by National Science Foundation Award #1349259, #1504432, and #1455349, as well as the Open Geospatial Consortium.

Author information

Authors and Affiliations

  1. School of Geographical Sciences and Urban Planning, Arizona State University, Tempe, AZ, USA

    Wenwen Li, Sheng Wu, Miaomiao Song & Xiran Zhou

Authors
  1. Wenwen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miaomiao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiran Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenwen Li.

Additional information

Communicated by: H. A. Babaie

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, W., Wu, S., Song, M. et al. A scalable cyberinfrastructure solution to support big data management and multivariate visualization of time-series sensor observation data. Earth Sci Inform 9, 449–464 (2016). https://doi.org/10.1007/s12145-016-0267-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12145-016-0267-1

Keywords

Search

Navigation