Skip to main content

Advertisement

SpringerLink
Log in

A scalable Cloud-based system for data-intensive spatial analysis

  • SW
  • Series B
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Advances in Cloud computing technology and the availability of affordable and easy to use Cloud services are enabling a multitude of scientific applications to use these resources as primary or secondary computing infrastructure. The urban and built environment research domain is one area that can benefit greatly from Cloud computing. The global population growth and increase in the size and population of cities raise many challenges for governments, planners and researchers alike. The Australian Urban Research Infrastructure Network (AURIN—http://www.aurin.org.au) project has been tasked with developing an advanced platform (e-Infrastructure) across Australia to tackle these challenges. The platform leverages large-scale Cloud resources to provide federated data access to, at present over 1100 data sets from major and often definitive government and industry data-rich organisations, and for scalable data processing and visualisation. The original AURIN tools were developed using the object modelling system (OMS) and supported integrated workflows to define and enact/re-enact scientific processes. More recently the work has evolved to focus more on delivery of a workbench offering a rich range of tools delivered through an extensible workflow environment. In this paper, we provide the background to AURIN including the scientific drivers that are shaping the work and the realisation of the Cloud-based AURIN environment. We focus in particular on the workflow environment and show how it seamlessly utilizes the Cloud for urban research processes focused especially on data-intensive spatial analysis. We illustrate the utilisation of this workflow environment across a range of case studies reflecting urban research activities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. A location quotient is a way of quantifying how concentrated a particular industry, cluster, occupation, or demographic group is in a region as compared to the nation. It can reveal what makes a particular region “unique” in comparison to the national average.

References

  1. Hey, A.J., Trefethen, A.E.: The data Deluge: An e-science perspective (2003)

  2. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I.: A view of cloud computing. Comm. ACM 53, 50–58 (2010)

    Article  Google Scholar 

  3. Hirschheim, R., Welke, R., Schwarz, A.: Service-oriented architecture: Myths, realities, and a maturity model. MIS Quart. Exec. 9(1), 37–48 (2010)

    Google Scholar 

  4. Hey, A.J., Trefethen, A.E.: Cyberinfrastructure for e-Science. Science 308, 817–821 (2005)

    Article  Google Scholar 

  5. Papajorgji, P., Beck, H.W., Braga, J.L.: An architecture for developing service-oriented and component-based environmental models. Ecol. Model. 179, 61–76 (2004)

    Article  Google Scholar 

  6. Shiers, J.: The worldwide LHC computing grid (worldwide LCG). Comp. Physic. Comm. 177(1), 219–223 (2007)

    Article  Google Scholar 

  7. Marx, V.: Biology: the big challenges of big data. Nature 498(7453), 255–260 (2013)

    Article  Google Scholar 

  8. Hoffa, C., Mehta, G., Freeman, T., Deelman, E., Keahey, K., Berriman, B., Good, J.: On the use of cloud computing for scientific workflows. In: eScience, 2008. eScience’08. IEEE Fourth International Conference on (pp. 640–645) [IEEE (2008, December)]

  9. Sinnott, R.O., Galang, G., Tomko, M., Stimson, R.: Towards an e-Infrastructure for Urban Research Across Australia, IEEE e-Science Conference, Stockholm, Sweden (December 2011)

  10. Stimson, R., Tomko, M., Sinnott, R.O.: The Australian Urban Research Infrastructure Network (AURIN) Initiative: A Platform Offering Data and Tools for Urban and Built Environment Researchers across Australia, State of Australian Cities, Melbourne, Australia (November 2011)

  11. Pettit, C., Stimson, R., Barton, J., Goldie, X., Sinnott, R.O., Kvan, T.: The Australian Urban Intelligence Network supporting Smart Cities, to appear in CUPUM 2015 Conference book on Smart Cities and Planning Support Systems, eds: Geertman, S., Stillwell, J., Ferreira, J., Goodspeed, R. (February 2015)

  12. Sinnott, R.O., Bayliss, C., Galang, G., Greenwood, P., Koetsier, G., Mannix, D., Morandini, L., Nino-Ruiz, M., Pettit, C., Tomko, M., Sarwar, M., Stimson, R., Voorsluys, W., Widjaja, I.: A Data-driven Urban Research Environment for Australia, IEEE e-Science Conference, Chicago USA, (October 2012)

  13. Goecks, J., Nekrutenko, A., Taylor, J.: Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biol. 11(8), R86 (2010)

    Article  Google Scholar 

  14. Javadi, B., Tomko, M., Sinnott, R.O.: Decentralised Orchestration of Data-centric Workflows in Cloud Environments. Future Gene. Comp. Syst. (2013). doi:10.1016/j.future.2013.01.008

  15. Deelman, E., Singh, G., Su, M., Blythe, J., Gil, Y., Kesselman, C., Katz, D.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)

    Google Scholar 

  16. Wolstencroft, K., Haines, R., Fellows, D., Williams, A., Withers, D., Owen, S., Goble, C.: The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucleic acids research, gkt328 (2013)

  17. Barker, A., Van Hemert, J.: Scientific workflow: a survey and research directions. In Parallel Processing and Applied Mathematics (pp. 746-753). Springer Berlin Heidelberg (2008)

  18. Sinnott, R.O., Chhetri, P., Gong, Y., Macaulay, A., Voorsluys, W.: Privacy-preserving Data Linkage through Blind Geo-spatial Data Aggregation, The IEEE International Symposium on Big Data Security on Cloud (BigDataSecurity 2015), New York, USA, (August 2015)

  19. Sinnott, R.O., Bayliss, C., Bromage, A., Galang, G., Grazioli, G., Greenwood, P., Macauley, G., Morandini, L., Nino-Ruiz, M., Nogoorani, G., Pettit, C., Tomko, M., Sarwar, M., Stimson, R., Voorsluys, W., Widjaja, I.: The Australian Urban Research Gateway. Journal of Concurrency and Computation: Practice and Experience (April 2014). doi:10.1002/cpe.3282

  20. Widjaja, I., Russo, P., Pettit, C., Sinnott, R.O., Tomko, M.: Modeling Coordinated Multiple Views of Heterogeneous Data Cubes for Urban Visual Analytics, International Journal of Digital Earth (October 2013)

Download references

Acknowledgments

The AURIN project is funded through the Australian Education Investment Fund SuperScience initiative. The project began in July 2010 and is due to run to mid-2015. We gratefully acknowledge their support and the wider research community on their feedback. We are grateful to the rest of the AURIN team including those within the Melbourne eResearch Group (Dr. Glenn Jayaputera, Philip Greenwood, Luca Morandini, Dr. Ivo Widjaja, Sulman Sarwar, Dr. Marcos Nino-Ruiz, Dr. Hossein Pursultani, Christopher Bayliss, Ghazal Nogoorani, Gerson Galang, Andrew Bromage, Daghan Acay, Davis Marques, Rosana Rabanal), those within the AURIN Office (Jack Barton, Xavier Goldie, Chis Pettit, Bob Stimson, Stewart Wallace) and the wider network of AURIN collaborators. Map tiles in figures by Stamen Design (stamen.com) under CC BY 3.0 (http://creativecommons.org/licenses/by/3.0).

Author information

Authors and Affiliations

  1. Department of Computing and Information Systems, University of Melbourne, Melbourne, Australia

    R. O. Sinnott & W. Voorsluys

Authors
  1. R. O. Sinnott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. W. Voorsluys
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. O. Sinnott.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sinnott, R.O., Voorsluys, W. A scalable Cloud-based system for data-intensive spatial analysis. Int J Softw Tools Technol Transfer 18, 587–605 (2016). https://doi.org/10.1007/s10009-015-0398-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-015-0398-6

Keywords

Search

Navigation