Skip to main content
SpringerLink
Log in

Distributing Data in Real Time Spatial Data Warehouse

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12453))

  • 1850 Accesses

Abstract

Nowadays, there are many real-time spatial applications like location-aware services and traffic monitoring and the need for real time spatial data processing becomes more and more important. As a result, there is a tremendous amount of real-time spatial data in real-time spatial data warehouse. The continuous growth in the amount of data seems to outspeed the advance of the traditional centralized real-time spatial data warehouse. As a solution, many organizations use distributed real-time spatial data warehouse (DRTSDW) as a powerful technique to achieve OLAP (On Line Analytical Processing) analysis and business intelligence (BI). Distributing data in real time data warehouse is divided into two steps: partitioning data and their allocation into sites. Several works have proposed many algorithms for partitioning and allocation data. But with a huge amount of real-time spatial data generated, the system performance degrades rapidly, especially in overload situations. In order to deal with this volumetry and to increase query efficiency, we propose a novel approach for partitioning data in real-time spatial data warehouse to find the right number of clusters and to divides the RTSDW into partitions using the horizontal partitioning. Secondly, we suggest our allocation strategy to place the partitions on the sites where they are most used, to minimize data transfers between sites. We have evaluated those proposed approaches using the new TPC-DS (Transaction processing performance council, http://www.tpc.org, 2014) benchmark. The preliminary results show that the approach is quite interesting.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bai, L., Lin, Z., Xu, C.: Spatiotemporal operations on spatiotemporal XML data using XQuery. In: 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 1278–1282. IEEE (August 2016)

    Google Scholar 

  2. Bernardino, J., Madeira, H.: Experimental evaluation of a new distributed partitioning technique for data warehouses. In: International Database Engineering and Applications Symposium, pp. 312–321 (2001)

    Google Scholar 

  3. Gorawski, M., Malczok, R.: Distributed spatial data warehouse indexed with virtual memory aggregation tree. In: STDBM, pp. 25–32 (2004)

    Google Scholar 

  4. Hadjieleftheriou, M., Kollios, G., Gunopulos, D., Tsotras, V.J.: On-line discovery of dense areas in spatio-temporal databases. In: Hadzilacos, T., Manolopoulos, Y., Roddick, J., Theodoridis, Y. (eds.) SSTD 2003. LNCS, vol. 2750, pp. 306–324. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45072-6_18

    Chapter  Google Scholar 

  5. Hamdi, S., Bouazizi, E., Faiz, S.: A speculative concurrency control in real-time spatial big data using real-time nested spatial transactions and imprecise computation. In: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), pp. 534–540. IEEE (October 2017)

    Google Scholar 

  6. Huang, C.Y., Liang, S.H.: LOST-Tree: a spatio-temporal structure for efficient sensor data loading in a sensor web browser. Int. J. Geogr. Inf. Sci. 27(6), 1190–1209 (2013)

    Article  Google Scholar 

  7. Inmon, W.H.: Building the Data Warehouse. Wiley, Hoboken (2005)

    Google Scholar 

  8. Iwerks, G.S., Samet, H., Smith, K.: Continuous k-nearest neighbor queries for continuously moving points with updates. In: Proceedings of the 29th International Conference on Very Large Data Bases, vol. 29, pp. 512–523. VLDB Endowment (September 2003)

    Google Scholar 

  9. Lee, M.L., Hsu, W., Jensen, C.S., Cui, B., Teo, K.L.: Supporting frequent updates in R-trees: a bottom-up approach. In: Proceedings of the 29th International Conference on Very Large Data Bases, vol. 29, pp. 608–619. VLDB Endowment (September 2003)

    Google Scholar 

  10. Mokbel, M.F., Xiong, X., Aref, W.G.: SINA: scalable incremental processing of continuous queries in spatio-temporal databases. In: Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, pp. 623–634. ACM (June 2004)

    Google Scholar 

  11. Mokbel, M.F., Xiong, X., Aref, W.G., Hambrusch, S.E., Prabhakar, S., Hammad, M.A.: PALACE: a query processor for handling real-time spatio-temporal data streams. In: Proceedings of the 13th International Conference on Very Large Data Bases, vol. 30, pp. 1377–1380. VLDB Endowment (August 2004)

    Google Scholar 

  12. Noaman, A.Y., Barker, K.: A horizontal fragmentation algorithm for fact relation in a distributed data warehouse. In: Proceedings of the 8th International Conference on Information and Knowledge Management, CIKM 1999, pp. 154–161 (November 1999)

    Google Scholar 

  13. Noaman, A.Y., Barker, K.: Distributed data warehouse architectures. J. Data Warehouse. 2(2), 37–50 (1997)

    Google Scholar 

  14. Phansalkar, S., Ahirrao, S.: Survey of data partitioning algorithms for big data stores. In: 2016 4th International Conference on Parallel, Distributed and Grid Computing (PDGC), pp. 163–168. IEEE (December 2016)

    Google Scholar 

  15. Saltenis, S., Jensen, C.S., Leutenegger, S.T., Lopez, M.A.: Indexing the positions of continuously moving objects. ACM SIGMOD Rec. 29(2), 331–342 (2000)

    Article  Google Scholar 

  16. Tao, Y., Papadias, D., Shen, Q.: Continuous nearest neighbor search. In: Proceedings of the 28th International Conference on Very Large Data Bases, pp. 287–298. VLDB Endowment (August 2002)

    Google Scholar 

  17. White, C.: A technical architecture for data warehousing. InfoDB J. 9(1), 5–11 (1995)

    Google Scholar 

  18. Zhou, S., Zhou, A., Tao, X., Hu, Y.: Hierarchically distributed data warehouse. In: Proceedings of the 4th International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region, Beijing, China, pp. 848–53 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Manouba University, Campus Universitaire de la Manouba, 2010, Manouba, Tunisia

    Wael Hamdi

  2. LTSIRS Laboratory, BP 37 Le Belvedere, 1002, Tunis, Tunisia

    Sami Faiz

Authors
  1. Wael Hamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sami Faiz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wael Hamdi .

Editor information

Editors and Affiliations

  1. Columbia University, New York, NY, USA

    Meikang Qiu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hamdi, W., Faiz, S. (2020). Distributing Data in Real Time Spatial Data Warehouse. In: Qiu, M. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2020. Lecture Notes in Computer Science(), vol 12453. Springer, Cham. https://doi.org/10.1007/978-3-030-60239-0_1

Download citation

Publish with us

Policies and ethics

Search

Navigation