Skip to main content
SpringerLink
Log in

Processing of extreme moving-object update and query workloads in main memory

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

The efficient processing of workloads that interleave moving-object updates and queries is challenging. In addition to the conflicting needs for update-efficient versus query-efficient data structures, the increasing parallel capabilities of multi-core processors yield challenges. To prevent concurrency anomalies and to ensure correct system behavior, conflicting update and query operations must be serialized. In this setting, it is a key concern to avoid that operations are blocked, which leaves processing cores idle. To enable efficient processing, we first examine concurrency degrees from traditional transaction processing in the context of our target domain and propose new semantics that enable a high degree of parallelism and ensure up-to-date query results. We define the new semantics for range and \(k\)-nearest neighbor queries. Then, we present a main-memory indexing technique called parallel grid that implements the proposed semantics as well as two other variants supporting different semantics. This enables us to quantify the effects that different degrees of consistency have on performance. We also present an alternative time-partitioning approach. Empirical studies with the above and three existing proposals conducted on modern processors show that our proposals scale near-linearly with the number of hardware threads and thus are able to benefit from increasing on-chip parallelism.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. The Minkowski sum of the query range and the circle with radius \(\delta \) must be performed.

  2. We use unordered_map from the Standard C++ library.

  3. The cell locking in the scanning algorithm is related to safe memory reclamation that can be implemented completely lock-free using advanced techniques such as atomic double compare-and-swap (DCAS) operations (that are, however, not supported by commodity hardware) or by multi-threaded memory allocators (which are implemented as complex libraries). Since we did not observe any contention or performance penalty due to this brief cell locking, we do not consider such techniques.

  4. Note that we achieved this atomicity only with 128-bit values. That is, our experiments break once we use 256-bit values and ymm registers.

  5. Available at http://moto.sourceforge.net.

  6. For instance, at the expense of accuracy, object data can be packed into 64-bit values. Then, object reads and writes are atomic on a 64-bit architecture.

References

  1. Akman, V., Franklin, W.R., Kankanhalli, M., Narayanaswami, C.: Geometric computing and the uniform grid data technique. CAD 21(7), 410–420 (1989)

    Google Scholar 

  2. Anderson, E., Tucek, J.: Efficiency matters!. SIGOPS Oper. Syst. Rev. 44(1), 40–45 (2010)

    Article  Google Scholar 

  3. Berenson, H., Bernstein, P., Gray, J., Melton, J., O’Neil, E., O’Neil, P.: A critique of ANSI SQL isolation levels. In: SIGMOD, pp. 1–10 (1995)

  4. Bernstein, P.A., Goodman, N.: Multiversion concurrency control—theory and algorithms. ACM TODS 8(4), 465–483 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bernstein, P.A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in Database Systems. Addison-Wesley, Boston (1987)

    Google Scholar 

  6. Brinkhoff, T.: A framework for generating network-based moving objects. GeoInformatica 6(2), 153–180 (2002)

    Article  MATH  Google Scholar 

  7. Cha, S.K., Hwang, S., Kim, K., Kwon, K.: Cache-conscious concurrency control of main-memory indexes on shared-memory multiprocessor systems. In: VLDB, pp. 181–190 (2001)

  8. Chakrabarti, K., Mehrotra, S.: Efficient concurrency control in multidimensional access methods. In: SIGMOD, pp. 25–36 (1999)

  9. Chen, S., Jensen, C.S., Lin, D.: A benchmark for evaluating moving object indexes. PVLDB 1(2), 1574–1585 (2008)

    Google Scholar 

  10. Civilis, A., Jensen, C.S., Pakalnis, S.: Techniques for efficient road-network-based tracking of moving objects. IEEE TKDE 17(5), 698–712 (2005)

    Google Scholar 

  11. Dittrich, J., Blunschi, L., Salles, M.A.V.: Indexing moving objects using short-lived throwaway indexes. In: SSTD, pp. 189–207 (2009)

  12. Dittrich, J., Blunschi, L., Vas Salles, M.: Movies: indexing moving objects by shooting index images. GeoInformatica 15(4), 727–767 (2011)

    Article  Google Scholar 

  13. Fekete, A., Liarokapis, D., O’Neil, E., O’Neil, P., Shasha, D.: Making snapshot isolation serializable. ACM TODS 30(2), 492–528 (2005)

    Article  Google Scholar 

  14. Fekete, A., O’Neil, E., O’Neil, P.: A read-only transaction anomaly under snapshot isolation. SIGMOD Rec. 33(3), 12–14 (2004)

    Article  Google Scholar 

  15. Gray, J., Reuter, A.: Transaction processing: concepts and techniques. Morgan Kaufmann Publishers, Burlington, MA (1993)

    MATH  Google Scholar 

  16. Gray, J.N., Lorie, R.A., Putzolu, G.R.: Granularity of locks and degrees of consistency in a shared data base. In: VLDB, pp. 428–451 (1975)

  17. Hwang, S., Kwon, K., Cha, S., Lee, B.: Performance evaluation of main-memory R-tree variants. In: SSTD, pp. 10–27 (2003)

  18. Intel 64 and IA-32 Architectures Software Developers Manual, Volume 3A: System Programming Guide, Part 1 (2011)

  19. Jensen, C.S., Lin, D., Ooi, B.C.: Query and update efficient B+-tree based indexing of moving objects. In: VLDB, pp. 768–779 (2004)

  20. Jiang, J., Bao, H., Chang, E.Y., Li, Y.: MOIST: a scalable and parallel moving object indexer with school tracking. PVLDB 5(12), 1838–1849 (2012)

    Google Scholar 

  21. Kalashnikov, D., Prabhakar, S., Hambrusch, S.: Main memory evaluation of monitoring queries over moving objects. Distrib. Parallel Databases 15(2), 117–135 (2004)

    Article  Google Scholar 

  22. Kim, K., Cha, S.K., Kwon, K.: Optimizing multidimensional index trees for main memory access. In: SIGMOD, pp. 139–150 (2001)

  23. Kornacker, M., Banks, D.: High-concurrency locking in R-trees. In: VLDB, pp. 134–145 (1995)

  24. Kornacker, M., Mohan, C., Hellerstein, J.M.: Concurrency and recovery in generalized search trees. In: SIGMOD, pp. 62–72 (1997)

  25. Kraftfahrt-Bundesamt (Federal Motor Transport Authority): Annual Report. Kraftfahrt-Bundesamt, Germany (2009)

    Google Scholar 

  26. Lee, M.L., Hsu, W., Jensen, C.S., Cui, B., Teo, K.L.: Supporting frequent updates in R-trees: a bottom-up approach. In: VLDB, pp. 608–619 (2003)

  27. Lehman, P.L., Yao, S.B.: Efficient locking for concurrent operations on B-trees. ACM TODS 6, 650–670 (1981)

    Article  MATH  Google Scholar 

  28. Molka, D., Hackenberg, D., Schone, R., Muller, M.S.: Memory performance and cache coherency effects on an intel nehalem multiprocessor system. In: PACT, pp. 261–270 (2009)

  29. Mouratidis, K., Papadias, D., Hadjieleftheriou, M.: Conceptual partitioning: an efficient method for continuous nearest neighbor monitoring. In: SIGMOD, pp. 634–645 (2005)

  30. Ng, V., Kameda, T.: Concurrent access to R-trees. In: SSD, pp. 142–161 (1993)

  31. Nguyen-Dinh, L.-V., Aref, W.G., Mokbel, M.F.: Spatio-temporal access methods: part 2 (2003–2010). IEEE Data Eng. Bull. 33(2), 46–55 (2010)

    Google Scholar 

  32. Nishimura, S., Das, S., Agrawal, D., Abbadi, A.E.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware services. In: MDM, pp. 7–16 (2011)

  33. Patel, J.M., Chen, Y., Chakka, V.P.: Stripes: an efficient index for predicted trajectories. In: SIGMOD, pp. 635–646 (2004)

  34. POSIX.1-2008. The open group base specifications (2008)

  35. Rastogi, R., Seshadri, S., Bohannon, P., Leinbaugh, D.W., Silberschatz, A., Sudarshan, S.: Logical and physical versioning in main memory databases. In: VLDB, pp. 86–95 (1997)

  36. Salzberg, B.: Grid file concurrency. Inf. Syst. 11(3), 235–244 (1986)

    Article  MathSciNet  Google Scholar 

  37. Šidlauskas, D., Jensen, C. S., Šaltenis, S.: A comparison of the use of virtual versus physical snapshots for supporting update-intensive workloads. In: DaMoN, pp. 1–8 (2012)

  38. Šidlauskas, D., Ross, K.A., Jensen, C.S., Šaltenis, S.: Thread-level parallel indexing of update intensive moving-object workloads. In: SSTD, pp. 186–204 (2011)

  39. Šidlauskas, D., Šaltenis, S., Christiansen, C.W., Johansen, J.M., Šaulys, D.: Trees or grids?: indexing moving objects in main memory. In: GIS, pp. 236–245 (2009)

  40. Šidlauskas, D., Šaltenis, S., Jensen, C.S.: Parallel main-memory indexing for moving-object query and update workloads. In: SIGMOD, pp. 37–48 (2012)

  41. Song, S.I., Kim, Y.H., Yoo, J.S.: An enhanced concurrency control scheme for multidimensional index structures. IEEE TKDE 16(1), 97–111 (2004)

    MathSciNet  Google Scholar 

  42. Stonebraker, M., Madden, S., Abadi, D.J., Harizopoulos, S., Hachem, N., Helland, P.: The end of an architectural era: (it’s time for a complete rewrite). In: VLDB, pp. 1150–1160 (2007)

  43. Tao, Y., Papadias, D., Sun, J.: The TPR*-tree: an optimized spatio-temporal access method for predictive queries. In: VLDB, pp. 790–801 (2003)

  44. Tzoumas, K., Yiu, M.L., Jensen, C.S.: Workload-aware indexing of continuously moving objects. PVLDB 1(2), 1186–1197 (2009)

    Google Scholar 

  45. Wolfson, O., Yin, H.: Accuracy and resource consumption in tracking and location prediction. In: SSTD, pp. 325–343 (2003)

  46. Wu, W., Tan, K.-L.: iSEE: Efficient continuous k-nearest-neighbor monitoring over moving objects. In SSDBM, p. 36 (2007)

  47. Xiong, X., Mokbel, M., Aref, W.: SEA-CNN: Scalable processing of continuous k-nearest neighbor queries in spatio-temporal databases. In: ICDE, pp. 643–654 (2005)

  48. Yiu, M., Tao, Y., Mamoulis, N.: The B\(^{dual}\)-tree: indexing moving objects by space filling curves in the dual space. PVLDB 17(3), 379–400 (2008)

    Google Scholar 

Download references

Acknowledgments

We thank the reviewers for their helpful comments. The research was supported in part by the Danish National Research Foundation grant DNRF84 through Center for Massive Data Algorithmics (MADALGO) and by a grant from the Obel Family Foundation.

Author information

Authors and Affiliations

  1. Department of Computer Science, Aarhus University, Aarhus, Denmark

    Darius Šidlauskas

  2. Department of Computer Science, Aalborg University, Aalborg, Denmark

    Simonas Šaltenis & Christian S. Jensen

Authors
  1. Darius Šidlauskas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simonas Šaltenis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian S. Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darius Šidlauskas.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 102 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Šidlauskas, D., Šaltenis, S. & Jensen, C.S. Processing of extreme moving-object update and query workloads in main memory. The VLDB Journal 23, 817–841 (2014). https://doi.org/10.1007/s00778-014-0353-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-014-0353-2

Keywords

Search

Navigation