Haozhou Wang
ABSTRACT
The last decade has witnessed the prevalence of sensor and GPS technologies that produce a high volume of trajectory data representing the motion history of moving objects. However some characteristics of trajectories such as variable lengths and asynchronous sampling rates make it difficult to fit into traditional database systems that are disk-based and tuple-oriented. Motivated by the success of column store and recent development of in-memory databases, we try to explore the potential opportunities of boosting the performance of trajectory data processing by designing a novel trajectory storage within main memory. In contrast to most existing trajectory indexing methods that keep consecutive samples of the same trajectory in the same disk page, we partition the database into frames in which the positions of all moving objects at the same time instant are stored together and aligned in main memory. We found this column-wise storage to be surprisingly well suited for in-memory computing since most frames can be stored in highly compressed form, which is pivotal for increasing the memory throughput and reducing CPU-cache miss. The independence between frames also makes them natural working units when parallelizing data processing on a multi-core environment. Lastly we run a variety of common trajectory queries on both real and synthetic datasets in order to demonstrate advantages and study the limitations of our proposed storage.
- A. C. Ammann, M. Hanrahan, and R. Krishnamurthy. Design of a memory resident DBMS. In COMPCON, pages 54--58, 1985.Google Scholar
- J. Baulier, P. Bohannon, S. Gogate, C. Gupta, and S. Haldar. DataBlitz storage manager: main-memory database performance for critical applications. In SIGMOD, pages 519--520, 1999. Google ScholarDigital Library
- C. Binnig, S. Hildenbrand, and F. Färber. Dictionary-based order-preserving string compression for main memory column stores. In SIGMOD, pages 283--296, 2009. Google ScholarDigital Library
- D. Bitton, M. Hanrahan, and C. Turbyfill. Performance of complex queries in main memory database systems. In ICDE, pages 72--81, 1987. Google ScholarDigital Library
- P. A. Boncz, M. Zukowski, and N. Nes. Monetdb/x100: Hyper-pipelining query execution. In CIDR, pages 225--237, 2005.Google Scholar
- V. Botea, D. Mallett, M. A. Nascimento, and J. Sander. PIST: an efficient and practical indexing technique for historical spatio-temporal point data. GeoInformatica, 12(2):143--168, 2008. Google ScholarDigital Library
- V. P. Chakka, A. C. Everspaugh, and J. M. Patel. Indexing large trajectory data sets with SETI. In CIDR, 2003.Google Scholar
- P. Cudre-Mauroux, E. Wu, and S. Madden. Trajstore: An adaptive storage system for very large trajectory data sets. In ICDE, pages 109--120, 2010.Google ScholarCross Ref
- D. Gawlick and D. Kinkade. Varieties of concurrency control in IMS/VS fast path. DEB, 8(2):3--10, 1985.Google Scholar
- A. Guttman. R-trees: a dynamic index structure for spatial searching. In SIGMOD, pages 47--57, 1984. Google ScholarDigital Library
- S. Héman, M. Zukowski, N. J. Nes, L. Sidirourgos, and P. Boncz. Positional update handling in column stores. In SIGMOD, pages 543--554, 2010. Google ScholarDigital Library
- M. G. Ivanova, M. L. Kersten, N. J. Nes, and R. A. Gonçalves. An architecture for recycling intermediates in a column-store. TODS, 35(4):24:1--24:43, 2010. Google ScholarDigital Library
- J. Krueger, C. Kim, M. Grund, N. Satish, D. Schwalb, J. Chhugani, H. Plattner, P. Dubey, and A. Zeier. Fast updates on read-optimized databases using multi-core CPUs. PVLDB, 5(1):61--72, 2011. Google ScholarDigital Library
- C. Lemke, K.-U. Sattler, F. Faerber, and A. Zeier. Speeding up queries in column stores. In DaWaK, pages 117--129, 2010. Google ScholarDigital Library
- S. Manegold, P. Boncz, and M. L. Kersten. Generic database cost models for hierarchical memory systems. In PVLDB, pages 191--202, 2002. Google ScholarDigital Library
- N. Meratnia and R. By. Spatiotemporal compression techniques for moving point objects. In EDBT, pages 765--782, 2004.Google ScholarCross Ref
- D. Pfoser, C. S. Jensen, Y. Theodoridis, et al. Novel approaches to the indexing of moving object trajectories. In Proceedings of VLDB, pages 395--406, 2000. Google ScholarDigital Library
- H. Plattner. A common database approach for OLTP and OLAP using an in-memory column database. In SIGMOD, pages 1--2, 2009. Google ScholarDigital Library
- H. Plattner. SanssouciDb: An in-memory database for processing enterprise workloads. In BTW, volume 20, pages 2--21, 2011.Google Scholar
- J. Rao and K. A. Ross. Making B+- trees cache conscious in main memory. In SIGMOD, pages 475--486, 2000. Google ScholarDigital Library
- S. Rasetic, J. Sander, J. Elding, and M. A. Nascimento. A trajectory splitting model for efficient spatio-temporal indexing. In Proceedings of VLDB, pages 934--945, 2005. Google ScholarDigital Library
- C. S. J. Simonas Saltenis, S. T. Leutenegger, and M. A. Lopez. Indexing the positions of continuously moving objects. In SIGMOD, pages 331--342, 2000. Google ScholarDigital Library
- M. Stonebraker, D. J. Abadi, A. Batkin, X. Chen, M. Cherniack, M. Ferreira, E. Lau, A. Lin, S. Madden, E. O'Neil, P. O'Neil, A. Rasin, N. Tran, and S. Zdonik. C-store: a column-oriented DBMS. In VLDB, pages 553--564, 2005. Google ScholarDigital Library
- H. Su, K. Zheng, H. Wang, J. Huang, and X. Zhou. Calibrating trajectory data for similarity-based analysis. In SIGMOD, pages 833--844, 2013. Google ScholarDigital Library
- Y. Tao, D. Papadias, and J. Sun. The TPR*-tree: an optimized spatio-temporal access method for predictive queries. In PVLDB, pages 790--801, 2003. Google ScholarDigital Library
Index Terms
- SharkDB: An In-Memory Column-Oriented Trajectory Storage
Recommendations
SharkDB: An In-Memory Storage System for Massive Trajectory Data
SIGMOD '15: Proceedings of the 2015 ACM SIGMOD International Conference on Management of DataAn increasing amount of motion history data, which is called trajectory, is being collected from different sources such as GPS-enabled mobile devices, surveillance cameras and social networks. However it is hard to store and manage trajectory data in ...
SharkDB: an in-memory column-oriented storage for trajectory analysis
The last decade has witnessed the prevalence of sensor and GPS technologies that produce a high volume of trajectory data representing the motion history of moving objects. However some characteristics of trajectories such as variable lengths and ...
Temporally enhanced network-constrained (TENC) R-tree
MobiGIS '16: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Mobile Geographic Information SystemsThis paper describes a new Network-constrained Moving objects indexing structure, which extends the state-of-the-art for this kind of data. The indexing structure we propose is called Temporally Enhanced Network-Constrained R-tree (TENC R-tree), which ...
Comments