Skip to main content

Advertisement

SpringerLink
Log in

Sub-trajectory clustering with deep reinforcement learning

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Sub-trajectory clustering is a fundamental problem in many trajectory applications. Existing approaches usually divide the clustering procedure into two phases: segmenting trajectories into sub-trajectories and then clustering these sub-trajectories. However, researchers need to develop complex human-crafted segmentation rules for specific applications, making the clustering results sensitive to the segmentation rules and lacking in generality. To solve this problem, we propose a novel algorithm using the clustering results to guide the segmentation, which is based on reinforcement learning (RL). The novelty is that the segmentation and clustering components cooperate closely and improve each other continuously to yield better clustering results. To devise our RL-based algorithm, we model the procedure of trajectory segmentation as a Markov decision process (MDP). We apply Deep-Q-Network (DQN) learning to train an RL model for the segmentation and achieve excellent clustering results. Experimental results on real datasets demonstrate the superior performance of the proposed RL-based approach over state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Algorithm 2
Fig. 7
Algorithm 3
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Agarwal, P.K., Fox, K., Munagala, K., Nath, A., Pan, J., Taylor, E.: Subtrajectory clustering: Models and algorithms. In: SIGMOD, pp. 75–87 (2018)

  2. Alt, H., Godau, M.: Computing the fréchet distance between two polygonal curves. Int. J. Comput. Geom. Appl 5(01–02), 75–91 (1995)

    Article  Google Scholar 

  3. Anagnostopoulos, A., Vlachos, M., Hadjieleftheriou, M., Keogh, E., Yu, P.S.: Global distance-based segmentation of trajectories. In: SIGKDD, pp. 34–43 (2006)

  4. Ankerst, M., Breunig, M.M., Kriegel, H.P., Sander, J.: Optics: Ordering points to identify the clustering structure. ACM SIGMOD Rec. 28(2), 49–60 (1999)

    Article  Google Scholar 

  5. Buchin, K., Buchin, M., Gudmundsson, J., Löffler, M., Luo, J.: Detecting commuting patterns by clustering subtrajectories. Int. J. Comput. Geom. Appl. 21(03), 253–282 (2011)

    Article  MathSciNet  Google Scholar 

  6. Buchin, M., Driemel, A., Van Kreveld, M., Sacristán, V.: Segmenting trajectories: a framework and algorithms using spatiotemporal criteria. J. Spatial Inf. Sci. 3, 33–63 (2011)

    Google Scholar 

  7. Chen, L., Gao, Y., Fang, Z., Miao, X., Jensen, C.S., Guo, C.: Real-time distributed co-movement pattern detection on streaming trajectories. In: Proceedings of the VLDB Endowment (2019)

  8. Dutta, S., Das, A., Patra, B.K.: Clustmosa: Clustering for gps trajectory data based on multi-objective simulated annealing to develop mobility application. Appl. Soft Comput. 130, 109655 (2022)

    Article  Google Scholar 

  9. Ester, M., Kriegel, H.P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: SIGKDD, pp. 226–231 (1996)

  10. Etemad, M., Júnior, A.S., Hoseyni, A., Rose, J., Matwin, S.: A trajectory segmentation algorithm based on interpolation-based change detection strategies. In: EDBT/ICDT Workshops (2019)

  11. Fang, Z., Du, Y., Chen, L., Hu, Y., Gao, Y., Chen, G.: E 2 dtc: An end to end deep trajectory clustering framework via self-training. In: ICDE, pp. 696–707 (2021)

  12. Ferreira, N., Klosowski, J.T., Scheidegger, C.E., Silva, C.T.: Vector field k-means: Clustering trajectories by fitting multiple vector fields. In: Computer Graphics Forum, vol. 32, pp. 201–210. Wiley Online Library (2013)

  13. Frentzos, E., Gratsias, K., Theodoridis, Y.: Index-based most similar trajectory search. In: ICDE (2007)

  14. Gaffney, S., Smyth, P.: Trajectory clustering with mixtures of regression models. In: SIGKDD, pp. 63–72 (1999)

  15. Grünwald, P.D., Myung, I.J., Pitt, M.A.: Advances in minimum description length: theory and applications (2005)

  16. Gu, T., Feng, K., Cong, G., Long, C., Wang, Z., Wang, S.: A reinforcement learning based r-tree for spatial data indexing in dynamic environments. arXiv preprint arXiv:2103.04541 (2021)

  17. Lee, J.G., Han, J., Li, X., Gonzalez, H.: Traclass: trajectory classification using hierarchical region-based and trajectory-based clustering. VLDB 1(1), 1081–1094 (2008)

    Google Scholar 

  18. Lee, J.G., Han, J., Whang, K.Y.: Trajectory clustering: a partition-and-group framework. In: SIGMOD, pp. 593–604 (2007)

  19. Li, Y., Luo, J., Chow, C.Y., Chan, K.L., Ding, Y., Zhang, F.: Growing the charging station network for electric vehicles with trajectory data analytics. In: ICDE, pp. 1376–1387 (2015)

  20. Li, Z., Lee, J.G., Li, X., Han, J.: Incremental clustering for trajectories. In: International conference on database systems for advanced applications, pp. 32–46 (2010)

  21. Lloyd, S.: Least squares quantization in pcm. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)

    Article  MathSciNet  Google Scholar 

  22. Marcus, R., Negi, P., Mao, H., Tatbul, N., Alizadeh, M., Kraska, T.: Bao: Making learned query optimization practical. In: SIGMOD, pp. 1275–1288 (2021)

  23. Meratnia, N., et al.: Spatiotemporal compression techniques for moving point objects. In: EDBT (2004)

  24. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

  25. Pelekis, N., Tampakis, P., Vodas, M., Panagiotakis, C., Theodoridis, Y.: In-dbms sampling-based sub-trajectory clustering. In: EDBT, pp. 632–643 (2017)

  26. Puterman, M.L.: Markov decision processes: discrete stochastic dynamic programming. Wiley, New York (2014)

    Google Scholar 

  27. Qian, W.N., Zhou, A.Y.: Analyzing popular clustering algorithms from different viewpoints. J. Softw. 13(8), 1382–1394 (2002)

  28. Qiao, D., Yang, X., Liang, Y., Hao, X.: Rapid trajectory clustering based on neighbor spatial analysis. Pattern Recogn. Lett. 156, 167–173 (2022)

    Article  Google Scholar 

  29. Schiller, P.L., Kenworthy, J.R.: An introduction to sustainable transportation: Policy, planning and implementation. Routledge (2017)

  30. Soares Júnior, A., Moreno, B.N., Times, V.C., Matwin, S., Cabral, L.D.A.F.: Grasp-uts: an algorithm for unsupervised trajectory segmentation. Int. J. Geogr. Inf. Sci. 29(1), 46–68 (2015)

  31. Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, Cambridge (2018)

  32. Tampakis, P., Doulkeridis, C., Pelekis, N., Theodoridis, Y.: Distributed subtrajectory join on massive datasets. ACM Trans. Spatial Algorith. Syst. (TSAS) 6(2), 1–29 (2020)

    Article  Google Scholar 

  33. Tampakis, P., Pelekis, N., Doulkeridis, C., Theodoridis, Y.: Scalable distributed subtrajectory clustering. In: 2019 IEEE international conference on big data (Big Data), pp. 950–959. IEEE (2019)

  34. Wang, S., Bao, Z., Culpepper, J.S., Sellis, T., Qin, X.: Fast large-scale trajectory clustering. VLDB 13(1), 29–42 (2019)

    Google Scholar 

  35. Wang, S., Bao, Z., Culpepper, J.S., Sellis, T., Sanderson, M., Qin, X.: Answering top-k exemplar trajectory queries. In: ICDE, pp. 597–608 (2017)

  36. Wang, S., Shen, Y., Bao, Z., Qin, X.: Intelligent traffic analytics: from monitoring to controlling. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 778–781 (2019)

  37. Wang, W., Xia, F., Nie, H., Chen, Z., Gong, Z., Kong, X., Wei, W.: Vehicle trajectory clustering based on dynamic representation learning of internet of vehicles. IEEE Trans. Intell. Transp. Syst. 22(6), 3567–3576 (2020)

    Article  Google Scholar 

  38. Wang, Z., Long, C., Cong, G.: Trajectory simplification with reinforcement learning. In: ICDE, pp. 684–695 (2021)

  39. Wang, Z., Long, C., Cong, G., Liu, Y.: Efficient and effective similar subtrajectory search with deep reinforcement learning. VLDB 13(12), 2312–2325 (2020)

  40. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)

    Article  Google Scholar 

  41. Wei, H., Zheng, G., Gayah, V., Li, Z.: Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD Explorat. Newsl. 22(2), 12–18 (2021)

    Article  Google Scholar 

  42. Xia, Y., Zhou, L.: Improved clustering algorithm based on hypercube. In: 2022 International Conference on Machine Learning, Control, and Robotics (MLCR), pp. 32–37 (2022)

  43. Yang, Z., Chandramouli, B., Wang, C., Gehrke, J., Li, Y., Minhas, U.F., Larson, P.Å., Kossmann, D., Acharya, R.: Qd-tree: Learning data layouts for big data analytics. In: SIGMOD, pp. 193–208 (2020)

  44. Yao, D., Zhang, C., Zhu, Z., Huang, J., Bi, J.: Trajectory clustering via deep representation learning. In: IJCNN, pp. 3880–3887 (2017)

  45. Yi, B.K., Jagadish, H.V., Faloutsos, C.: Efficient retrieval of similar time sequences under time warping. In: ICDE, pp. 201–208 (1998)

  46. Yu, X., Li, G., Chai, C., Tang, N.: Reinforcement learning with tree-lstm for join order selection. In: ICDE, pp. 1297–1308 (2020)

  47. Yuan, J., Zheng, Y., Zhang, C., Xie, W., Xie, X., Sun, G., Huang, Y.: T-drive: driving directions based on taxi trajectories. In: SIGSPATIAL, pp. 99–108 (2010)

  48. Zhang, D., Chang, Z., Yang, D., Li, D., Tan, K.L., Chen, K., Chen, G.: Squid: subtrajectory query in trillion-scale gps database. VLDB J. pp. 1–18 (2023)

  49. Zhang, T., Ramakrishnan, R., Livny, M.: Birch: an efficient data clustering method for very large databases. ACM SIGMOD Rec. 25(2), 103–114 (1996)

    Article  Google Scholar 

  50. Zhang, X., Meng, F., Xu, J.: Perfinsight: A robust clustering-based abnormal behavior detection system for large-scale cloud. In: IEEE CLOUD, pp. 896–899 (2018)

  51. Zheng, Y., Xie, X., Ma, W.Y., et al.: Geolife: A collaborative social networking service among user, location and trajectory. IEEE Data Eng. Bull. 33(2), 32–39 (2010)

    Google Scholar 

  52. Zygouras, N., Gunopulos, D.: Corridor learning using individual trajectories. In: MDM, pp. 155–160 (2018)

Download references

Acknowledgements

This work was supported by the NSFC (61832017), Alibaba Group through Alibaba Innovative Research (AIR) Program, the National Key Research and Development Program of China (2020YFB1710200), and Hangzhou Qianjiang Distinguished Expert Program.

Author information

Authors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, China

    Anqi Liang, Bin Yao & Bo Wang

  2. Alibaba Group, Hangzhou, China

    Yinpei Liu, Zhida Chen, Jiong Xie & Feifei Li

  3. Hangzhou Institute of Advanced Technology, Hangzhou, China

    Bin Yao

Authors
  1. Anqi Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yinpei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhida Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiong Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Feifei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Yao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, A., Yao, B., Wang, B. et al. Sub-trajectory clustering with deep reinforcement learning. The VLDB Journal 33, 685–702 (2024). https://doi.org/10.1007/s00778-023-00833-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-023-00833-w

Keywords

Search

Navigation