Abstract
Vessel traffic surveillance in inland waterways extensively relies on the Automatic Identification Syst em (AIS) and video cameras. While video data only captures the visual appearance of vessels, AIS data serves as a valuable source of vessel identity and motion information, such as position, speed, and heading. To gain a comprehensive understanding of the behavior and motion of known-identity vessels, it is necessary to fuse the AIS-based and video-based trajectories. An important step in this fusion is to obtain the correspondence between moving targets by trajectory association. Thus, we focus solely on trajectory association in this work and propose a trajectory association method based on deep graph matching. We formulate trajectory association as a graph matching problem and introduce an attention-based flexible context aggregation mechanism to exploit the semantic features of trajectories. Compared to traditional methods that rely on manually designed features, our approach captures complex patterns and correlations within trajectories through end-to-end training. The introduced dustbin mechanism can effectively handle outliers during matching. Experimental results on synthetic and real-world datasets demonstrate the exceptional performance of our method in terms of trajectory association accuracy and robustness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, J., Hu, Q., Zhao, R., Guojun, P., Yang, C.: Tracking a vessel by combining video and ais reports. In: 2008 Second International Conference on Future Generation Communication and Networking, vol. 2, pp. 374–378. IEEE (2008)
Cho, M., Lee, J., Lee, K.M.: Reweighted random walks for graph matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 492–505. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15555-0_36
Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Gold, S., Rangarajan, A.: A graduated assignment algorithm for graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 18(4), 377–388 (1996)
Guo, Y., Liu, R.W., Qu, J., Lu, Y., Zhu, F., Lv, Y.: Asynchronous trajectory matching-based multimodal maritime data fusion for vessel traffic surveillance in inland waterways. IEEE Trans. Intell. Transp. Syst. (2023)
Hartmanis, J.: Computers and intractability: a guide to the theory of np-completeness (michael r. garey and david s. johnson). Siam Rev. 24(1), 90 (1982)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. Logist. Quart. 2(1–2), 83–97 (1955)
Loiola, E.M., De Abreu, N.M.M., Boaventura-Netto, P.O., Hahn, P., Querido, T.: A survey for the quadratic assignment problem. Eur. J. Oper. Res. 176(2), 657–690 (2007)
Lu, Y., et al.: Fusion of camera-based vessel detection and AIS for maritime surveillance. In: 2021 26th International Conference on Automation and Computing (ICAC), pp. 1–6. IEEE (2021)
Müller, M.: Dynamic time warping. In: Information Retrieval for Music and Motion, pp. 69–84. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74048-3_4
Qu, J., Liu, R.W., Guo, Y., Lu, Y., Su, J., Li, P.: Improving maritime traffic surveillance in inland waterways using the robust fusion of AIS and visual data. Ocean Eng. 275, 114198 (2023)
Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: Superglue: learning feature matching with graph neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4938–4947 (2020)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, R., Yan, J., Yang, X.: Learning combinatorial embedding networks for deep graph matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3056–3065 (2019)
Yan, J., Yang, S., Hancock, E.R.: Learning for graph matching and related combinatorial optimization problems. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20. International Joint Conferences on Artificial Intelligence Organization, pp. 4988–4996 (2020)
Yu, T., Wang, R., Yan, J., Li, B.: Learning deep graph matching with channel-independent embedding and hungarian attention. In: International Conference on Learning Representations (2020)
Zanfir, A., Sminchisescu, C.: Deep learning of graph matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2684–2693 (2018)
Acknowledgements
This work is supported partly by National Key R &D Program of China (grant 2020AAA0108902), partly by National Natural Science Foundation (NSFC) of China (grants 61973301, 61972020), and partly by Youth Innovation Promotion Association CAS.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lu, Y. et al. (2024). A Deep Graph Matching-Based Method for Trajectory Association in Vessel Traffic Surveillance. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14448. Springer, Singapore. https://doi.org/10.1007/978-981-99-8082-6_32
Download citation
DOI: https://doi.org/10.1007/978-981-99-8082-6_32
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8081-9
Online ISBN: 978-981-99-8082-6
eBook Packages: Computer ScienceComputer Science (R0)