Skip to main content
SpringerLink
Log in

Vehicle re-identification based on keypoint segmentation of original image

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In the earlier days, part segmentation methods for vehicle re-id were based on segmenting the feature map of the last convolutional layer. However, by calculating the receptive field, we can see that the size of the receptive field of each point in the feature map of the last convolutional layer exceeds that of the original input image. Therefore, it is very difficult to segment vehicle parts according to feature map. In order to overcome such difficulty, we propose a vehicle re-identification method based on keypoint segmentation of the original image (KSOI). We segment the original image into two parts, which are termed as the window part (upper part) and the below-window part (lower part). Then we use three branches to extract the features of the window part image, the below-window part image and the original vehicle image respectively, which will be fused in the inference stage. In order to achieve accurate segmentation, we label the orientations for all the images in the training set and the keypoint coordinates of the two bottom vertices of the rectangle bounding boxes of the visible windows for some images in the training set. We then train an orientation extraction network and a keypoint detection network to obtain the visible keypoints, and segment the original image with the coordinates of visible keypoints. The experimental results of the proposed KSOI reach the state-of-the-art level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Liu X, Liu W, Ma H et al (2016) Large-scale vehicle re-identification in urban surveillance videos. 2016 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 1–6

  2. Liu X, Liu W, Mei T et al (2016) A deep learning-based approach to progressive vehicle re-identification for urban surveillance. European conference on computer vision. Springer, Cham, pp 869–884

  3. Wu F, Yan S, Smith JS et al (2018) Joint semi-supervised learning and re-ranking for vehicle re-identification. 2018 24th International Conference on Pattern Recognition (ICPR). IEEE, pp 278–283

  4. Liu H, Tian Y, Yang Y, Pang L, Huang T (2016) Deep relative distance learning: Tell the difference between similar vehicles. In: roceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2167–2175

  5. Yan K, Tian Y, Wang Y et al (2017) Exploiting multi-grain ranking constraints for precisely searching visually-similar vehicles. Proceedings of the IEEE international conference on computer vision, pp 562–570

  6. He B, Li J, Zhao Y et al (2019) Part-regularized near-duplicate vehicle re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3997–4005

  7. Qian J, Jiang W, Luo H et al (2020) Stripe-based and attribute-aware network: A two-branch deep model for vehicle re-identification. Measur Sci Technol 31(9):095401

  8. Wang H, Peng J, Jiang G et al (2021) Discriminative feature and dictionary learning with part-aware model for vehicle re-identification. Neurocomputing 438:55–62

    Article  Google Scholar 

  9. Luo W, Li Y, Urtasun R et al (2016) Understanding the effective receptive field in deep convolutional neural networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, pp 4905–4913

  10. Chu R, Sun Y, Li Y et al (2019) Vehicle re-identification with viewpoint-aware metric learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 8282–8291

  11. Zapletal D, Herout A (2016) Vehicle re-identification for automatic video traffic surveillance. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 25–31

  12. Zhang Y, Liu D, Zha ZJ (2017) Improving triplet-wise training of convolutional neural network for vehicle re-identification. 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 1386–1391

  13. Shen Y, Xiao T, Li H et al (2017) Learning deep neural networks for vehicle re-id with visual-spatio-temporal path proposals. Proceedings of the IEEE International Conference on Computer Vision, pp 1900–1909

  14. Zhou Y, Shao L (2018) Vehicle re-identification by adversarial bi-directional lstm network. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 653–662

  15. Zhou Y, Shao L (2017) Cross-view GAN Based Vehicle Generation for Re-identification. BMVC 1:1–12

    Google Scholar 

  16. Zhou Y, Shao L (2018) Aware attentive multi-view inference for vehicle re-identification. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6489–6498

  17. Zhou Y, Liu L, Shao L (2018) Vehicle re-identification by deep hidden multi-view inference. IEEE Trans Image Process 27(7):3275–3287

    Article  MathSciNet  Google Scholar 

  18. Wu CW, Liu CT, Chiang CE et al (2018) Vehicle re-identification with the space-time prior. Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 121–128

  19. Rajamanoharan G, Kanaci A, Li M et al (2019) Multi-task mutual learning for vehicle re-identification. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  20. Tang Z, Naphade M, Liu MY et al (2019) Cityflow: a city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8797–8806

  21. Peng J, Wang H, Xu F et al (2020) Cross domain knowledge learning with dual-branch adversarial network for vehicle re-identification. Neurocomputing 401:133–144

    Article  Google Scholar 

  22. Luo H, Gu Y, Liao X et al (2019) Bag of tricks and a strong baseline for deep person re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 0–0

  23. Cheng Y, Zhang C, Gu K et al (2020) Multi-Scale Deep feature fusion for vehicle Re-Identification. ICASSP 2020-2020. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 1928–1932

  24. Shen F, Zhu J, Zhu X et al (2021) Exploring spatial significance via hybrid pyramidal graph network for vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems

  25. Liu X, Liu W, Zheng J et al (2020) Beyond the Parts: Learning Multi-view Cross-part Correlation for Vehicle Re-identification. Proceedings of the 28th ACM International Conference on Multimedia, pp 907–915

  26. Zhang S, Lin C, Ma S (2020) Large margin metric learning for multi-view vehicle re-identification. Neurocomputing 447:118–128

    Article  Google Scholar 

  27. Teng S, Zhang S, Huang Q et al (2020) Multi-view spatial attention embedding for vehicle re-identification. IEEE Transactions on Circuits and Systems for Video Technology

  28. Wang Q, Min W, Han Q et al (2021) Viewpoint adaptation learning with cross-view distance metric for robust vehicle re-identification. Inf Sci 564:71–84

    Article  MathSciNet  Google Scholar 

  29. Zhu X, Luo Z, Fu P et al (2020) VOC-ReID: Vehicle re-identification based on vehicle-orientation-camera. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 602–603

  30. Chen X, Sui H, Fang J et al (2020) Multi-Proxy Constraint loss for vehicle Re-Identification. Sensors 20(18):5142

    Article  Google Scholar 

  31. Zhuge C, Peng Y, Li Y et al (2020) Attribute-guided Feature Extraction and Augmentation Robust Learning for Vehicle Re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 618–619

  32. Sun Z, Nie X, Xi X et al (2020) CFVMNEt: A Multi-branch Network for Vehicle Re-identification Based on Common Field of View. Proceedings of the 28th ACM International Conference on Multimedia, pp 3523–3531

  33. Guo H, Zhu K, Tang M, Wang J (2019) Two-level attention network with multi-grain ranking loss for vehicle re-identifification. IEEE Transactions on Image Processing

  34. Teng S, Liu X, Zhang S, Huang Q (2018) Scan: Spatial and channel attention network for vehicle re-identifification. In: Pacifific Rim Conference on Multimedia. Springer, pp 350– 361

  35. Wang Z, Tang L, Liu X, Yao Z, Wang X (2017) Orientation invariant feature embedding and spatial temporal regularization for vehicle re-identification. In: IEEE International Conference on Computer Vision

  36. Pan M, Zhu X, Li Y et al (2020) MRNEt: A Keypoint Guided Multi-scale Reasoning Network for Vehicle Re-identification. International Conference on Neural Information Processing. Springer, Cham, pp 469–478

  37. Khorramshahi P, Peri N, Kumar A, Shah A, Chellappa R (2019) Attention driven vehicle re-identifification and unsupervised anomaly detection for traffic understanding. In: roceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 239–246

  38. Zheng B, Lei Z, Tang C et al (2021) OERFF: A vehicle Re-Identification method based on orientation estimation and regional feature fusion. IEEE Access 9:66661–66674

    Article  Google Scholar 

  39. Khorramshahi P, Kumar A, Peri N, Rambhatla SS, Chen J-C, Chellappa R (2019) A dual path modelwith adaptive attention for vehicle re-identification. arXiv:1905.03397

  40. Zhu J, Zeng H, Huang J et al (2019) Vehicle re-identification using quadruple directional deep learning features. IEEE Trans Intell Transp Syst 21(1):410–420

    Article  Google Scholar 

  41. Liu X, Zhang S, Huang Q et al (2018) Ram: a region-aware deep model for vehicle re-identification. 2018 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 1–6

  42. Chen Y, Ma B, Chang H (2020) Part alignment network for vehicle re-identification. Neurocomputing 418:114–125

    Article  Google Scholar 

  43. Chen TS, Lee MY, Liu CT et al (2020) Viewpoint-aware channel-wise attentive network for vehicle re-identification. Proc. CVPR Workshops, Seattle

  44. Zhang F, Ma Y, Yuan G et al (2020) Multiview image generation for vehicle reidentification. Appl Intell:1–18

  45. Gu J, Jiang W, Luo H et al (2021) An efficient global representation constrained by Angular Triplet loss for vehicle re-identification. Pattern Anal Appl 24(1):367–379

    Article  Google Scholar 

  46. Tumrani S, Deng Z, Lin H et al (2020) Partial attention and multi-attribute learning for vehicle re-identification. Pattern Recogn Lett 138:290–297

    Article  Google Scholar 

  47. Katsaros E, Bouma H, van Rooijen A et al (2020) A triplet-learnt coarse-to-fine reranking for vehicle re-identification. ICPRAM:518–525

  48. Chen TS, Liu CT, Wu CW et al (2020) Orientation-aware vehicle re-identification with semantics-guided part attention network. European Conference on Computer Vision. Springer, Cham, pp 330–346

  49. Sun K, Xiao B, Liu D et al (2019) Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5693–5703

  50. Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: European Conference on Computer Vision, pp 483–499

  51. Ke L, Chang M-C, Qi H, Lyu S (2018) Multi-scale structure-aware network for human pose estimation. CoRR, arXiv:1803.09894

  52. Tang W, Yu P, Wu Y (2018) Deeply learned compositional models for human pose estimation. In: ECCV

  53. Kingma DP, Ba J (2014) Adam A method for stochastic optimization. arXiv:1412.6980

Download references

Acknowledgements

This work is supported by Shenzhen Key Laboratory of Visual Object Detection and Recognition (No.ZDSYS2019 0902093015527), National Natural Science Foundation of China (No. 61876051) and deep network based high-performance image object detection research (No. JCYJ20180306172101694), Guizhou Provincial Department of Education Youth Science and Technology Talents Growth Project(Project Nos.qianjiaoheK-Yzi[2017]251.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China

    Zhijun Hu, Xianjing Cheng, Lilei Sun & Lian Wu

  2. College of Mathematics and Statistics, Guangxi Normal University, Guilin, 541004, China

    Zhijun Hu

  3. Shenzhen Key Laboratory of Visual Object Detection and Recognition, Shenzhen, Guangdong, 518055, China

    Yong Xu

  4. Department of Computer Science and Engineering of Vellore Institute of Technology, Vellore, Tamilnadu, India

    Raja Soosaimarian Peter Raj

  5. Zunyi Normal University, Zunyi, 563000, China

    Xianjing Cheng

Authors
  1. Zhijun Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raja Soosaimarian Peter Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianjing Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lilei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhijun Hu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, Z., Xu, Y., Raj, R.S.P. et al. Vehicle re-identification based on keypoint segmentation of original image. Appl Intell 53, 2576–2592 (2023). https://doi.org/10.1007/s10489-022-03192-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03192-1

Keywords

Search

Navigation