Skip to main content
SpringerLink
Log in

PAII: A Pose Alignment Network with Information Interaction for Person Re-identification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

As an important part of intelligent surveillance systems, person re-identification (re-ID) has a wide range of application prospects in smart cities. However, due to occlusion, viewpoint variation, and background shift, the misalignment problem always decreases the re-ID systems’ effects. To solve this problem, a pose alignment network with information interaction (PAII) is proposed. This approach consists of three cascaded modules. First, guided by a pretrained pose estimator, the backbone with a dual attention block is used to obtain local features corresponding to different pose keypoints along with the global feature. Then, a pose alignment module is constructed to group these local features into different parts and fuse them with a hyperparameter \(\lambda \), which provides the possibility to achieve semantic alignment. Finally, since different semantic features are extracted, an information interaction module consisting of graph attention layers is made to conduct message passing between different semantic features. All semantic features and the global feature are used to calculate the loss functions. Our approach considers multi-scale representations and information interaction of semantic features, which makes it more robust to misalignment problems. Thus, the proposed PAII method achieves better performance than most existing methods on multiple popular re-ID datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Huang Y, Zha Z J, Fu X, Zhang W (2019) Illumination-invariant person re-identification. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 365–373

  2. Sun X, Zheng L (2019) Dissecting person re-identification from the viewpoint of viewpoint. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 608–617

  3. Li D, Hu R, Huang W, Li D, Wang X, Hu C (2021) Trajectory association for person re-identification. Neural Process Lett 53(5):3267–3285

    Article  Google Scholar 

  4. Zhuo J, Chen Z, Lai J, Wang G (2018) Occluded person re-identification. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE

  5. Zeng Z, Wang Z, Wang Z, Zheng Y, Chuang YY, Satoh S (2020) Illumination-adaptive person re-identification. IEEE Trans Multimedia 22(12):3064–3074

    Article  Google Scholar 

  6. Hou R, Ma B, Chang H, Gu X, Shan S, Chen X (2021) Feature completion for occluded person re-identification. IEEE Transactions on Pattern Analysis and Machine Intelligence

  7. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 480–496

  8. Yao H, Zhang S, Hong R, Zhang Y, Xu C, Tian Q (2019) Deep representation learning with part loss for person re-identification. IEEE Trans Image Process 28(6):2860–2871

    Article  MathSciNet  MATH  Google Scholar 

  9. Fu Y, Wei Y, Zhou Y, Shi H, Huang G, Wang X, Yao Z, Huang T (2019) Horizontal pyramid matching for person re-identification. Proc AAAI Conf Artif Intell 33:8295–8302

    Google Scholar 

  10. Wang C, Song L, Wang G, Zhang Q, Wang X (2020) Multi-scale multi-patch person re-identification with exclusivity regularized softmax. Neurocomputing 382:64–70

    Article  Google Scholar 

  11. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: A benchmark. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1116–1124

  12. Ristani E, Solera F, Zou R, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp. 17–35. Springer

  13. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3754–3762

  14. Wei L, Zhang S, Yao H, Gao W, Tian Q (2018) Glad: Global-local-alignment descriptor for scalable person re-identification. IEEE Trans Multimedia 21(4):986–999

    Article  Google Scholar 

  15. Luo H, Jiang W, Zhang X, Fan X, Qian J, Zhang C (2019) Alignedreid++: Dynamically matching local information for person re-identification. Pattern Recogn 94:53–61

    Article  Google Scholar 

  16. Chen B, Deng W, Hu J (2019) Mixed high-order attention network for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 371–381

  17. Li S, Bak S, Carr P, Wang X (2018) Diversity regularized spatiotemporal attention for video-based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 369–378

  18. Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1077–1085

  19. Moskvyak O, Maire F, Dayoub F, Baktashmotlagh M (2021) Keypoint-aligned embeddings for image retrieval and re-identification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 676–685

  20. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. arXiv preprint arXiv:1710.10903

  21. Li W, Zhao R, Xiao T, Wang X (2014) Deepreid: Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 152–159

  22. Zhong Z, Zheng L, Cao D, Li S (2017) Re-ranking person re-identification with k-reciprocal encoding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1318–1327

  23. Yu J, Tao D, Wang M, Rui Y (2014) Learning to rank using user clicks and visual features for image retrieval. IEEE trans cybernet 45(4):767–779

    Article  Google Scholar 

  24. Yu J, Tan M, Zhang H, Tao D, Rui Y (2019) Hierarchical deep click feature prediction for fine-grained image recognition. IEEE transactions on pattern analysis and machine intelligence

  25. Hong C, Yu J, Zhang J, Jin X, Lee KH (2018) Multimodal face-pose estimation with multitask manifold deep learning. IEEE Trans Industr Inf 15(7):3952–3961

    Article  Google Scholar 

  26. Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24(12):5659–5670

    Article  MathSciNet  MATH  Google Scholar 

  27. Hong C, Yu J, Tao D, Wang M (2014) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Industr Electron 62(6):3742–3751

    Google Scholar 

  28. Song L, Wang C, Zhang L, Du B, Zhang Q, Huang C, Wang X (2020) Unsupervised domain adaptive re-identification: Theory and practice. Pattern Recogn 102:107173

    Article  Google Scholar 

  29. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi S C (2021) Deep learning for person re-identification: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence

  30. Li Z, Chang S, Liang F, Huang T S, Cao L, Smith J R (2013) Learning locally-adaptive decision functions for person verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3610–3617

  31. Li W, Wang X (2013) Locally aligned feature transforms across views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3594–3601

  32. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

  33. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141

  34. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 510–519

  35. Woo S, Park J, Lee J Y, Kweon I S (2018) Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19

  36. Kipf T N, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  37. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29:3844–3852

    Google Scholar 

  38. Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1993–2001

  39. Hamilton W L, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1025–1035

  40. Sun Y, Xu Q, Li Y, Zhang C, Li Y, Wang S, Sun J (2019) Perceive where to focus: Learning visibility-aware part-level features for partial person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 393–402

  41. Xu J, Zhao R, Zhu F, Wang H, Ouyang W (2018) Attention-aware compositional network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2119–2128

  42. Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5693–5703

  43. Lin T Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick C L (2014) Microsoft coco: Common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer

  44. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR

  45. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  46. Li D, Chen X, Zhang Z, Huang K (2017) Learning deep context-aware features over body and latent parts for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 384–393

  47. Sun Y, Zheng L, Deng W, Wang S (2017) Svdnet for pedestrian retrieval. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3800–3808

  48. Liu J, Ni B, Yan Y, Zhou P, Cheng S, Hu J (2018) Pose transferrable person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4099–4108

  49. Qian X, Fu Y, Xiang T, Wang W, Qiu J, Wu Y, Jiang Y G, Xue X (2018) Pose-normalized image generation for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 650–667

  50. Wang G, Yang S, Liu H, Wang Z, Yang Y, Wang S, Yu G, Zhou E, Sun J (2020) High-order information matters: Learning relation and topology for occluded person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6449–6458

  51. Yang F, Yan K, Lu S, Jia H, Xie X, Gao W (2019) Attention driven person re-identification. Pattern Recogn 86:143–155

    Article  Google Scholar 

  52. Quan R, Dong X, Wu Y, Zhu L, Yang Y (2019) Auto-reid: Searching for a part-aware convnet for person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3750–3759

  53. Bai X, Yang M, Huang T, Dou Z, Yu R, Xu Y (2020) Deep-person: Learning discriminative deep features for person re-identification. Pattern Recogn 98:107036

    Article  Google Scholar 

  54. Li Z, Lv J, Chen Y, Yuan J (2021) Person re-identification with part prediction alignment. Comput Vis Image Underst 205:103172

    Article  Google Scholar 

  55. Wang C, Zhang Q, Huang C, Liu W, Wang X (2018) Mancs: A multi-task attentional network with curriculum sampling for person re-identification. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 365–381

  56. Jin H, Lai S, Zhao G, Qian X (2021) Hashing person re-id with self-distilling smooth relaxation. Neurocomputing 455:111–124

    Article  Google Scholar 

  57. Zhong Z, Zheng L, Zheng Z, Li S, Yang Y (2018) Camstyle: A novel data augmentation method for person re-identification. IEEE Trans Image Process 28(3):1176–1190

    Article  MathSciNet  Google Scholar 

  58. Miao J, Wu Y, Liu P, Ding Y, Yang Y (2019) Pose-guided feature alignment for occluded person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 542–551

  59. Liu Z, Wang J, Gong S, Lu H, Tao D (2019) Deep reinforcement active learning for human-in-the-loop person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6122–6131

  60. Liu C, Chang X, Shen Y D (2020) Unity style transfer for person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6887–6896

  61. Serbetci A, Akgul YS (2020) End-to-end training of cnn ensembles for person re-identification. Pattern Recogn 104:107319

    Article  Google Scholar 

  62. Liu M, Yan X, Wang C, Wang K (2021) Segmentation mask-guided person image generation. Appl Intell 51(2):1161–1176

    Article  Google Scholar 

  63. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2020) Random erasing data augmentation. Proc AAAI Conf Artif Intell 34:13001–13008

    Google Scholar 

  64. Wang C, Song L, Wang G, Zhang Q, Wang X (2020) Multi-scale multi-patch person re-identification with exclusivity regularized softmax. Neurocomputing 382:64–70

    Article  Google Scholar 

  65. Zhang T, Sun X, Li X, Yi Z (2021) Image generation and constrained two-stage feature fusion for person re-identification. Appl Intell 51(11):7679–7689

    Article  Google Scholar 

  66. Xu F, Ma B, Chang H, Shan S (2020) Isosceles constraints for person re-identification. IEEE Trans Image Process 29:8930–8943

    Article  MathSciNet  MATH  Google Scholar 

  67. Li W, Zhu X, Gong S (2018) Harmonious attention network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2285–2294

  68. Song C, Huang Y, Ouyang W, Wang L (2018) Mask-guided contrastive attention model for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1179–1188

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (61573114).

Author information

Authors and Affiliations

  1. College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin, 150001, People’s Republic of China

    Chunyan Lyu, Tong Xu, Wu Ning, Qi Cheng & Kejun Wang

  2. School of Information Technology, Beijing Institute of Technology, Zhuhai, People’s Republic of China

    Kejun Wang

  3. Department of Statistics, University of California, Los Angeles, 90095-1554, USA

    Chenhui Wang

Authors
  1. Chunyan Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wu Ning
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qi Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kejun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chenhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kejun Wang.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lyu, C., Xu, T., Ning, W. et al. PAII: A Pose Alignment Network with Information Interaction for Person Re-identification. Neural Process Lett 55, 1455–1477 (2023). https://doi.org/10.1007/s11063-022-10947-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10947-x

Keywords

Search

Navigation