Skip to main content
Log in

Unsupervised person re-identification via K-reciprocal encoding and style transfer

International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper, we study the unsupervised person re-identification (re-ID) problem, which does not require any annotation information. Our approach considers three aspects in unsupervised re-ID task, i.e., variance across various cameras, label allocation to unlabeled images and hard negative mining. First, an unsupervised style transfer model is adopted to generate style-transferred images with different camera styles, which contributes to reduce the variance across various cameras. Then we apply k-reciprocal encoding method to obtain k-reciprocal nearest neighbors. According to the feature similarity of the probe person with its neighbors, soft pseudo labels are allocated to the probe person iteratively. Due to lack of annotation information to pairwise images, we propose the k-reciprocal nearest neighbors loss (KNNL) to learn discriminative features. Furthermore, a hard negative mining strategy is adopted to improve the accuracy and robustness of our framework. We conduct experiments on three large-scale datasets: Market-1501, DukeMTMC-reID and MSMT17. Results show that our method not only outperforms the state-of-the-art unsupervised re-ID approaches, but also is superior to unsupervised domain adaptation methods (UDA) and semi-supervised learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 95–104

  2. Chen D, Yuan L, Liao J, Yu N, Hua G (2017) Stylebank: an explicit representation for neural image style transfer. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2770–2779

  3. Chen Y, Zhu X, Gong S (2019) Instance-guided context rendering for cross-domain person re-identification. In: 2019 IEEE/CVF International Conference on computer vision (ICCV), vol 1, pp 232–242

  4. Choi Y, Choi MJ, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 8789–8797

  5. Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on computer vision and pattern recognition, vol 1, pp 248–255

  6. Deng W, Zheng L, Kang G, Yang Y, Ye Q, Jiao J (2018) Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 994–1003

  7. Fan H, Zheng L, Yang Y (2018) Unsupervised person re-identification. ACM Trans Multimed Comput Commun Appl (TOMM) 14:1–18

    Article  Google Scholar 

  8. Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: 2010 IEEE Computer Society Conference on computer vision and pattern recognition, vol 1, pp 2360–2367

  9. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2414–2423

  10. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial networks. In: Proceedings of the International Conference on neural information processing systems, vol 27, pp 2672–2680

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer vision and pattern recognition (CVPR), vol 1, pp 770–778

  12. Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737

  13. Hoffman J, Tzeng E, Darrell T, Saenko K (2015) Simultaneous deep transfer across domains and tasks. 2015 IEEE International Conference on computer vision (ICCV), vol 1, pp 4068–4076

  14. Huang G, Liu Z, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR) 1:2261–2269

  15. Jégou H, Harzallah H, Schmid C (2007) A contextual dissimilarity measure for accurate and efficient image search. In: 2007 IEEE Conference on computer vision and pattern recognition, vol 1, pp 1–8

  16. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European Conference on computer vision, pp 694–711

  17. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  18. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Neural Inf Proces Syst. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  19. Liao S, Hu Y, Xiangyu Zhu, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2197–2206

  20. Lin Y, Dong X, Zheng L, Yan Y, Yang Y (2019) A bottom-up clustering approach to unsupervised person re-identification. Proc AAAI Conf Artif Intell 33:8738–8745. https://doi.org/10.1609/aaai.v33i01.33018738

    Article  Google Scholar 

  21. Lin Y, Wu Y, Yan C, Xu M, Yang Y (2020) Unsupervised person re-identification via cross-camera similarity exploration. IEEE Trans Image Process 29:5481–5490

    Article  Google Scholar 

  22. Lin Y, Xie L, Wu Y, Yan C, Tian Q (2020) Unsupervised person re-identification via softened similarity learning. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 3387–3396

  23. Lisanti G, Masi I, Bagdanov AD, Bimbo AD (2015) Person re-identification by iterative re-weighted sparse ranking. IEEE Trans Pattern Anal Mach Intell 37(8):1629–1642

    Article  Google Scholar 

  24. Qin D, Gammeter S, Bossard L, Quack T, Gool L (2011) Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors. CVPR 2011:777–784

    Google Scholar 

  25. Ristani E, Solera F, Zou RS, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: ECCV Workshops, pp 17–35

  26. Sun Y, Zheng L, Deng W, Wang S (2017) Svdnet for pedestrian retrieval. In: 2017 IEEE International Conference on computer vision (ICCV), vol 1, pp 3820–3828

  27. Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on computer vision (ECCV), pp 501–518

  28. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 1–9

  29. Taigman Y, Polyak A, Wolf L (2017) Unsupervised Cross-Domain Image Generation. arXiv preprint arXiv:1611.02200

  30. Wang, D., Zhang, S.: Unsupervised person re-identification via multi-label classification. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 10978–10987

  31. Wang J, Zhu X, Gong S, Li W (2018) Transferable joint attribute-identity deep learning for unsupervised person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 2275–2284

  32. Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer gan to bridge domain gap for person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 79–88

  33. Wu A, Zheng W, Lai J (2019) Unsupervised person re-identification by camera-aware similarity consistency learning. In: 2019 IEEE/CVF International Conference on computer vision (ICCV), vol 1, pp 6921–6930

  34. Wu Y, Lin Y, Dong X, Yan Y, Bian W, Yang Y (2019) Progressive learning for person re-identification with one example. IEEE Trans Image Process 28:2872–2881

    Article  MathSciNet  Google Scholar 

  35. Wu Y, Lin Y, Dong X, Yan Y, Ouyang W, Yang Y (2018) Exploit the unknown gradually: one-shot video-based person re-identification by stepwise learning. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 5177–5186

  36. Xiao T, Li S, Wang B, Lin L, Wang X (2017) Joint detection and identification feature learning for person search. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 3376–3385

  37. Yu HX, Zheng W, Wu A, Guo X, Gong S, Lai J (2019) Unsupervised person re-identification by soft multilabel learning. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2143–2152

  38. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. 2015 IEEE International Conference on computer vision (ICCV), vol 1, pp 1116–1124

  39. Zheng L, Yang Y, Hauptmann AG (2016) Person re-identification: past, present and future. arXiv preprint arXiv:1610.02984

  40. Zheng Z, Yang X, Yu Z, Zheng L, Yang Y, Kautz J (2019) Joint discriminative and generative learning for person re-identification. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2133–2142

  41. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: 2017 IEEE International Conference on computer vision (ICCV), vol 1, pp 3774–3782

  42. Zheng Z, Zheng L, Yang Y (2018) A discriminatively learned cnn embedding for person reidentification. ACM Trans Multimed Comput Commun Appl (TOMM) 14:1–20

    Article  Google Scholar 

  43. Zhong Z, Zheng L, Cao D, Li S (2017) Re-ranking person re-identification with k-reciprocal encoding. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 3652–3661

  44. Zhong Z, Zheng L, Li S, Yang Y (2018) Generalizing a person retrieval model hetero- and homogeneously. In: Proceedings of the European Conference on computer vision (ECCV), pp 172–188

  45. Zhong Z, Zheng L, Luo Z, Li S, Yang Y (2019) Invariance matters: exemplar memory for domain adaptive–person re-identification. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 598–607

  46. Zhong Z, Zheng L, Zheng Z, Li S, Yang Y (2019) Camstyle: a novel data augmentation method for person re-identification. IEEE Trans Image Process 28:1176–1190

    Article  MathSciNet  Google Scholar 

  47. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on computer vision (ICCV), vol 1, pp 2242–2251

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China No. 61872153, the National Science Foundation of Guangdong Province No. 2018A030313318 and the Key-Area Research and Development Program of Guangdong Province No. 2019B111101001.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Jing Xiao or Jingjing Li.

Ethics declarations

Conflicts of interest

The authors declare that there is no conflict of interests regarding the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, K., Wu, Y., Xiao, J. et al. Unsupervised person re-identification via K-reciprocal encoding and style transfer. Int. J. Mach. Learn. & Cyber. 12, 2899–2916 (2021). https://doi.org/10.1007/s13042-021-01376-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01376-8

Keywords

Navigation