Abstract
In this paper, we study the unsupervised person re-identification (re-ID) problem, which does not require any annotation information. Our approach considers three aspects in unsupervised re-ID task, i.e., variance across various cameras, label allocation to unlabeled images and hard negative mining. First, an unsupervised style transfer model is adopted to generate style-transferred images with different camera styles, which contributes to reduce the variance across various cameras. Then we apply k-reciprocal encoding method to obtain k-reciprocal nearest neighbors. According to the feature similarity of the probe person with its neighbors, soft pseudo labels are allocated to the probe person iteratively. Due to lack of annotation information to pairwise images, we propose the k-reciprocal nearest neighbors loss (KNNL) to learn discriminative features. Furthermore, a hard negative mining strategy is adopted to improve the accuracy and robustness of our framework. We conduct experiments on three large-scale datasets: Market-1501, DukeMTMC-reID and MSMT17. Results show that our method not only outperforms the state-of-the-art unsupervised re-ID approaches, but also is superior to unsupervised domain adaptation methods (UDA) and semi-supervised learning methods.
References
Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 95–104
Chen D, Yuan L, Liao J, Yu N, Hua G (2017) Stylebank: an explicit representation for neural image style transfer. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2770–2779
Chen Y, Zhu X, Gong S (2019) Instance-guided context rendering for cross-domain person re-identification. In: 2019 IEEE/CVF International Conference on computer vision (ICCV), vol 1, pp 232–242
Choi Y, Choi MJ, Kim M, Ha JW, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 8789–8797
Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on computer vision and pattern recognition, vol 1, pp 248–255
Deng W, Zheng L, Kang G, Yang Y, Ye Q, Jiao J (2018) Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 994–1003
Fan H, Zheng L, Yang Y (2018) Unsupervised person re-identification. ACM Trans Multimed Comput Commun Appl (TOMM) 14:1–18
Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: 2010 IEEE Computer Society Conference on computer vision and pattern recognition, vol 1, pp 2360–2367
Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2414–2423
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial networks. In: Proceedings of the International Conference on neural information processing systems, vol 27, pp 2672–2680
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer vision and pattern recognition (CVPR), vol 1, pp 770–778
Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737
Hoffman J, Tzeng E, Darrell T, Saenko K (2015) Simultaneous deep transfer across domains and tasks. 2015 IEEE International Conference on computer vision (ICCV), vol 1, pp 4068–4076
Huang G, Liu Z, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR) 1:2261–2269
Jégou H, Harzallah H, Schmid C (2007) A contextual dissimilarity measure for accurate and efficient image search. In: 2007 IEEE Conference on computer vision and pattern recognition, vol 1, pp 1–8
Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European Conference on computer vision, pp 694–711
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. Neural Inf Proces Syst. https://doi.org/10.1145/3065386
Liao S, Hu Y, Xiangyu Zhu, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2197–2206
Lin Y, Dong X, Zheng L, Yan Y, Yang Y (2019) A bottom-up clustering approach to unsupervised person re-identification. Proc AAAI Conf Artif Intell 33:8738–8745. https://doi.org/10.1609/aaai.v33i01.33018738
Lin Y, Wu Y, Yan C, Xu M, Yang Y (2020) Unsupervised person re-identification via cross-camera similarity exploration. IEEE Trans Image Process 29:5481–5490
Lin Y, Xie L, Wu Y, Yan C, Tian Q (2020) Unsupervised person re-identification via softened similarity learning. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 3387–3396
Lisanti G, Masi I, Bagdanov AD, Bimbo AD (2015) Person re-identification by iterative re-weighted sparse ranking. IEEE Trans Pattern Anal Mach Intell 37(8):1629–1642
Qin D, Gammeter S, Bossard L, Quack T, Gool L (2011) Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors. CVPR 2011:777–784
Ristani E, Solera F, Zou RS, Cucchiara R, Tomasi C (2016) Performance measures and a data set for multi-target, multi-camera tracking. In: ECCV Workshops, pp 17–35
Sun Y, Zheng L, Deng W, Wang S (2017) Svdnet for pedestrian retrieval. In: 2017 IEEE International Conference on computer vision (ICCV), vol 1, pp 3820–3828
Sun Y, Zheng L, Yang Y, Tian Q, Wang S (2018) Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In: Proceedings of the European Conference on computer vision (ECCV), pp 501–518
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 1–9
Taigman Y, Polyak A, Wolf L (2017) Unsupervised Cross-Domain Image Generation. arXiv preprint arXiv:1611.02200
Wang, D., Zhang, S.: Unsupervised person re-identification via multi-label classification. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 10978–10987
Wang J, Zhu X, Gong S, Li W (2018) Transferable joint attribute-identity deep learning for unsupervised person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 2275–2284
Wei L, Zhang S, Gao W, Tian Q (2018) Person transfer gan to bridge domain gap for person re-identification. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 79–88
Wu A, Zheng W, Lai J (2019) Unsupervised person re-identification by camera-aware similarity consistency learning. In: 2019 IEEE/CVF International Conference on computer vision (ICCV), vol 1, pp 6921–6930
Wu Y, Lin Y, Dong X, Yan Y, Bian W, Yang Y (2019) Progressive learning for person re-identification with one example. IEEE Trans Image Process 28:2872–2881
Wu Y, Lin Y, Dong X, Yan Y, Ouyang W, Yang Y (2018) Exploit the unknown gradually: one-shot video-based person re-identification by stepwise learning. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, vol 1, pp 5177–5186
Xiao T, Li S, Wang B, Lin L, Wang X (2017) Joint detection and identification feature learning for person search. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 3376–3385
Yu HX, Zheng W, Wu A, Guo X, Gong S, Lai J (2019) Unsupervised person re-identification by soft multilabel learning. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2143–2152
Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. 2015 IEEE International Conference on computer vision (ICCV), vol 1, pp 1116–1124
Zheng L, Yang Y, Hauptmann AG (2016) Person re-identification: past, present and future. arXiv preprint arXiv:1610.02984
Zheng Z, Yang X, Yu Z, Zheng L, Yang Y, Kautz J (2019) Joint discriminative and generative learning for person re-identification. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 2133–2142
Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: 2017 IEEE International Conference on computer vision (ICCV), vol 1, pp 3774–3782
Zheng Z, Zheng L, Yang Y (2018) A discriminatively learned cnn embedding for person reidentification. ACM Trans Multimed Comput Commun Appl (TOMM) 14:1–20
Zhong Z, Zheng L, Cao D, Li S (2017) Re-ranking person re-identification with k-reciprocal encoding. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), vol 1, pp 3652–3661
Zhong Z, Zheng L, Li S, Yang Y (2018) Generalizing a person retrieval model hetero- and homogeneously. In: Proceedings of the European Conference on computer vision (ECCV), pp 172–188
Zhong Z, Zheng L, Luo Z, Li S, Yang Y (2019) Invariance matters: exemplar memory for domain adaptive–person re-identification. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), vol 1, pp 598–607
Zhong Z, Zheng L, Zheng Z, Li S, Yang Y (2019) Camstyle: a novel data augmentation method for person re-identification. IEEE Trans Image Process 28:1176–1190
Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on computer vision (ICCV), vol 1, pp 2242–2251
Acknowledgements
This work is supported by the National Natural Science Foundation of China No. 61872153, the National Science Foundation of Guangdong Province No. 2018A030313318 and the Key-Area Research and Development Program of Guangdong Province No. 2019B111101001.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflicts of interest
The authors declare that there is no conflict of interests regarding the publication of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xie, K., Wu, Y., Xiao, J. et al. Unsupervised person re-identification via K-reciprocal encoding and style transfer. Int. J. Mach. Learn. & Cyber. 12, 2899–2916 (2021). https://doi.org/10.1007/s13042-021-01376-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-021-01376-8