Skip to main content
SpringerLink
Log in

Lightweight convolutional neural network-based pedestrian detection and re-identification in multiple scenarios

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Pedestrian detection and re-identification technology is a research hotspot in the field of computer vision. This technology currently has issues such as insufficient pedestrian expression ability, occlusion, diverse pedestrian attitude, and difficulty of small-scale pedestrian detection. In this paper, we proposed an end-to-end pedestrian detection and re-identification model in real scenes, which can effectively solve these problems. In our model, the original images are processed with a non-overlapped image blocking data augmentation method, and then input them into the YOLOv3 detector to obtain the object position information. LCNN-based pedestrian re-identification model is used to extract the features of the object. Furthermore, the eigenvectors of the object and the detected pedestrians are calculated, and the similarity between them are used to determine whether they can be marked as target pedestrians. Our method is lightweight and end-to-end, which can be applied to the real scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Oren, M., Papageoriou, C., Sinha, P., Osuna, E., Poggio, T.: Pedestrian detection using wavelet templates. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 193–199 (1997)

  2. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: European Conference on Computer Vision. Springer, Berlin, Heidelberg (2006)

  3. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 886–893, vol. 1 (2005)

  4. Dollár, P., Appel, R., Belongie, S.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1532–1545 (2014)

    Article  Google Scholar 

  5. Krizhevsky, A., Sutskever, I., Hinton, G. E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)

  6. Girshick, R., Donahue, J., Darrell, T.: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 38(1), 142–158 (2015)

    Article  Google Scholar 

  7. Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision, 1440–1448 (2015)

  8. Ren, S., He, K., Girshick, R.: Faster R-CNN: towards real-time object detection with region proposal networks. In: International Conference on Neural Information Processing Systems. MIT Press, 91–99 (2015)

  9. Liu, W., Anguelov, D., Erhan, D.: SSD: single shot MultiBox detector. In: European Conference on Computer Vision. Springer International Publishing, 21–37 (2016)

  10. Redmon, J., Divvala, S., Girshick, R.: You only look once: unified, real-time object detection. In: IEEE Computer Vision and Pattern Recognition, 779–788 (2016)

  11. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6517–6525 (2016)

  12. He, K., Gkioxari, G., Dollar, P.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 99, 1–1 (2017)

    Google Scholar 

  13. Gheissari, N., Sebastian, T. B., Hartley, R.: Person reidentification using spatiotemporal appearance. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1528–1535 (2006)

  14. Liao, S., Hu, Y., Zhu, X.: Person re-identification by local maximal occurrence representation and metric learning. In: IEEE Conference on Computer Vision and Pattern Recognition, 2197–2206 (2015)

  15. Yi, D., Lei, Z., Liao, S.: Deep metric learning for person re-identification. In: International Conference on Pattern Recognition. IEEE, 34–39 (2014)

  16. Li, W., Zhao, R., Xiao, T.: DeepReID: deep filter pairing neural network for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, 152–159 (2014)

  17. Xiao, T., Li, H., Ouyang, W.: Learning deep feature representations with domain guided dropout for person re-identification. IEEE Conference on Computer Vision and Pattern Recognition, 1249–1258 (2016)

  18. Zheng, L., Zhang, H., Sun, S.: Person re-identification in the wild. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 3346–3355 (2017)

  19. Zheng, L., Bie, Z., Sun, Y.: MARS: a video benchmark for large-scale person re-Identification. In: European Conference on Computer Vision. Springer International Publishing, 868–884 (2016)

  20. Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: Joint detection and identification feature learning for person search. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)

  21. Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: End-to-end deep learning for person search. IEEE Conference on Computer Vision and Pattern Recognition (2016)

  22. Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10(1), 207–244 (2009)

    MATH  Google Scholar 

  23. Roth, P. M., Wohlhart, P., Hirzer, M.: Large scale metric learning from equivalence constraints. In: IEEE Conference on Computer Vision and Pattern Recognition, 2288–2295 (2012)

  24. Pedagadi, S., Orwell, J., Velastin, S.: Local fisher discriminant analysis for pedestrian re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, 3318–3325 (2013)

  25. Girshick, R., Donahue, J., Darrell, T.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, 580–587 (2014)

  26. Uijlings, J.R., Sande, K.E.A.V.D., Gevers, T.: Selective search for object recognition. Int. J. Comput. Vis. 104(2), 154–171 (2013)

    Article  Google Scholar 

  27. He, K., Zhang, X., Ren, S.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)

    Article  Google Scholar 

  28. Girshick, R.: Fast R-CNN. In: IEEE Conference on Computer Vision and Pattern Recognition, 1440–1448 (2015)

  29. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. (2018). arXiv:1804.02767

  30. Taigman, Y., Yang, M., Ranzato, M. A., Deepface: closing the gap to human-level performance in face verification. In: IEEE Conference on Computer Vision and Pattern Recognition, 1701–1708 (2014)

  31. Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: IEEE Conference on Computer Vision and Pattern Recognition, 1891–1898 (2014)

  32. Sun, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. Adv. Neural Inf. Process. Syst. 27, 1988–1996 (2014)

  33. Sun, Y., Wang, X., Tang, X.: Deeply learned face representations are sparse, selective, and robust. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, pp. 2892--2900 (2015). https://doi.org/10.1109/CVPR.2015.7298907

  34. Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, 815–823 (2015)

  35. Wu, X., He, R., Sun, Z.: A lightened CNN for deep face representation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 4 (2015)

  36. Huang, G. B., Ramesh, M., Berg, T.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report, 07–49, University of Massachusetts, Amherst

  37. Munasinghe, S., Fookes, C., Sridharan, S.: Human-level face verification with intra-personal factor analysis and deep face representation. IET Biometrics 7(5), 467–473 (2018)

    Article  Google Scholar 

  38. Göngör, F., Tutsoy, Ö.: Design and implementation of a facial character analysis algorithm for humanoid robots. Robotica 37, 1850–1866 (2019)

    Article  Google Scholar 

  39. Göngör, F., Tutsoy, O.: Eigenface based emotion analysis algorithm and implementation to humanoid robot. In: International Science and Academic Congress (2018)

  40. Cheng, K., Xu, F., Tao, F, et al.: Data-driven pedestrian re-identification based on hierarchical semantic representation. Concurr. Comput. Pract. Exp, e4403 (2017)

  41. Sathish, P.K., Balaji, S.: A complete person re-identification model using Kernel-PCA-based Gabor-filtered hybrid descriptors. Int. J. Multimed. Inf. Retrieval 1, 1–9 (2018)

    Google Scholar 

  42. Felzenszwalb, PF., Girshick, RB., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010). https://doi.org/10.1109/tpami.2009.167

    Article  Google Scholar 

  43. Luo, J., Wang, Y.: Real-time pedestrian detection method based on improved YOLOv3. J. Phys. Conf. Ser. 1453, 012149 (2020)

    Article  Google Scholar 

  44. Yi, Z., Yongliang, S., Jun, Z.: An improved tiny-yolov3 pedestrian detection algorithm. Optik Int. J. Light Electron Opt. (2019)

  45. Qu, H., Yuan, T., Sheng, Z., et al.: A pedestrian detection method based on YOLOv3 model and image enhanced by retinex. In: International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) (2018)

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61972097, Grant 61502105, in part by the Technology Guidance Project of Fujian Province under Grant 2017H0015, in part by the Natural Science Foundation of Fujian Province under Grant 2018J1798, in part by the University Production Project of Fujian Province under Grant 2017H6008, in part by the Fujian Collaborative Innovation Center for Big Data Application in Governments, and in part by the Fujian Engineering Research Center of Big Data Analysis and Processing. The authors also gratefully acknowledge the helpful comments and suggestions of the reviewers.

Author information

Authors and Affiliations

  1. Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing, College of Mathematics and Computer Science, Fuzhou University, Fuzhou, 350116, China

    Xiao Ke, Xinru Lin & Liyun Qin

  2. Key Laboratory of Spatial Data Mining and Information Sharing, Ministry of Education, Fuzhou, 350003, China

    Xiao Ke

Authors
  1. Xiao Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinru Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liyun Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Ke.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ke, X., Lin, X. & Qin, L. Lightweight convolutional neural network-based pedestrian detection and re-identification in multiple scenarios. Machine Vision and Applications 32, 46 (2021). https://doi.org/10.1007/s00138-021-01169-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01169-7

Keywords

Search

Navigation