Skip to main content
Log in

Anthropometric salient points and convolutional neural network (CNN) for 3D human body classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, we introduce a 3D body shape biometric for recognizing a person as one of C possible individuals stored in a database in 3D free form (scatter of 3D data points) using a couple of canonical images (front and side) taken of that individual. The first step is to reconstruct the full body 3D shape model of the individual based on their frontal and profile silhouette images. This is done by using a 3D generic model that gets morphed in accordance with the canonical images of the individual. Starting with a small set of anthropometric interconnected ordered intrinsic control points residing on the silhouette boundary of the projections of the generic model onto the frontal and profile image spaces, corresponding control points on two canonical images of the person are automatically found. This imports equivalent saliency between the two sets. The positions of these control points on the canonical images are attained using deep convolutional neural networks (CNNs) that have been trained offline on a set of images of different individuals. Further equivalent saliencies between the projected points from the generic model and the canonical images are established through loop-subdivision. To personalize the generic model, points on the generic model are morphed to be consistent with their equivalent points on the canonical images. The 3D reconstruction yields sub-resolution errors when tested on the CAESAR data set with 700 different individuals. Classification based on the error between salient points with identical anthropometric meaning residing on nested sets of boundaries in the frontal and side projections, achieves an accuracy of 96%. This is to be compared to an accuracy of 72% when using KNN nearest distance point classification between test and base models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

N/A

References

  1. Ahmed E, Jones M, Marks TK (2015) "an improved deep learning architecture for person re-identification," 2015 IEEE conference on computer vision and pattern recognition (CVPR). MA, Boston, pp 3908–3916. https://doi.org/10.1109/CVPR.2015.7299016

  2. Aquino G, … Zacarias A (2020) Novel nonlinear hypothesis for the Delta parallel robot modeling. IEEE Access 8:46324–46334. https://doi.org/10.1109/ACCESS.2020.2979141

    Article  Google Scholar 

  3. H. Bay, and T. Tuytelaars and L.J. Van Gool, SURF: Speeded Up Robust Features, European Conference on Computer Vision, 2006 pp. 404–417.

  4. Y. Bergeon, I. Hadda, V. Křivánek, J. Motsch and A. Štefek, "Low cost 3D mapping for indoor navigation," Int Conf Military Technol (ICMT) 2015, Brno, 2015, pp. 1–5, https://doi.org/10.1109/MILTECHS.2015.7153749.

  5. Brunelli R, Falavigna D (1995) Person identification using multiple cues. IEEE Trans Pattern Anal Mach Intell 17(10):955–966

    Article  Google Scholar 

  6. V. Bushaev, “Adam - latest trends in deep learning optimization.,” Medium, 24-Oct-2018. [Online]. Available: https://towardsdatascience.com/adam-latest-trends-in-deep-learning-optimization-6be9a291375c. [Accessed: 22-Apr-2021].

  7. Chen Y, Cheng L, Li M, Wang J, Tong L, Yang K (2014) Multiscale grid method for detection and reconstruction of building roofs from airborne LiDAR data. IEEE J Selected Topics Appl Earth Observ Remote Sensing 7(10):4081–4094. https://doi.org/10.1109/JSTARS.2014.2306003

    Article  Google Scholar 

  8. Chiang H, Chen M, Huang Y (2019) Wavelet-based EEG processing for epilepsy detection using fuzzy entropy and associative petri net. IEEE Access 7:103255–103262. https://doi.org/10.1109/ACCESS.2019.2929266

    Article  Google Scholar 

  9. Collins RT, Gross R, Shi J (2002) Silhouette-based human identification from body shape and gait. Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition, Washington, DC, USA, pp 366–371. https://doi.org/10.1109/AFGR.2002.1004181

  10. Cootes TF, Edwards GJ, Taylor CJ et al (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell (TPAMI) 23(6):681–685

    Article  Google Scholar 

  11. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models-their training and application. Comp Vision Image Underst (CVIU) 61(1):38–59

    Article  Google Scholar 

  12. CS231n Convolutional Neural Networks for Visual Recognition. [Online]. Available: https://cs231n.github.io/convolutional-networks/. [Accessed: 22-Apr-2021].

  13. de Jesus Rubio J (Dec. 2009) SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17(6):1296–1309. https://doi.org/10.1109/TFUZZ.2009.2029569

    Article  Google Scholar 

  14. Elias I, Rubio JJ, Martinez DI, Vargas TM, Garcia V, Mujica-Vargas D, Meda-Campaña JA, Pacheco J, Gutierrez GJ, Zacarias A (2020) Genetic algorithm with radial basis mapping network for the electricity consumption modeling. Appl Sci 10:4239

    Article  Google Scholar 

  15. Gheissari, N. & Sebastian, Thomas & Hartley, Richard. (2006). Person Reidentification Using Spatiotemporal Appearance. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2. 1528–1535. https://doi.org/10.1109/CVPR.2006.223.

  16. A.Godil, P. Grother, and S. Ressler, "Human identification from body shape," in Proc. Fourth Int'l Conf. on 3-D Digital Imaging and Modeling (3DIM'03), 2003, pp. 386–392.

  17. Green RD, Guan L (2004) Quantifying and recognizing human movement patterns from monocular video images – part II: applications to biometrics. IEEE Tran CSVT 14(2):191–198

    Google Scholar 

  18. L. Gu and T. Kanade, "A generative shape regularization model for robust face alignment", European Conference on Computer Vision (ECCV), pp. 413–426, 2008.

  19. Z. He, M. Kan, J. Zhang, X. Chen and S. Shan, "A Fully End-to-End Cascaded CNN for Facial Landmark Detection," 2017 12th IEEE international conference on Automatic Face & Gesture Recognition (FG 2017), Washington, DC, 2017, pp. 200–207. https://doi.org/10.1109/FG.2017.33.

  20. Hernández G, Zamora E, Sossa H, Téllez G, Furlán F (accepted/in press). Hybrid neural networks for big data classification. Neurocomputing. https://doi.org/10.1016/j.neucom.2019.08.095

  21. A. Kanazawa, M. J. Black, D. W. Jacobs and J. Malik, "End-to-end recovery of human shape and pose", IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7122–7131, 2018.

  22. Kim H, Lee K, Lee D, Baek N (2019) "3D reconstruction of leg bones from X-ray images using CNN-based feature analysis," 2019 international conference on information and communication technology convergence (ICTC). Jeju Island, Korea (South), pp 669–672. https://doi.org/10.1109/ICTC46691.2019.8939984

    Book  Google Scholar 

  23. Lanitis A, Stylianou G (2009) Isualizing the 3D structure of medical objects based on 2D data. 2009 9th Int Conf Inform Technol Appl Biomed, Larnaca:1–4. https://doi.org/10.1109/ITAB.2009.5394337

  24. Lee B, Tian L-F, Ping C, Mo H-Q, Mao Z-Y (2005) A fast accurate 3D surface reconstruction method of medical image based on modularization. 2005 Int Conf Mach Learn Cybern, Guangzhou, China 8:4942–4945. https://doi.org/10.1109/ICMLC.2005.1527813

    Article  Google Scholar 

  25. Li C, Cohen F (2020) In-home application (app) for 3D virtual garment fitting dressing room. Springer-Verlag, Journal of Multimedia tools and Applications. https://doi.org/10.1007/s11042-020-09989

  26. X. Li and X. Li, "Human body dimensions extraction from 3D scan data," 2010 Int Conference Intell Computation Technol Autom, Changsha, 2010, pp. 441–444. https://doi.org/10.1109/ICICTA.2010.849.

  27. Yueh-Ling Lin and M. J. Wang, "Constructing 3D human model from 2D images," 2010 IEEE 17Th International Conference on Industrial Engineering and Engineering Management, Xiamen, 2010, pp. 1902–1906. https://doi.org/10.1109/ICIEEM.2010.5645897.

  28. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  29. N. Luncher and J. Zelek, “Deep Learning Whole Body Point Cloud Scans from a Single Depth Map,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018.

  30. Matthews I, Baker S (2004) Active appearance models revisited. Int J Comp Vision (IJCV) 60(2):135–164

    Article  Google Scholar 

  31. I. Mebsout, “Convolutional Neural Networks' mathematics,” Medium, 03-Oct-2020. [Online]. Available: https://towardsdatascience.com/convolutional-neural-networks-mathematics-1beb3e6447c0. [Accessed: 22-Apr-2021].

  32. Meda-Campaña JA (2018) On the estimation and control of nonlinear systems with parametric uncertainties and Noisy outputs. IEEE Access 6:31968–31973. https://doi.org/10.1109/ACCESS.2018.2846483

    Article  Google Scholar 

  33. G. Medioni et al., “Identifying non-cooperative subjects at a distance using face images and inferred three-dimensional face models,” IEEE Trans Syst, Man, Cybern A, v. 39, n. 1, Jan 2009, pp. 12–24.

  34. “MPII Human Shape,” MPII Human Shape. [Online]. Available: http://humanshape.mpi-inf.mpg.de/. [Accessed: 03-Jun-2020].

  35. Munaro, Matteo & Fossati, Andrea & Basso, Alberto & Menegatti, Emanuele & Van Gool, Luc. (2014). One-Shot Person Re-Identification with a Consumer Depth Camera. https://doi.org/10.1007/978-1-4471-6296-4_8.

  36. G. Nishad, “Facial Keypoint Detection: Detect relevant features of face in a go using CNN & your own dataset,” Medium, 24-Mar-2019. [Online]. Available: https://towardsdatascience.com/facial-keypoint-detection-detect-relevant-features-of-face-in-a-go-using-cnn-your-own-dataset-e09cf359c2bc. [Accessed: 03-Jun-2020].

  37. Ober, D., Neugebauer, S., Sallee, P.: Training and feature-reduction techniques for human identification using anthropometry. In: Biometrics: Theory Applications and Systems (BTAS), 2010 Fourth IEEE International Conference on, pp. 1–8 (2010)

  38. H. Pang, J. Li, J. Peng, X. Zhong and X. Cai, "Personalized full-body reconstruction based on single kinect," 2015 8th International Congress on Image and Signal Processing (CISP), Shenyang, 2015, pp. 979–983. https://doi.org/10.1109/CISP.2015.7408021.

  39. Pavlakos G, Zhu L, Zhou X, Daniilidis K (2018) Learning to estimate 3D human pose and shape from a single color image. Proceedings of the IEEE conference on computer vision and pattern recognition pp 459-468. https://doi.org/10.1109/CVPR.2018.00055.

  40. B. M. Smith, V. Chari, A. Agrawal, J. M. Rehg and R. Sever, "Towards Accurate 3D Human Body Reconstruction from Silhouettes," 2019 International conference on 3D vision (3DV), Québec City, QC, Canada, 2019, pp. 279–288. https://doi.org/10.1109/3DV.2019.00039.

  41. Y. Sun, X. Wang and X. Tang, "Deep Convolutional Network Cascade for Facial Point Detection," 2013 IEEE conference on computer vision and pattern recognition, Portland, OR, 2013, pp. 3476–3483. https://doi.org/10.1109/CVPR.2013.446.

  42. H. Temiz, B. Gökberk and L. Akarun, "Multi-view Reconstruction of 3D Human Pose with Procrustes Analysis," 2019 Ninth international conference on image processing theory, tools and applications (IPTA), Istanbul, Turkey, 2019, pp. 1–5. https://doi.org/10.1109/IPTA.2019.8936071.

  43. Wang WA, Lin M-C (2010) A fast method in reconstruction 3D computed tomography medical images. 2010 IEEE 17Th Int Conf Industrial Eng Eng Manag, Xiamen:1877–1881. https://doi.org/10.1109/ICIEEM.2010.5645895

  44. Z. Wang, M. Sun, G. Ren and F. Meng, "High-Resolution Textured 3D Human Modeling from Images," 2010 International conference on multimedia technology, Ningbo, 2010, pp. 1–4. https://doi.org/10.1109/ICMULT.2010.5631431.

  45. Chengze Yang and Zhiquan Cheng, "3D human reconstruction from multi-image," 2011 International Conference on Multimedia Technology, Hangzhou, 2011, pp. 6–11. https://doi.org/10.1109/ICMT.2011.6001814.

  46. Yang S, Fan Y (2017) IEEE Int Conf Consumer Electron - Taiwan (ICCE-TW), Taipei 2017:127–128. https://doi.org/10.1109/ICCE-China.2017.7991028

    Article  Google Scholar 

  47. Zhang J, Liu M, Shen D (2017) Detecting anatomical landmarks from limited medical imaging data using two-stage task-oriented deep neural networks. IEEE Trans Image Process 26(10):4753–4764. https://doi.org/10.1109/TIP.2017.2721106

    Article  MathSciNet  Google Scholar 

  48. Zhong Q, Zhao J (2014) "research on 3D reconstruction for robot based on SIFT feature," 2014 IEEE workshop on advanced research and Technology in Industry Applications (WARTIA). Ottawa, ON, pp 976–979. https://doi.org/10.1109/WARTIA.2014.6976437

Download references

Code availability

N/A

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Semanti Basu.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Basu, S., Li, C. & Cohen, F. Anthropometric salient points and convolutional neural network (CNN) for 3D human body classification. Multimed Tools Appl 81, 10497–10527 (2022). https://doi.org/10.1007/s11042-022-12284-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12284-6

Keywords

Navigation