Skip to main content
SpringerLink
Log in

A new deep model for family and non-family photo identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human trafficking is a global issue of the world and the problems related to human trafficking remain unsolved. This paper presents a new method for the identification of photos of different types of families and non-families such that the method can assist investigation team to find a solution to such issue. We believe that parts of human beings are the main resources for representing family and non-family photos. Based on this intuition, we propose to segment hair, head, cloth, torso, and skin regions from each human in input photos by exploring a self-correlation for human parsing method. This step results in region of interest (ROI). Motivated by ability of deep learning models in solving complex issues and special property of MobileNet, which is light weight model, we further explore MobileNetv2 for the identification of photos of different families and non-families by considering ROI as the input. For the experiment of this work, we consider a dataset of ten classes, which include five family classes, namely, Couple, Nuclear Family, Multi-Cultural Family, Father–Child, Mother–Child and five more non-family classes, namely, Male Friends, Female Friends, Mixed Friends, Male Celebrity, Female Celebrity. The results of the proposed method are demonstrated by testing on our dataset of family and non-family photos classification. Comparative results with the existing methods show that our proposed method outperforms existing methods in terms of classification rate and F-Score.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Almuashi M, Hashim SZM, Mohammad D, Alkawaz MH, Ali A (2017) Automated kinship verification and identification through human facial images: a survey. Multimed Tools Appl 76:265–307

    Article  Google Scholar 

  2. Andrade DOSD, Maia LF, Figueiredo HFD, Viana W, Trina F, Baptista CDS (2018) Photo annotation: a survey, Multimed Tools Appl 423–457

  3. Bastian BT, Jiji CV (2020) Integrated feature set using aggregate channel features and histogram of sparse codes for human detection. Multimed Tools and Appl 79:2931–2944

    Article  Google Scholar 

  4. Cai G, Hio C, Bermingham L, Lee K, Lee I (2014) Sequential pattern mining of geo-tagged photos with an arbitrary regions-of-interest detection method. Expert Syst Appl 41:3514–3526

    Article  Google Scholar 

  5. Dai Q, Carr P, Sigal L, Hoiem D (2015) Family member identification from photo collections. In Proceedings of WCACV, pp 982–989

  6. Dandekar AR, Nimbarte MS (2014) A survey: verification of family relationship from parents and child facial images. In: Proceedings of SCEECS

  7. Grouver A, Shivakumara P, Kaljahi MA, Chetty B, Pal U, Lu T, Kumar GH (2019) A spatial density and phase angle based correlation for multi-type family photo identification. In: Proceedings of ACPR, pp 76–89

  8. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications, pp 1–9

  9. Ioffe SA (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167

  10. Karpagavalli P, Ramprasad AV (2020) Automatic multiple human tracking using adaptive hybrid GMM based detection in a crowd. Multimed Tools Appl. https://doi.org/10.1007/s11042-019-08181-0

    Article  Google Scholar 

  11. Kingma DP (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  12. Li P, Xu Y, Wei Y, Yang Y (2019) Self-correction for human parsing. arXiv preprint arXiv:1910.09777

  13. Maryam AK, Shivakumara P, Tianping H, Hamid AJ, Rabha WI, Michale B, Tong L, Mohamad NBA (2019) A geometric and fractional entropy-based method for family photo classification. Expert Syst Appl. https://doi.org/10.1016/j.eswax.2019.100008

    Article  Google Scholar 

  14. Paracchini M, Marcon M, Villa F, Tubaro S (2020) Deep skin detection on low resolution gray scale images. Pattern Recognit. https://doi.org/10.1016/j.patrec.2019.12.021

    Article  Google Scholar 

  15. Stylianou A, Schreier J, Souvenir R, Pless R (2017) TraffickCam: crowdsourced and computer vision based approaches to fighting sex trafficking. In: Proceedings of AIPR

  16. Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: understanding the core of a family. IEEE Trans Multimed 17:1855–1867

    Article  Google Scholar 

  17. Robinson JP, Shao M, Wu LHY, Gillis T, Fu Y (2018) Visual kinship recognition of families in the wild. IEEE Trans PAMI 40:2624–2637

    Article  Google Scholar 

  18. Wang X, Guo G, Rohith M, Kambhamettu C (2015) Leveraging geometry and appearance cues for recognizing family photos. In: Proceedings of ICWAFG, pp 1–8

  19. Wang X, Guo G, Merler M, Codella NC, Rohith M, Smith JR, Kambhamettu C (2017) Leveraging multiple cues for recognizing family photos. Image Vis Comput 58:61–75

    Article  Google Scholar 

  20. Xia S, Pan H, Qin AK (2014) Face clustering in photo album. In: Proceedings of ICPR, pp 2844–2848

Download references

Acknowledgements

The authors of this paper thank to the anonymous reviewers for their constructive comments and suggestions, which help us to improve the quality and clarify of the proposed work. This work received the support from the Faculty Grant (GPF096A-2020, GPF096B-2020, GPF096C-2020), University of Malaya, Malaysia. This work is also supported by the National Science Foundation of China under Grant 61672273.

Author information

Authors and Affiliations

  1. Computer Vision and Pattern Recognition Unit, Indian Statistical Institute, Kolkata, India

    Tapan Karnik, Pinaki Nath Chowdhury & Umapada Pal

  2. Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Palaiahnakote Shivakumara & Nor Badrul Anuar

  3. National Key Lab for Novel Software Technology, Nanjing University, Nanjing, China

    Tong Lu

Authors
  1. Tapan Karnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Palaiahnakote Shivakumara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pinaki Nath Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Umapada Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nor Badrul Anuar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tong Lu.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karnik, T., Shivakumara, P., Chowdhury, P.N. et al. A new deep model for family and non-family photo identification. Multimed Tools Appl 81, 1765–1785 (2022). https://doi.org/10.1007/s11042-021-11631-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11631-3

Keywords

Search

Navigation