Skip to main content
SpringerLink
Log in

An industrial portrait background removal solution based on knowledge infusion

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Background removal of an identity (ID) picture consists in separating the foreground (face, body, hair and clothes) from the background of the image. It is a necessary groundwork for all modern identity documents that also has many benefits for improving ID security. State of the art image processing techniques encountered several segmentation issues and offer only partial solutions. It is due to the presence of erratic components like hairs, poor contrast, luminosity variation, shadow, color overlap between clothes and background. In this paper, a knowledge infused approach is proposed that hybridizes smart image processing tasks and prior knowledge. The research is based on a divide and conquer strategy aiming at simulating the sequential attention of human when performing a manual segmentation. Knowledge is infused by considering the spatial relation between anatomic elements of the ID image (face feature, forehead, body and hair) as well as their “signal properties”. The process consists in first determining a convex hull around the person’s body including all the foreground while keeping very close to the contour between the background and the foreground. Then, a body map generated from biometric analysis associated to an automatic grab cut process is applied to reach a finer segmentation. Finally, a heuristic-based post-processing step consisting in correcting potential hair and fine boundary issues leads to the final segmentation. Experimental results show that the newly proposed architecture achieves better performances than tested current state-of-the-art methodologies including active contours, generalist popular deep learning techniques, and also two other ones considered as the smartest for portrait segmentation. This new technology has been adopted by an international company as its industrial ID foreground solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Abbas Q, Celebi ME, García IF (2011) Hair removal methods: a comparative study for dermoscopy images. Biomedical Signal Processing and Control 6(4):395–404

    Article  Google Scholar 

  2. Amar M, Harba R, Douzi H, Ros F, El Hajji M, Riad R, Gourrame K (2016) A jnd model using a texture-edge selector based on faber-schauder wavelet lifting scheme. In: International Conference on Image and Signal Processing. Springer, pp 328–336

  3. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans on Pattern Analysis and Machine Intelligence 39(12):2481–2495

    Article  Google Scholar 

  4. Bouwmans T (2014) Traditional and recent approaches in background modeling for foreground detection: An overview. Computer science review 11:31–66

    Article  Google Scholar 

  5. Bresenham JE (1965) Algorithm for computer control of a digital plotter. IBM Systems Journal 4(1):25–30

    Article  Google Scholar 

  6. Çeliktutan O, Ulukaya S (2013) Sankur B (2013) A comparative study of face landmarking techniques. EURASIP Journal on Image and Video Processing 1:13

    Article  Google Scholar 

  7. Chan TF, Vese LA (2001) Active contours without edges. IEEE Transactions on Image Processing 10(2):266–277

    Article  Google Scholar 

  8. Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans on Pattern Analysis and Machine Intelligence 40(4):834–848

    Article  Google Scholar 

  9. Cootes T, Taylor C, Cooper D, Graham J (1995) Active shape models-their training and application. Computer Vision and Image Understanding 61(1):38–59

    Article  Google Scholar 

  10. Ferrara M, Franco A, Maio D (2012) A multi-classifier approach to face image segmentation for travel documents. Expert Systems with Applications 39(9):8452–8466

    Article  Google Scholar 

  11. Jayaram M, Fleyeh H (2016) Convex hulls in image processing: A scoping review. American Journal of Intelligent Systems 6(2):48–58

    Google Scholar 

  12. Julian P, Dehais C, Lauze F, Charvillat V, Bartoli A, Choukroun A (2010) Automatic hair detection in the wild. In: 2010 20th International Conference on Pattern Recognition. IEEE, pp 4617–4620

  13. Kumar A, Kaur A, Kumar M (2019) Face detection techniques: a review. Artificial Intelligence Review 52(2):927–948

    Article  Google Scholar 

  14. Li Y, Sun J, Tang CK, Shum HY (2004) Lazy snapping. ACM Trans Graph 23(3):303–308

    Article  Google Scholar 

  15. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440

  16. Milborrow S, Nicolls F (2008) Locating facial features with an extended active shape model. In: Forsyth D, Torr P, Zisserman A (eds) Computer Vision - ECCV 2008. Springer, Heidelberg, pp 504–513

    Chapter  Google Scholar 

  17. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics 9(1):62–66

    Article  Google Scholar 

  18. Park H, Sjosund L, Yoo Y, Monet N, Bang J, Kwak N (2020) Sinet: Extreme lightweight portrait segmentation networks with spatial squeeze module and information blocking decoder. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

  19. Riad R, Harba R, Douzi H, Ros F, Elhajji M (2016) Robust fourier watermarking for id images on smart card plastic supports. Advances In Electrical and Computer Engineering 16(4):23–30

    Article  Google Scholar 

  20. Rodriguez A, Laio A (2014) Clustering by fast search and find of density peaks. Science 344(6191):1492–1496

    Article  Google Scholar 

  21. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  22. Rother C, Kolmogorov V, Blake A (2004) Grabcut -interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (SIGGRAPH)

  23. Sengupta S, Jayaram V, Curless B, Seitz SM, Kemelmacher-Shlizerman I (2020) Background matting: The world is your green screen. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  24. Shen X, Hertzmann A, Jia J, Paris S, Price B, Shechtman E, Sachs I (2016) Automatic portrait segmentation for image stylization. Computer Graphics Forum, Wiley Online Library 35:93–102

    Article  Google Scholar 

  25. Shi D, Yao Y, Yu W (2017) Comparison of preoperative hair removal methods for the reduction of surgical site infections: a meta-analysis. Journal of clinical nursing 26(19–20):2907–2914

    Article  Google Scholar 

  26. Sklansky J (1982) Finding the convex hull of a simple polygon. Pattern Recognition Letters 1(2):79–83

    Article  Google Scholar 

  27. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271), pp 839–846

  28. Wang Z, Wang E, Zhu Y (2020) Image segmentation evaluation: a survey of methods. Artificial Intelligence Review pp 1–38

  29. Xie X, Niu J, Liu X, Chen (2020) A survey on domain knowledge powered deep learning for med. image analysis. arXiv preprint arXiv:200412150

  30. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Learning a discriminative feature network for semantic segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1857–1866

  31. Zhang SH, Dong X, Li H, Li R, Yang YL (2019) Portraitnet: Real-time portrait segmentation network for mobile device. Computers & Graphics 80:104–113

    Article  Google Scholar 

  32. Zhang Z, Zhang X, Peng C, Xue X, Sun J (2018) Exfuse: Enhancing feature fusion for semantic segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 269–284

  33. Zuiderveld K (1994) Contrast Limited Adaptive Histogram Equalization. Academic Press Professional Inc, USA, pp 474–485

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ERMAM-FPO, Ibn Zhor University, Agadir, Morocco

    Rabia Riad

  2. PRISME, University of Orlans, Orléans, France

    Frédéric Ros & Rachid Harba

  3. IRF-SIC, Ibn Zhor University, Agadir, Morocco

    Mohamed El hajji

Authors
  1. Rabia Riad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frédéric Ros
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed El hajji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rachid Harba
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rabia Riad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Riad, R., Ros, F., hajji, M.E. et al. An industrial portrait background removal solution based on knowledge infusion. Appl Intell 52, 11592–11605 (2022). https://doi.org/10.1007/s10489-021-03099-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03099-3

Keywords

Search

Navigation