Skip to main content
SpringerLink
Log in

Automatic localization and annotation of facial features using machine learning techniques

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Content-based image retrieval (CBIR) systems traditionally find images within a database that are similar to query image using low level features, such as colour histograms. However, this requires a user to provide an image to the system. It is easier for a user to query the CBIR system using search terms which requires the image content to be described by semantic labels. However, finding a relationship between the image features and semantic labels is a challenging problem to solve. This paper aims to discover semantic labels for facial features for use in a face image retrieval system. Face image retrieval traditionally uses global face-image information to determine similarity between images. However little has been done in the field of face image retrieval to use local face-features and semantic labelling. Our work aims to develop a clustering method for the discovery of semantic labels of face-features. We also present a machine learning based face-feature localization mechanism which we show has promise in providing accurate localization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Notes

  1. The ffs and d notation is not included in the diagram for clarity.

  2. Known as Stratified K-fold cross validation.

  3. Detailed in Sect. 2.2.1.

References

  • Ai H, Liang L, Xiao X, Xu G (2001) Face indexing and retrieval in personal digital album. In: Proceedings of 2nd IEEE Pacific rim conference on multimedia, vol 2195. Springer, London, pp 48–54

  • Asthana A, Goecke R, Quadrianto N, Gedeon T (2009) Learning based automatic face annotation for arbitrary poses and expressions from frontal images only. In: Proceedings of IEEE conference on computer vis and pattern recognit. pp 1635–1642

  • Belhumeur PN, Hespanha J, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  • Cai D, He X, Han J, Zhang H-J (2006) Orthogonal laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614

    Article  Google Scholar 

  • Carneiro G, Chan AB, Moreno PJ, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans Pattern Anal Mach Intell 29(3):394–410

    Article  Google Scholar 

  • Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm

  • Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):1–60

    Article  Google Scholar 

  • Gao Y, Qi Y (2005) Robust visual similarity retrieval in single model face databases. Pattern Recognit 38(7):1009–1020

    Article  Google Scholar 

  • Hanif SM, Prevost L, Belaroussi R, Milgram M (2008) Real-time facial feature localization by combining space displacement neural networks. Pattern Recognit Lett 29(8):1094–1104

    Article  Google Scholar 

  • He X, Cai D, Han J (2008) Learning a maximum margin subspace for image retrieval. IEEE Trans Knowl Data Eng 20(2):189–201

    Article  Google Scholar 

  • Heisele B, Serre T, Poggio T (2007) A component-based framework for face detection and identification. Int J Comput Vis 74(2):167–181

    Article  Google Scholar 

  • Heisele B, Serre T, Pontil M, Poggio T (2001) Component-based face detection. In: Proceedings of IEEE conference on computer visual and pattern recognition, vol 1. IEEE Computer Society, Los Alamitos, CA, USA, pp 657–662

  • Hsu CW, Chang CC, Lin C-J (2005) A practical guide to support vector classification

  • Hsu RL, Abdel Mottaleb M, Jain AK (2002) Face detection in color images. IEEE Trans Pattern Anal Mach Intell 24(5):696–706

    Article  Google Scholar 

  • Hsu RL, Jain AK (2002) Semantic face matching. In: Proceedings of IEEE international conference on multimedia and expo, vol 2, pp 145–148

  • Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international conference on neural networks, vol 2. IEEE, pp 985–990

  • Ito H, Koshimizu H (2006) Face image retrieval and annotation based on two latent semantic spaces in FIARS. In:Proceedings of 8th IEEE international multimedia. IEEE Computer Society, Washington, DC, USA, pp 831–836

  • Lai PJ, Wang JH (2003) Facial image database for law enforcement application: an implementation. In: Proceedings of 37th IEEE international conference on security technology. Taipei, Taiwan, pp 285–289

  • Levenberg K (1944) A method for the solution of certain problems in least squares. Quart Appl Math 2:164–168

    MathSciNet  MATH  Google Scholar 

  • Li CM, Li YS, Zhuang QD, Xiao ZZ (2004) The face localization and regional features extraction. In: Proceedings of international conference on machine learn and cybernetics, vol 6. pp 3835–3840

  • Lin SH, Kung SY, Lin LJ (1997) Face recognition/detection by probabilistic decision-based neural network. IEEE Trans Neural Netw 8(1):114–132

    Article  Google Scholar 

  • Lu Y, Guo H, Feldkamp L (1998) Robust neural learning from unbalanced data samples. In: Proceedings of IEEE international joint conference on neural networks, vol 3. Anchorage, Alaska, USA, pp 1816–1821

  • MacQueen JB (1967) Some methods for classification and analysis of multivariate observations. In: Cam LL, Neyman J (eds.) Proceedings of fifth Berkeley symposium on math statist and prob, vol 1. University of California, pp 281–297

  • Nguyen D, Widrow B (1990) Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In: Proceedings of international joint conference on neural networks, vol 3, pp 21–26

  • Rasiwasia N, Moreno PL, Vasconcelos N (2007) Bridging the gap: query by semantic example. IEEE Trans Multimed 9(5):923–938

    Article  Google Scholar 

  • Sahbi H (2008) A particular gaussian mixture model for clustering and its application to image retrieval. Soft Comput 12(7):667–676

    Article  Google Scholar 

  • Sheikholeslami G, Chang W, Zhang A (2002) SemQuery: semantic clustering and querying on heterogeneous features for visual data. IEEE Trans Knowl Data Eng 14(5):988–1002

    Article  Google Scholar 

  • Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  • Sridharan K, Nayak S, Chikkerur S, Govindaraju V (2005) A probabilistic approach to semantic face retrieval system. In: Kanade T, Jain A, Ratha NK (eds) Audio- and video-based biometric person authentication, vol 3546 of Lecture Notes in Computer Science. Springer, Berlin, pp 977–986

  • Tan SC, Rao MVC, Lim CP (2008) A hybrid neural network classifier combining ordered fuzzy artmap and the dynamic decay adjustment algorithm. Soft Comput 12(8):765–775

    Article  Google Scholar 

  • Tesic J, Smith JR (2006) Semantic labeling of multimedia content clusters. In: Proceedings of IEEE international conference on multimedia and expo. pp 1493–1496

  • Turk M, Pentland A (1991) Eigenfaces for recognition. Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  • Wang DH (2006) ELM-based multiple classifier systems. In: Proceedings of international conference on control, sutom, robot and vis. pp 1–5

  • Wang DH, Kim Y-S, Park SC, Lee CS, Han YK (2007) Learning based neural similarity metrics for multimedia data mining. Soft Comput 11(4):335–340

    Article  Google Scholar 

  • Wang DH, Ma XH (2005) A hybrid image retrieval system with user’s relevance feedback using neurocomputing. Informatica 29:271–279

    Google Scholar 

  • Wu B, Ai H, Huang C (2004a) Facial Image Retrieval based on Demographic Classification. In: Proceedings of 17th international conference on pattern recognition, vol 3. pp 914–917

  • Wu TF, Lin CJ, Weng RC (2004b) Probability estimates for multi-class classification by pairwise coupling. Machine Learn Res 5:975–1005

    MathSciNet  Google Scholar 

  • Xu R, Wunsch DI (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678

    Article  Google Scholar 

  • Yang M-H, Kriegman DJ, Ahuja N (2002) Detecting Faces in Images: A Survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58

    Article  Google Scholar 

  • Zhou H, Yuan Y, Sadka AH (2008) Application of semantic features in face recognition. Pattern Recogn 41(10):3251–3256

    Article  MATH  Google Scholar 

  • Zuo F, de With PH (2008) Facial feature extraction by a cascade of model-based algorithms. Signal Process Image Commun 23(3):194–211

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, VIC, 3086, Australia

    Paul C. Conilione & Dianhui Wang

Authors
  1. Paul C. Conilione
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dianhui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dianhui Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Conilione, P.C., Wang, D. Automatic localization and annotation of facial features using machine learning techniques. Soft Comput 15, 1231–1245 (2011). https://doi.org/10.1007/s00500-010-0586-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-010-0586-y

Keywords

Search

Navigation