Skip to main content
Log in

FaceCAPTCHA: a CAPTCHA that identifies the gender of face images unrecognized by existing gender classifiers

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Computers tend to fail to classify human faces by gender, especially upon changes in viewpoint or upon occlusion that make it more difficult to extract the necessary image features. In contrast, humans are good at identifying gender but have difficulties in dealing with a large number of images. Accounting for this gap, we proposed FaceCAPTCHA, a novel image-based CAPTCHA that asks users to identify the gender of face images whose gender cannot be recognized by computers (gender-indiscernible faces). By converting the manual gender classification task into a CAPTCHA test, FaceCAPTCHA was designed to not only continuously identify the gender of gender-indiscernible faces but also differentiate between humans and computers and generate new test images. Our user studies showed that FaceCAPTCHA reliably identifies gender-indiscernible faces. A single eight-image FaceCAPTCHA test was completed in 12.41 s on average with a human success rate of 86.51 %, which can be further increased by filtering error-prone test images. In contrast, the probability of passing a FaceCAPTCHA test by random guessing was 0.006 %. We could therefore conclude that FaceCAPTCHA is robust against malicious attacks and easy enough for practical use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://vasc.ri.cmu.edu/idb/html/face/profile_images/index.html

  2. http://www.flickr.com

  3. http://www.itl.nist.gov/iad/humanid/feret

  4. http://vis-www.cs.umass.edu/lfw

  5. http://www.mturk.com

  6. http://www.sheffield.ac.uk/eee/research/iel/research/face

  7. This system is no longer available since the Web site HotCAPTCHA.com was closed down in 2009.

  8. http://www.archer-group.com/2010/development/we-dont-talk-anymore

  9. http://www.creativecommons.org

  10. http://press.liacs.nl/mirflickr

  11. http://developers.face.com

  12. http://code.google.com/appengine

References

  1. Alexandre LA (2010) Gender recognition: a multiscale decision fusion approach. Pattern Recognit Lett 31(11):1422–1427. doi:10.1016/j.patrec.2010.02.010

    Article  Google Scholar 

  2. Bekios-Calfa J, Buenaposada JM, Baumela L (2011) Revisiting linear discriminant techniques in gender recognition. IEEE Trans Pattern Anal Mach Intell 33(4):858–864

    Article  Google Scholar 

  3. Brown E, Perrett DI (1993) What gives a face its gender. Perception 22:829–840

    Article  Google Scholar 

  4. Chellapilla K, Larson K, Simard P, Czerwinski M (2005) Designing human friendly human interaction proofs (HIPs). Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Portland, Oregon, USA

  5. Chew M, Tygar J (2004) Image recognition CAPTCHAs. Paper presented at the Proceedings of the 7th international information security conference, Palo Alto, CA, USA

  6. Chow R, Golle P, Jakobsson M, Wang L, Wang XF (2008) Making captchas clickable. Paper presented at the Proceedings of the 9th workshop on Mobile computing systems and applications, Napa Valley, CA, USA

  7. Elson J, Douceur JR, Howell J, Saul J (2007) Asirra: a CAPTCHA that exploits interest-aligned manual image categorization. Paper presented at the Proceedings of the 14th ACM conference on Computer and communications security, Alexandria, VA, USA

  8. Gonçalves D, Jesus R, Correia N (2008) A gesture based game for image tagging. Paper presented at the CHI ’08 extended abstracts on Human factors in computing systems, Florence, Italy

  9. Gossweiler R, Kamvar M, Baluja S (2009) What’s up CAPTCHA?: a CAPTCHA based on image orientation. Paper presented at the Proceedings of the 18th international conference on World wide web, Madrid, Spain

  10. Guanglei S, Wenze L (2012) Face detection in complex background using AdaBoost algorithm. In: Internet Computing for Science and Engineering (ICICSE), 2012 Sixth International Conference on, 21–23 April 2012. pp 149–154. doi:10.1109/ICICSE.2012.23

  11. Guo G, Mu G, Fu Y (2010) Gender from body: a biologically-inspired approach with manifold learning. Computer vision - ACCV 2009. Lect Notes Comput Sci 5996:236–245

    Article  Google Scholar 

  12. Kalsoom S, Ziauddin S, Abbasi AR (2012) An image-based CAPTCHA scheme exploiting human appearance characteristics. KSII Trans Internet Inf 6(2):734–750. doi:10.3837/tiis.2012.02.017

    Google Scholar 

  13. Khot RA, Srinathan K (2009) iCAPTCHA: Image Tagging for Free. Paper presented at the Proceedings of the 3rd international conference on Usable software and interface design, Hyderabad, India

  14. Kim JW, Chung WK, Cho HG (2010) A new image-based CAPTCHA using the orientation of the polygonally cropped sub-images. Vis Comput 26(6):1135–1143

    Google Scholar 

  15. Kumar N, Berg A, Belhumeur PN, Nayar S (2011) Describable visual attributes for face verification and image search. IEEE Trans Pattern Anal Mach Intell 33(10):1962–1977. doi:10.1109/TPAMI.2011.48

    Article  Google Scholar 

  16. Lian XC, Lu BL (2009) Gender classification by combining facial and hair information. Advances in neuro-information processing. Lect Notes Comput Sci 5507:647–654

    Article  Google Scholar 

  17. Marlow C, Naaman M, Boyd D, Davis M (2006) HT06, tagging paper, taxonomy, Flickr, academic article, to read. Paper presented at the Proceedings of the 17th conference on Hypertext and hypermedia, Odense, Denmark

  18. Misra D, Gaj K (2006) Face recognition CAPTCHAs. Paper presented at the Advanced International Conference on Telecommunications and International Conference on Internet and Web Applications and Services, Guadeloupe, French Caribbean

  19. Morrison D, Marchand-Maillet S, Bruno É (2009) TagCAPTCHA: annotating images with CAPTCHAs. Paper presented at the Proceedings of the international conference on Multimedia, Firenze, Italy

  20. Naaman M, Harada S, Wang QY, Garcia-Molina H, Paepcke A (2004) Context data in geo-referenced digital photo collections. Paper presented at the Proceedings of the 12th annual ACM international conference on Multimedia, New York, NY, USA

  21. Quinn AJ, Bederson BB (2011) Human computation: a survey and taxonomy of a growing field. Paper presented at the Proceedings of the 2011 annual conference on Human factors in computing systems, Vancouver, BC

  22. Rui Y, Liu Z (2004) ARTiFACIAL: automated reverse Turing test using FACIAL features. Multimedia Systems 9(6):493–502

    Google Scholar 

  23. Russell B, Torralba A, Murphy K, Freeman W (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77(1–3):157–173. doi:10.1007/s11263-007-0090-8

    Article  Google Scholar 

  24. Shakhnarovich G, Viola PA, Moghaddam B (2002) A unified learning framework for real time face detection and classification. In: Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on, 20–21 May 2002. pp 14–21. doi:10.1109/AFGR.2002.1004124

  25. Shan C (2012) Learning local binary patterns for gender classification on real-world face images. Pattern Recognit Lett 33(4):431–437. doi:10.1016/j.patrec.2011.05.016

    Article  Google Scholar 

  26. Thaler S, Siorpaes K, Mear D, Simperl E, Goodman C (2011) Seafish: a game for collaborative and visual image annotation and interlinking. The semanic web: research and applications. Lect Notes Comput Sci 6644:466–470

    Article  Google Scholar 

  27. Toews M, Arbel T (2009) Detection, localization, and sex classification of faces from arbitrary viewpoints and under occlusion. IEEE Trans Pattern Anal Mach Intell 31(9):1567–1581

    Article  Google Scholar 

  28. Ueki K, Komatsu H, Imaizumi S, Kaneko K, Sekine N, Katto J, Kobayashi T (2004) A method of gender classification by integrating facial, hairstyle, and clothing images. Paper presented at the Proceedings of the 17th international conference on Pattern recognition, Cambridge, England, UK

  29. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  30. von Ahn L, Blum M, Hopper NJ, Langford J (2003) CAPTCHA: using hard AI problems for security. Advances in cryptology - Eurocrypt 2003. Lect Notes Comput Sci 2656:294–311

    Article  Google Scholar 

  31. von Ahn L, Dabbish L (2004) Labeling images with a computer game. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Vienna, Austria

  32. von Ahn L, Ginosar S, Kedia M, Blum M (2007) Improving image search with PHETCH. Paper presented at the IEEE international conference on Acoustics, speech and signal processing, Honolulu, Hawaii, USA

  33. von Ahn L, Liu R, Blum M (2006) Peekaboom: a game for locating objects in images. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Montréal, Canada

  34. von Ahn L, Maurer B, McMillen C, Abraham D, Blum M (2008) reCAPTCHA: human-based character recognition via web security measures. Science 321(5895):1465–1468

    Article  MATH  MathSciNet  Google Scholar 

  35. Wolf L, Hassner T, Taigman Y (2010) Similarity scores based on background samples. Computer Vision-ACCV 2009. Lect Notes Comput Sci 5995:88–97

    Article  Google Scholar 

  36. Xiangxin Z, Ramanan D (2012) Face detection, pose estimation, and landmark localization in the wild. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 16–21 June. pp 2879–2886. doi:10.1109/CVPR.2012.6248014

  37. Zheng J, Lu B-L (2011) A support vector machine classifier with automatic confidence and its application to gender classification. Neurocomputing 74(11):1926–1935. doi:10.1016/j.neucom.2010.07.032

    Article  Google Scholar 

  38. Zhu BB, Yan J, Li Q, Yang C, Liu J, Xu N, Yi M, Cai K (2010) Attacks and design of image recognition CAPTCHAs. Paper presented at the Proceedings of the 17th ACM conference on Computer and communications security, Chicago, IL, USA

Download references

Acknowledgment

The authors would like to thank Hyungeun Jo and Professor Youn-kyung Lim for their support and useful discussions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jonghak Kim.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kim, J., Kim, S., Yang, J. et al. FaceCAPTCHA: a CAPTCHA that identifies the gender of face images unrecognized by existing gender classifiers. Multimed Tools Appl 72, 1215–1237 (2014). https://doi.org/10.1007/s11042-013-1422-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1422-z

Keywords

Navigation