Abstract
This paper proposes an ensemble of different state-of-art algorithms for realizing a face search system aimed at achieving higher accuracies compared to any single algorithm. This is achieved by leveraging most promising deep networks (Facenet, OpenFace, DeepFace, and VGGFace – originally trained for face recognition) and different Approximate Nearest Neighbor Search (ANNS) algorithms (Annoy and LSHash). Face images in the database are subjected to feature extraction (embeddings computed by deep networks) and indexing (in set structure for faster search) by ANNS algorithms. An input face query image is processed in the following four stages. First, the face region is detected from the query image and appropriately aligned for further processing. Second, the facial features are extracted using multiple deep networks. Third, the ANNS algorithms perform fast search by efficiently shrinking the gallery size from millions to a few hundred faces. Fourth, a fine matching is performed using two different methods separately to produce the final search results. These are (a) cosine similarity and (b) score-based matching and re-ranking of results. The experimental results demonstrate the diversity in results obtained by use of multiple feature extractors and ANNS techniques and the accuracy achieved by using the proposed ensemble framework.
S. Vats and S. Jain—These authors have an equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amos, B., Ludwiczuk, B., Satyanarayanan, M., et al.: Openface: a general-purpose face recognition library with mobile applications. CMU School Comput. Sci. 6(2) (2016)
Babenko, A., Lempitsky, V.: The inverted multi-index. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1247–1260 (2014)
Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM, 509–517 (1975)
Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: Twentieth Annual Symposium on Computational Geometry, pp. 253–262 (2004)
Friedman, J.H., Bentley, J.L., Finkel, R.A.: An algorithm for finding best matches in logarithmic expected time. ACM Trans. Math. Softw. 3, 209–226 (1977)
Ge, T., He, K., Ke, Q., Sun, J.: Optimized product quantization. IEEE Trans. Pattern Anal. Mach. Intell. 744–755
Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition, October 2008
Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with gpus. IEEE Trans. Big Data 1 (2019)
Jones, M.J., Viola, P.A.: Fast multi-view face detection. CTIT Technical Reports Series (2003)
Junjie Yan, Zhen Lei, Dong Yi, Li, S.Z.: Towards incremental and large scale face recognition. In: 2011 International Joint Conference on Biometrics, pp. 1–6 (2011)
Jégou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 117–128 (2011)
Kalantidis, Y., Avrithis, Y.: Locally optimized product quantization for approximate nearest neighbor search. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2321–2328 (2014)
Li, W., Zhang, Y., Sun, Y., Wang, W., Zhang, W., Lin, X.: Approximate nearest neighbor search on high dimensional data - experiments, analyses, and improvement (v1.0). CoRR abs/1610.02455 (2016)
Liu, T., Moore, A.W., Gray, A.: New algorithms for efficient high-dimensional nonparametric classification. J. Mach. Learn. Res. 1135–1158, June 2006
Norouzi, M., Punjani, A., Fleet, D.J.: Fast exact search in hamming space with multi-index hashing. IEEE Trans. Pattern Anal. Mach. Intell. 6, 1107–1119 (2014)
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference, pp. 41.1–41.12, September 2015
Qi, C., Liu, Z., Su, F.: Accurate and efficient similarity search for large scale face recognition. CoRR abs/1806.00365 (2018)
Ranjan, R., Patel, V.M., Chellappa, R.: Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41, 121–135 (2017)
Sadovnik, A., Gharbi, W., Vu, T., Gallagher, A.: Finding your lookalike: measuring face similarity rather than face identity. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 2408–24088 (2018)
Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823, June 2015
Song, J., Yang, Y., Huang, Z., Shen, H.T., Hong, R.: Multiple feature hashing for real-time large scale near-duplicate video retrieval. In: 19th ACM International Conference on Multimedia, pp. 423–432 (2011)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. vol. 1, p. I (2001)
Wang, D., Otto, C., Jain, A.K.: Face search at scale. IEEE Trans. Pattern Anal. Mach. Intell. 6, 1122–1136 (2017)
Wang, F., Xiang, X., Cheng, J., Yuille, A.L.: Normface: L\({}_{\text{2}}\) hypersphere embedding for face verification. CoRR abs/1704.06369 (2017)
Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. CoRR abs/1411.7923 (2014)
Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23(10), 1499–1503 (2016)
Zou, F., et al.: Fast large scale deep face search. Pattern Recogn. Lett. 83–90 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Vats, S., Jain, S., Guha, P. (2021). A Novel Ensemble Framework for Face Search. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12665. Springer, Cham. https://doi.org/10.1007/978-3-030-68821-9_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-68821-9_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68820-2
Online ISBN: 978-3-030-68821-9
eBook Packages: Computer ScienceComputer Science (R0)
