Abstract
This paper proposes a novel Eigen face recognition that is aided by fusion of visible and thermal face images to improve the face recognition accuracy. We adopt three different fusion schemes where in the face information is fused by the optimal weights obtained by different optimization algorithms. The first two fusion approaches operate in the dual tree discrete wavelet transform (DT-DWT), while the third one operates in the Curvelet transform (CT) domain. We employ particle swarm optimization (PSO), self-tuning particle swarm optimization (STPSO) and brain storm optimization algorithm (BSO) to find optimal fusion coefficients. The proposed fusion aided face recognition approaches are evaluated through extensive experiments using OCTVBS benchmark face database and the Eigen face detection methodology. Simulation results show that proposed face recognition techniques have significant performance improvement in recognition accuracy suggesting fusion aided face recognition approach that deserves further study and consideration whenever high recognition accuracy is desired.
Similar content being viewed by others
References
Ahmad A, Riaz MM, Ghafoor A, Zaidi T (2016) Noise resistant fusion for multi-exposure sensors. IEEE Sensors J 16(13):5123–5124
Arivazhagan S, Mumtaj J (2007) Face Recognition using Multi-Resolution Transform. International Conference on Computational Intelligence and Multimedia Applications, 301–306
Bebis G, Gyaourova A, Singh S, Pavlidis I (2006) Face recognition by fusing thermal infrared and visible imagery. Image Vis Comput 24:727–742
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs.fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720
BenAbdelkader C, Griffin P (2005) A local region-based approach to gender classification from face images. Proc. IEEE Comp. Society Conf. Computer Vision and Pattern Recognition, pages 52–57
Brunelli R, Poggio T (1993) Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052
Chen W, Er MJ, Wu S (2006) “Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain” IEEE transactions on systems, man, and cybernetics. Part B (Cybernetics) 36(2):458–466
Chen YL, Jahanshahi MR, Manjunatha P, Gan WP, Abdelbarr M, Masri SF, Becerik-Gerber B, Caffrey JP (2016) Inexpensive multimodal sensor fusion system for autonomous data acquisition of road surface conditions. IEEE Sensors J 16(21):7731–7743
Choudhary A, Vig R (2018) Face Recognition Using Multiresolution Hybrid Kekre-DCT Wavelet Transform Features with Multiclass ECOC Framework. Proc Comp Sci 132:1781–1787
Choudhary A, Vig R (2018) Face recognition using multiresolution hybrid Kekre-DCT wavelet transform features with multiclass ECOC framework. Proc Comp Sci 132:1781–1787
Dakin SC, Watt RJ (2009) Biological `bar codes' in human faces. World Acad Sci Eng Technol 9:1–10
Ekenel HK, Sankur B (2005) Multiresolution face recognition. Image Vis Comput 23:469–477
Eleyan A, Demirel H (2007) Face Recognition using Multiresolution PCA. IEEE International Symposium on Signal Processing and Information Technology, 52–55.
Er MJ, Chen W, Wu S (2005) High-speed face recognition based on discrete cosine transform and RBF neural networks. IEEE Trans Neural Netw 16(3):679–691
Gao Y, Ma J (2017) Semi-Supervised Sparse Representation Based Classification for Face Recognition with Insufficient Labeled Samples. IEEE Transactions on Image Processing
Ghasemzadeh A, Demirel H (2017) 3D discrete wavelet transform-based feature extraction for hyperspectral face recognition. IET Biometrics 7(1):49–55
Grotschel M, Lovász L (1993) Combinatorial optimization: a survey, DIMACS technical report 93–29, Princeton University. In Internet
Guzman AM, Goryawala M, Wang J, Barreto A, Andrian J, Rishe N, Adjouadi M (2013) Thermal imaging as a biometrics approach to facial signature authentication. IEEE J Biomed Health Inf 17(1):214–222
Hermosilla G, Gallardo F, Farias G, San Martin C (2015) Fusion of visible and thermal descriptors using genetic algorithms for face recognition systems. Sensors 15:17944–17962. https://doi.org/10.3390/s150817944
Hizem W, Allano L, Mellakh A, Dorizzi B (2009) Face recognition from synchronised visible and near-infrared images. IET Signal Process 3(4):282–288
Hollingsworth KP, Darnell SS, Miller PE, Woodard DL, Bowyer KW, Flynn PJ (2012) Human and Machine Performance on Periocular Biometrics Under Near-Infrared Light and Visible Light. IEEE Transact Inf Foren Sec 7(2):588–601
IEEE OTCBVS WS Series Bench (n.d.); DOE University Research Program in Robotics under grant DOE-DE-FG02-86NE37968; DOD/TACOM/NAC/ARC Program under grant R01–1344-18; FAA/NSSA grant R01–1344-48/49; Office of Naval Research under grant #N000143010022
IEEE OTCBVS WS Series Bench (n.d.); DOE University Research Program in Robotics under grant DOE-DE-FG02-86NE37968; DOD/TACOM/NAC/ARC Program under grant R01–1344-18; FAA/NSSA grant R01–1344-48/49; Office of Naval Research under grant #N000143010022. http://vcipl-okstate.org/pbvs/bench/Data/02/download.html
Imtiaz H, Fattah SA (2011) A face recognition scheme using waveletbased dominant features. Signal & Image Processing : An International Journal (SIPIJ). 2(3)
Kim T-K, Kittler J (2005) Locally Linear Discriminant Analysis for Multi-modally Distributed Classes for Face Recognition with a Single Model Image. IEEE Trans Pattern Anal Mach Intell 27(3):318–327
Klare BF, Jain AK Heterogeneous Face Recognition Using Kernel Prototype Similarities. IEEE Trans Pattern Anal Mach Intell 35(6):1410–1422
Kong SG, Heo J, Boughorbel F, Zheng Y, Abidi BR, Koschan A, Yi M, Abidi MA (2007) Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition. Int J Comput Vis 71(2):215–233
Kong WW, Lei YJ, Lei Y, Zhang J (2010) Technique for image fusion based on non-subsampled contourlet transform domain improved NMF. SCIENCE CHINA Inf Sci 53(12):2429–2440
Kumar A, Zhang D (2006) Personal recognition using hand shape and texture. IEEE Trans Image Process 15(8):2454–2461
Laurenz W, Fellous JM, Krüger N, von der Malsburg C (1997) Face recognition by elastic bunch graph matching. 19(7): 775–779
Lawrence S, Giles CL, Tsoi A, Back AD (1998) Face recognition: a convolutional neural network approach. IEEE Trans Neural Netw 8(1):98–113
Li SZ, Chu RF, Liao SC, Zhang L (2007) Illumination invariant face recognition using near-infrared images. IEEE Trans Pattern Anal Mach Intell 29(4):627–639
Liu C (2004) Gabor-Based Kernel PCA with Fractional Power Polynomial Models for Face Recognition. IEEE Trans Pattern Anal Mach Intell 26(5):572–581
Ma J, Zhao J, Ma Y, Tian J (2015) Non-rigid visible and infrared face registration via regularized Gaussian fields criterion. Pattern Recogn 48:772–784
Ma J, Qiu W, Zhao J, Ma Y, Yuille AL (2015) Robust L2E Estimation of Transformation for Non-Rigid Registration, IEEE Transactions on Signal Processing. IEEE Transactions on Signal Processing, VOL. 63, NO. 5, MARCH 1, 2015, 1115–1129
Ma J, Chen C, Li C, Huang J (2016) Infrared and visible image fusion via gradient transfer and total variation minimization. Info Fusion 31:100–109
Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Info Fusion 45:153–178
Madheswari K, Venkateswaran N (2016) Swarm intelligence based optimisation in thermal image fusion using dual tree discrete wavelet transform. Quant Infra Thermograph J, First Online, 1–20
MartõÂnez AM, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233
Meraoumia A, Chitroub S, Bouridane A (2010) Gaussian modeling and Discrete Cosine Transform for efficient and automatic palmprint identification. Int Conf Mach Web Intell (ICMWI):121–125
Moghaddam B, Pentland A (1997) Probabilistic visual learning for object representation. IEEE Trans Pattern Analysis Mach Intell 18:696–710
Nicolò F, Schmid NA (2012) Long range cross-spectral face recognition: matching SWIR against visible light images. IEEE Transact Inf Foren Sec 7:1717–1725
Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection
Parkhi OM, Vedaldi A, Zisserman A (2015) Deep Face Recognition British Machine Vision Conference
Penev P, Atick J (1996) Local Feature Analysis: A general statistical theory for object representation
Petrovic V (2007) Subjective tests for image fusion evaluation and objective metric validation. Info Fusion 8(2):2018–2216
Raghavendra R, Dorizzi B, Rao A, Kumar GH (2011) Particle swarm optimization based fusion of near infrared and visible images for improved face verification. Pattern Recogn 44:401–411
Rajoub BA, Zwiggelaar R (2014) Thermal facial analysis for deception detection. IEEE Transact Inf Foren Sec 9(6):1015–1023
Schoelkopf B, Smola A, Muller KR (1997) Kernel principal components analysis”, artificial neural networks, ICANN97
Seal A, Bhattacharjee D, Nasipuri M (2016) Human face recognition using random forest based fusion of à-trous wavelet transform coefficients from thermal and visible images. Int J Electron Commun (AEÜ) 70:1041–1049
Shen LL, Bai L (2004) Gabor feature based face recognition using kernal methods. Proc IEEE Int Conf Auto Face Gest Recog:386–389
Tan H, Huang X, Tan H, He C Pixel-level Image Fusion Algorithm Based on Maximum Likelihood and Laplacian Pyramid Transformation. J Comput Inf Syst 9(1):327–334
Toet A (1989) Image fusion by a ratio of low-pass pyramid. Pattern Recogn Lett 9:245–253
Tripathi BK (2017) On the complex domain deep machine learning for face recognition. Appl Intell 47(2):382–396
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Sci:71–86
Turk M, Pentland A (1991) Eigen faces for recognition. IEEE Transact Cogn Neurosci 13(1):71–86
Turk MA, Pentland AP (1991) Face recognition using eigenfaces. Computer Vision and Pattern Recognition. Proceedings {CVPR'91.}, {IEEE} Computer Society Conference on 1991
Villegas-Quezada C, Climent J (2008) Holistic face recognition using multivariate approximation,genetic algorithms and AdaBoost classifier: preliminary results. World Acad Sci Eng Technol 44:802–806
Xiaozheng Z, Gao Y (2009) Face recognition across pose: A review. Pattern Recogn 42:2876–2896
Yang Y, Que Y (2016) Shuying Huang, and Pan Lin, ‘multimodal sensor medical image fusion based on Type-2 fuzzy logic in NSCT domain’. IEEE Sensors J 16(10):3735–3745
Yang J, Zhang D, Frangi AF, Yang J-y (2004) Two-Dimensional PCA : A New Approach to Appearance-Based Face Representation and Recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137
Zheng L, Blasch E, Xue Z, Zhao J, Laganiere R, Wei W (2012) Objective Assessment of Multi-resolution Image Fusion algorithms for Context enhancement in Night Vision: A Comparative Study. IEEE Trans Pattern Anal Mach Intell 34(1):94–109
Zheng L, Blasch E, ZhiyunXue JZ, Laganiere R, Wu W (2012) Objective assessment of multi-resolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans Pattern Anal Mach Intell 34(1)
Zhou M, Wei H (2006) Face verification using Gabor wavelets and adaboost. Proc Int Conf Pattern Recog:404–407
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kanmani, M., Narasimhan, V. Optimal fusion aided face recognition from visible and thermal face images. Multimed Tools Appl 79, 17859–17883 (2020). https://doi.org/10.1007/s11042-020-08628-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-08628-9