Abstract
Based on discriminant canonical correlation analysis of LDA, a new method of multimodal information analysis and fusion is proposed in this paper. We process data from two perspectives, single modality and cross-modal. More specifically, firstly, LDA is utilised to obtain the best projection matrix, this way, the data in each within-modal can be as centralized as possible. Secondly, the improved DCCA is used to process the output of first step in order to maximize within-class correlation and minimize between-class correlation. The above two steps prove beneficial to obtain the feature with higher discriminating ability which is essential for the average fusion recognition accuracy improvement. We show state-of-art results or better than state-of-art on widely used USM benchmarks against all existing results include CCA, LDA, DCCA, GCCA and KCCA.
Huabin Wang: The research work is supported by the National Natural Science Foundation of China (grant no.61372137).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, L., Jun, Z., Fei, J., et al.: An incremental face recognition system based on deep learning. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), pp. 238–241. IEEE (2017). https://doi.org/10.23919/mva.2017.7986845
Sun, Z., Zhang, H., Tan, T., et al.: Iris image classification based on hierarchical visual codebook. IEEE Trans. Pattern Anal. Mach. Intell. 36(6), 1120–1133 (2014). https://doi.org/10.1109/tpami.2013.234
Liu, S., Liu, Y., Yu, J., et al.: Hierarchical static hand gesture recognition by combining finger detection and HOG features. J. Image Graph (2015)
Seng, K., Ang, L.M., Ooi, C.: A combined rule-based and machine learning audio-visual emotion recognition approach. IEEE Trans. Affect. Comput. (2016). https://doi.org/10.1109/TAFFC.2016.2588488
Ahlawat, M., Kant, C.: An introduction to multimodal biometric system: an overview. Int. J. Sci. Res. Dev. 3(02), 2321–0613 (2015)
Zheng, H., Geng, X.: A multi-task model for simultaneous face identification and facial expression recognition. Neurocomputing 171, 515–523 (2016). https://doi.org/10.1016/j.neucom.20-15.06.079
Wen, H., Liu, Y., Rekik, I., et al.: Multi-modal multiple kernel learning for accurate identification of tourette syndrome children. Pattern Recogn. 63, 601–611 (2017). https://doi.org/10.1016/j.patcog.2016.09.039
Wang, S., Zhang, J., Zong, C.: Learning multimodal word representation via dynamic fusion methods. arXiv preprint arXiv:1801.00532 (2018)
Liu, Q., Tang, L., Zhao, B.-J., et al.: Infrared target tracking based on adaptive multiple features fusion and mean shift. J. Electron. Inf. Technol. 34(5), 1137–1141 (2012). https://doi.org/10.3724/SP.J.1146.2011.01077
Liu, H., Wu, Y., Sun, F., et al.: Weakly paired multimodal fusion for object recognition. IEEE Trans. Autom. Sci. Eng. 15(2), 784–795 (2018). https://doi.org/10.1109/TASE.2017.2692271
Zeng, M., Wu, Z., Tian, C., et al.: Fusing appearance statistical features for person re-identification. J. Electron. Inf. Technol. 36(8), 1845–1851 (2014). https://doi.org/10.3724/SP.J.1146.2013.01389
Meng, W., Wong, D.S., Furnell, S., et al.: Surveying the development of biometric user authentication on mobile phones. IEEE Commun. Surv. Tutorials 17(3), 1268–1293 (2015). https://doi.org/10.1109/COMST.2014.2386915
Haghighat, M., Abdel-Mottaleb, M., Alhalabi, W.: Discriminant correlation analysis for feature level fusion with application to multimodal biometrics. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1866–1870 (2016). https://doi.org/10.1109/icassp.2016.7472000
He, M., Horng, S.J., Fan, P., et al.: Performance evaluation of score level fusion in multimodal biometric systems. Pattern Recogn. 43(5), 1789–1800 (2010). https://doi.org/10.1016/j.patcog.2009.11.018
Liu, H., Li, S.: Decision fusion of sparse representation and support vector machine for SAR image target recognition. Neurocomputing 113, 97–104 (2013). https://doi.org/10.1016/j.neucom.2013.01.033
Liu, C., Wechsler, H.: A shape-and texture-based enhanced fisher classifier for face recognition. IEEE Trans. Image Process. 10(4), 598–608 (2001). https://doi.org/10.1109/83.913594
Yang, J., Yang, J., Zhang, D., et al.: Feature fusion: parallel strategy vs. serial strategy. Pattern Recogn. 36(6), 1369–1381 (2003). https://doi.org/10.1016/S0031-3203(02)00262-5
Sun, Q., Zeng, S., Yang, M., et al.: A new method of feature fusion and its application inimage recognition. Pattern Recogn. 38(12), 2437–2448 (2005). https://doi.org/10.1016/j.patcog.2004.12.013
Sun, Q., Zeng, S., Yang, M., et al.: Combined feature extraction based on canonical correlation analysis and face recognition. J. Comput. Res. Dev. 42(4), 614–621 (2005). https://doi.org/10.1360/crad20050413
Kim, T.K., Kittler, J., Cipolla, R.: Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1005–1018 (2007). https://doi.org/10.1109/TPAMI.2007.1037
Tenenhaus, A., Philippe, C., Guillemot, V., et al.: Variable selection for generalized canonical correlation analysis. Biostatistics. 15(3), 569–583 (2014). https://doi.org/10.1093/biostatistics/kxu001
Jia, Z.: Multi-feature combination face recognition based on kernel canonical correlation analysis. Int. J. Signal Process. Image Process. Pattern Recogn. 9(7), 221–230 (2016)
Haghighat, M., Abdel-Mottaleb, M., Alhalabi, W.: Discriminant correlation analysis: real-time feature level fusion for multimodal biometric recognition. IEEE Trans. Inf. Forensics Secur. 11(9), 1984–1996 (2016). https://doi.org/10.1109/TIFS.2016.2569061
Gao, L., Qi, L., Chen, E., et al.: Discriminative multiple canonical correlation analysis for information fusion. IEEE Trans. Image Process. 27(4), 1951–1965 (2018). https://doi.org/10.1109/TIP.2017.2765820
Hotelling, H.: Relations between two sets of variates. Biometrika 28(3–4), 321–377 (1992). https://doi.org/10.2307/2333955
Weenink, D.: Canonical correlation analysis. In: Proceedings of the Institute of Phonetic Sciences of the University of Amsterdam, vol. 25, pp. 81–99. University of Amsterdam (2003)
Sun, Q., Zeng, S., Yang, M., et al.: A new method of feature fusion and its application in image recognition. Pattern Recogn. 38(12), 2437–2448 (2005). https://doi.org/10.1016/j.patcog.2004.12.013
Correa, N.M., Adali, T., Li, Y.-O., Calhoun, V.D.: Canonicalcorrelation analysis for data fusion and group inferences. IEEE Signal Process. Mag. 27(4), 39–50 (2010). https://doi.org/10.1109/34.598228
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997). https://doi.org/10.1109/34.598228
Dan, K.: A singularly valuable decomposition: the SVD of a matrix. Coll. Math. J. 27(1), 2–23 (1996). https://doi.org/10.2307/2687269
Zhou, H., Chen, S.: Ordinal discriminative canonical correlation analysis. J. Softw. 25(9), 2018–2025 (2014). https://doi.org/10.13328/j.cnki.jos.004649
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, C., Wang, H., Liu, X., Tao, L. (2018). Enhanced Linear Discriminant Canonical Correlation Analysis for Cross-modal Fusion Recognition. In: Hong, R., Cheng, WH., Yamasaki, T., Wang, M., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science(), vol 11164. Springer, Cham. https://doi.org/10.1007/978-3-030-00776-8_77
Download citation
DOI: https://doi.org/10.1007/978-3-030-00776-8_77
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00775-1
Online ISBN: 978-3-030-00776-8
eBook Packages: Computer ScienceComputer Science (R0)