Skip to main content
SpringerLink
Log in

Multi-task clustering ELM for VIS-NIR cross-modal feature learning

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

Extreme learning machine (ELM) as a new emergent and efficient machine learning algorithm has shown its good performance in many real regression applications as well as large data classification. In this paper, we propose a new multi-task clustering ELM for cross-modal feature learning. Different to traditional face recognition methods, a coupled cross-modal feature learning based face descriptor is proposed to reduce the cross-modal differences, meanwhile, the multi-task learning is integrated with ELM for cross-modal classification. In this method, the discriminant feature learning is firstly proposed to learn the cross-modality feature representation. Then, common subspace learning based method is utilized to reduce the obtained cross-modality features. Finally, a multi-task clustering based ELM is proposed to improve the recognition accuracy by learning the shared information between tasks. Experiments conducted on two different VIS-NIR face recognition scenarios demonstrate the effectiveness of our proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face description with local binary patterns:application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(12), 2037–2041.

    Article  MATH  Google Scholar 

  • Cao, J., Chen, T., & Fan, J. (2016). Landmark recognition with compact BoW histogram and ensemble ELM. Multimedia Tools and Applications, 75(5), 2839–2857.

    Article  Google Scholar 

  • Cao, J., & Lin, Z. (2014). Bayesian signal detection with compressed measurements. Information Sciences, 289, 241–253.

    Article  MATH  Google Scholar 

  • Cao, J., & Lin, Z. (2015). Extreme learning machines on high dimensional and large data applications: A survey. Mathematical Problems in Engineering, 501, 103796.

    Google Scholar 

  • Cao, J., Lin, Z., Huang, G.-B., & Liu, N. (2012). Voting based extreme learning machine. Information Sciences, 185(1), 66–77.

    Article  MathSciNet  Google Scholar 

  • Cao, J., & Xiong, L. (2014). Protein sequence classification with improved extreme learning machine algorithms. BioMed Research International, 2014, Article ID 103054.

  • Cao, J., Zhao, Y., Lai, X., Ong, M., Yin, C., Koh, Z., et al. (2015). Landmark recognition with sparse representation classification and extreme learning machine. Journal of the Franklin Institute, 352(10), 4528–4545.

    Article  MathSciNet  Google Scholar 

  • Chang, L. C. J., & Libsvm, C. C. (2011). A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), Article 27.

  • Chen, J., Yi, D., J, Y., Zhao, G., Li, S., & Pietikainen, M. (2009). Learning mapping forface synthesis from near infrared to visual light images. In Proceedings of IEEE conference on computer vision and pattern recognition, pp. 156–163.

  • Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Conference on computer vision and pattern recognition (CVPR), pp. 886–893.

  • Gao, X., Zhong, J., Li, J., & Tian, C. (2005). Face sketch synthesis algorithm based on e-hmm and selective ensemble. IEEE Transactions on Circuits and Systems for Video Technology, 4, 487–496.

    Google Scholar 

  • Gu, Q., & Zhou, J. (2009). Learning the shared subspace for multi-task clustering and transductive transfer classification. In Ninth IEEE international conference on data mining, 2009. ICDM’09, pp. 159–168.

  • Hardoon, D., Szedmak, S., & Shawe-Taylor, J. (2004). Canonical correlation analysis: An overview with application to learning method. Neural Computing, 16, 2639–2664.

    Article  MATH  Google Scholar 

  • He, Q., Du, C., Zhuang, F., & Shi, Z. (2014). Clustering in extreme learning machine feature space. Neurocomputing, 128, 88–95.

  • Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28, 321–377.

    Article  MATH  Google Scholar 

  • Huang, X. S., Lei, Z., Fan, M. Y., Wang, X., & Li, S. Z. (2013). Regularized discriminative spectral regression method for heterogeneous face matching. IEEE Transactions on Image Processing, 22(1), 353–362.

    Article  MathSciNet  Google Scholar 

  • Huang, L. K., Lu, J. W., & Tan, Y.-P. (2012). Learning modality-invariant features for heterogeneous face recognition. In Proceedings of IEEE international conference on pattern recognition, pp. 1683–1686.

  • Huang, G.-B., Wang, D. H., & Lan, Y. (2011). Extreme learning machines: A survey. International Journal of Machine Learning and Cybernetics, 2(2), 107–122.

    Article  Google Scholar 

  • Huang, G.-B., Zhou, H., Ding, X., & Zhang, R. (2012). Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 42(2), 513–529.

    Article  Google Scholar 

  • Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006). Extreme learning machine theory and applications. Neurocomputing, 70(1), 489–501.

    Article  Google Scholar 

  • Jin, Y., Lu, J., & Ruan, Q. (2015). Coupled discriminative feature learning for heterogeneous face recognition. IEEE Transactions on Information Forensics and Security, 10(3), 640–652.

    Article  Google Scholar 

  • Jin, X., Zhuang, F., Xiong, H., Du, C., Luo, P., & He, Q. (2014). Multi-task multi-view learning for heterogeneous tasks. In Proceedings of the 23rd ACM international conference on information and knowledge management, CIKM ’14, pp. 441–450.

  • Juefei-Xu, F., Pal, D., & Savvides, M. (2015). Nir-vis heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 141–150.

  • Kan, M., Shan, S., Zhang, H., Lao, S., & Chen, X. (2012). Multi-view discriminant analysis. In Proceedings of The 12th European conference on computer vision, Vol. Part I, pp. 808–821.

  • Klare, B. F., & Jain, A. K. (2013). Heterogeneous face recognition using kernel prototype similarities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6), 1410–1422.

    Article  Google Scholar 

  • Klare, B. F., Li, Z., & Jain, A. K. (2011). Matching forensic sketches to mug shot photos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3), 639–646.

    Article  Google Scholar 

  • Lei, Z., & Li, S. Z. (2009). Coupled spectral regressoin for matching heterogeneous faces. In Proceedings of IEEE conference on computer vision and pattern recognition, pp. 1123–1128.

  • Lei, Z., Liao, S. C., Jain, A. K., & Li, S. Z. (2012). Coupled discriminant analysis for heterogeneous face recognition. IEEE Transactions on Information Forensics and Security, 7(6), 1707–1716.

    Article  Google Scholar 

  • Lei, Z., Pietikainen, M., & Li, S. Z. (2014). Learning discriminant face descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(2), 289–302.

    Article  Google Scholar 

  • Lei, Z., Yi, D., & Li, S. Z. (2012). Discriminant image filter learning for face recognition with local binary pattern like representation. In Proceedings of IEEE conference on computer vision and pattern recognition, pp. 2512–2517.

  • Li, S. (2009). Heterogeneous face bimoetrics, Encyclopedia of Biometreics. Berlin: Springer.

    Book  Google Scholar 

  • Li, Z., Gong, D., Qiao, Y., & Tao, D. (2014). Common feature discriminant analysis for matching infrared face images to optical face images. IEEE Transactions on Image Processing, 23(6), 2436–2445.

    Article  MathSciNet  Google Scholar 

  • Li, S., Lei, Z., & Ao, M. (2009). The hfb face database for heterogeneous face biometrics research. In Proceedings of IEEE computer society conference on computer vision and pattern recognition workshops, pp. 1–8.

  • Li, A., Shan, S., Chen, X., & Gao, W. (2011). Face recognition based on non-corresponding region matching. In Proceedings of IEEE international conference on computer vision, pp. 1060–1067.

  • Li, S.,Yi, D., Lei, Z., & Liao, S. (2013). The casia NIR-VIS 2.0 face database. In Proceedings of IEEE conference on computer vision and pattern recognition workshops, pp. 348–353.

  • Liao, S., Yi, D., Lei, Z., Qin, R., & Li, S. (2009). Heterogeneous face recognition from local structures of normalized appearance. In Proceedings of international conference on biometrics, pp. 209–218.

  • Lin, D., & Tang, X. (2006). Inter-modality face recognition. In Proceedings of the European conference on computer vision, pp. 13–26.

  • Lin, Z., Cao, J., Chen, T., Jin, Y., Sun, Z.-L., & Lendasse, A. (2015). Extreme learning machine on high dimensional and large data applications. Mathematical Problems in Engineering, 501, 624903.

    Google Scholar 

  • Liu, Q., Tang, X., Jin, H., Lu, H., & Ma, S. (2005). A nonlinear approach for face sketch synthesis and recognition. In Proceedings of IEEE conference on computer vision and pattern recognition, pp. 1005–1010.

  • Long, X., Lu, H., Peng, Y., & Li, W. (2014). Graph regularized discriminative non-negative matrix factorization for face recognition. Multimedia Tools and Applications, 72(3), 2679–2699.

    Article  Google Scholar 

  • Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Mao, W., Xu, J., Zhao, S., & Tian, M. (2013). Research of multi-task learning based on extreme learning machine. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 21(supp02), 75–85.

    Article  MathSciNet  MATH  Google Scholar 

  • Mohammed, A., Minhas, R., Wu, Q. J., & Sid-Ahmed, M. (2011). Human face recognition based on multidimensional pca and extreme learning machine. Pattern Recognition, 44(10–11), 2588–2597.

    Article  MATH  Google Scholar 

  • Sharma, A., & Jacobs, D.W. (2011). Bypassing synthesis: Pls for face recognition with pose, low-resolution and sketch. In Proceedings of IEEE conference on computer vision and pattern recognition, pp. 593–600.

  • Tang, X., & Wang, X. (2004). Face sketch recognition. IEEE Transactions on Circuits and Systems for Video Technology, 1, 50–57.

    Article  Google Scholar 

  • Tan, X., & Triggs, B. (2010). Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Transactions on Image Processing, 19(6), 1635–1650.

    Article  MathSciNet  Google Scholar 

  • Wang, X., & Tang, X. (2009). Face photo-sketch synthesis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1955–1967.

    Article  Google Scholar 

  • Wold, H. (1975). Quantitative sociology: International perspectives on mathematical and statistical modeling (quantitative studies in social relations), (Vol. 16, pp. 307–357), Academic Press edn. London: Academic Press.

  • Xie, S., Lu, H., & He, Y. (2012). Multi-task co-clustering via nonnegative matrix factorization. In 2012 21st international conference on pattern recognition (ICPR), pp. 2954–2958.

  • Yang, X., Kim, S., & Xing, E. P. (2009). Heterogeneous multitask learning with joint sparsity constraints. In Advances in neural information processing systems, Vol. 22, pp. 2151–2159.

  • Yang, Y., Ma, Z., Hauptmann, A. G., & Sebe, N. (2013). Feature selection for multimedia analysis by sharing information among multiple tasks. IEEE Transactions on Multimedia, 15(3), 661–669.

    Article  Google Scholar 

  • Yang, Y., Ma, Z., Yang, Y., Nie, F., & Shen, H. T. (2015). Multitask spectral clustering by exploring intertask correlation. IEEE Transactions on Cybernetics, 45(5), 1083–1094.

    Article  Google Scholar 

  • Yi, D., Lei, Z., Liao, S., Li, S. Z. Shared representation learning for heterogeneous face recognition. arXiv preprint arXiv:1406.1247.

  • Zhang, X.-L. (2015). Convex discriminative multitask clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(1), 28–40.

    Article  Google Scholar 

  • Zhang, Y., & Yeung, D.-Y. (2011). Multi-task learning in heterogeneous feature spaces. In AAAI conference on artificial intelligence, North America.

  • Zhang, J., & Zhang, C. (2011). Multitask bregman clustering. Neurocomputing, 74(10), 1720–1734.

    Article  MathSciNet  Google Scholar 

  • Zhou, J., Chen, J., & Malsar, J. Y. (2012). Multi-task learning via structural regularization. Arizona State University.

  • Zhu, J.-Y., Zheng, W.-S., Lai, J.-H., & Li, S. (2014). Matching NIR face to VIS face using transduction. IEEE Transactions on Information Forensics and Security, 9(3), 501–514.

    Article  Google Scholar 

  • Zong, W., & Huang, G.-B. (2011). Face recognition based on extreme learning machine. Neurocomputing, 74(16), 2541–2551.

    Article  Google Scholar 

  • Zong, W., Zhou, H., Huang, G.-B., & Lin, Z. (2011). Face recognition based on kernelized extreme learning machine. In Autonomous and Intelligent Systems, Vol. 6752 of Lecture Notes in Computer Science, pp. 263–272.

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Nos. 61403024, 61471032, 51505004, 61272352, 61502026) and the National Key Basic Research Program of China (2012CB316304), Beijing Natural Science Foundation (4142045,4163075), Beijing Higher Education Young Elite Teacher Project (YETP0547) and the Fundamental Research Funds for the Central Universities (2015JBM037).

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China

    Yi Jin, Jie Li, Congyan Lang & Qiuqi Ruan

  2. Beijing Key Lab of Traffic Data Analysis and Mining, Beijing, 100044, China

    Yi Jin, Jie Li, Congyan Lang & Qiuqi Ruan

Authors
  1. Yi Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Congyan Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiuqi Ruan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi Jin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, Y., Li, J., Lang, C. et al. Multi-task clustering ELM for VIS-NIR cross-modal feature learning. Multidim Syst Sign Process 28, 905–920 (2017). https://doi.org/10.1007/s11045-016-0401-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-016-0401-8

Keywords

Search

Navigation