Skip to main content
SpringerLink
Log in

A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Extreme learning machine (ELM) is a well-known cognitive model, that has been extended to cross-domain tasks. Nonetheless, most existing paradigms that are based on ELM either concern about a case in which a specific number of instances are labelled in the target domain or a learner is trained without sufficient capacity to eliminate the gap between domains. To cope with the scenario in which there are no target labels and to acquire a better adaptive learner, we propose a joint unsupervised cross-domain model via scalable discriminative ELM, which is abbreviated as JUC-SDELM. Within the framework, the scalable factor is integrated into discriminative ELM (DELM) to adjust the output margin, which strengthens the discriminative capacity of the ELM classifier. In addition, we follow the basic strategy of joint distribution adaptation (JDA) to align the subspaces generated by JUC-SDELM in terms of their statistics. The discrepancy across domains is alleviated after a few iterations. Moreover, a metric on the outputs of ELM is utilized to filter unreliable pseudo labels in the target domain, with the aim of eliminating the negative transfer effect. Results are obtained by comparing JUC-SDELM with state-of-the-art baseline methods on 16 cross-domain benchmarks that were constructed based on three combined datasets. Likewise, the outcomes in terms of key parameters are also examined. According to the experiments, our proposed model achieves competitive overall performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://people.eecs.berkeley.edu/~jhoffman/domainadapt/

  2. http://www.vision.caltech.edu/Image_Datasets/Caltech256/

  3. http://www-scf.usc.edu/~boqinggo/domainadaptation.html

  4. https://github.com/alshedivat/stsc/tree/master/data

  5. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php

  6. http://ise.thss.tsinghua.edu.cn/~mlong

  7. https://www.csie.ntu.edu.tw/~cjlin/libsvm/

  8. http://www.cad.zju.edu.cn/home/dengcai/Data/code/PCA.m

  9. http://users.cecs.anu.edu.au/~basura/DA_SA/

  10. http://www.ntu.edu.sg/home/egbhuang/elm_codes.html

References

  1. Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–1359.

    Article  Google Scholar 

  2. Gao S, Li H. A cross-domain adaptation method for sentiment classification using probabilistic latent analysis. Proceedings of the 20th ACM international conference on Information and knowledge management; 2011. p. 1047–1052.

  3. Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw. 2011;22(2):199–210.

    Article  PubMed  Google Scholar 

  4. Patel VM, Gopalan R, Li R, Chellappa R. Visual domain adaptation: a survey of recent advances. IEEE Signal Proc Mag. 2015;32(3):53–69.

    Article  Google Scholar 

  5. Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. IEEE conference on computer vision and pattern recognition (CVPR); 2012. p. 2066–2073.

  6. Yu J, Hong C, Rui Y, Tao D. Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron. 2017;65(6):5060–5068.

    Article  Google Scholar 

  7. Ando S, Suzuki E. Unsupervised cross-domain learning by interaction information co-clustering. Eighth IEEE international conference on data mining data mining; 2008. p. 13–22.

  8. Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans Image Process. 2016;25(10):4959–4973.

    Article  PubMed  Google Scholar 

  9. Zhang L, Zhang D. Domain adaptation extreme learning machines for drift compensation in E-nose systems. IEEE Trans Instrum Meas. 2015;64(7):1790–1801.

    Article  Google Scholar 

  10. Pan SJ, Kwok JT, Yang Q. Transfer learning via dimensionality reduction. Proceedings of the Twenty-Third AAAI conference on artificial intelligence; 2008. p. 677–682.

  11. Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Schölkopf B, Smola AJ. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics. 2006; 22(14):e49–e57.

  12. Jiang M, Huang W, Huang Z, Yen GG. Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE Trans Cybern. 2017;47(1):38–51.

    Article  PubMed  Google Scholar 

  13. Long M, Cao Y, Wang J, Jordan MI. Learning transferable features with deep adaptation networks. International conference on machine learning; 2015. p. 97–105.

  14. Fernando B, Habrard A, Sebban M, Tuytelaars T. Unsupervised visual domain adaptation using subspace alignment. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2960–2967.

  15. Jolliffe IT. Principal component analysis, 2nd ed. Berlin: Springer; 2002. http://www.springer.com/us/book/9780387954424.

    Google Scholar 

  16. Hong C, Zeng Z, Xie R, Zhuang W, Wang X. Domain adaptation with low-rank alignment for weakly supervised hand pose recovery. Signal Process. 2018;142:223–230.

    Article  Google Scholar 

  17. Long M, Wang J, Ding G, Sun J, Yu PS. Transfer feature learning with joint distribution adaptation. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2200–2207.

  18. Tahmoresnezhad J, Hashemi S. Visual domain adaptation via transfer feature learning. Knowl Inf Syst 2017; 50(2):585–605.

    Article  Google Scholar 

  19. Luo L, Wang X, Hu S, Chen L. Robust data geometric structure aligned close yet discriminative domain adaptation. 2017. arXiv:1705.08620v1.

  20. Long M, Wang J, Ding G, Pan SJ, Yu PS. Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng. 2014;26(5):1076–1089.

    Article  Google Scholar 

  21. Gheisari M, Baghshah MS. Joint predictive model and representation learning for visual domain adaptation. Eng Appl Artif Intel 2017;58:157–170.

    Article  Google Scholar 

  22. Wang X, Ma Y, Cheng Y. 2017. Domain adaptation network based on hypergraph regularized denoising autoencoder. Artif Intell Rev. https://doi.org/10.1007/s10462-017-9576-0.

  23. Lu B, Chellappa R, Nasrabadi NM. Incremental dictionary learning for unsupervised domain adaptation. British machine vision conference (BMVC); 2015. p. 108.1–108.12.

  24. Hou CA, Yeh YR, Wang YCF. An unsupervised domain adaptation approach for cross-domain visual classification. IEEE international conference on advanced video and signal based surveillance (AVSS); 2015. p. 1–6.

  25. Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006; 70(1):489–501.

    Article  Google Scholar 

  26. Huang GB, Zhou H, Ding X, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern. 2012;42(2):513–529.

    Article  PubMed  Google Scholar 

  27. Huang GB. What are extreme learning machines? Filling the gap between Frank Rosenblatts dream and John von Neumanns puzzle. Cogn Comput 2015;7(3):263–278.

    Article  Google Scholar 

  28. Guo T, Zhang L, Tan X. Neuron pruning-based discriminative extreme learning machine for pattern classification. Cogn Comput. 2017;9(4):581–595.

    Article  Google Scholar 

  29. Liu Y, Zhang L, Deng P, He Z. Common subspace learning via cross-domain extreme learning machine. Cogn Comput. 2017;9(4):555–563.

    Article  Google Scholar 

  30. Peng Y, Wang S, Long X, Lu B. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing. 2015;149:340–353.

    Article  Google Scholar 

  31. Huang G, Song S, Gupta JND, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 2014;44(12):2405–2417.

    Article  PubMed  Google Scholar 

  32. Iosifidis A, Tefas A, Pitas I. Graph embedded extreme learning machine. IEEE Trans Cybern 2016;46 (1):311–324.

    Article  PubMed  Google Scholar 

  33. Yan D, Chu Y, Zhang H, Liu D. 2016. Information discriminative extreme learning machine. Soft Comput. https://doi.org/10.1007/s00500-016-2372-y.

  34. Iosifidis A, Tefas A, Pitas I. Semi-supervised classification of human actions based on neural networks. 22nd international conference on pattern recognition (ICPR), IEEE; 2014. p. 1336–1341.

  35. Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst. 2012;23(11):1738–1754.

    Article  PubMed  Google Scholar 

  36. Xu Y, Fang X, Wu J, Li X, Zhang D. Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process. 2016;25(2):850–863.

    Article  PubMed  Google Scholar 

  37. Lei B, Yang P, Wang T, Chen S, Ni D. Relational-Regularized Discriminative sparse learning for alzheimers disease diagnosis. IEEE Trans Cybern. 2017;47(4):1102–1113.

    Article  PubMed  Google Scholar 

  38. Luo M, Zhang K. A hybrid approach combining extreme learning machine and sparse representation for image classification. Eng Appl Artif Intel 2014;27:228–235.

    Article  Google Scholar 

  39. Li X, Mao W, Jiang W. Extreme learning machine based transfer learning for data classification. Neurocomputing. 2016;174:203–210.

    Article  Google Scholar 

  40. Zhang L, He Z, Liu Y. Deep object recognition across domains based on adaptive extreme learning machine. Neurocomputing. 2017;239:194–203.

    Article  Google Scholar 

  41. Uzair M, Mian A. Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern. 2017;47(3):651–660.

    Article  PubMed  Google Scholar 

  42. Wang L, Zhang XY, Pan C. MSDLSR Margin Scalable discriminative least squares regression for multicategory classification. IEEE Trans Neural Netw Learn Syst. 2016;27(12):2711–2717.

    Article  PubMed  Google Scholar 

  43. Pan J, Wang X, Cheng Y, Cao G. Multi-source transfer ELM-based Q learning. Neurocomputing 2014; 137:57–64.

    Article  Google Scholar 

  44. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–297.

    Google Scholar 

  45. Lu Y, Lai Z, Fan Z, Cui J, Zhu Q. Manifold discriminant regression learning for image classification. Neurocomputing 2015;166:475–486.

    Article  Google Scholar 

  46. Al-Shedivat M, Wang JJY, Alzahrani M, Huang JZ, Gao X. Supervised transfer sparse coding. Proceedings of the Twenty-Eighth AAAI conference on artificial intelligence; 2014. p. 1665–1672.

  47. Iosifidis A, Mygdalis V, Tefas A, Pitas I. One-class classification based on extreme learning and geometric class information. Neural Process Lett. 2017;45(2):577–592.

    Article  Google Scholar 

  48. Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. European conference on computer vision (ECCV); 2010. p. 213–226.

  49. Griffin G, Holub AD, Perona P. Caltech-256 object category dataset. Caltech Technical Report. 2007.

  50. LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86(11):2278–2324.

    Article  Google Scholar 

  51. Hull JJ. A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 1994;16 (5):550–554.

    Article  Google Scholar 

  52. Nene SA, Nayar SK, Murase H. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96.

  53. Chang CC, Lin CJ. LIBSVM: A library for support vector machines. ACM TIST. 2011;2(3):27.

    Google Scholar 

  54. Ghifary M, Kleijn WB, Zhang M. Domain adaptive neural networks for object recognition. Pacific Rim international conference on artificial intelligence; 2014. p. 898–904.

  55. Yu Y, Sun Z. Sparse coding extreme learning machine for classification. Neurocomputing 2017;261:50–56.

    Article  Google Scholar 

  56. Niu J, Shi Y, Cai M, Cao Z, Wang D, Zhang Z, et al. 2017. Detection of sputum by interpreting the time-frequency distribution of respiratory sound signal using image processing techniques. Bioinformatics. https://doi.org/10.1093/bioinformatics/btx652.

  57. Cao J, Zhang K, Luo M, Yin C, Lai X. Extreme learning machine and adaptive sparse representation for image classification. Neural Networks. 2016;81(C):91–102.

    Article  PubMed  Google Scholar 

  58. Chen Y, Song S. Domain transfer extreme learning machine and its application on domain adaptation problems. China Sciencepaper. 2017;12(14):1565–1569 + 1609.

    Google Scholar 

Download references

Funding

This work was funded by SGCC (State Grid Corporation of China) Thousand Talents program special support project (EPRIPDKJ (2014)2863), and partially funded by the National Natural Science Foundation of China (61273165).

Author information

Authors and Affiliations

  1. School of Automation Science and Electrical Engineering, Beihang University, Beijing, 100191, China

    Boyang Zhang, Yingyi Liu, Haiwen Yuan & Lingjie Sun

  2. China Electric Power Research Institute, Beijing, 100192, China

    Zhao Ma

Authors
  1. Boyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haiwen Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lingjie Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhao Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingyi Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

No studies with human participants or animals are involved in this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, B., Liu, Y., Yuan, H. et al. A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine. Cogn Comput 10, 577–590 (2018). https://doi.org/10.1007/s12559-018-9555-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-018-9555-z

Keywords

Search

Navigation