Skip to main content
SpringerLink
Log in

Transferring variational autoencoders with coarse-and-fine alignment for open set broad classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Transfer learning aims to help target learners with a different but related source domain. Open set recognition extends the settings of transfer learning for identifying whether an instance belongs to an unseen category. However, it is often pragmatic and valuable to further classify the unseen categories in target domain. We present a new setting called open set broad classification (OSBC) to classify unseen target categories which are open within the broad classes of source domain. Aiming at adapting to the challenging domain shift between unseen categories and seen categories, we propose a variational autoencoders model with coarse-and-fine alignment (CFVA) to leverage the structural information in the OSBC setting. First, two-stream decoders are employed and coarsely aligned by a relaxed parameters regularizer, which can absorb domain shift on features to facilitate fine alignment. Then fine alignment at encoding level enhances discriminative power of the latent representation by mixing the distributional structure hinted by source domain. Experimental results demonstrate the effectiveness of our CFVA approach in improving the accuracies in both unsupervised and semi-supervised cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Availability of data and materials

The datasets generated during and/or analysed during the current study are available in the cifar-100 repository, http://www.cs.toronto.edu/~kriz/cifar.html.

References

  1. Scheirer WJ, Jain LP, Boult TE (2014) Probability models for open set recognition. IEEE Trans Pattern Anal Mach Intell 36(11):2317–2324

    Article  Google Scholar 

  2. Busto PP, Iqbal A, Gall J (2020) Open set domain adaptation for image and action recognition. IEEE Trans Pattern Anal Mach Intell 42(2):413–429

    Article  Google Scholar 

  3. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  4. Zhuang F, Cheng X, Luo P, Pan SJ, He Q (2017) Supervised representation learning with double encoding-layer autoencoder for transfer learning. ACM Trans Intell Syst Technol 9(2):16

    Google Scholar 

  5. Sun S, Liu H, Meng J, Chen CLP, Yang Y (2018) Substructural regularization with data-sensitive granularity for sequence transfer learning. IEEE Trans Neural Netw Learn Syst 29(6):2545–2557

    Article  MathSciNet  Google Scholar 

  6. Cevikalp H, Saglamlar H (2021) Polyhedral conic classifiers for computer vision applications and open set recognition. IEEE Trans Pattern Anal Mach Intell 43(2):608–622

    Article  Google Scholar 

  7. Geng C, Tao L, Chen S (2020) Guided CNN for generalized zero-shot and open-set recognition using visual and semantic prototypes. Pattern Recognit 263:102–107

    Google Scholar 

  8. Wang W, Zheng VW, Yu H, Miao C (2019) A survey of zero-shot learning: settings, methods, and applications. ACM Trans Intell Syst Technol 10(2):13–11337

    Article  Google Scholar 

  9. Wang C, Chen X, Min S, Sun X, Li H (2021) Task-independent knowledge makes for transferable representations for generalized zero-shot learning. In: Thirty-fifth AAAI conference on artificial intelligence (AAAI), pp 2710–2718

  10. Ge J, Xie H, Min S, Zhang Y (2021) Semantic-guided reinforced region embedding for generalized zero-shot learning. In: Thirty-fifth AAAI conference on artificial intelligence (AAAI), pp 1406–1414

  11. Luo Y, Wang X, Pourpanah F (2021) Dual vaegan: a generative model for generalized zero-shot learning. Appl Soft Comput 107:107352

    Article  Google Scholar 

  12. Blitzer J, Kakade SM, Foster DP (2011) Domain adaptation with coupled subspaces. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics. pp 173–181

  13. Wang W, Pan SJ (2018) Recursive neural structural correspondence network for cross-domain aspect and opinion co-extraction. In: Proceedings of the 56th annual meeting of the association for computational linguistics (ACL), pp 2171–2181

  14. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2960–2967

  15. Ding Z, Fu Y (2017) Robust transfer metric learning for image classification. IEEE Trans Image Process 26(2):660–670

    Article  MathSciNet  MATH  Google Scholar 

  16. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Proceedings of the thirtieth AAAI conference on artificial intelligence (AAAI), pp 2058–2065

  17. Chen Y, Song S, Li S, Wu C (2020) A graph embedding framework for maximum mean discrepancy-based domain adaptation algorithms. IEEE Trans Image Process 29:199–213

    Article  MathSciNet  MATH  Google Scholar 

  18. Ganapathiraman V, Shi Z, Zhang X, Yu Y (2018) Inductive two-layer modeling with parametric bregman transfer. In: Proceedings of the 35th international conference on machine learning (ICML), pp 1622–1631

  19. Zhang L, Liu J, Yang Y, Huang F, Nie F, Zhang D (2020) Optimal projection guided transfer hashing for image retrieval. IEEE Trans Circuits Syst Video Technol 30(10):3788–3802

    Article  Google Scholar 

  20. Roy J, Konidaris GD (2021) Visual transfer for reinforcement learning via wasserstein domain confusion. In: Thirty-Fifth AAAI conference on artificial intelligence (AAAI), pp 9454–9462

  21. Courty N, Flamary R, Tuia D, Rakotomamonjy A (2017) Optimal transport for domain adaptation. IEEE Trans Pattern Anal Mach Intell 39(9):1853–1865

    Article  Google Scholar 

  22. Long M, Wang J, Cao Y, Sun J (2016) Deep learning of transferable representation for scalable domain adaptation. IEEE Trans Knowl Data Eng 28(8):2027–2040

    Article  Google Scholar 

  23. Long M, Cao Y, Cao Z, Wang J, Jordan MI (2019) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell 41(12):3071–3085

    Article  Google Scholar 

  24. Qian K, Zhang Y, Chang S, Yang X, Hasegawa-Johnson M (2019) Autovc: zero-shot voice style transfer with only autoencoder loss. In: Proceedings of the 36th international conference on machine learning (ICML), pp 5210–5219

  25. Rozantsev A, Salzmann M, Fua P (2019) Beyond sharing weights for deep domain adaptation. IEEE Trans Pattern Anal Mach Intell 41(4):801–814

    Article  Google Scholar 

  26. Heinze-Deml C, Meinshausen N (2021) Conditional variance penalties and domain shift robustness. Mach Learn 110(2):303–348

    Article  MathSciNet  MATH  Google Scholar 

  27. Wang J, Cheng M-M, Jiang J (2021) Domain shift preservation for zero-shot domain adaptation. IEEE Trans Image Process 30:5505–5517

    Article  Google Scholar 

  28. Kamath A, Jia R, Liang P (2020) Selective question answering under domain shift. In: Proceedings of the 58th annual meeting of the association for computational linguistics. pp 5684–5696

  29. Wang C, Sennrich R (2020) On exposure bias, hallucination and domain shift in neural machine translation. In: Proceedings of the 58th annual meeting of the association for computational linguistics (ACL), pp 3544–3552

  30. Kalogeiton V, Ferrari V, Schmid C (2016) Analysing domain shift factors between videos and images for object detection. IEEE Trans Pattern Anal Mach Intell 38(11):2327–2334

    Article  Google Scholar 

  31. Daume III H (2007) Frustratingly easy domain adaptation. In: Proceedings of the 45th annual meeting of the association for computational linguistics (ACL), pp 256–263

  32. Qiu Q, Chellappa R (2015) Compositional dictionaries for domain adaptive face recognition. IEEE Trans Image Process 24(12):5152–5165

    Article  MathSciNet  MATH  Google Scholar 

  33. Rozantsev A, Salzmann M, Fua P ( 2018) Residual parameter transfer for deep domain adaptation. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 4339–4348

  34. Cao A, Luo Y, Klabjan D (2021) Open-set recognition with gaussian mixture variational autoencoders. In: Thirty-fifth AAAI conference on artificial intelligence (AAAI), pp 6877–6884

  35. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79(1–2):151–175

    Article  MathSciNet  MATH  Google Scholar 

  36. Ganin Y, Lempitsky VS ( 2015) Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd international conference on machine learning (ICML), pp 1180–1189

  37. Chidlovskii B, Clinchant S, Csurka G (2016) Domain adaptation in the absence of source domain data. In: Proceedings of the 22nd ACM international conference on knowledge discovery and data mining (SIGKDD), pp 451–460

  38. Dev K, Ashraf Z, Muhuri PK, Kumar S (2022) Deep autoencoder based domain adaptation for transfer learning. Multimed Tools Appl 81(16):22379–22405

    Article  Google Scholar 

  39. Na J, Han D, Chang HJ, Hwang W (2022) Contrastive vicinal space for unsupervised domain adaptation. In: 17th European conference on computer vision (ECCV), pp 92–110

  40. Pu Y, Gan Z, Henao R, Yuan X, Li C, Stevens A, Carin L (2016) Variational autoencoder for deep learning of images, labels and captions. In: Advances in neural information processing systems (NIPS), pp 2352–2360

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant nos. 61876031).

Funding

This work was supported by the National Natural Science Foundation of China (Grant nos. 61876031).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Dalian Minzu University, Development Zone Liaohexi Road 18, Dalian, 116600, Liaoning, China

    Shichang Sun, Di Zhao, Yuhai Yu & Jiana Meng

  2. School of Science, Dalian Minzu University, Development Zone Liaohexi Road 18, Dalian, 116600, Liaoning, China

    Yongdong Huang

Authors
  1. Shichang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongdong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Di Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuhai Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiana Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiana Meng.

Ethics declarations

Conflict of interest

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, S., Huang, Y., Zhao, D. et al. Transferring variational autoencoders with coarse-and-fine alignment for open set broad classification. Int. J. Mach. Learn. & Cyber. 14, 3655–3669 (2023). https://doi.org/10.1007/s13042-023-01856-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01856-z

Keywords

Search

Navigation