Skip to main content
SpringerLink
Log in

Hierarchical Triple-Level Alignment for Multiple Source and Target Domain Adaptation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Domain adaptation aims to bridge the domain gap between the source and target domains. Most existing approaches concentrate on one target domain setting adapted from one or multiple source domains while neglecting the importance of multitarget domain setting. This inevitably causes a problem with suboptimal solutions in practical applications. To address this problem, we focus on a challenging but realistic scenario, unsupervised multisource-multitarget domain adaptation (UMDA), where multiple labeled source domains and multiple unlabeled target domains are available. In this paper, we propose a Hierarchical Triple-level Alignment (HTA) method for UMDA in which domain label, class label, and data structure information can be incorporated into a unified framework for effective knowledge transfer. The innovative points of this paper are as follows: 1) we devise a triple-level alignment mechanism including domain-level alignment, class-level alignment, and structure-level alignment, which effectively reduces the domain shift among multiple source and target domains; and 2) we develop a novel hierarchical gradient synchronization strategy to enhance class-level alignment, which can greatly reduce class distribution differences among multiple domains and preserve their individual class discrimination. Similarly, the hierarchical gradient synchronization strategy is also applied to structure-level alignment. As such, structure discrepancy reduction and individual structure preservation can both be achieved. To the best of our knowledge, HTA is the first attempt to simultaneously consider domain label, class label, and data structure information in the UMDA setting and can be regarded as a well-performing baseline for UMDA tasks. Experimental results on three standard benchmarks demonstrate the superiority of the proposed framework for multiple source-and-target domain adaptation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Wu M, Wang S, Li Z, Zhang L, Wang L, Ren Z (2021) Joint latent low-rank and non-negative induced sparse representation for face recognition

  2. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: CVPR, pp 815–823

  3. Jia K, Chan T, Ma Y (2012) Robust and practical face recognition via structured sparsity. In: ECCV (4). Lecture Notes in computer science, vol. 7575, pp. 331–344

  4. Hong S, Noh H, Han B (2015) Decoupled deep neural network for semi-supervised semantic segmentation. In: NIPS, pp. 1495– 1503

  5. Pan T, Wang B, Ding G, Yong J (2017) Fully convolutional neural networks with full-scale-features for semantic segmentation. In: AAAI, pp. 4240–4246

  6. Zhu H, Wang B, Zhang X, Liu J (2020) Semantic image segmentation with shared decomposition convolution and boundary reinforcement structure

  7. Ren S, He K, Girshick RB, Sun J (2015) Faster r-CNN: towards real-time object detection with region proposal networks. In: NIPS, pp. 91–99

  8. Yang C, Wu W, Wang Y, Zhou H (2021) A novel feature-based model for zero-shot object detection with simulated attributes

  9. Tian G, Liu J, Zhao H, Yang W (2021) Small object detection via dual inspection mechanism for uav visual images

  10. Wang J, Fu J, Xu Y, Mei T (2016) Beyond object recognition: Visual sentiment analysis with deep coupled adjective and noun neural networks. In: IJCAI, pp. 3484–3490

  11. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440

  12. Chen J, Chen Y, He Y, Xu Y, Zhao S, Zhang Y (2021) A classified feature representation three-way decision model for sentiment analysis

  13. Weiss KR, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J. Big Data 3:9

    Article  Google Scholar 

  14. Deng Z, Luo Y, Zhu J (2019) Cluster alignment with a teacher for unsupervised domain adaptation. In: ICCV, pp. 9943–9952

  15. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22 (10):1345–1359

    Article  Google Scholar 

  16. Shu R, Bui HH, Narui H, Ermon S (2018) A DIRT-t approach to unsupervised domain adaptation. In: ICLR (Poster)

  17. Yang H, He H, Zhang W, Bai Y, Li T (2021) Lie group manifold analysis: an unsupervised domain adaptation approach for image classification

  18. Alipour N, Tahmoresnezhad J (2021) Heterogeneous domain adaptation with statistical distribution alignment and progressive pseudo label selection

  19. Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: Maximizing for domain invariance

  20. Yan H, Ding Y, Li P, Wang Q, Xu Y, Zuo W (2017) Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In: CVPR, pp. 945–954

  21. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: ICML. Proceedings of Machine Learning Research, vol. 70, pp. 2208–2217

  22. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial nets. In: NIPS, pp. 2672–2680

  23. Ganin Y, Lempitsky VS (2015) Unsupervised domain adaptation by backpropagation. In: ICML

  24. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. In: ICML, vol. 37, pp. 97–105

  25. Xie S, Zheng Z, Chen L, Chen C (2018) Learning semantic representations for unsupervised domain adaptation. In: ICML

  26. Shen J, Qu Y, Zhang W, Yu Y (2018) Wasserstein distance guided representation learning for domain adaptation. In: AAAI

  27. Hu L, Kan M, Shan S, Chen X (2020) Unsupervised domain adaptation with hierarchical gradient synchronization. In: CVPR

  28. Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D (2016) Domain separation networks. In: NIPS, pp. 343– 351

  29. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: CVPR

  30. Hoffman J, Mohri M, Zhang N (2018) Algorithms and theory for multiple-source adaptation. In: NeurIPS, pp. 8256–8266

  31. Zhao H, Zhang S, Wu G, Moura JMF, Costeira JP, Gordon GJ (2018) Adversarial multiple source domain adaptation. In: NeurIPS, pp. 8568–8579

  32. Ma X, Zhang T, Xu C (2019) GCAN: graph convolutional adversarial network for unsupervised domain adaptation. In: CVPR, pp. 8266–8276

  33. Gholami B, Sahu P, Rudovic O, Bousmalis K, Pavlovic V (2020) Unsupervised multi-target domain adaptation: An information theoretic approach. IEEE Trans Image Process 29:3993–4002

    Article  MATH  Google Scholar 

  34. Wang Y, Zhang Z, Hao W, Song C (2021) Attention guided multiple source and target domain adaptation. IEEE Trans Image Process 30:892–906

    Article  Google Scholar 

  35. Cicek S, Soatto S (2019) Unsupervised domain adaptation via regularized conditional alignment. In: ICCV

  36. Lee S, Kim D, Kim N, Jeong S (2019) Drop to adapt: Learning discriminative features for unsupervised domain adaptation. In: ICCV

  37. Gu X, Sun J, Xu Z (2020) Spherical space domain adaptation with robust pseudo-label loss. In: CVPR

  38. Yang G, Ding M, Zhang Y (2021) Bi-directional class-wise adversaries for unsupervised domain adaptation

  39. Chen C, Xie W, Huang W, Rong Y, Ding X, Huang Y, Xu T, Huang J (2019) Progressive feature alignment for unsupervised domain adaptation. In: CVPR

  40. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: CVPR

  41. Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1-2):74–93

    Article  MATH  Google Scholar 

  42. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: CVPR

  43. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: CVPR, pp. 5150–5158

  44. Liu H, Shao M, Ding Z, Fu Y (2019) Structure-preserved unsupervised domain adaptation. IEEE Trans Knowl. Data Eng 31(4):799–812

    Article  Google Scholar 

  45. Luo Y, Ren C, Ge P, Huang K, Yu Y (2020) Unsupervised domain adaptation via discriminative manifold embedding and alignment. In: AAAI, pp. 5029–5036

  46. Xia H, Ding Z (2020) Structure preserving generative cross-domain learning. In: CVPR

  47. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. In: AAAI

  48. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: ECCV (4), vol. 6314, pp. 213–226

  49. Griffin G, Holub A, Perona P (2006) Caltech256 image dataset

  50. Caputo B, Müller H, Martínez-Gȯmez J, Villegas M, Acar B, Patricia N, Marvasti NB, Üsku̇darli S, Paredes R, Cazorla M, García-Varea I, Morell V (2014) Imageclef 2014: Overview and analysis of the results. In: CLEF, vol. 8685, pp. 192–211

  51. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: CVPR, pp. 5385–5394

  52. Kinnunen T, Kamarainen J, Lensu L, Lankinen J, Kälviȧinen H (2010) Making visual object categorization more challenging: Randomized caltech-101 data set. In: ICPR, pp. 476–479

  53. Wei J, Liang J, He R, Yang J (2018) Learning discriminative geodesic flow kernel for unsupervised domain adaptation. In: ICME

  54. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp. 770–778. IEEE Computer Society

  55. Deng J, Dong W, Socher R, Li L, Li K, Li F (2009) Imagenet: A large-scale hierarchical image database. In: CVPR, pp. 248– 255

  56. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  57. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: CVPR, pp. 2066–2073

  58. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. In: ICML

  59. Gu X, Sun J, Xu Z (2020) Spherical space domain adaptation with robust pseudo-label loss. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  60. Meng M, Chen Q, Wu J (2021) Structure preservation adversarial network for visual domain adaptation. Inf. Sci. 579:266– 280

    Article  Google Scholar 

  61. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. In: NeurIPS, pp. 1647–1657

  62. Liu H, Long M, Wang J, Jordan MI (2019) Transferable adversarial training: A general approach to adapting deep classifiers. In: ICML. Proceedings of Machine Learning Research, vol. 97, pp. 4013–4022. PMLR

  63. Saito K, Watanabe K, Ushiku Y, Harada T (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In: CVPR, pp. 3723–3732

  64. Wang X, Li L, Ye W, Long M, Wang J (2019) Transferable attention for domain adaptation. In: AAAI, pp. 5345–5352

  65. Zhang Y, Tang H, Jia K, Tan M (2019) Domain-symmetric networks for adversarial domain adaptation

  66. Cui S, Wang S, Zhuo J, Li L, Huang Q, Tian Q (2020) Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In: CVPR, pp. 3940–3949. Computer Vision Foundation / IEEE

  67. Xu R, Chen Z, Zuo W, Yan J, Lin L (2018) Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In: CVPR, pp. 3964–3973

  68. Chen C, Xie W, Wen Y, Huang Y, Ding X (2020) Multiple-source domain adaptation with generative adversarial nets. Knowl Based Syst 105962:199

    Google Scholar 

  69. Zhang Y, Liu T, Long M, Jordan MI (2019) Bridging theory and algorithm for domain adaptation. In: ICML. Proceedings of Machine Learning Research, vol. 97, pp. 7404–7413. PMLR

  70. Cui S, Wang S, Zhuo J, Su C, Huang Q, Tian Q (2020) Gradually vanishing bridge for adversarial domain adaptation CVPR, pp. 12452–12461. Computer Vision Foundation / IEEE

  71. Rauber PE, Falcão AX, Telea AC (2016) Visualizing time-dependent data using dynamic t-sne. In: Eurovis (short papers), pp. 73–77

  72. Hu C, Wang Y, Gu J (2020) Cross-domain intelligent fault classification of bearings based on tensor-aligned invariant subspace learning and two-dimensional convolutional neural networks. Knowl Based Syst 106214:209

    Google Scholar 

  73. Hu C, He S, Wang Y (2021) A classification method to detect faults in a rotating machinery based on kernelled support tensor machine and multilinear principal component analysis. Appl Intell 51(4):2609–2621

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 62172109 and Grant 62072118 ,in part by the Natural Science Foundation of Guangdong Province under Grant 2020A1515011361 and Grant 2022A1515010322, in part by the High-Level Talents Programme of Guangdong Province under Grant 2017GC010556, and in part by the Guangdong Basic and Applied Basic Research Foundation under Grant 2021B1515120010.

Author information

Authors and Affiliations

  1. Department of Computer Science, Guangdong University of Technology, 100 WaiHuanxi Road, Guangzhou, 510006, China

    Zhuanghui Wu, Min Meng, Tianyou Liang & Jigang Wu

Authors
  1. Zhuanghui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianyou Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jigang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Meng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Z., Meng, M., Liang, T. et al. Hierarchical Triple-Level Alignment for Multiple Source and Target Domain Adaptation. Appl Intell 53, 3766–3782 (2023). https://doi.org/10.1007/s10489-022-03638-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03638-6

Keywords

Search

Navigation