Partial domain adaptation based on shared class oriented adversarial network

https://doi.org/10.1016/j.cviu.2020.103018Get rights and content

Highlights

  • A deep-based model for solving partial domain adaptation.

  • Combine adversarial network and Maximum Mean Discrepancy (MMD) to bridge domain gap.

  • Propose a weighted class sampler module to circumvent negative transfer.

  • Improve the generalization ability of the network via multi-classifier module.

  • Experiments on benchmark datasets verify the effectiveness of the proposed model.

Abstract

Most existing domain adaptation methods assume that the label space of the source domain is the same as the label space of the target domain. However, this assumption is generally untenable due to the differences between the two domains. Therefore, a novel domain adaptation paradigm called Partial Domain Adaptation (PDA), which only assumes that the source label space is large enough to subsume the target label space has been proposed recently to relax such strict assumption. Previous partial domain adaptation methods mainly utilize weighting mechanisms to alleviate negative transfer caused by outlier classes samples. Though these methods have achieved high performance in PDA tasks, all the heterogeneous data is retained during the whole training process, which still contributes to negative transfer. In this work, we propose a shared class oriented adversarial network (SCOAN) for partial domain adaptation. Outlier samples are excluded from training process via weighting strategy to entirely circumvent negative transfer and positive transfer is performed by combining adversarial network and Maximum Mean Discrepancy (MMD) to bridge domain gap. Multi-classifier module is proposed to further improve the generalization ability of the network. Extensive experiments show that SCOAN achieves state-of-the-art results on several benchmark partial domain adaptation datasets.

Introduction

Deep learning algorithms have achieved significant performance in various machine learning tasks (Goodfellow et al., 2014, He et al., 2016) and practical applications (Girshick et al., 2014, Long et al., 2015b, Ren et al., 2015). Such impressive performance highly depends on the number of labeled training samples. However, manual annotation of sufficient data is time-consuming and expensive. Domain adaptation, which transfers knowledge from a distinct yet related source domain to a target domain has been proposed to reduce the cost for labeling. Though this transfer learning paradigm seems promising, it suffers from distribution shift across different domains (Pan and Yang, 2010).

Most existing domain adaptation methods assume that the labeled source domain and the unlabeled target domain have the same label space yet different feature distributions. To enable positive knowledge transfer from one domain to another, previous works Tzeng et al., 2014, Long et al., 2015a and Sun and Saenko (2016) try to reduce the distribution discrepancy between two domains. In real world applications, we can hardly find two domains share exactly identical label space, and thus the assumption may no longer be valid. In the big data era, large-scale dataset (e.g. ImageNet, Deng et al., 2009) is available to improve the performance of our target tasks, and its label space generally contains the label space of most target tasks. Therefore, Cao et al. (2018a) propose a novel setting called partial domain adaptation that assumes the target label space is a subspace of the source label space and the set of classes in the target domain is unknown, as illustrated in Fig. 1. In this case, partial domain adaptation is more challenging compared with standard domain adaptation for the differences of both the label space and feature distributions.

In partial domain adaptation, directly matching the whole distributions between two domains may suffer from negative knowledge transfer caused by heterogeneous samples. To address such a problem, an ideal method is to entirely exclude the outlier samples whose labels are not contained in the target label space from the training phase to circumvent negative transfer. Furthermore, reducing the distribution discrepancy between the two domains is essential to enable positive transfer. In this work, we solve the problems of partial domain adaptation with an end-to-end Shared Class Oriented Adversarial Network (SCOAN) as shown in Fig. 2.

SCOAN overcomes the challenges caused by the differences of the label spaces and feature distribution respectively. First, we propose a class-level importance weighted module based on Cao et al. (2018a) to identify the shared categories between the two domains. Then a corresponding adaptive weighted sampler filters out heterogeneous samples during learning process. Thus effective negative transfer alleviation is performed with fewer outlier classes. Second, besides adopting adversarial learning to match the two domain distributions implicitly, we also leverage the Maximum Mean Discrepancy (MMD) objective to bridge the domain gap explicitly to promote positive transfer. Third, we propose a simple yet effective masked classification strategy to enable shared class oriented classification, which helps improve the performance. The generalization ability of SCOAN is further enhanced by embedding multiple classifiers.

The main contributions of this paper can be summarized as follows:

  • We propose a novel shared class sampler combined with weighting mechanisms to avoid negative transfer in partial domain adaptation.

  • Both adversarial network and distribution discrepancy metric (i.e. MMD) is adopted to further promote positive transfer.

  • The proposed masked classification enables shared class oriented classification while the adopted ensemble learning strategy improves the generalization ability of SCOAN.

  • Extensive experiments show that SCOAN achieves state-of-the-art performance on several partial domain adaptation datasets.

The remainder of this paper is organized as follows. Section 2 discusses the related works of SCOAN. Section 3 describes the proposed approach in detail. In Section 4, our method is evaluated on several benchmark datasets. Section 5 concludes this paper.

Section snippets

Related work

The rapid development of deep neural networks has greatly promoted the progress and development of image recognition (He et al., 2016), object detection (Cheng et al., 2019), semantic segmentation (Long et al., 2015b) and other related fields in recent years. Researches Bengio, 2012, Yosinski et al., 2014a and Donahue et al. (2014) have demonstrated that deep neural networks can disentangle explanatory factors of differences underlying samples and organize deep stratified representations based

Shared Class Oriented Adversarial Network

In this section, we propose a Shared Class Oriented Adversarial Network (SCOAN) for partial domain adaptation as shown in Fig. 2. The settings and notations of partial domain adaptation is described in Section 3.1. Section 3.2 discusses how to employ adversarial framework to learn domain-invariant features and use discriminators to quantify the transferability of each source class. Section 3.3 presents how to enable positive transfer by leveraging MMD to explicitly bridging domain distribution

Experiments

Experiments are conducted on several benchmark datasets to investigate the proposed approach against many state-of-the-art (partial) domain adaptation approaches. Further ablation study is also performed to evaluate the efficacy of proposed modules of SCOAN.

Conclusion

This paper proposes a Shared Class Oriented Adversarial Network (SCOAN) based on partial adversarial domain adaptation (Cao et al., 2018a). We leverage a weighted class sampler to effectively circumvent negative transfer caused by the outlier classes, and shared class oriented classification is realized by using the weighted sampler output as an associated mask. MMD metric is further utilized to enable positive transfer by minimizing the distribution discrepancy in the shared label space. To

CRediT authorship contribution statement

Wenjie Qiu: Conceptualization, Methodology, Software. Wendong Chen: Writing - original draft, Visualization, Validation. Haifeng Hu: Supervision, Writing - review & editing, Investigation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China (61673402, 61273270 and 60802069), the Natural Science Foundation of Guangdong Province (2017A030311029), and the Science and Technology Program of Guangzhou, China, under Grant 201704020180, and the Fundamental Research Funds for the Central Universities of China .

References (40)

  • DingM. et al.

    Cross-domain mapping learning for transductive zero-shot learning

    Comput. Vis. Image Underst.

    (2019)
  • Bengio, Y., 2012. Deep learning of representations for unsupervised and transfer learning. In: Unsupervised and...
  • Cao, Z., Long, M., Wang, J., Jordan, M.I., 2018a. Partial transfer learning with selective adversarial networks. In:...
  • Cao, Z., Ma, L., Long, M., Wang, J., 2018b. Partial adversarial domain adaptation. In:...
  • CaoZ. et al.

    Learning to transfer examples for partial domain adaptation

    CoRR

    (2019)
  • Chapelle, O., Zien, A., 2005. Semi-Supervised classification by low density separation. In: Proceedings of the Tenth...
  • ChengG. et al.

    Learning rotation-invariant and Fisher discriminative convolutional neural networks for object detection

    IEEE Trans. Image Process.

    (2019)
  • Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L., 2009. ImageNet: A Large-scale hierarchical image...
  • Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T., 2014. DeCAF: A deep convolutional...
  • GaninY. et al.

    Unsupervised domain adaptation by backpropagation

  • GaninY. et al.

    Domain-adversarial training of neural networks

    J. Mach. Learn. Res.

    (2016)
  • GirshickR. et al.

    Rich feature hierarchies for accurate object detection and semantic segmentation

  • Gong, B., Shi, Y., Sha, F., Grauman, K., 2012. Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE...
  • GoodfellowI. et al.

    Generative adversarial nets

  • Grandvalet, Y., Bengio, Y., 2004. Semi-supervised learning by entropy minimization. In:...
  • GrettonA. et al.

    A kernel two-sample test

    J. Mach. Learn. Res.

    (2012)
  • GrettonA. et al.

    Optimal kernel choice for large-scale two-sample tests

  • GriffinG. et al.

    Caltech-256 Object Category DatasetCalTech Report

    (2007)
  • He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. in: 2016 IEEE Conference on...
  • KrizhevskyA. et al.

    Imagenet classification with deep convolutional neural networks

  • Cited by (0)

    View full text