Informative discriminator for domain adaptation

https://doi.org/10.1016/j.imavis.2021.104180Get rights and content

Highlights

  • Proposemethodtousesourceinformationinascalablewayfordomaindiscriminator

  • Show that it helps to preserve the target samples mode information.

  • Propose a novel Sample Section Module

  • Provides additional insights into understanding our method

  • Results includes hierarchical class labels and statistical significance tests

  • Discrepancy distance and feature visualization for detailed analysis comprehensively

Abstract

In this paper, we consider the problem of domain adaptation for multi-class classification, where we are provided a labeled set of examples in a source dataset and target dataset with no supervision. We tackle the mode collapse problem in adapting the classifier across domains. In this setting, we propose an adversarial learning-based approach using an informative discriminator. Our observation relies on the analysis that shows if the discriminator has access to all the information available, including the class structure present in the source dataset, then it can guide the transformation of features of the target set of classes to a more structured adapted space. Further, by training the informative discriminator using the more robust source samples, we are able to obtain better domain invariant features. Using this formulation, we achieve state-of-the-art results for the standard evaluation on benchmark datasets. We also provide detailed analysis, which shows that using all the labeled information results in an improved domain adaptation.

Introduction

Many computer vision tasks such as object classification, detection, and segmentation have been solved by deep learning and machine learning frameworks. The major limitation of these models is that they required extensive training data and are not generalized well for the dataset bias. However, it was shown by Tzeng et al. [1] that while generically trained deep networks have a reduced dataset bias, there still exists a domain shift between different datasets, and it is required to adapt the features appropriately. One possible and successful approach to tackle the domain adaptation problem using adversarial learning. Ganin et al. [2] use a binary adversarial discriminator and solved this problem by adding an auxiliary task solves the problem of domain classification using the unsupervised domain adaptation through backpropagation The main observation for this method is that for classifiers to be adapted across domains, the domain classifier should fail. This can be easily achieved through a gradient reversal layer that modifies features to worsen the ability to classify domains, and it requires no labels to be available in the target dataset. But, due to the limited capacity of using a binary discriminator, it introduces a problem of mode collapse in the feature space for source and target domains. The binary discriminator tends to classify all the target (or source) samples into a single domain class. It can lead to loose the class discriminative property of features and introduce the mode collapse in the domain adaptation. In other words, all target images collapse to only one value and satisfy the domain invariant condition. This situation is known as mode collapse, which is very common in adversarial learning methods (GANs) [3], [4]. In contrast to a binary discriminator, the proposed informative discriminator considers all the source label information and encourages each target sample to be misclassified into only one source class. If target image features are misclassified into all the source classes equally, the target features lose their class variant property. This causes the mode collapse and is avoided by the proposed informative discriminator.

It helps that any target sample does not mix with multiple modes. For the source sample, the multiple modes are preserved by the regular classifier. The binary discriminator model [2] is considered as a baseline for the proposed model. Further, we provide more informative source samples to the discriminator based on the classifier's performance. The informative samples are referred here to the samples which have distinct class boundaries. If the classifier predicts the correct class label for the source samples, these samples are used to train the discriminator. This additional information also helps the discriminator get the mode information correctly, reducing the negative transfer introduced by the ambiguous source samples. Thus the target samples are cleanly separable class-wise by adversarial learning.

In Fig. 1. we provide an example of a binary classification problem, where features are domain invariant but have the mode collapse problem; domain invariant means that there are no domain classifier exits that can separate the source and target sample. In the figure, the red points correspond to the source sample, and the blue points are the target sample. Fig. 1(a) shows the adaptation after the baseline model, where the discriminator can access only the domain information, and Fig. 1(b) shows the adaptation by when the discriminator can access the source label information. From Fig. 1(a), it is clear that the target domain is inseparable with only some of the source samples (that are near the classifier's boundary). However, Fig. 1(b), when trained with an informative discriminator, all the target samples should also be invariant to all the source samples.

In this paper, we show that providing all the possible information of the source data to the adversarial discriminator at training time helping to improved performance. We obtain an impressive improvement of 9% and 8% over baseline (binary discriminator) on the Amazon-DSLR and Amazon-Webcam adaptation task, respectively. The proposed method also obtain an improvement of 6.21% over the very recently proposed approach that also considers introducing additional source label information [5].

To summarize, this paper makes the following contributions:

 We propose a scalable domain adaptation method that effectively utilizes the source data information to train the informative discriminator.

 We demonstrate that training the informative discriminator using the source class labels helps to preserve the target samples' mode information.

 Further, we show that providing only the source samples that are more informative about the mode information to the discriminator reduces the negative transfer.

 In this paper, we thoroughly analyze the proposed method by providing additional insights into the results. We provide the results for discrepancy distance, hyper-parameter sensitivity, statistical significance tests, feature visualization, and results on the inclusion of hierarchical class labels. These detailed analyses comprehensively support our claims regarding the efficacy of the proposed approach.

Section snippets

Motivation

In the adversarial domain adaptation models, the discriminator plays an essential role in guides the feature extractor to obtain the domain invariant features. The invariant features are obtained by misclassifying the source and target domain only by the discriminator's domain knowledge. In the domain adaptation problem, we have more rich information about the data distribution. For example, in the source data sample, we have the class information along with the domain knowledge. Further, by

Related work

The domain adaptation problem in the literature of computer vision is tackled by minimizing the discrepancy between source and target domain. The maximum means discrepancy (MMD) [1] uses the kernel distance between source and target distribution at the last convolution layer and tries to minimize it by the loss function. Similarly, the idea of domain adaptation network (DAN) [6] is based on minimizing the multi-kernel MMD between other layers. Residual domain adaptation network (RTN) [7]

Background: Discriminator for domain adaptation

In the generative adversarial network (GAN) [13] and its variants are based on the adversarial loss to train the generator. The objective of the adversarial loss to train the generator to learn the true data distributions. In this setting, only the discriminator can access the true data distribution. However, the binary discriminator is trained without considering the complex multimodal structures of the dataset, which results in the feature or images loose the datasets' class discriminative

Problem description

We define unsupervised domain adaptation setup where no labels present for the target domain during training time. We can access only the labeled source data and unlabeled target data. More formally, we are given data for a source domain, S = (xis, yis)i=1ns of ns labeled examples and a target domain, T = (xit)i=1nt of nt unlabeled examples. The labels are not provided. We further assume that source data is sampled from joint distributions P(Xs, Ys), while target data is sampled from Q(Xt, Yt). We

Results & experiments

For evaluation of the proposed model, we follow common protocol defined in [9]. We use the both Alexnet [48] and Alexnet [48] as a backbone for the adaptation model. Both the models are pretrained on Imagenet dataset. We compare our model on the state-of-art methods such as [1], [2], [5], [6], [7], [9], [15], [27], [31], [49], [50], [51], [52]. We use standard dataset such as Office-31 [53], Office-Home datasets [52], Caltech-Bing datasets [54] and ImageCLEF datasets. The other implementation

Analysis

We provide analysis of IDDA and IDDA-S using statistical significance test, distribution discrepancy, hyperparameter sensitivity, and analysis on the number of source data samples. We also provide the feature visualization using tSNE plots.

Conclusion

We propose a method for obtaining an informative discriminator that aids improved domain adaptation. We also show that we learn better domain invariant representations by providing the confident source samples to the discriminator. The thorough analysis of informed discriminators shows that this discriminator helps us obtain statistically significant improvement that the results can also justify. Sample selection module indeed helpful for making the discriminator's training robust. We further

Declarations of interest

none.

CRediT authorstatement

Vinod K. Kurmi: Conceptualization, Datacuration, Formalanalysis, Investigation, Methodology, Project administration, Validation,Visualization, Writing original draft, Writing review. Venkatesh K. Subramanian: Supervision, Validation Vinay P. Namboodiri: Supervision, Validation, Writing review and editing, Funding acquisition.

Declaration of Competing Interest

None.

References (78)

  • E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, T. Darrell, Deep domain confusion: Maximizing for domain invariance, arXiv...
  • Y. Ganin et al.

    Unsupervised domain adaptation by backpropagation

  • A. Srivastava, L. Valkov, C. Russell, M. U. Gutmann, C. Sutton, Veegan: reducing mode collapse in gans using implicit...
  • S.-W. Park et al.

    Began v3: avoiding mode collapse in gans using variational inference

    Electronics

    (2020)
  • Z. Pei et al.

    Multi-adversarial domain adaptation

  • M. Long et al.

    Learning transferable features with deep adaptation networks

  • M. Long et al.

    Unsupervised domain adaptation with residual transfer networks

  • B. Sun et al.

    Correlation alignment for unsupervised domain adaptation

    Domain Adaptation Comp. Vision Appl.

    (2017)
  • B. Sun et al.

    Deep coral: Correlation alignment for deep domain adaptation

  • M. Arjovsky, S. Chintala, L. Bottou, Wasserstein gan, arXiv preprint...
  • P. Haeusser et al.

    Associative domain adaptation, in: International Conference on Computer Vision (ICCV)

    Vol.

    (2017)
  • P.O. Pinheiro

    Unsupervised domain adaptation with similarity learning

  • I. Goodfellow et al.

    Generative adversarial nets

  • M. Ghifary et al.

    Domain generalization for object recognition with multi-task autoencoders

  • Z. Murez et al.

    Image to image translation for domain adaptation

  • K. Bousmalis et al.

    Unsupervised pixel-level domain adaptation with generative adversarial networks

  • Y. Choi et al.

    Stargan: Unified generative adversarial networks for multi-domain image-to-image translation

  • S. Sankaranarayanan et al.

    Generate to adapt: Aligning domains using generative adversarial networks

  • Y.-C. Liu et al.

    Detach and adapt: Learning cross-domain disentangled deep representation

  • J. Hoffman et al.

    Cycada: Cycle-consistent adversarial domain adaptation

  • J.-Y. Zhu et al.

    Unpaired image-to-image translation using cycle-consistent adversarial networks

  • K. Bousmalis et al.

    Domain separation networks

  • A. Rozantsev et al.

    Beyond Sharing Weights for Deep Domain Adaptation

    (2018)
  • K. Saito et al.

    Asymmetric tri-training for unsupervised domain adaptation

  • A. Rozantsev et al.

    Residual Parameter Transfer for Deep Domain Adaptation, in: Conference on Computer Vision and Pattern Recognition, no

    (2018)
  • Y. Li et al.

    Revisiting Batch Normalization for Practical Domain Adaptation

    (2016)
  • F.M. Carlucci et al.

    Autodial: Automatic domain alignment layers

  • K. Saito et al.

    Maximum classifier discrepancy for unsupervised domain adaptation

  • K. Saito et al.

    Adversarial dropout regularization

    (2017)
  • E. Tzeng et al.

    Adversarial discriminative domain adaptation

  • M. Long et al.

    Conditional adversarial domain adaptation

  • V.K. Kurmi et al.

    Attending to discriminative certainty for domain adaptation

  • H. Li et al.

    Domain generalization with adversarial feature learning

  • V. K. Kurmi, V. Bajaj, V. K. Subramanian, V. P. Namboodiri, Curriculum based dropout discriminator for domain...
  • W. Zhang et al.

    Self-Paced Collaborative and Adversarial Network for Unsupervised Domain Adaptation

    (2019)
  • R. Volpi et al.

    Adversarial feature augmentation for unsupervised domain adaptation

  • Q. Chen et al.

    Re-weighted adversarial adaptation network for unsupervised domain adaptation

  • J. Zhang et al.

    Importance weighted adversarial nets for partial domain adaptation

  • J. Wen et al.

    Bayesian uncertainty matching for unsupervised domain adaptation

  • Cited by (2)

    View full text