Yajing Liu
Skip Abstract Section
Abstract
Domain generalization aims at generalizing the network trained on multiple domains to unknown but related domains. Under the assumption that different domains share the same classes, previous works can build relationships across domains. However, in realistic scenarios, the change of domains is always followed by the change of categories, which raises a difficulty for collecting sufficient aligned categories across domains. Bearing this in mind, this article introduces union domain generalization (UDG) as a new domain generalization scenario, in which the label space varies across domains, and the categories in unknown domains belong to the union of all given domain categories. The absence of categories in given domains is the main obstacle to aligning different domain distributions and obtaining domain-invariant information. To address this problem, we propose category-stitch learning (CSL), which aims at jointly learning the domain-invariant information and completing missing categories in all domains through an improved variational autoencoder and generators. The domain-invariant information extraction and sample generation cross-promote each other to better generalizability. Additionally, we decouple category and domain information and propose explicitly regularizing the semantic information by the classification loss with transferred samples. Thus our method can breakthrough the category limit and generate samples of missing categories in each domain. Extensive experiments and visualizations are conducted on MNIST, VLCS, PACS, Office-Home, and DomainNet datasets to demonstrate the effectiveness of our proposed method.
- [1] . 2018. Metareg: Towards domain generalization using meta-regularization. In Proceedings of the Advances in Neural Information Processing Systems. 998–1008.Google Scholar
- [2] . 2012. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of the ICML Workshop on Unsupervised and Transfer Learning. 37–49.Google Scholar
- [3] . 2007. Analysis of representations for domain adaptation. In Proceedings of the Advances in Neural Information Processing Systems. 137–144.Google Scholar
- [4] . 2018. Partial adversarial domain adaptation. In Proceedings of the European Conference on Computer Vision. 135–150.Google ScholarDigital Library
- [5] . 2019. Learning to transfer examples for partial domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [6] . 2019. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2229–2238.Google ScholarCross Ref
- [7] . 2010. Exploiting hierarchical context on a large database of object categories. InProceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Google Scholar
- [8] . 2018. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8789–8797.Google ScholarCross Ref
- [9] . 2020. Adaptive exploration for unsupervised person re-identification. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 1(2020), 19 pages.
DOI: Google ScholarDigital Library - [10] . 2016. Tutorial on variational autoencoders. arXiv:1606.05908. Retrieved from https://arxiv.org/abs/1606.05908.Google Scholar
- [11] . 2010. The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88, 2 (2010), 303–338.Google ScholarDigital Library
- [12] . 2018. Self-ensembling for visual domain adaptation. In Proceedings of the International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=rkpoTaxA-.Google Scholar
- [13] . 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096–2030.Google ScholarCross Ref
- [14] . 2017. Scatter component analysis: A unified framework for domain adaptation and domain generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 7 (2017), 1414–1430.Google ScholarDigital Library
- [15] . 2015. Domain generalization for object recognition with multi-task autoencoders. In Proceedings of the IEEE International Conference on Computer Vision. 2551–2559.Google ScholarDigital Library
- [16] . 2007. Caltech-256 Object Category Dataset. California Institute of Technology.Google Scholar
- [17] . 2012. Undoing the damage of dataset bias. In Proceedings of the European Conference on Computer Vision. Springer, 158–171.Google ScholarDigital Library
- [18] . 2021. Bottom-up and layerwise domain adaptation for pedestrian detection in thermal images. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1(2021), 19 pages.
DOI: Google ScholarDigital Library - [19] . 2013. Auto-encoding variational bayes. arXiv:1312.6114. Retrieved from https://arxiv.org/abs/1312.6114.Google Scholar
- [20] . 2018. Learning latent subspaces in variational autoencoders. In Proceedings of the Advances in Neural Information Processing Systems. 6444–6454.Google Scholar
- [21] . 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems. 1097–1105.Google ScholarDigital Library
- [22] . 1997. Information Theory and Statistics. Courier Corporation.Google Scholar
- [23] . 2017. Deeper, broader and artier domain generalization. In Proceedings of the 2017 IEEE International Conference on Computer Vision. IEEE, 5543–5551.Google ScholarCross Ref
- [24] . 2018. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google ScholarCross Ref
- [25] . 2019. Episodic training for domain generalization. In Proceedings of the International Conference on Computer Vision. Institute of Electrical and Electronics Engineers (IEEE).Google ScholarCross Ref
- [26] . 2018. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision. 624–639.Google ScholarDigital Library
- [27] . 2019. Feature-critic networks for heterogeneous domain generalization. In Proceedings of the International Conference on Machine Learning. 3915–3924.Google Scholar
- [28] . 2019. Compact feature learning for multi-domain image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [29] . 2021. Domain Generalization via Encoding and Resampling in a Unified Latent Space. In IEEE Transactions on Multimedia.
DOI: Google ScholarCross Ref - [30] . 2021. Generalized domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1084–1093.Google ScholarCross Ref
- [31] . 2013. Domain generalization via invariant feature representation. In Proceedings of the International Conference on Machine Learning. 10–18.Google Scholar
- [32] . 2017. Conditional image synthesis with auxiliary classifier gans. In Proceedings of the International Conference on Machine Learning. PMLR, 2642–2651.Google Scholar
- [33] . 2009. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2009), 1345–1359.Google ScholarDigital Library
- [34] . 2020. Exploring category-agnostic clusters for open-set domain adaptation. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.13864–13872.
DOI: Google ScholarCross Ref - [35] . 2019. Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2234–2242.
DOI: Google ScholarCross Ref - [36] . 2020. Learning explanations that are hard to vary. (2020).Google Scholar
- [37] . 2019. Moment matching for multi-source domain adaptation. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 1406–1415.
DOI: Google ScholarCross Ref - [38] . 2019. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision. 1406–1415.Google ScholarCross Ref
- [39] . 2008. LabelMe: A database and web-based tool for image annotation. International Journal of Computer Vision 77, 1–3 (2008), 157–173.Google ScholarDigital Library
- [40] . 2019. Gradient matching generative networks for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2168–2178.Google ScholarCross Ref
- [41] . 2018. Generalizing across domains via cross-gradient training. In Proceedings of the International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=r1Dx7fbCW.Google Scholar
- [42] . 2015. Learning structured output representation using deep conditional generative models. In Proceedings of the Advances in Neural Information Processing Systems. 3483–3491.Google Scholar
- [43] . 2016. Generalized deep transfer networks for knowledge propagation in heterogeneous domains. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 4s(2016), 22 pages.
DOI: Google ScholarDigital Library - [44] . 2018. Latent domain transfer: Crossing modalities with bridging autoencoders. In Proceedings of the ICLR 2019 Conference on Blind Submission.Google Scholar
- [45] . 2011. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1521–1528.Google ScholarDigital Library
- [46] . 2017. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5018–5027.Google ScholarCross Ref
- [47] . 2019. SDIT: Scalable and diverse cross-domain image translation. In Proceedings of the 27th ACM International Conference on Multimedia. 1267–1276.Google ScholarDigital Library
- [48] . 2018. Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3964–3973.Google ScholarCross Ref
- [49] . 2014. Exploiting low-rank structure from latent domains for domain generalization. In Proceedings of the European Conference on Computer Vision. Springer, 628–643.Google ScholarCross Ref
- [50] . 2019. Heterogeneous domain adaptation via soft transfer network. In Proceedings of the 27th ACM International Conference on Multimedia. 1578–1586.Google ScholarDigital Library
- [51] . 2021. Equivariant adversarial network for image-to-image translation. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 2s(2021), 14 pages.
DOI: Google ScholarDigital Library - [52] . 2020. Domain generalization via entropy regularization. Advances in Neural Information Processing Systems 33 (2020).Google Scholar
Index Terms
- Category-Stitch Learning for Union Domain Generalization
Recommendations
MultiMatch: Multi-task Learning for Semi-supervised Domain Generalization
Domain generalization (DG) aims at learning a model on source domains to well generalize on the unseen target domain. Although it has achieved great success, most of the existing methods require the label information for all training samples in source ...
Learning Domain-Invariant Representations from Text for Domain Generalization
Pattern Recognition and Computer VisionAbstractDomain generalization (DG) aims to transfer the knowledge learned in the source domain to the unseen target domain. Most DG methods focus on studying how to learn domain-invariant representations that remain invariant across different domains. For ...
Achieving domain generalization for underwater object detection by domain mixup and contrastive learning
AbstractThe performance of existing underwater object detection methods severely degrades when they face the domain shift caused by complicated underwater environments. Due to the limited domain diversity in collected data, deep detectors ...
Comments