Skip to main content
SpringerLink
Log in

Cross-Domain Gated Learning for Domain Generalization

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Domain generalization aims to improve the generalization capacity of a model by leveraging useful information from the multi-domain data. However, learning an effective feature representation from such multi-domain data is challenging, due to the domain shift problem. In this paper, we propose an information gating strategy, termed cross-domain gating (CDG), to address this problem. Specifically, we try to distill the domain-invariant feature by adaptively muting the domain-related activations in the feature maps. This feature distillation process prevents the network from overfitting to the domain-related detailed information, and thereby improves the generalization ability of learned feature representation. Extensive experiments are conducted on three public datasets. The experimental results show that the proposed CDG training strategy can excellently enforce the network to exploit the intrinsic features of objects from the multi-domain data, and achieve a new state-of-the-art domain generalization performance on these benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. The ResNet-50 achieves higher Top-1 accuracy than AlexNet on the ImageNet dataset; therefore, ResNet-50 is seen as the network with the higher capacity.

References

  • Amjad, R. A., & Geiger, B. C. (2020). Learning representations for Neural Network-Based Classification using the information bottleneck principle. Transactions on Pattern Analysis and Machine Intelligence, 42, 2225–2239.

    Article  Google Scholar 

  • Balaji, Y., Sankaranarayanan, S., & Chellappa, R. (2018). MetaReg: Towards domain generalization using meta-regularization. In Advances in neural information processing systems.

  • Carlucci, F.M., D’Innocente, A., Bucci, S., Caputo, B., Tommasi, T. (2019). Domain generalization by solving Jigsaw Puzzles. In conference on computer vision and pattern recognition.

  • Chattopadhyay, A., Sarkar, A., Howlader, P., & Balasubramanian, V.N. (2018). Grad-CAM++: generalized gradient-based visual explanations for Deep Convolutional Networks. In IEEE winter conference on applications of computer vision.

  • Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., & Chua, T. (2017). SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834–848.

    Article  Google Scholar 

  • Choi, M.J., Lim, J.J., Torralba, A., & Willsky, A.S. (2010). Exploiting hierarchical context on a large database of object categories. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Devries, T., & Taylor, G.W. (2017). Improved regularization of convolutional neural networks with Cutout. Preprint retrieved from arXiv: 1708.04552

  • Dou, Q., de Castro, D.C., Kamnitsas, K., & Glocker, B. (2019). Domain generalization via model-agnostic learning of semantic features. In Advances in Neural Information Processing Systems.

  • Du, D., Wang, L., Wang, H., Zhao, K., & Wu, G. (2019). Translate-to-recognize networks for RGB-D scene recognition. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Du, Y., Xu, J., Xiong, H., Qiu, Q., Zhen, X., Snoek, CGM., & Shao, L. (2020). Learning to learn with variational information bottleneck for domain generalization. European Conference on Computer Vision. Springer.

  • Everingham, M., Gool, L. V., Williams, C. K. I., Winn, J. M., & Zisserman, A. (2010). The Pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88, 303–338.

    Article  Google Scholar 

  • Fang, C., Xu, Y., Rockmore, D.N. (2013). Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In IEEE International Conference on Computer Vision.

  • Federici, M., Dutta, A., Forré, P., Kushman, N., & Akata, Z. (2020). Learning Robust Representations via Multi-View Information Bottleneck. Preprint retrieved from arXiv:2002.07017.

  • Ganin, Y., & Lempitsky, V.S. (2015). Unsupervised domain adaptation by Backpropagation. In International Conference on Machine Learning.

  • Ghiasi, G., Lin, T., Le, Q.V. (2018). DropBlock: A regularization method for convolutional networks. In: Advances in Neural Information Processing Systems.

  • Ghifary, M., Kleijn, W.B., Zhang, M., & Balduzzi, D. (2015). Domain generalization for object recognition with multi-task autoencoders. In IEEE International Conference on Computer Vision.

  • Girshick, R.B. (2015). Fast R-CNN. In IEEE International Conference on Computer Vision.

  • Gong, B., Grauman, K., & Sha, F. (2014). Learning kernels for unsupervised domain adaptation with applications to visual object recognition. International Journal of Computer Vision, 109, 3–27.

    Article  MathSciNet  Google Scholar 

  • Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014). Learning rich features from RGB-D images for object detection and segmentation. In European Conference on Computer Vision, Springer, pp. 345–360.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition.

  • He, K., Gkioxari, G., Dollár, P., & Girshick, R. B. (2020). Mask R-CNN. IEEE Transaction on Pattern Analysis and Machine Intelligence, 42(2), 386–397.

    Article  Google Scholar 

  • Huang, Z., Wang, H., Xing, E.P., & Huang, D. (2020). Self-challenging improves cross-domain generalization. In European Conference on Computer Vision.

  • Kolchinsky, A., Tracey, B. D., & Kuyk, S. V. (2019). Caveats for information bottleneck in deterministic scenarios. Preprint retrieved from arXiv:1808.07593.

  • Krizhevsky, A., Sutskever, I., Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems.

  • LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. https://doi.org/10.1109/5.726791.

    Article  Google Scholar 

  • Li, F., Fergus, R., & Perona, P. (2004). Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.

  • Li, H., Pan, S.J., Wang, S., Kot, A.C. (2018). Domain generalization with adversarial feature learning. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Li, D., Yang, Y., Song, Y., & Hospedales, T.M. (2017). Deeper, Broader and Artier domain generalization. In International Conference on Computer Vision.

  • Li, Y., Yang, Y., Zhou, W., & Hospedales, T. M. (2019). Feature-critic networks for heterogeneous domain generalization. InInternational Conference on Machine Learning (pp. 3915-3924). PMLR

  • Li, D., Zhang, J., Yang, Y., Liu, C., Song, Y., Hospedales, T.M. (2019a). Episodic training for domain generalization. In IEEE International Conference on Computer Vision.

  • Li, H., Wan, R., Wang, S., & Kot, A. C. (2020). Unsupervised domain adaptation in the wild via disentangling representation learning. International Journal of Computer Vision, 129, 267–283.

    Article  MathSciNet  Google Scholar 

  • Long, M., Cao, Y., Wang, J., & Jordan, M.I. (2015b). Learning transferable features with deep adaptation networks. In International Conference on Machine Learning.

  • Long, J., Shelhamer, E., & Darrell, T. (2015a). Fully convolutional networks for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition.

  • McIlraith, S.A., & Weinberger, K.Q. (2017). Learning to generalize: Meta-learning for domain generalization. In AAAI Conference on Artificial Intelligence.

  • Moreno-Torres, J. G., Raeder, T., Alaíz-Rodríguez, R., Chawla, N. V., & Herrera, F. (2012). A unifying view on dataset shift in classification. Pattern Recognition, 45, 521–530.

    Article  Google Scholar 

  • Motiian, S., Piccirilli, M., Adjeroh, D.A., & Doretto, G. (2017). Unified deep supervised domain adaptation and generalization. In IEEE International Conference on Computer Vision.

  • Muandet, K., Balduzzi, D., & Schölkopf, B. (2013). Domain generalization via invariant feature representation. In International Conference on Machine Learning.

  • Omeiza, D., Speakman, S., Cintas, C., & Weldemariam, K. (2019). Smooth Grad-CAM++: An enhanced inference level visualization technique for deep convolutional neural network models. Preprint retrieved from arXiv: 1908.01224.

  • Park, S., Kwak, N. (2016). Analysis on the Dropout effect in convolutional neural networks. In Asian Conference on Computer Vision.

  • Park, S., Park, J., Shin, S., & Moon, I. (2018). Adversarial Dropout for supervised and semi-supervised learning. In AAAI Conference on Artificial Intelligence.

  • Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., & Chintala. S. (2019). PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems.

  • Peng, X. B., Kanazawa, A., Toyer, S., Abbeel, P., & Levine, S. (2019). Variational discriminator bottleneck: Improving imitation learning, inverse RL, and GANs by constraining information flow. Preprint retrieved from arXiv:1810.00821

  • Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008). LabelMe: A database and web-based tool for image annotation. International Journal of Computer Vision, 77, 157–173.

    Article  Google Scholar 

  • Saito, K., Kim, D., Sclaroff, S., Darrell, T., & Saenko, K. (2019). Semi-supervised domain adaptation via minimax entropy. In IEEE International Conference on Computer Vision.

  • Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision.

  • Shankar, S., Piratla, V., Chakrabarti, S., Chaudhuri, S., Jyothi, P., & Sarawagi, S. (2018). Generalizing across domains via cross-gradient training. Preprint retrieved from arXiv:1804.10745

  • Shwartz-Ziv, R., & Tishby, N. (2017). Opening the Black Box of Deep Neural Networks via information. Preprint retrieved from arXiv:1703.00810.

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Preprint retrieved from arXiv:1409.1556.

  • Simonyan, K., & Zisserman, A. (2014a). Two-stream convolutional networks for action recognition in videos. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, & K.Q. Weinberger (Eds.) NIPS, pp. 568–576.

  • Singh, K.K., Lee, Y.J. (2017). Hide-and-Seek: Forcing a network to be meticulous for weakly-supervised object and action localization. In IEEE International Conference on Computer Vision.

  • Tishby, N., & Zaslavsky, N. (2015). Deep learning and the information Bottleneck principle. In Information Theory Workshop.

  • Tishby, N., Pereira, F.C.N., & Bialek, W. (2000). The information bottleneck method. Preprint retrieved from arXiv:physics/0004057

  • Tompson, J., Goroshin, R., Jain, A., LeCun, Y., & Bregler, C. (2015). Efficient object localization using convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition.

  • van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9, 2579–2605.

    MATH  Google Scholar 

  • Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Wang, H., Ge, S., Lipton, Z.C., & Xing, E.P. (2019a). Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems.

  • Wang, H., He, Z., Lipton, Z. C., & Xing, E. P. (2019). Learning robust representations by projecting superficial statistics out. Preprint retrieved from arXiv:1903.06256

  • Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., & Hu, X. (2020a). Score-CAM: Score-weighted visual explanations for convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, Workshops.

  • Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L.V. (2016). Temporal segment networks: Towards good practices for deep action recognition. In European Conference on Computer Vision.

  • Wang, S., Yu, L., Li, C., Fu, C., & Heng, P. (2020b). Learning from extrinsic and intrinsic supervisions for domain generalization. In European Conference on Computer Vision.

  • Wang, L., Guo, S., Huang, W., Xiong, Y., & Qiao, Y. (2017). Knowledge guided disambiguation for large-scale scene classification with multi-resolution CNNs. IEEE Trans Image Process, 26(4), 2055–2068.

    Article  MathSciNet  Google Scholar 

  • You, Q., Jin, H., Wang, Z., Fang, C., & Luo, J. (2016). Image captioning with semantic attention. In IEEE Conference on Computer Vision and Pattern Recognition.

  • Yue, X., Zhang, Y., Zhao, S., Sangiovanni-Vincentelli, A.L., Keutzer, K., & Gong, B. (2019). Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data. In IEEE International Conference on Computer Vision.

  • Zakharov, S., Kehl, W., & Ilic, S. (2019). DeceptionNet: Network-driven domain randomization. In IEEE International Conference on Computer Vision.

  • Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., & Torralba, A. (2016a). Learning deep features for discriminative localization. In Conference on Computer Vision and Pattern Recognition.

  • Zhou B, Khosla A, Lapedriza À, Torralba A, Oliva A (2016b) Places: An image database for deep scene understanding. Preprint retrieved from arXiv: 1610.02055

  • Zhou K, Yang Y, Hospedales TM, Xiang T (2020) Learning to generate novel domains for domain generalization. In European Conference on Computer Vision

Download references

Acknowledgements

This work is supported by the National Science Foundation of China (No. 62076119, No. 61921006), National Key R & D Program of China (2018YFC2000702), the Scientific and Technical Innovation 2030-“New Generation Artificial Intelligence” Project (No. 2020AAA0104100), the Fundamental Research Funds for the Central Universities (No. 020214380091), Collaborative Innovation Center of Novel Software Technology and Industrialization, the Key-Area Research and Development Program of Guangdong Province (No. 2018B010111001). Part of this work was done when Dapeng Du was an intern at Tencent Jarvis Lab.

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

    Dapeng Du, Gangshan Wu & Limin Wang

  2. Tencent Jarvis Lab, Shenzhen, China

    Jiawei Chen, Yuexiang Li, Kai Ma & Yefeng Zheng

Authors
  1. Dapeng Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiawei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuexiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gangshan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yefeng Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Limin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yefeng Zheng or Limin Wang.

Additional information

Communicated by Judy Hoffman.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Du, D., Chen, J., Li, Y. et al. Cross-Domain Gated Learning for Domain Generalization. Int J Comput Vis 130, 2842–2857 (2022). https://doi.org/10.1007/s11263-022-01674-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-022-01674-w

Keywords

Search

Navigation