Abstract
In sim-to-real task, domain adaptation is one of the basic challenge topic as it can reduce the huge performance variation caused by domain shift. Domain adaptation can effectively transfer knowledge from a labeled source domain to an unlabeled target domain. Existing DA methods always consider to match cross-domain local features, however, only consider local features may lead to negative transfer. To alleviate this problem, in this paper, we propose a novel non-local networks for sim-to-real adversarial augmentation transfer (AAT). Our method leverages attention mechanism and semantic data augmentation to focus on global features and augmented features. Specifically, to focus on the global features, we leverage the non-local attention mechanism to weight the extracted features which can effectively eliminate the influence of untransferable features. Additionally, in order to enhance the ability of classifier adaptation, semantic data augmentation is leveraged to augment source features toward target features. We also give an upper bound of the divergence between the augmented features and the source features. Although our method is simple, it consistently improves the generalization performance of the popular domain adaptation and sim-to-real benchmarks, i.e., Office-31, Office-Home, ImageCLEF-DA and VisDA-2017.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Etemadi, N.: An elementary proof of the strong law of large numbers. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 55(1), 119–122 (1981). https://doi.org/10.1007/BF01013465
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International conference on machine learning. pp. 1180–1189. PMLR (2015)
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. The journal of machine learning research 17(1), 2096–2030 (2016)
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
Li, J., Chen, E., Ding, Z., Zhu, L., Lu, K., Shen, H.T.: Maximum density divergence for domain adaptation. IEEE transactions on pattern analysis and machine intelligence (2020)
Li, S., Xie, M., Gong, K., Liu, C.H., Wang, Y., Li, W.: Transferable semantic augmentation for domain adaptation. arXiv preprint arXiv:2103.12562 (2021)
Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with deep adaptation networks. In: International conference on machine learning. pp. 97–105. PMLR (2015)
Long, M., Cao, Z., Wang, J., Jordan, M.I.: Conditional adversarial domain adaptation. arXiv preprint arXiv:1705.10667 (2017)
Long, M., Zhu, H., Wang, J., Jordan, M.I.: Deep transfer learning with joint adaptation networks. In: International conference on machine learning. pp. 2208–2217. PMLR (2017)
Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of machine learning research 9(11) (2008)
Nguyen, A.T., Tran, T., Gal, Y., Baydin, A.G.: Domain invariant representation learning with domain density transformations. arXiv preprint arXiv:2102.05082 (2021)
Pei, Z., Cao, Z., Long, M., Wang, J.: Multi-adversarial domain adaptation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 32 (2018)
Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., Saenko, K.: Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924 (2017)
Román-Flores, H., Flores-Franulič, A., Chalco-Cano, Y.: A jensen type inequality for fuzzy integrals. Information Sciences 177(15), 3192–3201 (2007)
Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: European conference on computer vision. pp. 213–226. Springer (2010)
Sun, D., Wang, M., Ma, X., Zhang, T., Yin, N., Yu, W., Luo, Z.: A focally discriminative loss for unsupervised domain adaptation. In: Neural Information Processing - 28th International Conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, Proceedings, Part I. vol. 13108, pp. 54–64 (2021)
Tanwani, A.K.: Domain-invariant representation learning for sim-to-real transfer. arXiv preprint arXiv:2011.07589 (2020)
Venkateswara, H., Eusebio, J., Chakraborty, S., Panchanathan, S.: Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5018–5027 (2017)
Wang, H., Yang, W., Wang, J., Wang, R., Lan, L., Geng, M.: Pairwise similarity regularization for adversarial domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. pp. 2409–2418 (2020)
Wang, M., Li, P., Shen, L., Wang, Y., Wang, S., Wang, W., Zhang, X., Chen, J., Luo, Z.: Informative pairs mining based adaptive metric learning for adversarial domain adaptation. Neural Networks pp. 238–249 (2022)
Wang, M., Wang, W., Li, B., Zhang, X., Lan, L., Tan, H., Liang, T., Yu, W., Luo, Z.: Interbn: Channel fusion for adversarial unsupervised domain adaptation. In: MM ’21: ACM Multimedia Conference, Virtual Event, China, October 20–24, 2021. pp. 3691–3700 (2021)
Wang, S., Zhang, L.: Self-adaptive re-weighted adversarial domain adaptation. IJCAI (2020)
Wang, S., Zhang, L., Wang, P., Wang, M., Zhang, X.: Bp-triplet net for unsupervised domain adaptation: A bayesian perspective. Pattern Recognition p. 108993 (2022)
Wang, Y., Huang, G., Song, S., Pan, X., Xia, Y., Wu, C.: Regularizing deep networks with semantic data augmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)
Yang, X., Wang, M., Tao, D.: Person re-identification with metric learning using privileged information. IEEE Transactions on Image Processing 27(2), 791–805 (2017)
Yang, X., Wang, S., Dong, J., Dong, J., Wang, M., Chua, T.S.: Video moment retrieval with cross-modal neural architecture search. TIP 31, 1204–1216 (2022)
Yang, X., Zhou, P., Wang, M.: Person reidentification via structural deep metric learning. IEEE transactions on neural networks and learning systems 30(10), 2987–2998 (2018)
Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. In: International conference on machine learning. pp. 7354–7363. PMLR (2019)
Zhang, J., Tai, L., Liu, P.Y.Y.X.M., Burgard, J.B.W.: Supplement file of vr-goggles for robots: Real-to-sim domain adaptation for visual control. Training 853(840), 715 (2018)
Zhang, W., Ouyang, W., Li, W., Xu, D.: Collaborative and adversarial network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3801–3809 (2018)
Zhang, W., Zhang, X., Liao, Q., Yang, W., Lan, L., Luo, Z.: Robust normalized squares maximization for unsupervised domain adaptation. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. pp. 2317–2320 (2020)
Zhang, Y., Tang, H., Jia, K., Tan, M.: Domain-symmetric networks for adversarial domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5031–5040 (2019)
Zhang, Y., Liu, T., Long, M., Jordan, M.: Bridging theory and algorithm for domain adaptation. In: International Conference on Machine Learning. pp. 7404–7413. PMLR (2019)
Acknowledgment
This work is supported by National Natural Science Fund of China (No. 62106003), the University Synergy Innovation Program of Anhui Province (No.GXXT-2021-005) and Open Fund of Chongqing Key Laboratory of Bio-perception and Intelligent Information Processing (No.2020CKL-BPIIP001).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, M., Wang, S., Yan, T., Luo, Z. (2023). AAT: Non-local Networks for Sim-to-Real Adversarial Augmentation Transfer. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Communications in Computer and Information Science, vol 1791. Springer, Singapore. https://doi.org/10.1007/978-981-99-1639-9_19
Download citation
DOI: https://doi.org/10.1007/978-981-99-1639-9_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1638-2
Online ISBN: 978-981-99-1639-9
eBook Packages: Computer ScienceComputer Science (R0)