Abstract
The purpose of unsupervised domain adaptation is to use the knowledge of the source domain whose data distribution is different from that of the target domain for promoting the learning task in the target domain. The key bottleneck in unsupervised domain adaptation is how to obtain higher-level and more abstract feature representations between source and target domains which can bridge the chasm of domain discrepancy. Recently, deep learning methods based on autoencoder have achieved sound performance in representation learning, and many dual or serial autoencoder-based methods take different characteristics of data into consideration for improving the effectiveness of unsupervised domain adaptation. However, most existing methods of autoencoders just serially connect the features generated by different autoencoders, which pose challenges for the discriminative representation learning and fail to find the real cross-domain features. To address this problem, we propose a novel representation learning method based on an integrated autoencoders for unsupervised domain adaptation, called IAUDA. To capture the inter- and inner-domain features of the raw data, two different autoencoders, which are the marginalized autoencoder with maximum mean discrepancy (mAEMMD) and convolutional autoencoder (CAE) respectively, are proposed to learn different feature representations. After higher-level features are obtained by these two different autoencoders, a sparse autoencoder is introduced to compact these inter- and inner-domain representations. In addition, a whitening layer is embedded for features processed before the mAEMMD to reduce redundant features inside a local area. Experimental results demonstrate the effectiveness of our proposed method compared with several state-of-the-art baseline methods.
Similar content being viewed by others
References
Pan S J, Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359
Xin J, Cui Z, Zhao P, He T. Active transfer learning of matching query results across multiple sources. Frontiers of Computer Science, 2015, 9(4): 595–607
Weiss K, Khoshgoftaar T M, Wang D. A survey of transfer learning. Journal of Big Data, 2016, 3(1): 9
Zhang Y, Chu G, Li P, Hu X, Wu X. Three-layer concept drifting detection in text data streams. Neurocomputing, 2017, 260: 393–403
Du B, Xiong W, Wu J, Zhang L, Zhang L, Tao D. Stacked convolutional denoising auto-encoders for feature representation. IEEE Transactions on Cybernetics, 2017, 47(4): 1017–1027
Zhu Y, Wu X, Li P, Zhang Y, Hu X. Transfer learning with deep manifold regularized auto-encoders. Neurocomputing, 2019, 369: 145–154
Caron M, Bojanowski P, Joulin A, Douze M. Deep clustering for unsupervised learning of visual features. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). 2018, 139–156
Zhang H, Zhang Y, Geng X. Practical age estimation using deep label distribution learning. Frontiers of Computer Science, 2021, 15(3): 153318
Qiang J, Qian Z, Li Y, Yuan Y, Wu X. Short text topic modeling techniques, applications, and performance: a survey. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(3): 1427–1445
Zhu Y, Hu X, Zhang Y, Li P. Transfer learning with stacked reconstruction independent component analysis. Knowledge-Based Systems, 2018, 152: 100–106
Li R, Jiao Q, Cao W, Wong H S, Wu S. Model adaptation: Unsupervised domain adaptation without source data. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 9638–9647
Chen M, Xu Z, Weinberger K Q, Sha F. Marginalized denoising autoencoders for domain adaptation. In: Proceedings of the 29th International Conference on Machine Learning. 2012, 1627–1634
Yang S, Zhang Y, Zhu Y, Li P, Hu X. Representation learning via serial autoencoders for domain adaptation. Neurocomputing, 2019, 351: 1–9
Wang J, Feng W, Chen Y, Yu H, Huang M, Yu P S. Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM International Conference on Multimedia. 2018, 402–410
Iovanac N C, Savoie B M. Simpler is better: how linear prediction tasks improve transfer learning in chemical autoencoders. The Journal of Physical Chemistry A, 2020, 124(18): 3679–3685
Wang X, Ma Y, Cheng Y. Domain adaptation network based on autoencoder. Chinese Journal of Electronics, 2018, 27(6): 1258–1264
Zhuang F, Cheng X, Luo P, Pan S J, He Q. Supervised representation learning with double encoding-layer autoencoder for transfer learning. ACM Transactions on Intelligent Systems and Technology, 2018, 9(2): 16
Sun C, Ma M, Zhao Z, Tian S, Yan R, Chen X. Deep transfer learning based on sparse autoencoder for remaining useful life prediction of tool in manufacturing. IEEE Transactions on Industrial Informatics, 2019, 15(4): 2416–2425
Li C, Zhang S, Qin Y, Estupinan E. A systematic review of deep transfer learning for machinery fault diagnosis. Neurocomputing, 2020, 407: 121–135
Sevakula R K, Singh V, Verma N K, Kumar C, Cui Y. Transfer learning for molecular cancer classification using deep neural networks. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2019, 16(6): 2089–2100
Sun M, Wang H, Liu P, Huang S, Fan P. A sparse stacked denoising autoencoder with optimized transfer learning applied to the fault diagnosis of rolling bearings. Measurement, 2019, 146: 305–314
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P A. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 2010, 11: 3371–3408
Yan H, Ding Y, Li P, Wang Q, Xu Y, Zuo W. Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 945–954
Lin W W, Mak M W, Chien J T. Multisource I-vectors domain adaptation using maximum mean discrepancy based autoencoders. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018, 26(12): 2412–2422
Yang S, Wang H, Zhang Y, Li P, Zhu Y, Hu X. Semi-supervised representation learning via dual autoencoders for domain adaptation. Knowledge-Based Systems, 2020, 190: 105161
Glorot X, Bordes A, Bengio Y. Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 513–520
Jin X, Zhuang F, Xiong H, Du C, Luo P, He Q. Multi-task multi-view learning for heterogeneous tasks. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. 2014, 441–450
Roy S, Siarohin A, Sangineto E, Bulò S R, Sebe N, Ricci E. Unsupervised domain adaptation using feature-whitening and consensus loss. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 9463–9472
Pan S J, Tsang I W, Kwok J T, Yang Q. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 2011, 22(2): 199–210
Sun B, Feng J, Saenko K. Return of frustratingly easy domain adaptation. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016, 2058–2065
Cao Y, Long M, Wang J. Unsupervised domain adaptation with distribution matching machines. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 341
Zhang J, Li W, Ogunbona P. Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5150–5158
Chen Z, Chen C, Jin X, Liu Y, Cheng Z. Deep joint two-stream wasserstein auto-encoder and selective attention alignment for unsupervised domain adaptation. Neural Computing and Applications, 2020, 32(11): 7489–7502
Ben-David S, Blitzer J, Crammer K, F. Pereira. Analysis of representations for domain adaptation. In: Proceedings of the 19th International Conference on Neural Information Processing Systems. 2007, 137–144
Yang S, Zhang Y, Wang H, Li P, Hu X. Representation learning via serial robust autoencoder for domain adaptation. Expert Systems with Applications, 2020, 160: 113635
Hoffman J, Rodner E, Donahue J, Kulis B, Saenko K. Asymmetric and category invariant feature transformations for domain adaptation. International Journal of Computer Vision, 2014, 109(1–2): 28–41
Tsai Y H, Sohn K, Schulter S, Chandraker M. Domain adaptation for structured output via discriminative patch representations. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 1456–1465
Sharma R, Bhattacharyya P, Dandapat S, Bhatt H S. Identifying transferable information across domains for cross-domain sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 968–978
Chen M, Zhao S, Liu H, Cai D. Adversarial-learned loss for domain adaptation. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 3521–3528
Fan H, Zheng L, Yan C, Yang Y. Unsupervised person re-identification: clustering and fine-tuning. ACM Transactions on Multimedia Computing, Communications, and Applications, 2018, 14(4): 83
Fan H, Liu P, Xu M, Yang Y. Unsupervised visual representation learning via dual-level progressive similar instance selection. IEEE Transactions on Cybernetics, 2022, 52(9): 8851–8861
Qiang J, Wu X. Unsupervised statistical text simplification. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(4): 1802–1806
Qiang J, Chen P, Ding W, Wang T, Xie F, Wu X. Heterogeneous-length text topic modeling for reader-aware multi-document summarization. ACM Transactions on Knowledge Discovery from Data, 2019, 13(4): 42
Su J C, Tsai Y H, Sohn K, Liu B, Maji S, Chandraker M. Active adversarial domain adaptation. In: Proceedings of 2020 IEEE Winter Conference on Applications of Computer Vision. 2020, 728–737
Gholami B, Sahu P, Rudovic O, Bousmalis K, Pavlovic V. Unsupervised multi-target domain adaptation: an information theoretic approach. IEEE Transactions on Image Processing, 2020, 29: 3993–4002
Carlucci F M, Porzi L, Caputo B, Ricci E, Buló S R. MultiDIAL: domain alignment layers for (multisource) unsupervised domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(12): 4441–4452
Luo L, Chen L, Hu S, Lu Y, Wang X. Discriminative and geometry-aware unsupervised domain adaptation. IEEE Transactions on Cybernetics, 2020, 50(9): 3914–3927
Vincent P, Larochelle H, Bengio Y, Manzagol P A. Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning. 2008, 1096–1103
Wei P, Ke Y, Goh C K. Deep nonlinear feature coding for unsupervised domain adaptation. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence. 2016, 2189–2195
Wang D, Cui P, Zhu W. Deep asymmetric transfer network for unbalanced domain adaptation. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. 2018, 55
Acknowledgements
This research was partially supported by the National Natural Science Foundation of China (Grant Nos. 61906060, 62076217, 62120106008), the Yangzhou University Interdisciplinary Research Foundation for Animal Husbandry Discipline of Targeted Support (yzuxk202015), the Opening Foundation of Key Laboratory of Huizhou Architecture in Anhui Province (HPJZ-2020-02), and the Open Project Program of Joint International Research Laboratory of Agriculture and Agri-Product Safety (JILAR-KF202104).
Author information
Authors and Affiliations
Corresponding author
Additional information
Yi Zhu is currently an assistant professor in the School of Information Engineering, at Yangzhou University, China. He received the BS degree from Anhui University, China in 2006, the MS degree from the University of Science and Technology of China, China in 2012, and the PhD degree from Hefei University of Technology, China in 2018. His research interests include data mining and recommendation systems.
Xindong Wu is a Professor in the School of Computer Science and Information Engineering at the Hefei University of Technology, China, and a fellow of IEEE and AAAS. He received his BS and MS degrees in computer science from the Hefei University of Technology, China in 1984 and 1987, and his PhD degree in artificial intelligence from the University of Edinburgh, Britain in 1993. His research interests include data mining, big data analytics, and knowledge-based systems.
Jipeng Qiang is currently an associate professor in the School of Information Engineering, at Yangzhou University, China. He received his PhD degree in computer science and technology from Hefei University of Technology, China in 2016. He was a PhD visiting student in the Artificial Intelligence Lab at the University of Massachusetts Boston, USA from 2014 to 2016. He has published more than 40 papers, including AAAI, TKDE, TKDD, and TASLP. His research interests mainly include natural language processing and data mining.
Yunhao Yuan is currently an associate professor in the School of Information Engineering, Yangzhou University, China. He received the MEng degree in computer science and technology from Yangzhou University, China in 2009, and the PhD degree in pattern recognition and intelligence system from Nanjing University of Science and Technology, China in 2013. His research interests include pattern recognition, data mining, and image processing.
Yun Li is currently a professor in the School of Information Engineering, Yangzhou University, China. He received the MS degree in computer science and technology from Hefei University of Technology, China in 1991, and the PhD degree in control theory and control engineering from Shanghai University, China in 2005. He has published more than 100 scientific papers. His research interests include data mining and cloud computing.
Electronic Supplementary Material
Rights and permissions
About this article
Cite this article
Zhu, Y., Wu, X., Qiang, J. et al. Representation learning via an integrated autoencoder for unsupervised domain adaptation. Front. Comput. Sci. 17, 175334 (2023). https://doi.org/10.1007/s11704-022-1349-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-022-1349-5