Skip to main content
SpringerLink
Log in

Attentive recurrent adversarial domain adaptation with Top-k pseudo-labeling for time series classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The key challenge of Unsupervised Domain Adaptation (UDA) for analyzing time series data is to learn domain-invariant representations by capturing complex temporal dependencies. In addition, existing unsupervised domain adaptation methods for time series data are designed to align marginal distribution between source and target domains. However, existing UDA methods (e.g. R-DANN Purushotham et al. (2017), VRADA Purushotham et al. (2017), CoDATS Wilson et al. (2020)) neglect the conditional distribution discrepancy between two domains, leading to misclassification of the target domain. Therefore, to learn domain-invariant representations by capturing the temporal dependencies and to reduce the conditional distribution discrepancy between two domains, a novel Attentive Recurrent Adversarial Domain Adaptation with Top-k time series pseudo-labeling method called ARADA-TK is proposed in this paper. In the experiments, our proposed method was compared with the state-of-the-art UDA methods (R-DANN, VRADA and CoDATS). Experimental results on four benchmark datasets revealed that ARADA-TK achieves superior classification accuracy when it is compared to the competing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. For the purpose of illustration, we depict the unfolded diagram of a RNN layer in Fig. 2

References

  1. Anguita D, Ghio A, Oneto L, Parra Perez X, Reyes Ortiz JL (2013) A public domain dataset for human activity recognition using smartphones. In: Proceedings of the 21th international European symposium on artificial neural networks, computational intelligence and machine learning, pp 437–442

  2. Chen C, Miao Y, Lu CX, Xie L, Blunsom P, Markham A, Trigoni N (2019) Motiontransformer: transferring neural inertial tracking between domains. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 8009–8016

  3. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  4. Deng W, Liao Q, Zhao L, Guo D, Kuang G, Hu D, Liu L (2021) Joint clustering and discriminative feature alignment for unsupervised domain adaptation. IEEE Trans Image Process 30:7842–7855

    Article  Google Scholar 

  5. Du Y, Wang J, Feng W, Pan S, Qin T, Xu R, Wang C (2021) Adarnn: adaptive learning and forecasting of time series. In: Proceedings of the 30th ACM international conference on information & knowledge management, pp 402–411

  6. Fawaz HI, Forestier G, Weber J, Idoumghar L, Muller PA (2018) Transfer learning for time series classification. In: 2018 IEEE international conference on big data (big data). IEEE, pp 1367–1376

  7. He QQ, Pang PCI, Si YW (2019) Transfer learning for financial time series forecasting. In: Pacific rim international conference on artificial intelligence. Springer, pp 24–36

  8. Hussein A, Hajj H (2022) Domain adaptation with representation learning and nonlinear relation for time series. ACM Trans Internet Things, vol 3(2). https://doi.org/10.1145/3502905

  9. Jiang X, Lao Q, Matwin S, Havaei M (2020) Implicit class-conditioned domain alignment for unsupervised domain adaptation. In: International conference on machine learning. PMLR, pp 4816–4827

  10. Johnson DA, Trivedi MM (2011) Driving style recognition using a smartphone as a sensor platform. In: 2011 14th International IEEE conference on intelligent transportation systems (ITSC). IEEE, pp 1609–1615

  11. Kwapisz JR, Weiss GM, Moore SA (2011) Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter 12(2):74–82

    Article  Google Scholar 

  12. Li S, Huang J, Hua XS, Zhang L (2021) Category dictionary guided unsupervised domain adaptation for object detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 1949–1957

  13. Liu J, Zhong L, Wickramasuriya J, Vasudevan V (2009) uwave: accelerometer-based personalized gesture recognition and its applications. Pervasive Mobile Comput 5(6):657–675

    Article  Google Scholar 

  14. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning. PMLR, pp 97–105

  15. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res, vol 9(11)

  16. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  17. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037

    Google Scholar 

  18. Purushotham S, Carvalho W, Nilanon T, Liu Y (2017) Variational recurrent adversarial deep domain adaptation. In: ICLR

  19. Ronao CA, Cho SB (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 59:235–244

    Article  Google Scholar 

  20. Stisen A, Blunck H, Bhattacharya S, Prentow TS, Kjærgaard MB, Dey A, Sonne T, Jensen MM (2015) Smart devices are different: assessing and mitigatingmobile sensing heterogeneities for activity recognition. In: Proceedings of the 13th ACM conference on embedded networked sensor systems, SenSys ’15. Association for computing machinery, New York, pp 127–140

  21. Sun B, Saenko K (2016) Deep coral: correlation alignment for deep domain adaptation. In: European conference on computer vision. Springer, pp 443–450

  22. Vercruyssen V, Meert W, Davis J (2017) Transfer learning for time series anomaly detection. In: Proceedings of the workshop and tutorial on interactive adaptive learning@ ECMLPKDD 2017. CEUR Workshop Proceedings, vol 1924, pp 27–37

  23. Wang J, Chen Y, Feng W, Yu H, Huang M, Yang Q (2020) Transfer learning with dynamic distribution adaptation. ACM Trans Intell Syst Technol (TIST) 11(1):1–25

    Google Scholar 

  24. Wang Q, Luo H, Wang J, Sun L, Ma Z, Zhang C, Fu M, Zhao F (2022) Recent advances in pedestrian navigation activity recognition: a review. IEEE Sensors J 22(8):7499–7518. https://doi.org/10.1109/JSEN.2022.3153610

    Article  Google Scholar 

  25. Wilcoxon F (1992) Individual comparisons by ranking methods. In: Breakthroughs in statistics. Springer, pp 196–202

  26. Wilson G, Doppa JR, Cook DJ (2020) Multi-source deep domain adaptation with weak supervision for time-series sensor data. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1768–1778

  27. Xu T, Chen W, Pichao W, Wang F, Li H, Jin R (2021) Cdtrans: cross-domain transformer for unsupervised domain adaptation. In: International conference on learning representations

  28. Yao S, Hu S, Zhao Y, Zhang A, Abdelzaher T (2017) Deepsense: a unified deep learning framework for time-series mobile sensing data processing. In: Proceedings of the 26th international conference on world wide web, WWW ’17. International world wide web conferences steering committee, Republic and Canton of Geneva, CHE, pp 351–360. https://doi.org/10.1145/3038912.3052577

  29. Ye R, Dai Q (2021) Implementing transfer learning across different datasets for time series forecasting. Pattern Recogn 109:107617

    Article  Google Scholar 

  30. Ye R, Dai Q (2022) A relationship-aligned transfer learning algorithm for time series forecasting. Inf Sci 593:17–34

    Article  Google Scholar 

  31. Yin W, Kann K, Yu M, Schütze H (2017) Comparative study of cnn and rnn for natural language processing. arXiv:1702.01923

  32. Zhao M, Yue S, Katabi D, Jaakkola TS, Bianchi MT (2017) Learning sleep stages from radio signals: a conditional adversarial architecture. In: International conference on machine learning. PMLR, pp 4100–4109

Download references

Acknowledgements

This research was funded by the University of Macau (file no. MYRG2019-00136-FST).

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Macau, Macau, China

    Qi-Qiao He & Yain-Whar Si

  2. Institute of Science and Environment, University of Saint Joseph, Macau, China

    Shirley Weng In Siu

Authors
  1. Qi-Qiao He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shirley Weng In Siu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yain-Whar Si
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yain-Whar Si.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

: Appendix A: Label proportions for the participants of HAR, HHAR, uWave and WISDM AR datasets

: Appendix A: Label proportions for the participants of HAR, HHAR, uWave and WISDM AR datasets

Fig. 7
figure 7

Label proportions for the participants of the HAR dataset

Full size image
Fig. 8
figure 8

Label proportions for the participants of the HHAR dataset

Full size image
Fig. 9
figure 9

Label proportions for the participants of the uWave dataset

Full size image
Fig. 10
figure 10

Label proportions for the participants of the WISDM AR dataset

Full size image

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, QQ., Siu, S.W.I. & Si, YW. Attentive recurrent adversarial domain adaptation with Top-k pseudo-labeling for time series classification. Appl Intell 53, 13110–13129 (2023). https://doi.org/10.1007/s10489-022-04176-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04176-x

Keywords

Search

Navigation