Skip to main content
SpringerLink
Log in

SVM-based subspace optimization domain transfer method for unsupervised cross-domain time series classification

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Time series classification on edge devices has received considerable attention in recent years, and it is often conducted on the assumption that the training and testing data are drawn from the same distribution. However, in practical IoT applications, this assumption does not hold due to variations in installation positions, precision error, and sampling frequency of edge devices. To tackle this problem, in this paper, we propose a new SVM-based domain transfer method called subspace optimization transfer support vector machine (SOTSVM) for cross-domain time series classification. SOTSVM aims to learn a domain-invariant SVM classifier by which (1) global projected distribution alignment jointly exploits the marginal distribution discrepancy, geometric structure, and distribution scatter to reduce the global distribution discrepancy between the source and target domains; (2) feature grouping is used to divide the features into highly transferable features (HTF) and lowly transferable features (LTF), where the importance of HTF is preserved and importance of LTF is suppressed in the domain-invariant classifier training; and (3) empirical risk minimization is constructed for improving the discrimination of the SOTSVM. In this paper, we formulate a minimization problem that integrates global projected distribution alignment, feature grouping and empirical risk minimization into the joint SVM framework, giving an effective optimization algorithm. Furthermore, we present the extension of multiple kernel SOTSVM. Experimental results on three sets of cross-domain time series datasets show that our method outperforms some state-of-the-art conventional transfer learning methods and no transfer learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://pan.baidu.com/s/1zMnozCBXREEJrCNuf9SJrw(Password:1qef).

References

  1. Aghabozorgi S, Shirkhorshidi AS, Wah TY (2015) Time-series clustering-a decade review. Inf Syst 53:16–38

    Article  Google Scholar 

  2. Barshan B, Yuksek M (2013) Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. Comput J 57:1649–1667

    Article  Google Scholar 

  3. Belkin M, Niyogi P (2001) Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in neural information processing systems, vol 14

  4. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(Nov):2399–2434

    MATH  Google Scholar 

  5. Bock C, Togninalli M, Ghisu E, Gumbsch T, Rieck B, Borgwardt K (2019) A wasserstein subsequence kernel for time series. In: 2019 IEEE international conference on data mining (ICDM). IEEE, pp 964–969

  6. Cao Y, Long M, Wang J (2018) Unsupervised domain adaptation with distribution matching machines. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  7. Chavarriaga R, Sagha H, Calatroni A, Digumarti ST, Tröster G, Millán JDR, Roggen D (2013) The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recognit Lett 34(15):2033–2042. https://doi.org/10.1016/j.patrec.2012.12.014

    Article  Google Scholar 

  8. Dai W, Yang Q, Xue G-R, Yu Y (2008) Self-taught clustering. In: Proceedings of the 25th international conference on machine learning, pp 200–207

  9. Duan L, Tsang IW, Xu D, Chua T-S (2009) Domain adaptation from multiple sources via auxiliary classifiers. In: Proceedings of the 26th annual international conference on machine learning, pp 289–296

  10. Duan L, Tsang IW, Xu D, Maybank SJ (2009) Domain transfer SVM for video concept detection. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 1375–1381

  11. Evgeniou A, Pontil M (2007) Multi-task feature learning. In: Advances in neural information processing systems, vol 19, pp 41

  12. Fawaz HI, Forestier G, Weber J, Idoumghar L, Muller P-A (2019) Deep learning for time series classification: a review. Data Min Knowl Discov 33(4):917–963

    Article  MATH  Google Scholar 

  13. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967

  14. Fukunaga K (2013) Introduction to statistical pattern recognition. Academic Press, Cambridge

    MATH  Google Scholar 

  15. Geler Z, Kurbalija V, Ivanović M, Radovanović M (2020) Weighted KNN and constrained elastic distances for time-series classification. Expert Syst Appl 162:113829

    Article  Google Scholar 

  16. Geurts P (2001) Pattern extraction for time series classification. In: European conference on principles of data mining and knowledge discovery. Springer, pp 115–127

  17. Ghifary M, Balduzzi D, Kleijn WB, Zhang M (2016) Scatter component analysis: a unified framework for domain adaptation and domain generalization. IEEE Trans Pattern Anal Mach Intell 39(7):1414–1430

    Article  Google Scholar 

  18. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073

  19. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT Press, Cambridge

    MATH  Google Scholar 

  20. Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(1):723–773

    MATH  Google Scholar 

  21. Huang J, Gretton A, Borgwardt K, Schölkopf B, Smola A (2006) Correcting sample selection bias by unlabeled data. In: Advances in neural information processing systems, vol 19, pp 601–608

  22. Ismail Fawaz H, Lucas B, Forestier G, Pelletier C, Schmidt DF, Weber J, Webb GI, Idoumghar L, Muller P-A, Petitjean F (2020) Inceptiontime: finding alexnet for time series classification. Data Min Knowl Discov 34(6):1936–1962

    Article  Google Scholar 

  23. Jia X, Zhao M, Di Y, Yang Q, Lee J (2017) Assessment of data suitability for machine prognosis using maximum mean discrepancy. IEEE Trans Ind Electron 65(7):5872–5881

    Article  Google Scholar 

  24. Jiang W, Zavesky E, Chang S-F, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: 2008 15th IEEE international conference on image processing. IEEE, pp 161–164

  25. Kampouraki A, Manis G, Nikou C (2008) Heartbeat time series classification with support vector machines. IEEE Trans Inf Technol Biomed 13(4):512–518

    Article  Google Scholar 

  26. Keogh E, Ratanamahatana CA (2005) Exact indexing of dynamic time warping. Knowl Inf Syst 7(3):358–386

    Article  Google Scholar 

  27. Li H, Jiang T, Zhang K (2003) Efficient and robust feature extraction by maximum margin criterion. In: Advances in neural information processing systems, vol 16

  28. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207

  29. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417

  30. Lucas B, Shifaz A, Pelletier C, O’Neill L, Zaidi N, Goethals B, Petitjean F, Webb GI (2019) Proximity forest: an effective and scalable distance-based classifier for time series. Data Min Knowl Discov 33(3):607–635

    Article  Google Scholar 

  31. Olszewski RT, Maxion R, Siewiorek D (2001) Generalized feature extraction for structural pattern recognition in time-series data. PhD thesis, USA

  32. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  33. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  34. Quanz B, Huan J (2009) Large margin transductive transfer learning. In: Proceedings of the 18th ACM conference on information and knowledge management, pp 1327–1336

  35. Raina R, Battle A, Lee H, Packer B, Ng AY (2007) Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th international conference on machine learning, pp 759–766

  36. Reiss A, Stricker D (2012) Introducing a new benchmarked dataset for activity monitoring. In: Proceedings of the 2012 16th annual international symposium on wearable computers. IEEE Computer Society, USA. https://doi.org/10.1109/ISWC.2012.13

  37. Shi W, Cao J, Zhang Q, Li Y, Xu L (2016) Edge computing: vision and challenges. IEEE Internet Things J 3(5):637–646

    Article  Google Scholar 

  38. Si S, Tao D, Geng B (2009) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942

    Article  Google Scholar 

  39. Song P (2019) Transfer linear subspace learning for cross-corpus speech emotion recognition. IEEE Ann Hist Comput 02:265–275

    Google Scholar 

  40. Sun B, Feng J, Saenko K (2015) Return of frustratingly easy domain adaptation. arXiv preprint arXiv:1511.05547

  41. Sun B, Feng J, Saenko K (2017) Correlation alignment for unsupervised domain adaptation. In: Domain adaptation in computer vision applications. Springer, pp 153–171

  42. Sun B, Saenko K (2015) Subspace distribution alignment for unsupervised domain adaptation. In: BMVC, vol 4, pp 24–1

  43. Sun Z, Ampornpunt N, Varma M, Vishwanathan S (2010) Multiple kernel learning and the SMO algorithm. In: Advances in neural information processing systems, vol 23

  44. Tan B, Zhang Y, Pan SJ, Yang Q (2017) Distant domain transfer learning. In: AAAI, vol 300, pp 301–302

  45. Tao J, Chung F-L, Wang S (2012) On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recognit 45(11):3962–3984

    Article  MATH  Google Scholar 

  46. Wang J, Chen Y, Hu L, Peng X, Philip SY (2018) Stratified transfer learning for cross-domain activity recognition. In: 2018 IEEE international conference on pervasive computing and communications (PerCom). IEEE, pp 1–10

  47. Wang J, Chen Y, Yu H, Huang M, Yang Q (2019) Easy transfer learning by exploiting intra-domain structures. In: 2019 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1210–1215

  48. Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM international conference on multimedia, pp 402–410

  49. Wang J, Shen X, Pan W (2007) On transductive support vector machines. Contem Math 443:7–20

    Article  MATH  Google Scholar 

  50. Wang Z, Song Y, Zhang C (2008) Transferred dimensionality reduction. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 550–565

  51. Wang Z, Yan W, Oates T (2017) Time series classification from scratch with deep neural networks: a strong baseline. In: 2017 International joint conference on neural networks (IJCNN). IEEE, pp 1578–1585

  52. Wu Y, Liu Y (2007) Robust truncated hinge loss support vector machines. J Am Stat Assoc 102(479):974–983

    Article  MATH  Google Scholar 

  53. Yang C-HH, Tsai Y-Y, Chen P-Y (2021) Voice2series: reprogramming acoustic models for time series classification. In: International conference on machine learning. PMLR, pp 11808–11819

  54. Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive SVMS. In: Proceedings of the 15th ACM international conference on multimedia, pp 188–197

  55. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1859–1867

  56. Zhang L, Fu J, Wang S, Zhang D, Dong Z, Chen CP (2019) Guide subspace learning for unsupervised domain adaptation. IEEE Trans Neural Netw Learn Syst 31(9):3374–3388

    Article  Google Scholar 

  57. Zhao Z, Chen Y, Liu J, Shen Z, Liu M (2011) Cross-people mobile-phone based activity recognition. In: Twenty-second international joint conference on artificial intelligence

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant No. 61672115; the Chongqing Technology & Application Development Project (Nos. cstc2019jscx-gksbX0038 and cstc2020jscx-dxwtBX0055) and the Fundamental Research Funds for the Central Universities, China (No. 2022CDJYGRH-001). We would like to thank Yanai Wang for her help on English writing enhancement.

Author information

Authors and Affiliations

  1. College of Computer Science, Chongqing University, Chongqing, China

    Fei Ma, Chengliang Wang & Zhuo Zeng

Authors
  1. Fei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengliang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhuo Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengliang Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, F., Wang, C. & Zeng, Z. SVM-based subspace optimization domain transfer method for unsupervised cross-domain time series classification. Knowl Inf Syst 65, 869–897 (2023). https://doi.org/10.1007/s10115-022-01784-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01784-4

Keywords

Search

Navigation