Skip to main content
SpringerLink
Log in

Representation and analysis of time-series data via deep embedding and visual exploration

  • Regular Paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

Time-series representation is fundamental for various analysis tasks on time-series data, such as similarity analysis, clustering, and classification. Traditional representation methods produce indistinguishable representations based on statistical information. Given the good performance of deep learning techniques in feature extraction, we apply it to the task of time-series data representation. However, such methods are usually evaluated only through a few quantitative metrics that reflect the limited model performance and lack further explanation. In this paper, we design TSRNet, a deep neural network containing two weight-sharing auto-encoders to learn representations effectively. To better analyze the representation, we design an interactive visual analytics system TSRVis in collaboration with seven experts from the fields of visual analytics and machine learning. Experimental evaluations on 18 datasets verify the effectiveness of TSRNet in terms of clustering accuracy. Three case studies demonstrate that TSRVis is practicable for assisting analysts to perform overview, comparison, and point-of-interest analyses and assess the validity of representations from multiple perspectives.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org

  • Ahn Y, Lin Y-R (2019) Fairsight: visual analytics for fairness in decision making. IEEE Trans Vis Comput Gr 26(1):1086–1095

    Google Scholar 

  • Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271

  • Bai Z, Tao Y, Lin H (2020) Time-varying volume visualization: a survey. J Vis 23(5):745–761

    Article  Google Scholar 

  • Bernard J, Hutter M, Zeppelzauer M, Fellner D, Sedlmair M (2017) Comparing visual-interactive labeling with active learning: an experimental study. IEEE Trans Vis Comput Gr 24(1):298–308

    Article  Google Scholar 

  • Boniol P, Linardi M, Roncallo F, Palpanas T (2020) Automated anomaly detection in large sequences. In: 2020 IEEE 36th international conference on data engineering (ICDE), pp. 1834–1837. IEEE

  • Cao K, Liu M, Su H, Wu J, Zhu J, Liu S (2020) Analyzing the noise robustness of deep neural networks. IEEE Trans Vis Comput Gr 27(7):3289–3304

    Article  Google Scholar 

  • Chen C, Yuan J, Lu Y, Liu Y, Su H, Yuan S, Liu S (2020) Oodanalyzer: interactive analysis of out-of-distribution samples. IEEE Trans Vis Comput Gr 27(7):3335–3349

    Article  Google Scholar 

  • Chen Y, Garcia EK, Gupta MR, Rahimi A, Cazzanti L (2009) Similarity-based classification: concepts and algorithms. J Mach Learn Res 10(3):747–776

    MathSciNet  MATH  Google Scholar 

  • Dau HA, Bagnall A, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S, Ratanamahatana CA, Keogh E (2019) The ucr time series archive. IEEE/CAA J Autom Sin 6(6):1293–1305

    Article  Google Scholar 

  • Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  • Franceschi J-Y, Dieuleveut A, Jaggi M (2019) Unsupervised scalable representation learning for multivariate time series. In: Wallach H, Larochelle H, Beygelzimer A, d'Alché-Buc F, Fox E, Garnett R (eds) Advances in neural information processing systems, vol 32. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2019/file/53c6de78244e9f528eb3e1cda69699bb-Paper.pdf

  • Fujiwara T, Sakamoto N, Nonaka J, Yamamoto K, Ma K-L et al (2020) A visual analytics framework for reviewing multivariate time-series data with dimensionality reduction. IEEE Trans Vis Comput Gr 27(2):1601–1611

    Article  Google Scholar 

  • Fujiwara T, Wei X, Zhao J, Ma K-L (2021) Interactive dimensionality reduction for comparative analysis. IEEE Trans Vis Comput Gr 28(1):758–768

    Article  Google Scholar 

  • Gogolou A, Tsandilas T, Palpanas T, Bezerianos A (2018) Comparing similarity perception in time series visualizations. IEEE Trans Vis Comput Gr 25(1):523–533

    Article  Google Scholar 

  • Goldin DQ, Kanellakis PC (1995) On similarity queries for time-series data: Constraint specification and implementation. In: Montanari U, Rossi F (eds) Principles and practice of constraint programming - CP’95, first international conference, CP’95, Cassis, France, September 19-22, Proceedings, vol. 976 of Lecture Notes in Computer Science, pp. 137–153. Springer https://doi.org/10.1007/3-540-60299-2_9

  • Gratzl S, Lex A, Gehlenborg N, Pfister H, Streit M (2013) Lineup: visual analysis of multi-attribute rankings. IEEE Trans Vis Comput Gr 19(12):2277–2286. https://doi.org/10.1109/TVCG.2013.173

    Article  Google Scholar 

  • Guo Y, Guo S, Jin Z, Kaul S, Gotz D, Cao N (2021) A survey on visual analysis of event sequence data. IEEE Trans Vis Comput Gr 28(12):5091–5112. https://doi.org/10.1109/TVCG.2021.3100413

    Article  Google Scholar 

  • Han D, Pan J, Guo F, Luo X, Wu Y, Zheng W, Chen W (2019) Rankbrushers: interactive analysis of temporal ranking ensembles. J Vis 22(6):1241–1255

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  • Hoffman P, Grinstein G, Marx K, Grosse I, Stanley E (1997) Dna visual and analytic data mining. In: Proceedings. Visualization’97 (Cat. No. 97CB36155), pp. 437–441. IEEE

  • Ismail Fawaz H, Lucas B, Forestier G, Pelletier C, Schmidt DF, Weber J, Webb GI, Idoumghar L, Muller P-A, Petitjean F (2020) Inceptiontime: finding alexnet for time series classification. Data Min Knowl Discov 34(6):1936–1962

    Article  MathSciNet  Google Scholar 

  • Jiang L, Liu S, Chen C (2019) Recent research advances on interactive machine learning. J Vis 22(2):401–417

    Article  Google Scholar 

  • Keogh E, Chakrabarti K, Pazzani M, Mehrotra S (2001) Dimensionality reduction for fast similarity search in large time series databases. Knowl Inf Syst 3(3):263–286

    Article  MATH  Google Scholar 

  • Kim W, Shim C, Chung YD (2021) Skyflow: a visual analysis of high-dimensional skylines in time-series. J Vis 24(5):1033–1050

    Article  Google Scholar 

  • Kong L, Tang X, Zhu J, Wang Z, Li J, Wu H, Wu Q, Chen H, Zhu L, Wang W, Liu B, Wang Q, Chen D, Pan Y, Song T, Li F, Zheng H, Jia G, Lu M, Wu L, Carmichael GR (2021) A 6-year-long (2013–2018) high-resolution air quality reanalysis dataset in China based on the assimilation of surface observations from CNEMC. Earth Syst Sci Data 13(2):529–570. https://doi.org/10.5194/essd-13-529-2021

    Article  Google Scholar 

  • Kowsari K, Jafari Meimandi K, Heidarysafa M, Mendu S, Barnes L, Brown D (2019) Text classification algorithms: a survey. Information 10(4):150

    Article  Google Scholar 

  • Kwon BC, Eysenbach B, Verma J, Ng K, De Filippi C, Stewart WF, Perer A (2017) Clustervision: visual supervision of unsupervised clustering. IEEE Trans Vis Comput Gr 24(1):142–151

    Article  Google Scholar 

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  • Lei H, Xia J, Guo F, Zou Y, Chen W, Liu Z (2016) Visual exploration of latent ranking evolutions in time series. J Vis 19(4):783–795

    Article  Google Scholar 

  • Lin H, Gao S, Gotz D, Du F, He J, Cao N (2017) Rclens: interactive rare category exploration and identification. IEEE Trans Vis Comput Gr 24(7):2223–2237

    Article  Google Scholar 

  • Lin J, Keogh E, Lonardi S, Chiu B (2003) A symbolic representation of time series, with implications for streaming algorithms. In: Proceedings of the 8th ACM SIGMOD workshop on research issues in data mining and knowledge discovery, pp. 2–11

  • Liu S, Cui W, Wu Y, Liu M (2014) A survey on information visualization: recent advances and challenges. Vis Comput 30(12):1373–1393

    Article  Google Scholar 

  • Lv C, Ren K, Zhang H, Fu J, Lin Y (2022) PEVis: visual analytics of potential anomaly pattern evolution for temporal multivariate data.Springer. J Vis 25(3):575–591

    Article  Google Scholar 

  • Malhotra P, TV V, Vig L, Agarwal P, Shroff G (2017) Timenet: Pre-trained deep recurrent neural network for time series classification. arXiv Preprint arXiv:1706.08838

  • Nonato LG, Aupetit M (2018) Multidimensional projection for visual analytics: Linking techniques with distortions, tasks, and layout enrichment. IEEE Trans Vis Comput Gr 25(8):2650–2673

    Article  Google Scholar 

  • Paparrizos J, Gravano L (2015) k-shape: efficient and accurate clustering of time series. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp. 1855–1870

  • Qu D, Lin X, Ren K, Liu Q, Zhang H (2020) Airexplorer: visual exploration of air quality data based on time-series querying. J Vis 23(6):1129–1145

    Article  Google Scholar 

  • Rakthanmanon T, Keogh EJ, Lonardi S, Evans S (2011) Time series epenthesis: Clustering time series streams requires ignoring some data. In: 2011 IEEE 11th international conference on data mining, pp. 547–556. IEEE

  • Richardson S, Green PJ (1997) On bayesian analysis of mixtures with an unknown number of components (with discussion). J R Stat Soc Ser B (Stat Methodol) 59(4):731–792

    Article  MATH  Google Scholar 

  • Sacha D, Zhang L, Sedlmair M, Lee JA, Peltonen J, Weiskopf D, North SC, Keim DA (2016) Visual interaction with dimensionality reduction: a structured literature analysis. IEEE Trans Vis Comput Gr 23(1):241–250

    Article  Google Scholar 

  • Shi C, Cui W, Liu S, Xu P, Chen W, Qu H (2012) Rankexplorer: visualization of ranking changes in large time series data. IEEE Trans Vis Comput Gr 18(12):2669–2678. https://doi.org/10.1109/TVCG.2012.253

    Article  Google Scholar 

  • Turkay C, Kaya E, Balcisoy S, Hauser H (2016) Designing progressive and interactive analytics processes for high-dimensional data analysis. IEEE Trans Vis Comput Gr 23(1):131–140

    Article  Google Scholar 

  • Wang Q, Palpanas T (2021) Deep learning embeddings for data series similarity search. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp. 1708–1716

  • Yuan J, Chen C, Yang W, Liu M, Xia J, Liu S (2021) A survey of visual analytics techniques for machine learning. Comput Vis Med 7(1):3–36

    Article  Google Scholar 

  • Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp. 818–833

  • Zhao Y, Ge L, Xie H, Bai G, Zhang Z, Wei Q, Lin Y, Liu Y, Zhou F (2022) Astf: visual abstractions of time-varying patterns in radio signals. IEEE Trans Vis Comput Gr. https://doi.org/10.1109/TVCG.2022.3209469

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant U1836114.

Author information

Authors and Affiliations

  1. School of Big Data and Software Engineering, Chongqing University, Chongqing, China

    Yixuan Zhou, Runfeng Jiang & Haibo Hu

  2. College of Computer Science, Chongqing University, Chongqing, China

    Hongxing Qin

Authors
  1. Yixuan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Runfeng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongxing Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haibo Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haibo Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Jiang, R., Qin, H. et al. Representation and analysis of time-series data via deep embedding and visual exploration. J Vis 26, 593–610 (2023). https://doi.org/10.1007/s12650-022-00890-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12650-022-00890-3

Keywords

Search

Navigation