Skip to main content
SpringerLink
Log in

Visual time period analysis: a multimedia analytics application for summarizing and analyzing eye-tracking experiments

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, an increasing need for sophisticated multimedia analytics tools has been observed, which is triggered by a rapid growth of multimedia collections and by an increasing number of scientific fields embedding images in their studies. Although temporal data is ubiquitous and crucial in many applications, such tools typically do not support the analysis of data along the temporal dimension, especially for time periods. An appropriate visualization and comparison of period data associated with multimedia collections would help users to infer new information from such collections. In this paper, we present a novel multimedia analytics application for summarizing and analyzing temporal data from eye-tracking experiments. The application combines three different visual approaches: Time∘diff, visual-information-seeking mantra, and multi-viewpoint. A qualitative evaluation with domain experts confirmed that our application helps decision makers to summarize and analyze multimedia collections containing period data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. https://www.tobiipro.com/

  2. https://shiny.rstudio.com

References

  1. Aigner W, Miksch S (2006) Carevis: integrated visualization of computerized protocols and temporal patient data. Artif Intell Med 37(3):203–218

    Article  Google Scholar 

  2. Aigner W, Miksch S, Thurnher B, Biffl S (2005) Planninglines: novel glyphs for representing temporal uncertainties and their evaluation. In: IV, pp 457–463

  3. Aigner W, Miksch S, Müller W, Schumann H, Tominski C (2008) Visual methods for analyzing time-oriented data. IEEE Trans Vis Comput Graph 14(1):47–60

    Article  Google Scholar 

  4. André P, Wilson M, Russell A, Smith D, Owens A, Schraefel M (2007) Continuum: designing timelines for hierarchies, relationships and scale, pp 101–110

  5. Ankerst M, Kao A, Tjoelker R, Wang C (2008) Datajewel: integrating visualization with temporal data mining. In: Visual data mining. Springer, pp 312–330

  6. Behrend A, Schmiegelt P, Xie J, Fehling R, Ghoneimy A, Liu ZH, Chan ES, Gawlick D (2014) Temporal state management for supporting the real-time analysis of clinical data. In: ADBIS, pp 159–170

  7. Benito A, Losada AG, Therón R, Dorn A, Seltmann M, Wandl-Vogt E (2016) A spatio-temporal visual analysis tool for historical dictionaries. In: Proceedings of the fourth international conference on technological ecosystems for enhancing multiculturality, TEEM ’16. ACM, New York, pp 985–990

  8. Böhlen MH, Dignös A, Gamper J, Jensen CS (2018) Database technology for processing temporal data (invited paper). In: TIME, vol 120 of LIPIcs, pp 2:1–2:7. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik

  9. Böhlen MH, Dignös A, Gamper J, Jensen CS (2018) Temporal data management – an overview. In: Business intelligence and big data. Springer International Publishing, pp 51–83

  10. Burch M, Pompe D, Weiskopf D (2015) An analysis and visualization tool for dblp data. volume 2015-September, pp 163–170 Institute of Electrical and Electronics Engineers Inc.

  11. Chen H, Atabakhsh H, Tseng C, Marshall B, Kaza S, Eggers S, Gowda H, Shah A, Petersen T, Violette C (2005) Visualization in law enforcement. In: DG.O, pp 229–230

  12. Chinchor N, Thomas J, Wong P, Christel M, Ribarsky W (2010) Multimedia analysis + visual analytics = multimedia analytics. IEEE Comput Graph Appl 30(5):52–60

    Article  Google Scholar 

  13. Chittaro L, Combi C (2003) Visualizing queries on databases of temporal histories: new metaphors and their evaluation. Data Knowl Eng 44(2):239–264

    Article  Google Scholar 

  14. Combi C, Oliboni B (2012) Visually defining and querying consistent multi-granular clinical temporal abstractions. Artif Intell Med 54(2):75–101

    Article  Google Scholar 

  15. De Chiara D, Del Fatto V, Laurini R, Sebillo M, Vitiello G (2011) A chorem-based approach for visually analyzing spatial data. J Vis Lang Comput 22 (3):173–193

    Article  Google Scholar 

  16. De Chiara D, Del Fatto V, Sebillo M (2012) Visualizing geographical information through tag clouds. Springer, New York

    Book  Google Scholar 

  17. De Chiara D, Del Fatto V, Sebillo M, Tortora G, Vitiello G (2012) Tag@map: a web-based application for visually analyzing geographic information through georeferenced tag clouds. In: Proceedings of the 11th international conference on web and wireless geographical information systems, w2GIS’12. Springer-Verlag, Berlin, pp 72–81

  18. Del Fatto V, Dignös A, Gamper J (2018) Time ∘diff: a visual approach to compare period data. In: iV2018. IEEE Computer Society, p 7

  19. Dignös A., Böhlen MH, Gamper J (2012) Temporal alignment. In: SIGMOD, pp 433–444

  20. Dignös A., Böhlen MH, Gamper J (2013) Query time scaling of attribute values in interval timestamped databases. In: ICDE, pp 1304–1307

  21. Dignös A, Böhlen MH, Gamper J, Jensen CS (2016) Extending the kernel of a relational DBMS with comprehensive support for sequenced temporal queries. ACM Trans Database Syst 41(4):26:1–26:46

    Article  MathSciNet  Google Scholar 

  22. Dignös A, Glavic B, Niu X, Böhlen MH, Gamper J (2019) Snapshot semantics for temporal multiset relations. PVLDB 12(6):639–652

    Google Scholar 

  23. Fischer F, Fuchs J, Vervier P-A, Mansmann F, Thonnard O (2012) Vistracer: a visual analytics tool to investigate routing anomalies in traceroutes, pp 80–87

  24. Gamper J, Böhlen MH, Jensen CS (2009) Temporal aggregation. In: Encyclopedia of database systems, pp 2924–2929. Springer US

  25. Gregersen H, Jensen CS (1999) Temporal entity-relationship models - a survey. IEEE Trans Knowl Data Eng 11(3):464–497

    Article  Google Scholar 

  26. Hochheiser H, Shneiderman B (2004) Dynamic query tools for time series data sets Timebox widgets for interactive exploration. Inf Vis 3(1):1–18

    Article  Google Scholar 

  27. Jensen CS, Snodgrass RT (1999) Temporal data management. IEEE Trans Knowl Data Eng 11(1):36–44

    Article  Google Scholar 

  28. Jensen M (2003) Visualizing complex semantic timelines. Technical Report MSU-CSE-06-2 NewsBlip

  29. Kaptelinin V, Czerwinski M (2007) Beyond the desktop metaphor: designing integrated digital work environments. MIT Press, Cambridge

    Book  Google Scholar 

  30. Keim DA, Mansmann F, Schneidewind J, Ziegler H, Thomas J Visual analytics: scope and challenges. December 2008. Visual data mining: theory, techniques and tools for visual analytics, Springer, Lecture Notes in Computer Science (lncs)

  31. Keim ED, Kohlhammer J, Ellis G (2010), Mastering the information age: solving problems with visual analytics, eurographics association

  32. Kulkarni KG, Michels J-E (2012) Temporal features in sql: 2011. SIGMOD Record 41:34–43

    Article  Google Scholar 

  33. Lee JH, Ostwald MJ (2018) Measuring cognitive complexity in parametric design. International Journal of Design Creativity and Innovation 0(0):1–21

    Google Scholar 

  34. Liu T, Bouali F, Venturini G (2014) Technical section Exod: a tool for building and exploring a large graph of open datasets. Comput Graph 39:117–130

    Article  Google Scholar 

  35. Luo X, Tian F, Liu W, Teng D, Dai G, Wang H (2010) Visualizing time-series data in processlines: design and evaluation of a process enterprise application. In: SAC, pp 1165–1172

  36. Maccioni L, Borgianni Y, Basso D (2019) Value perception of green products: an exploratory study combining conscious answers and unconscious behavioral aspects. Sustainability 11(5), 1226; https://doi.org/10.3390/su11051226

  37. Mahlknecht G, Böhlen MH, Dignös A, Gamper J (2017) VISOR: Visualizing summaries of ordered data. In: SSDBM, pp 40:1–40:5

  38. Mahlknecht G, Dignös A, Gamper J (2017) A scalable dynamic programming scheme for the computation of optimal k-segments for ordered data. Inf Syst 70:2–17

    Article  Google Scholar 

  39. Meghdadi A, Irani P (2013) Interactive exploration of surveillance video through action shot summarization and trajectory visualization. IEEE Trans Vis Comput Graph 19(12):2119–2128

    Article  Google Scholar 

  40. Melton J (2006) Database language SQL. Springer, Berlin, pp 105–132

    Google Scholar 

  41. Olsson J, Boldt M (2009) Computer forensic timeline visualization tool. Digit Investig 6(SUPPL.):S78–S87

    Article  Google Scholar 

  42. Plaisant C, Milash B, Rose A, Widoff S, Shneiderman B (1996) Lifelines: visualizing personal histories. In: CHI, pp 221–227

  43. Richter HA, Brotherton JA, Abowd GD, Truong KN (1999) A multi-scale timeline slider for stream visualization and control. Technical Report GIT-GVU TR 99-30 Georgia Institute of Technology

  44. Rooij O, Van Wijk J, Worring M (2010) Mediatable: interactive categorization of multimedia collections. IEEE Comput Graph Appl 30(5):42–51

    Article  Google Scholar 

  45. Schüller G, Schmiegelt P, Behrend A (2015) Air traffic monitoring using datastream analysis techniques. In: 18th International Conference on Information Fusion, FUSION 2015, Washington, DC, USA, July 6-9, 2015. IEEE, pp 1238–1245

  46. Seyfert M, Viola I (2017) Dynamic word clouds. In: Proceedings of the 33rd spring conference on computer graphics, SCCG ’17. ACM, New York, pp 7:1–7:8

  47. Shmueli G, Jank W, Aris A, Plaisant C, Shneiderman B (2006) Exploring auction databases through interactive visualization. Decis Support Syst 42(3):1521–1538

    Article  Google Scholar 

  48. Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings 1996 IEEE symposium on visual languages, pp 336–343

  49. Silva SF, Catarci T (2000) Visualization of linear time-oriented data a survey. In: WISE, pp 310–319

  50. Steele J, Iliinsky N (2010) Beautiful visualization: looking at data through the eyes of experts. O’Reilly Media, Inc. 1st edition

  51. Tominski C, Abello J, Schumann H (2004) Axes-based visualizations with radial layouts. In: SAC, pp 1242–1247

  52. Weber M, Alexa M, Müller W (2001) Visualizing time-series on spirals. In: INFOVIS, pp 7–14

  53. Worring M, Engl A, Smeria C (2012) A multimedia analytics framework for browsing image collections in digital forensics. In: Proceedings of the 20th ACM international conference on multimedia, MM ’12. ACM, New York, pp 289–298

  54. Yang J, Luo D, Liu Y (2010) Newdle: interactive visual exploration of large online news collections. IEEE Comput Graph Appl 30(5):32–41

    Article  Google Scholar 

  55. Zahálka J, Worring M (2014) Towards interactive, intelligent, and integrated multimedia analytics. In: 2014 IEEE conference on visual analytics science and technology (VAST), pp 3–12

  56. Zhou X, Wang F, Zaniolo C (2006) Efficient temporal coalescing query support in relational database systems. In: DEXA, pp 676–686

Download references

Acknowledgements

We would like to thank our colleague Prof. Demis Basso, Faculty of Education of the Free University of Bozen-Bolzano, who provided insight and expertise that greatly assisted the research.

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Free University of Bozen-Bolzano, Bolzano, Italy

    Vincenzo Del Fatto, Anton Dignös, Guerriero Raimato & Johann Gamper

  2. Faculty of Science and Technology, Free University of Bozen-Bolzano, Bolzano, Italy

    Lorenzo Maccioni & Yuri Borgianni

Authors
  1. Vincenzo Del Fatto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anton Dignös
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guerriero Raimato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lorenzo Maccioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuri Borgianni
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Johann Gamper
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincenzo Del Fatto.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the projects VCTP (RTD call 2017) and EYE-TRACK (CRC call 2017) of the Free University of Bozen-Bolzano.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Del Fatto, V., Dignös, A., Raimato, G. et al. Visual time period analysis: a multimedia analytics application for summarizing and analyzing eye-tracking experiments. Multimed Tools Appl 78, 32779–32804 (2019). https://doi.org/10.1007/s11042-019-07950-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07950-1

Keywords

Search

Navigation