Skip to main content
SpringerLink
Log in

Modeling and Assessing User Interaction in Big Data Visualization Systems

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2023 (INTERACT 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14143))

Included in the following conference series:

Abstract

When users interact with large data through a visualization system, its response time is crucial in keeping engagement and efficacy as high as possible, and latencies as low as 500 ms can be detrimental to the correct execution of the analysis. This can be due to several causes: (i) for large data or high query rates, database management systems (DBMS) may fail to meet the performance needs; (ii) modeling all the interactions with a visualization system is challenging due to their exploratory nature, where not all of them are equally demanding in terms of computation time; (iii) there is a lack of models for integrating optimizations in a holistic way, hampering consistent evaluation across systems. In response to these problems, we propose a conceptual interaction-driven framework that enhances the visualization pipeline by adding a new Translation layer between the Data-, Visualization- and Interaction- layers, leveraging the modeling of interactions with augmented statecharts. This new layer aims to collect information about queries and rendering computations, linking such values to interactions in the statechart. To make the Translation layer actionable, we contribute a software component to automatically model the user interactions for a generic web-based visualization system through augmented statecharts, in which each interaction is labeled with its latency threshold. We first demonstrate its generality on ten state-of-the-art visualization systems. Then we perform a user study (n = 50), collecting traces by asking users to perform already established exploratory tasks on the well-known Crossfilter interface. Finally, we replay those traces over its generated statechart, assessing the capability to model the user interactions correctly and describing violations in the latency thresholds.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://osf.io/79hsw/?view_only=be4e5107a18145e6b86a7eaf6109cb60

References

  1. Ali, S.M., Gupta, N., Nayak, G.K., Lenka, R.K.: Big data visualization: tools and challenges. In: 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), pp. 656–660 (2016). https://doi.org/10.1109/IC3I.2016.7918044

  2. Amar, R.A., Eagan, J., Stasko, J.T.: Low-level components of analytic activity in information visualization, pp. 111–117. IEEE (2005). https://doi.org/10.1109/INFVIS.2005.1532136

  3. Angelini, M., Blasilli, G., Farina, L., Lenti, S., Santucci, G.: Nemesis (network medicine analysis): Towards visual exploration of network medicine data (2019)

    Google Scholar 

  4. Angelini, M., Blasilli, G., Lenti, S., Palleschi, A., Santucci, G.: Towards enhancing radviz analysis and interpretation. In: 2019 IEEE Visualization Conference (VIS), pp. 226–230 (2019). https://doi.org/10.1109/VISUAL.2019.8933775

  5. Angelini, M., Blasilli, G., Lenti, S., Palleschi, A., Santucci, G.: CrossWidgets: enhancing complex data selections through modular multi attribute selectors (2020). https://doi.org/10.1145/3399715.3399918

  6. Angelini, M., Blasilli, G., Lenti, S., Santucci, G.: STEIN: speeding up evaluation activities with a seamless testing environment INtegrator. In: Johansson, J., Sadlo, F., Schreck, T. (eds.) EuroVis 2018 - Short Papers. The Eurographics Association (2018). https://doi.org/10.2312/eurovisshort.20181083

  7. Angelini, M., Blasilli, G., Lenti, S., Santucci, G.: A visual analytics conceptual framework for explorable and steerable partial dependence analysis. IEEE Trans. Visual. Comput. Graph. 1–16 (2023)

    Google Scholar 

  8. Angelini, M., Catarci, T., Santucci, G.: Ivan: an interactive Herlofson’s nomogram visualizer for local weather forecast. Computers 8(3) (2019). https://doi.org/10.3390/computers8030053

  9. Angelini, M., Lenti, S., Santucci, G.: Crumbs: a cyber security framework browser. In: 2017 IEEE Symposium on Visualization for Cyber Security (VizSec), pp. 1–8 (2017). https://doi.org/10.1109/VIZSEC.2017.8062194

  10. Angelini, M., Santucci, G., Schumann, H., Schulz, H.J.: A review and characterization of progressive visual analytics. Informatics 5(3) (2018). https://doi.org/10.3390/informatics5030031

  11. Baeza-Yates, R., Ribeiro-Neto, B., et al.: Modern Information Retrieval, vol. 463. ACM Press, New York (1999)

    Google Scholar 

  12. Battle, L., et al.: Database benchmarking for supporting real-time interactive querying of large data. In: SIGMOD ’20, New York, NY, USA, pp. 1571–1587, June 2020. https://doi.org/10.1145/3318464.3389732

  13. Benvenuti, D., Buda, E., Fraioli, F., Marrella, A., Catarci, T.: Detecting and explaining usability issues of consumer electronic products. In: Ardito, C., et al. (eds.) INTERACT 2021. LNCS, vol. 12935, pp. 298–319. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85610-6_18

    Chapter  Google Scholar 

  14. Bigelow, A., Nobre, C., Meyer, M., Lex, A.: Origraph: interactive network wrangling. In: 2019 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 81–92. IEEE (2019)

    Google Scholar 

  15. Blanch, R., Beaudouin-Lafon, M.: Programming rich interactions using the hierarchical state machine toolkit. In: AVI ’06, pp. 51–58. ACM, New York, NY, USA (2006). https://doi.org/10.1145/1133265.1133275

  16. Bostock, M., Ogievetsky, V., Heer, J.: D\(^3\) data-driven documents. IEEE Trans. Vis. Comput. Graph. 17(12), 2301–2309 (2011). https://doi.org/10.1109/TVCG.2011.185

    Article  Google Scholar 

  17. Brehmer, M., Munzner, T.: A multi-level typology of abstract visualization tasks. IEEE Trans. Visual Comput. Graph. 19(12), 2376–2385 (2013). https://doi.org/10.1109/TVCG.2013.124

    Article  Google Scholar 

  18. Card, S., Mackinlay, J.: The structure of the information visualization design space. In: Proceedings of VIZ ’97: Visualization Conference, Information Visualization Symposium and Parallel Rendering Symposium, pp. 92–99 (1997). https://doi.org/10.1109/INFVIS.1997.636792

  19. Class, B.: Summit: scaling deep learning interpretability by visualizing activation and attribution summarizations

    Google Scholar 

  20. Conner, C., Samuel, J., Garvey, M., Samuel, Y., Kretinin, A.: Conceptual frameworks for big data visualization: discussion of models, methods, and artificial intelligence for graphical representations of data. In: Handbook of Research for Big Data, pp. 197–234. Apple Academic Press (2021)

    Google Scholar 

  21. Dabrowski, J.R., Munson, E.V.: Is 100 milliseconds too fast? In: CHI’01 Extended Abstracts on Human Factors in Computing Systems, pp. 317–318 (2001)

    Google Scholar 

  22. Desolda, G., Esposito, A., Lanzilotti, R., Costabile, M.F.: Detecting emotions through machine learning for automatic UX evaluation. In: Ardito, C., et al. (eds.) INTERACT 2021. LNCS, vol. 12934, pp. 270–279. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85613-7_19

    Chapter  Google Scholar 

  23. Edwards, M., Aspinall, D.: The synthesis of digital systems using ASM design techniques. In: Computer Hardware Description Languages and their Applications, pp. 55–64 (1983)

    Google Scholar 

  24. Erraissi, A., Mouad, B., Belangour, A.: A big data visualization layer meta-model proposition. In: 2019 8th International Conference on Modeling Simulation and Applied Optimization (ICMSAO), pp. 1–5 (2019). https://doi.org/10.1109/ICMSAO.2019.8880276

  25. Fekete, J.D., Fisher, D., Nandi, A., Sedlmair, M.: Progressive data analysis and visualization (Dagstuhl Seminar 18411). Dagstuhl Rep. 8(10), 1–40 (2019). https://doi.org/10.4230/DagRep.8.10.1, http://drops.dagstuhl.de/opus/volltexte/2019/10346

  26. Ferrentino, A., AB, F.: State machines and their semantics in software engineering (1977)

    Google Scholar 

  27. Feyock, S.: Transition diagram-based cai/help systems. Int. J. Man Mach. Stud. 9(4), 399–413 (1977)

    Article  Google Scholar 

  28. Galletta, A., Carnevale, L., Bramanti, A., Fazio, M.: An innovative methodology for big data visualization for telemedicine. IEEE Trans. Industr. Inf. 15(1), 490–497 (2019). https://doi.org/10.1109/TII.2018.2842234

    Article  Google Scholar 

  29. Golfarelli, M., Rizzi, S.: A model-driven approach to automate data visualization in big data analytics. Inf. Visual. 19(1), 24–47 (2020). https://doi.org/10.1177/1473871619858933, https://doi.org/10.1177/1473871619858933

  30. Han, F., Xu, T., Tian, C., Hou, Z.: Investigation on human visual response latency. In: 2010 International Conference On Computer Design and Applications, vol. 1, pp. V1–602. IEEE (2010)

    Google Scholar 

  31. Harel, D.: Statecharts: a visual formalism for complex systems. Sci. Comput. Program. 8(3), 231–274 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  32. Huot, S., Dumas, C., Dragicevic, P., Fekete, J.D., Hégron, G.: The MaggLite post-WIMP toolkit: draw it, connect it and run it. In: Proceedings of ACM Symposium on User Interface Software and Technology. UIST ’04, pp. 257–266. ACM, New York, NY, USA (2004)

    Google Scholar 

  33. Jacob, R.J.: Using formal specifications in the design of a human-computer interface. Commun. ACM 26(4), 259–264 (1983)

    Article  Google Scholar 

  34. Keim, D., Andrienko, G., Fekete, J.-D., Görg, C., Kohlhammer, J., Melançon, G.: Visual analytics: definition, process, and challenges. In: Kerren, A., Stasko, J.T., Fekete, J.-D., North, C. (eds.) Information Visualization. LNCS, vol. 4950, pp. 154–175. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-70956-5_7

    Chapter  Google Scholar 

  35. Keim, D., Qu, H., Ma, K.L.: Big-data visualization. IEEE Comput. Graph. Appl. 33(4), 20–21 (2013). https://doi.org/10.1109/MCG.2013.54

    Article  Google Scholar 

  36. Liu, Z., Heer, J.: The effects of interactive latency on exploratory visual analysis. IEEE Trans. Vis. Comput. Graph. 20(12), 2122–2131 (2014). https://doi.org/10.1109/TVCG.2014.2346452

    Article  Google Scholar 

  37. Livny, M., et al.: Devise: integrated querying and visual exploration of large datasets. In: SIGMOD ’97, pp. 301–312. Association for Computing Machinery, New York, NY, USA (1997)

    Google Scholar 

  38. Marrella, A., Catarci, T.: Measuring the learnability of interactive systems using a petri net based approach. In: Proceedings of the 2018 Designing Interactive Systems Conference. In: DIS ’18, pp. 1309–1319 (2018). https://doi.org/10.1145/3196709.3196744

  39. Miller, R.B.: Response time in man-computer conversational transactions. In: Proceedings of the December 9–11, 1968, Fall Joint Computer Conference, Part I, pp. 267–277 (1968)

    Google Scholar 

  40. Mohammed, L.T., AlHabshy, A.A., ElDahshan, K.A.: Big data visualization: a survey, In: 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1–12 (2022). https://doi.org/10.1109/HORA55278.2022.9799819

  41. Moreland, K.: A survey of visualization pipelines. IEEE Trans. Vis. Comput. Graph. 19(3), 367–378 (2013). https://doi.org/10.1109/TVCG.2012.133

    Article  Google Scholar 

  42. Moritz, D., Howe, B., Heer, J.: Falcon: balancing interactive latency and resolution sensitivity for scalable linked visualizations. In: CHI ’19. ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300924

  43. Myers, B.A.: Separating application code from toolkits: eliminating the spaghetti of call-backs. In: Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology. UIST ’91, pp. 211–220. ACM, New York, NY, USA (1991)

    Google Scholar 

  44. Nah, F.F.H.: A study on tolerable waiting time: how long are web users willing to wait? Behav. Inf. Technol. 23(3), 153–163 (2004)

    Article  Google Scholar 

  45. Nielsen, J.: Usability Engineering. Morgan Kaufmann, Burlington (1993)

    Google Scholar 

  46. Oney, S., Myers, B., Brandt, J.: Interstate: a language and environment for expressing interface behavior. In: Proceedings of ACM Symposium on User Interface Software and Technology. UIST ’14, pp. 263–272. ACM, New York, NY, USA (2014)

    Google Scholar 

  47. Parnas, D.L.: On the use of transition diagrams in the design of a user interface for an interactive computer system. In: Proceedings of the 1969 24th National Conference, pp. 379–385 (1969)

    Google Scholar 

  48. Qin, X., Luo, Y., Tang, N., Li, G.: Deepeye: an automatic big data visualization framework. Big Data Min. Anal. 1(1), 75–82 (2018). https://doi.org/10.26599/BDMA.2018.9020007

  49. Raghav, R.S., Pothula, S., Vengattaraman, T., Ponnurangam, D.: A survey of data visualization tools for analyzing large volume of data in big data platform. In: 2016 International Conference on Communication and Electronics Systems (ICCES), pp. 1–6 (2016). https://doi.org/10.1109/CESYS.2016.7889976

  50. Sacha, D., Stoffel, A., Stoffel, F., Kwon, B.C., Ellis, G., Keim, D.A.: Knowledge generation model for visual analytics. IEEE Trans. Visual Comput. Graph. 20(12), 1604–1613 (2014). https://doi.org/10.1109/TVCG.2014.2346481

    Article  Google Scholar 

  51. Shin, M., Soen, A., Readshaw, B.T., Blackburn, S.M., Whitelaw, M., Xie, L.: Influence flowers of academic entities. In: 2019 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 1–10. IEEE (2019)

    Google Scholar 

  52. Shneiderman, B.: Response time and display rate in human performance with computers. ACM Comput. Surv. (CSUR) 16(3), 265–285 (1984)

    Article  Google Scholar 

  53. Shneiderman, B.: The eyes have it: a task by data type taxonomy for information visualizations, pp. 364–371. Interactive Technologies (2003). https://doi.org/10.1016/B978-155860915-0/50046-9

  54. Shneiderman, B., Plaisant, C., Cohen, M., Jacobs, S., Elmqvist, N., Diakopoulos, N.: Designing the User Interface: Strategies for Effective Human-Computer Interaction, 6th edn. Pearson, London (2016)

    Google Scholar 

  55. Strahl, J., Peltonen, J., Floréen, P.: Directing and combining multiple queries for exploratory search by visual interactive intent modeling. In: Ardito, C., et al. (eds.) INTERACT 2021. LNCS, vol. 12934, pp. 514–535. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85613-7_34

    Chapter  Google Scholar 

  56. Vredenburg, K., Mao, J.Y., Smith, P.W., Carey, T.: A survey of user-centered design practice. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’02, pp. 471–478. Association for Computing Machinery, New York, NY, USA (2002)

    Google Scholar 

  57. Waloszek, G., Kreichgauer, U.: User-centered evaluation of the responsiveness of applications. In: Gross, T., et al. (eds.) INTERACT 2009. LNCS, vol. 5726, pp. 239–242. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03655-2_29

    Chapter  Google Scholar 

  58. Woodburn, L., Yang, Y., Marriott, K.: Interactive visualisation of hierarchical quantitative data: an evaluation. In: 2019 IEEE Visualization Conference (VIS), pp. 96–100. IEEE (2019)

    Google Scholar 

  59. Yang, J., Bäuerle, A., Moritz, D., Çağatay Demiralp: Vegaprof: Profiling vega visualizations (2022)

    Google Scholar 

  60. Yi, J.S., ah Kang, Y., Stasko, J.T., Jacko, J.A.: Toward a deeper understanding of the role of interaction in information visualization. IEEE Trans. Vis. Comput. Graph. (2007)

    Google Scholar 

  61. Zgraggen, E., Galakatos, A., Crotty, A., Fekete, J.D., Kraska, T.: How progressive visualizations affect exploratory analysis. IEEE Trans. Vis. Comput. Graph. 23(8), 1977–1987 (2017). https://doi.org/10.1109/TVCG.2016.2607714

    Article  Google Scholar 

  62. Zhang, T., Ramakrishnan, R., Livny, M.: Birch: an efficient data clustering method for very large databases. SIGMOD ’96, pp. 103–114. Association for Computing Machinery, New York, NY, USA (1996)

    Google Scholar 

  63. Zhang, Y., Chanana, K., Dunne, C.: IDMVis: temporal event sequence visualization for type 1 diabetes treatment decision support. IEEE Trans. Visual Comput. Graph. 25(1), 512–522 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

We thank Dr. Leilani Battle and Dr. Jean-Daniel Fekete for the initial discussion and precious suggestions on this work. This study was carried out within the MICS (Made in Italy - Circular and Sustainable) Extended Partnership and received funding from the European Union Next-GenerationEU (PIANO NAZIONALE DI RIPRESA E RESILIENZA (PNRR) - MISSIONE 4 COMPONENTE 2, INVESTIMENTO 1.3 - D.D. 1551.11- 10-2022, PE00000004).

The work of Dario Benvenuti is supported by the H2020 project DataCloud (Grant number: 101016835)

Author information

Authors and Affiliations

  1. Sapienza, University of Rome, Rome, Italy

    Dario Benvenuti, Matteo Filosa, Tiziana Catarci & Marco Angelini

Authors
  1. Dario Benvenuti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matteo Filosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tiziana Catarci
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Angelini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dario Benvenuti .

Editor information

Editors and Affiliations

  1. University of West London, London, UK

    José Abdelnour Nocera

  2. Reykjavik University, Reykjavik, Iceland

    Marta Kristín Lárusdóttir

  3. University of York, York, UK

    Helen Petrie

  4. University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  5. Université Côte d’Azur, Sophia Antipolis Cedex, France

    Marco Winckler

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Benvenuti, D., Filosa, M., Catarci, T., Angelini, M. (2023). Modeling and Assessing User Interaction in Big Data Visualization Systems. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14143. Springer, Cham. https://doi.org/10.1007/978-3-031-42283-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42283-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42282-9

  • Online ISBN: 978-3-031-42283-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation