Skip to main content
Springer Nature Link
Log in

CrossIndex: Memory-Friendly and Session-Aware Index for Supporting Crossfilter in Interactive Data Exploration

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13245))

Included in the following conference series:

  • 3443 Accesses

Abstract

Crossfilter, a typical application for interactive data exploration (IDE), is widely used in data analysis, BI, and other fields. However, with the scale-up of the dataset, the real-time response of crossfilter can be hardly fulfilled. In this paper, we propose a memory-friendly and session-aware index called CrossIndex, which can support crossfilter-style queries with low latency. We first analyze a large number of query workloads generated by previous work and find that queries in the data exploration workload are inter-dependent, which means these queries have overlapped predicates. Based on this observation, this paper defines the inter-dependent queries as a session and builds a hierarchical index that can be used to accelerate crossfilter-style query processing by utilizing the overlapped property of the session to reduce unnecessary search space. Extensive experiments show that CrossIndex outperforms almost all other approaches and meanwhile keeps a low building cost.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://community.amstat.org/jointscsg-section/dataexpo/dataexpo2009.

  2. 2.

    http://www.tpc.org/tpch/.

  3. 3.

    http://www.tpc.org/tpcds/.

  4. 4.

    https://www.cs.umb.edu/~poneil/StarSchemaB.PDF.

  5. 5.

    http://kylin.apache.org/.

  6. 6.

    https://www.omnisci.com/.

References

  1. Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., Stoica, I.: BlinkDB: queries with bounded errors and bounded response times on very large data. In: EuroSys, pp. 29–42 (2013)

    Google Scholar 

  2. Battle, L., Chang, R., Stonebraker, M.: Dynamic prefetching of data tiles for interactive visualization. In: SIGMOD, pp. 1363–1375 (2016)

    Google Scholar 

  3. Battle, L., et al.: Database benchmarking for supporting real-time interactive querying of large data. In: SIGMOD, pp. 1571–1587 (2020)

    Google Scholar 

  4. Battle, L., Heer, J.: Characterizing exploratory visual analysis: a literature review and evaluation of analytic provenance in tableau. In: CGF, vol. 38, pp. 145–159 (2019)

    Google Scholar 

  5. Boncz, P.A., Zukowski, M., Nes, N.: Monetdb/x100: hyper-pipelining query execution. In: CIDR, vol. 5, pp. 225–237 (2005)

    Google Scholar 

  6. Chaudhuri, S., Ding, B., Kandula, S.: Approximate query processing: no silver bullet. In: SIGMOD, pp. 511–519 (2017)

    Google Scholar 

  7. Ding, B., Huang, S., Chaudhuri, S., Chakrabarti, K., Wang, C.: Sample+ seek: approximating aggregates with distribution precision guarantee. In: SIGMOD, pp. 679–694 (2016)

    Google Scholar 

  8. Doshi, P.R., Rundensteiner, E.A., Ward, M.O.: Prefetching for visual data exploration. In: DASFAA, pp. 195–202 (2003)

    Google Scholar 

  9. Eichmann, P., Zgraggen, E., Binnig, C., Kraska, T.: IDEBench: a benchmark for interactive data exploration. In: SIGMOD, pp. 1555–1569 (2020)

    Google Scholar 

  10. Fekete, J., Fisher, D., Nandi, A., Sedlmair, M.: Progressive data analysis and visualization (Dagstuhl seminar 18411). Dagstuhl Rep. 8(10), 1–40 (2018)

    Google Scholar 

  11. Fisher, D., Popov, I., Drucker, S., Schraefel, M.: Trust me, I’m partially right: incremental visualization lets analysts explore large datasets faster. In: SIGCHI, pp. 1673–1682 (2012)

    Google Scholar 

  12. Gray, J., et al.: Data cube: a relational aggregation operator generalizing group-by, cross-tab, and sub-totals. DMKD 1(1), 29–53 (1997)

    Google Scholar 

  13. Hellerstein, J.M., Haas, P.J., Wang, H.J.: Online aggregation. In: SIGMOD, pp. 171–182 (1997)

    Google Scholar 

  14. Kalinin, A., Cetintemel, U., Zdonik, S.: Interactive data exploration using semantic windows. In: SIGMOD, pp. 505–516 (2014)

    Google Scholar 

  15. Kandel, S., Parikh, R., Paepcke, A., Hellerstein, J.M., Heer, J.: Profiler: integrated statistical analysis and visualization for data quality assessment. In: AVI, pp. 547–554 (2012)

    Google Scholar 

  16. Li, L., et al.: BinDex: a two-layered index for fast and robust scans. In: SIGMOD, pp. 909–923 (2020)

    Google Scholar 

  17. Lins, L., Klosowski, J.T., Scheidegger, C.: NanoCubes for real-time exploration of spatiotemporal datasets. TVCG 19(12), 2456–2465 (2013)

    Google Scholar 

  18. Liu, Z., Heer, J.: The effects of interactive latency on exploratory visual analysis. TVCG 20(12), 2122–2131 (2014)

    Google Scholar 

  19. Liu, Z., Jiang, B., Heer, J.: imMens: real-time visual querying of big data. In: CGF, vol. 32, pp. 421–430 (2013)

    Google Scholar 

  20. Moritz, D., Howe, B., Heer, J.: Falcon: balancing interactive latency and resolution sensitivity for scalable linked visualizations. In: SIGCHI, pp. 1–11 (2019)

    Google Scholar 

  21. Psallidas, F., Wu, E.: Provenance for interactive visualizations. In: HILDA, pp. 1–8 (2018)

    Google Scholar 

  22. Psallidas, F., Wu, E.: Smoke: fine-grained lineage at interactive speed. Proc. VLDB Endow. (2018)

    Google Scholar 

  23. Satyanarayan, A., Russell, R., Hoffswell, J., Heer, J.: Reactive vega: a streaming dataflow architecture for declarative interactive visualization. TVCG 22(1), 659–668 (2015)

    Google Scholar 

  24. Vartak, M., Rahman, S., Madden, S., Parameswaran, A., Polyzotis, N.: SeeDB: efficient data-driven visualization recommendations to support visual analytics. Proc. VLDB Endow. 8(13), 2182–2193 (2015)

    Article  Google Scholar 

  25. Wu, Z., Jing, Y., He, Z., Guo, C., Wang, X.S.: POLYTOPE: a flexible sampling system for answering exploratory queries. World Wide Web 23(1), 1–22 (2019). https://doi.org/10.1007/s11280-019-00685-x

    Article  Google Scholar 

  26. Yang, Z., et al.: iExplore: accelerating exploratory data analysis by predicting user intention. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds.) DASFAA 2018. LNCS, vol. 10828, pp. 149–165. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91458-9_9

    Chapter  Google Scholar 

  27. Zhang, Y., Zhang, H., He, Z., Jing, Y., Zhang, K., Wang, X.S.: Parrot: a progressive analysis system on large text collections. Data Sci. Eng. 6(1), 1–19 (2021)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by the NSFC (No. 61732004, No. U1836207 and No. 62072113), the National Key R&D Program of China (No. 2018YFB1004404) and the Zhejiang Lab (No. 2021PE0AC01).

Author information

Authors and Affiliations

  1. School of Computer Science, Fudan University, Shanghai, China

    Tianyu Xia, Hanbing Zhang, Yinan Jing, Zhenying He, Kai Zhang & X. Sean Wang

  2. Shanghai Key Laboratory of Data Science, Shanghai, China

    Tianyu Xia, Hanbing Zhang, Yinan Jing, Zhenying He, Kai Zhang & X. Sean Wang

  3. Shanghai Institute of Intelligent Electronics and Systems, Shanghai, China

    X. Sean Wang

Authors
  1. Tianyu Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanbing Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinan Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenying He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. X. Sean Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinan Jing .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Kanpur, Kanpur, India

    Arnab Bhattacharya

  2. National University of Singapore, Singapore, Singapore

    Janice Lee Mong Li

  3. University of California, Santa Barbara, Santa Barbara, CA, USA

    Divyakant Agrawal

  4. IIIT Hyderabad, Hyderabad, India

    P. Krishna Reddy

  5. Indraprastha Institute of Information Technology Delhi, New Delhi, India

    Mukesh Mohania

  6. Ashoka University, Sonepat, Haryana, India

    Anirban Mondal

  7. Indraprastha Institute of Information Technology Delhi, New Delhi, India

    Vikram Goyal

  8. University of Aizu, Aizu, Japan

    Rage Uday Kiran

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xia, T., Zhang, H., Jing, Y., He, Z., Zhang, K., Wang, X.S. (2022). CrossIndex: Memory-Friendly and Session-Aware Index for Supporting Crossfilter in Interactive Data Exploration. In: Bhattacharya, A., et al. Database Systems for Advanced Applications. DASFAA 2022. Lecture Notes in Computer Science, vol 13245. Springer, Cham. https://doi.org/10.1007/978-3-031-00123-9_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-00123-9_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-00122-2

  • Online ISBN: 978-3-031-00123-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics