Skip to main content
SpringerLink
Log in

Past and Future Steps for Adaptive Storage Data Systems: From Shallow to Deep Adaptivity

  • Conference paper
  • First Online:
Real-Time Business Intelligence and Analytics (BIRTE 2015, BIRTE 2016, BIRTE 2017)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 337))

  • 381 Accesses

Abstract

Data systems with adaptive storage can autonomously change their behavior by altering how data is stored and accessed. Such systems have been studied primarily for the case of adaptive indexing to automatically create the right indexes at the right granularity. More recently work on adaptive loading and adaptive data layouts brought even more flexibility. We survey this work and describe the need for even deeper adaptivity that goes beyond adjusting knobs in a single architecture; instead it can adapt the fundamental architecture of a data system to drastically alter its behavior.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, D.J., Boncz, P., Harizopoulos, S., Idreos, S., Madden, S.: The design and implementation of modern column-oriented database systems. Found. Trends Databases 5(3), 197–280 (2013)

    Article  Google Scholar 

  2. Alagiannis, I., Borovica, R., Branco, M., Idreos, S., Ailamaki, A.: NoDB: efficient query execution on raw data files. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 241–252 (2012)

    Google Scholar 

  3. Alagiannis, I., Borovica, R., Branco, M., Idreos, S., Ailamaki, A.: NoDB in action: adaptive query processing on raw data. Proc. VLDB Endow. 5(12), 1942–1945 (2012)

    Article  Google Scholar 

  4. Alagiannis, I., Borovica-Gajic, R., Branco, M., Idreos, S., Ailamaki, A.: NoDB: efficient query execution on raw data files. Commun. ACM 58(12), 112–121 (2015)

    Article  Google Scholar 

  5. Alagiannis, I., Idreos, S., Ailamaki, A.: H2O: a hands-free adaptive store. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 1103–1114 (2014)

    Google Scholar 

  6. Alvarez, V., Schuhknecht, F.M., Dittrich, J., Richter, S.: Main memory adaptive indexing for multi-core systems. In: Tenth International Workshop on Data Management on New Hardware, DaMoN 2014, 23 June 2014, Snowbird, UT, USA, pp. 3:1–3:10 (2014)

    Google Scholar 

  7. Arulraj, J., Pavlo, A., Menon, P.: Bridging the archipelago between row-stores and column-stores for hybrid workloads. In: Proceedings of the ACM SIGMOD International Conference on Management of Data (2016)

    Google Scholar 

  8. Athanassoulis, M., Idreos, S.: Design tradeoffs of data access methods. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, Tutorial (2016)

    Google Scholar 

  9. Bruno, N., Chaudhuri, S.: To tune or not to tune?: a lightweight physical design alerter. In: Proceedings of the International Conference on Very Large Data Bases (VLDB), pp. 499–510 (2006)

    Google Scholar 

  10. Bruno, N., Chaudhuri, S.: An online approach to physical design tuning. In: Proceedings of the IEEE International Conference on Data Engineering (ICDE), pp. 826–835 (2007)

    Google Scholar 

  11. Chaudhuri, S., Narasayya, V.R.: An efficient cost-driven index selection tool for Microsoft SQL server. In: Proceedings of the International Conference on Very Large Data Bases (VLDB), pp. 146–155 (1997)

    Google Scholar 

  12. Deshpande, A., Hellerstein, J.M., Raman, V.: Adaptive query processing: why, how, when, what next. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, 27–29 June 2006, Chicago, Illinois, USA, pp. 806–807 (2006)

    Google Scholar 

  13. Graefe, G., Halim, F., Idreos, S., Kuno, H., Manegold, S.: Concurrency control for adaptive indexing. Proc. VLDB Endow. 5(7), 656–667 (2012)

    Article  Google Scholar 

  14. Graefe, G., Halim, F., Idreos, S., Kuno, H.A., Manegold, S., Seeger, B.: Transactional support for adaptive indexing. VLDB J. 23(2), 303–328 (2014)

    Article  Google Scholar 

  15. Graefe, G., Idreos, S., Kuno, H., Manegold, S.: Benchmarking adaptive indexing. In: Nambiar, R., Poess, M. (eds.) TPCTC 2010. LNCS, vol. 6417, pp. 169–184. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18206-8_13

    Chapter  Google Scholar 

  16. Graefe, G., Kuno, H.: Self-selecting, self-tuning, incrementally optimized indexes. In: Proceedings of the International Conference on Extending Database Technology (EDBT), pp. 371–381 (2010)

    Google Scholar 

  17. Halim, F., Idreos, S., Karras, P., Yap, R.H.C.: Stochastic database cracking: towards robust adaptive indexing in main-memory column-stores. Proc. VLDB Endow. 5(6), 502–513 (2012)

    Article  Google Scholar 

  18. Hellerstein, J.M., et al.: Adaptive query processing: technology in evolution. IEEE Data Eng. Bull. 23(2), 7–18 (2000)

    Google Scholar 

  19. Hellerstein, J.M., Stonebraker, M., Hamilton, J.R.: Architecture of a database system. Found. Trends Databases 1(2), 141–259 (2007)

    Article  Google Scholar 

  20. Idreos, S.: Database cracking: towards auto-tuning database kernels. Ph.D. thesis, University of Amsterdam (2010)

    Google Scholar 

  21. Idreos, S., Alagiannis, I., Johnson, R., Ailamaki, A.: Here are my data files. Here are my queries. Where are my results? In: Proceedings of the Biennial Conference on Innovative Data Systems Research (CIDR), pp. 57–68 (2011)

    Google Scholar 

  22. Idreos, S., Kersten, M.L., Manegold, S.: Database cracking. In: Proceedings of the Biennial Conference on Innovative Data Systems Research (CIDR) (2007)

    Google Scholar 

  23. Idreos, S., Kersten, M.L., Manegold, S.: Updating a cracked database. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 413–424 (2007)

    Google Scholar 

  24. Idreos, S., Kersten, M.L., Manegold, S.: Self-organizing tuple reconstruction in column-stores. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 297–308 (2009)

    Google Scholar 

  25. Idreos, S., Manegold, S., Kuno, H., Graefe, G.: Merging what’s cracked, cracking what’s merged: adaptive indexing in main-memory column-stores. Proc. VLDB Endow. 4(9), 586–597 (2011)

    Article  Google Scholar 

  26. Idreos, S., Papaemmanouil, O., Chaudhuri, S.: Overview of data exploration techniques. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, Tutorial, pp. 277–281 (2015)

    Google Scholar 

  27. Karras, P., Nikitin, A., Saad, M., Bhatt, R., Antyukhov, D., Idreos, S.: Adaptive indexing over encrypted numeric data. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 171–183 (2016)

    Google Scholar 

  28. Liu, Z., Idreos, S.: Main memory adaptive denormalization. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 2253–2254 (2016)

    Google Scholar 

  29. Lu, Y., Shanbhag, A., Jindal, A., Madden, S.: AdaptDB: adaptive partitioning for distributed joins. PVLDB 10(5), 589–600 (2017)

    Google Scholar 

  30. Petraki, E., Idreos, S., Manegold, S.: Holistic indexing in main-memory column-stores. In: Proceedings of the ACM SIGMOD International Conference on Management of Data (2015)

    Google Scholar 

  31. Pirk, H., Petraki, E., Idreos, S., Manegold, S., Kersten, M.L.: Database cracking: fancy scan, not poor man’s sort! In: Proceedings of the International Workshop on Data Management on New Hardware (DAMON), pp. 1–8 (2014)

    Google Scholar 

  32. Qin, W., Idreos, S.: Adaptive data skipping in main-memory systems. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 2255–2256 (2016)

    Google Scholar 

  33. Richter, S., Quiané-Ruiz, J.-A., Schuh, S., Dittrich, J.: Towards zero-overhead static and adaptive indexing in Hadoop. VLDB J. 23(3), 469–494 (2013)

    Article  Google Scholar 

  34. Schuh, S., Dittrich, J.: AIR: adaptive index replacement in Hadoop. In: 31st IEEE International Conference on Data Engineering Workshops, ICDE Workshops 2015, 13–17 April 2015, Seoul, South Korea, pp. 22–29 (2015)

    Google Scholar 

  35. Schuhknecht, F.M., Jindal, A., Dittrich, J.: The uncracked pieces in database cracking. Proc. VLDB Endow. 7(2), 97–108 (2013)

    Article  Google Scholar 

  36. Schuhknecht, F.M., Jindal, A., Dittrich, J.: An experimental evaluation and analysis of database cracking. Very Large Database J. VLDBJ 25(1), 27–52 (2016)

    Article  Google Scholar 

  37. Zoumpatianos, K., Idreos, S., Palpanas, T.: Indexing for interactive exploration of big data series. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 1555–1566 (2014)

    Google Scholar 

  38. Zoumpatianos, K., Idreos, S., Palpanas, T.: ADS: the adaptive data series index. Very Large Database J. VLDBJ 25(6), 843–866 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Harvard University, Cambridge, USA

    Stratos Idreos, Manos Athanassoulis, Niv Dayan, Demi Guo, Mike S. Kester, Lukas Maas & Kostas Zoumpatianos

Authors
  1. Stratos Idreos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manos Athanassoulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Niv Dayan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Demi Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mike S. Kester
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lukas Maas
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kostas Zoumpatianos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stratos Idreos .

Editor information

Editors and Affiliations

  1. Teradata, Santa Clara, CA, USA

    Malu Castellanos

  2. University of Pittsburgh, Pittsburgh, PA, USA

    Panos K. Chrysanthis

  3. University of Pittsburgh, Pittsburgh, PA, USA

    Konstantinos Pelechrinis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Idreos, S. et al. (2019). Past and Future Steps for Adaptive Storage Data Systems: From Shallow to Deep Adaptivity. In: Castellanos, M., Chrysanthis, P., Pelechrinis, K. (eds) Real-Time Business Intelligence and Analytics. BIRTE BIRTE BIRTE 2015 2016 2017. Lecture Notes in Business Information Processing, vol 337. Springer, Cham. https://doi.org/10.1007/978-3-030-24124-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-24124-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-24123-0

  • Online ISBN: 978-3-030-24124-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation