Skip to main content
SpringerLink
Log in

Scheduling Parallel Data Transfers in Multi-tiered Persistent Storage

  • Conference paper
  • First Online:
Recent Challenges in Intelligent Information and Database Systems (ACIIDS 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1716))

Included in the following conference series:

  • 844 Accesses

Abstract

Multi-tiered persistent storage provides a logical view where all available storage is distributed over a number of levels of different speeds and capacities. Efficient scheduling of parallel data transfers in multi-tiered persistent storage is a significant problem for pipelined data processing. This work considers a class of database applications implemented as sequences of operations that transfer data between persistent storage tiers. We show how to partition the sets of data transfers to reduce the number of conflicts when data transfers are performed in parallel. The paper proposes the new rule-based algorithms for allocating parallel data transfer to the processors to minimize total processing time. The new algorithms evenly distribute the workload among the processors and reduce their idle times. We describe a number of experiments that validate the efficiency of parallel data transfer plans generated by the algorithms presented in the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Blazewicz, J., Ecker, K.H., Pesh, E., Schmidt, G., Sterna, M., Weglarz, J.: Handbook on Scheduling From theory to Practice, 2nd edn. Springer, Cham (2019)

    Google Scholar 

  2. Data Storage Trends in 2020 and Beyond. https://www.spiceworks.com/marketing/-reports/storage-trends-in-2020-and-beyond/. Accessed 30 April 2021

  3. Frachtenberg, E., Feitelson, D.G., Petrini, F., Fernandez, J.: Adaptive parallel job scheduling with flexible coscheduling. In: IEEE TPDS, vol. 16, no. 11, pp. 1066–1077 (2005)

    Google Scholar 

  4. Li, J., Naughton, J.F., Nehme, R.V.: Resource bricolage and resource selection for parallel database systems. VLDB J. 26(1), 31–54 (2016). https://doi.org/10.1007/s00778-016-0435-4

    Article  Google Scholar 

  5. Nehme, R., Bruno, N.: Automated partitioning design in parallel database systems. In: SIGMOD, Association for Computing Machinery, New York, NY, USA, pp. 1137–1148 (2011). https://doi.org/10.1145/1989323.1989444

  6. Noon, N.N., Getta, J.R.: Automated performance tuning of data management systems with materializations and indices. J. Comput. Commun. 4, 46–52 (2016). https://doi.org/10.4236/jcc.2016.45007

    Article  Google Scholar 

  7. Noon, N.N., Getta, J.R.: Optimisation of query processing with multilevel storage. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, T.-P. (eds.) ACIIDS 2016. LNCS (LNAI), vol. 9622, pp. 691–700. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49390-8_67

    Chapter  Google Scholar 

  8. Noon, N.N., Getta, J.R., Xia, T.: Optimization query processing for multi-tiered persistent storage. In: 2021 IEEE 4th International Conference on Computer and Communication Engineering Technology (CCET), pp. 131–135 (2021). https://doi.org/10.1109/CCET52649.2021.9544285

  9. Stöhr, T., Märtens, H., Rahm, E.: Multi-dimensional database allocation for parallel data warehouses. In: Proceedings of the 26th International Conference on Very Large Databases, pp. 273–284 (2000)

    Google Scholar 

  10. Tiered Storage. https://searchstorage.techtarget.com/definition/tiered-storage Accessed 30 April 2021

  11. Wang, K., Choi, S.H., Qin, H., Huang, Y.: A cluster-based scheduling model using SPT and SA for dynamic hybrid flow shop problems. Int. J. Adv. Manuf. Technol. 67(9), 2243–2258 (2013). https://doi.org/10.1007/s00170-012-4645-7

    Article  Google Scholar 

  12. Zhang, Y., Franke, H., Moreira, J., Sivasubramaniam, A.: An integrated approach to parallel scheduling using gang-scheduling, backfilling, and migration. IEEE Trans. Parallel Distrib. Syst. 14(3), 23–247 (2003)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing and Information Technology, University of Wollongong, Wollongong, Australia

    Nan Noon Noon, Janusz R. Getta & Tianbing Xia

Authors
  1. Nan Noon Noon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janusz R. Getta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianbing Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nan Noon Noon .

Editor information

Editors and Affiliations

  1. University of Newcastle Australia, Newcastle, NSW, Australia

    Edward Szczerbicki

  2. Wrocław University of Science and Technology, Wrocław, Poland

    Krystian Wojtkiewicz

  3. International University - VNU-HCM, Ho Chi Minh City, Vietnam

    Sinh Van Nguyen

  4. Wrocław University of Science and Technology, Wrocław, Poland

    Marcin Pietranik

  5. Wrocław University of Science and Technology, Wrocław, Poland

    Marek Krótkiewicz

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Noon, N.N., Getta, J.R., Xia, T. (2022). Scheduling Parallel Data Transfers in Multi-tiered Persistent Storage. In: Szczerbicki, E., Wojtkiewicz, K., Nguyen, S.V., Pietranik, M., Krótkiewicz, M. (eds) Recent Challenges in Intelligent Information and Database Systems. ACIIDS 2022. Communications in Computer and Information Science, vol 1716. Springer, Singapore. https://doi.org/10.1007/978-981-19-8234-7_34

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-8234-7_34

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-8233-0

  • Online ISBN: 978-981-19-8234-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation