Loading [a11y]/accessibility-menu.js
Modular Data Analytics | IEEE Journals & Magazine | IEEE Xplore

Modular Data Analytics


Abstract:

Relational online analytical processing (ROLAP) often involves complex queries with many intermediate steps, which can be expensive to run on large datasets. Modularizing...Show More

Abstract:

Relational online analytical processing (ROLAP) often involves complex queries with many intermediate steps, which can be expensive to run on large datasets. Modularizing SQL queries improves readability, testability, and incremental execution. While SQL common table expressions (CTEs) and views help in modularization, they have availability, performance, or development limitations. To address these, simple-rolap, an open-source framework, automates dependency analysis and orchestrates the execution of modular queries using GNU make. It uses a main database containing the infrequently modified base data and a secondary database for caching intermediate results. Simple-rolap ensures efficient, maintainable ROLAP queries by splitting complex queries into simple ones that create intermediate tables or report results, which can be unit-tested. It uses the make tool to manage dependencies and execution order, enhancing workflow control, testing, visualization, and troubleshooting.
Published in: IEEE Software ( Volume: 41, Issue: 5, Sept.-Oct. 2024)
Page(s): 20 - 23
Date of Publication: 07 August 2024

ISSN Information:


References

References is not available for this document.