Abstract
Data warehouses integrate external data sources (EDSs), which very often change their data structures (schemas). In many cases, such changes cause an erroneous execution of an already deployed ETL workflow. Structural changes of EDSs are frequent, therefore an automatic reparation of an ETL workflow, after such changes, is of a high importance. This paper presents a framework for handling the evolution of an ETL layer – E − ETL. Detection of changes in EDSs causes a reparation of the fragment of ETL workflow which interacts with the changed EDS. The proposed framework was developed as a module external to an ETL engine, accessing the engine by means of API. The innovation of this framework are algorithms for semi-automatic reparation of an ETL workflow.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Eder, J., Koncilia, C., Morzy, T.: The COMET Metamodel for Temporal Data Warehouses. In: Pidduck, A.B., Mylopoulos, J., Woo, C.C., Ozsu, M.T. (eds.) CAiSE 2002. LNCS, vol. 2348, pp. 83–99. Springer, Heidelberg (2002)
Papastefanatos, G., Vassiliadis, P., Simitsis, A., Sellis, T., Vassiliou, Y.: Rule-Based Management of Schema Changes at ETL Sources. In: Grundspenkis, J., Kirikova, M., Manolopoulos, Y., Novickis, L. (eds.) ADBIS 2009. LNCS, vol. 5968, pp. 55–62. Springer, Heidelberg (2010)
Papastefanatos, G., Vassiliadis, P., Simitsis, A., Vassiliou, Y.: Policy-Regulated Management of ETL Evolution. J. Data Semantics, 147–177 (2009)
Rundensteiner, E.A., Koeller, A., Zhang, X.: Maintaining data warehouses over changing information sources. Communications of the ACM 43(6), 57–62 (2000)
Rundensteiner, E.A., Koeller, A., Zhang, X., Lee, A.J., Nica, A., Van Wyk, A., Lee, Y.: Evolvable View Environment (EVE): Non-Equivalent View Maintenance under Schema Changes. In: Proc. of ACM Int. Conf. on Management of Data, SIGMOD, pp. 553–555. ACM Press (1999)
Wojciechowski, A.: E-ETL: Framework For Managing Evolving ETL Processes. In: Proc. of Ph.D. Students in Information and Knowledge Management Workshop (PIKM), pp. 59–66. ACM Press (2011)
Wojciechowski, A., Wrembel, R.: Research Problems of the ETL Technology. Foundations of Computing and Decision Sciences 35(5), 283–306 (2010)
Wrembel, R.: On handling the evolution of external data sources in a data warehouse architecture. In: Taniar, D., Chen, L. (eds.) Data Mining and Database Technologies: Innovative Approaches. IGI Group (2011)
Wrembel, R., Bębel, B.: The Framework for Detecting and Propagating Changes from Data Sources Structure into a Data Warehouse. Foundations of Computing & Decision Sciences 30(4), 361–372 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wojciechowski, A. (2013). E-ETL: Framework For Managing Evolving ETL Processes. In: Pechenizkiy, M., Wojciechowski, M. (eds) New Trends in Databases and Information Systems. Advances in Intelligent Systems and Computing, vol 185. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32518-2_42
Download citation
DOI: https://doi.org/10.1007/978-3-642-32518-2_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32517-5
Online ISBN: 978-3-642-32518-2
eBook Packages: EngineeringEngineering (R0)