A Transactional Approach to Parallel Data Warehouse Maintenance

Abstract

Data Warehousing is becoming an increasingly important technology for information integration and data analysis.Given the dynamic nature of modern distributed environments, both source data and schema changes are likely to occur autonomously and even concurrently in different sources.We have thus developed a comprehensive solution approach, called TxnWrap,that successfully maintains the warehouse views under any type of concurrent source updates.In this work, we now overcome TxnWrap’s restriction that the maintenance is processed one by one for each source update, since that limits the performance. To overcome this limitation, we exploit the transactional approach of TxnWrap to achieve parallel data warehouse maintenance. For this, we first identify the read/write conflicts among the different warehouse maintenance processes. We then propose a parallel maintenance scheduler (PMS)that generates legal schedules that resolve these conflicts.PMS has been implemented and incorporated into our TxnWrap system.The experimental results confirm that our parallel maintenance scheduler significantly improves the performance of data warehouse maintenance.

This work was supported in part by several grants from NSF, namely, the NSF NYI grant #IRI 97.96264, the NSF CISE Instrumentation grant #IRIS 97.29878, and the NSF grant #IIS 9988776.