Abstract
In this paper, we study the problem of mining sequential patterns in a large database of customer transactions. Since finding sequential patterns has to handle a large amount of customer transaction data and requires multiple passes over the database, it is expected that parallel algorithms help to improve the performance significantly. We consider the parallel algorithms for mining sequential patterns on a shared-nothing environment. Three parallel algorithms (Non Partitioned Sequential Pattern Mining(NPSPM), Simply Partitioned Sequential Pattern Mining(SPSPM) and Hash Partitioned Sequential Pattern Mining(HPSPM)) are proposed. In NPSPM, the candidate sequences are just copied among all the nodes, which can lead to memory overflow for large databases. The remaining two algorithms partition the candidate sequences over the nodes, which can efficiently exploit the total system's memory as the number of nodes in increased. If it is partitioned simply, customer transaction data has to be broadcasted to all nodes. HPSPM partitions the candidate sequences among the nodes using hash function, which eliminates the customer transaction data broadcasting and reduces the comparison workload. We describe the implementation of these algorithms on a shared-nothing parallel computer IBM SP2 and its performance evaluation results. Among three algorithms HPSPM attains best performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R.Agrawal and R.Srikant. Mining sequential patterns. In Proc. of the 11th Int. Conf. on Data Engineering, pages 3–14, March 1995.
R.Srikant and R.Agrawal. Mining sequential patterns: Generalizations and performance improvements. In Proc. of the 5th Int. Conf. on Extending Database Technology, March 1996.
S.-J.Yen and A.L.P.Chen. An efficient approach to discovering knowledge from large databases. In Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, pages 82-18, December 1996.
T.Shintani and M.Kitsuregawa. Hash based parallel algorithms for mining association rules. In Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, pages 19–30, December 1996.
J.S.Park, M.-S.Chen, and P.S.Yu. Efficient parallel data mining for association rules. In Proc. of the 4th Conf. on Information and Knowledge Management, pages 31–36, November 1995.
D.W.Cheung, J.Han, V.T.Ng, A.W.Fu, and Y.Fu. A fast distributed algorithms for mining association rules. In Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, pages 31–42, December 1996.
R.Agrawal and J.C.Shafer. Parallel mining of association rules. In IEEE Trans. on Knowledge and Data Engineering, Vol.8, No.6, pages 962–969, December 1996.
D.W.Cheung, V.T.Ng, A.W.Fu, and Y.Fu. Efficient mining of association rules in distributed databases. In IEEE Trans. on Knowledge and Data Engineering, Vol.8, No.6, pages 911–922, December 1996.
E.-H.(Sam)Han, G.Kaxypis, and V.Kumar. Scalable parallel data mining for association rules. In Proc. of 1997 ACM SIGMOD Int. Conf. on Management of Data, pages 277–288, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shintani, T., Kitsuregawa, M. (1998). Mining algorithms for sequential patterns in parallel : Hash based approach. In: Wu, X., Kotagiri, R., Korb, K.B. (eds) Research and Development in Knowledge Discovery and Data Mining. PAKDD 1998. Lecture Notes in Computer Science, vol 1394. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-64383-4_24
Download citation
DOI: https://doi.org/10.1007/3-540-64383-4_24
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64383-8
Online ISBN: 978-3-540-69768-8
eBook Packages: Springer Book Archive