Skip to main content
SpringerLink
Log in

An Adaptive Algorithm for Mining Association Rules on Shared-Memory Parallel Machines

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

Mining association rules from large databases is very costly. We propose to develop parallel algorithms for this task on shared-memory multiprocessor (SMP). All proposed parallel algorithms for other paradigms follow the conventional level-wise approach: they need as many iterations as the length of the maximum large itemset. To make matter worse, they impose a synchronization in every iteration which would cause serious I/O contention on shared-memory parallel system. An adaptive asynchronous parallel mining algorithm APM has been proposed for SMP. All processors generate candidates dynamically and count itemset supports independently without synchronization. Two optimization techniques have been proposed for the reduction of database scanning and the number of candidates. The algorithm APM has been implemented on a Sun Enterprise 4000 shared-memory multiprocessor with 12 nodes. The experiments show that the optimizations have very good effects and APM has a substantial lead in performance over other proposed level-wise algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. R. Agrawal, T. Imielinski, and A. Swami, “Mining association rules between sets of items in large databases,” in Proc. of 1993 ACM-SIGMOD Int. Conf. On Management of Data, Washington, D.C., 1993, pp. 207–216.

  2. R. Agrawal and R. Srikant, “Fast algorithms for mining association rules,” in Proc. of the 20th VLDB Conference, Santiago, Chile, 1994, pp. 487–499.

  3. R. Agrawal and J.C. Shafer, “Parallel mining of association rules: Design, implementation and experience,” special issue in Data Mining, IEEE Trans. on Knowledge and Data Engineering, IEEE Computer Society, vol.8, no.6, pp. 962–969, Dec. 1996.

    Google Scholar 

  4. S. Brin, R. Motwani, J. Ullman, and S. Tsur, “Dynamic itemset counting and implication rules for market basket data,” in Proc. of 1997 ACM-SIGMOD Int. Conf. On Management of Data, Tucson, Arizona, 1997, pp. 255–264.

  5. D.W. Cheung, J. Han, V.T. Ng, A.W. Fu, and Y. Fu, “A fast distributed algorithm for mining association rules,” in Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, Miami Beach, Florida, December, 1996, pp. 31–43.

  6. J. Han and Y. Fu, “Discovery of multiple-level association rules from large databases,” in Proc. of the 21th VLDB Conference, Zurich, Switzerland, 1995, pp. 420–431.

  7. E. Han, G. Karypis, and V. Kumar, “Scalable parallel data mining for association rules,” in Proc. of 1997 ACM-SIGMOD Int. Conf. On Management of Data, Tucson, Arizona, 1997, pp. 277–288.

  8. M.A.W. Houtsma and A.N. Swami, “Set-oriented mining for association rules in relational databases,” in Proc. of the 11th Int. Conf. on Data Eng., Taipei, Taiwan, 1995, pp. 25–33.

  9. T. Kohonen, J. Hynninen, J. Kangas, and J. Laaksonen, “The self-organizing map program package, version 3.1, SOM programming team of the Helsinki university of technology laboratory of computer and information science, 1995.

  10. J.B. MacQueen, “Some methods for classification and analysis of multivariate observations,” in Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281–297.

  11. H. Mannila, H. Toivonen, and A.I. Verkamo, “Efficient algorithms for discovering association rules,” in KDD-94: AAAI Workshop on Knowledge Discovery in Databases, July 1994.

  12. J.S. Park, M.S. Chen, and P.S. Yu, “An effective hash-based algorithm for mining association rules,” in Proc. of 1995 ACM-SIGMOD Int. Conf. on Management of Data, San Jose, CA, May 1995, pp. 175–186.

  13. J.S. Park, M.S. Chen, and P.S. Yu, “Efficient parallel mining for association rules,” in Proc. of the 4th Int. Conf. on Information and Knowledge Management, Baltimore, Maryland, 1995.

  14. A. Savasere, E. Omiecinski, and S. Navathe, “An efficient algorithm for mining association rules in large databases,” in Proc. of the 21th VLDB Conference, Zurich, Switzerland, 1995, pp. 432–444.

  15. T. Shintani and M. Kitsuregawa, “Hash based parallel algorithms for mining association rules,” in Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, Miami Beach, Florida, 1996.

  16. H. Toivonen, “Sampling large databases for association rules,” in Proc. 1996 Int. Conf. Very Large Data Bases, Bombay, India, Sept. 1996, pp. 134–145.

  17. M.J. Zaki, M. Ogihara, S. Parthasarathy, and W. Li, “Parallel data mining for association rules on sharedmemory multi-processors,” Supercomputing'96, Pittsburg, PA, Nov 17- 22, 1996.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Systems, The University of Hong Kong, Hong Kong

    David W. Cheung

  2. Department of Automation, Tsinghua University, Beijing

    Kan Hu

  3. Department of Automation, Tsinghua University, Beijing

    Shaowei Xia

Authors
  1. David W. Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaowei Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheung, D.W., Hu, K. & Xia, S. An Adaptive Algorithm for Mining Association Rules on Shared-Memory Parallel Machines. Distributed and Parallel Databases 9, 99–132 (2001). https://doi.org/10.1023/A:1018951022124

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018951022124

Search

Navigation