Abstract
The notion of similarity is an important one in data mining. It can be used to pro vide useful structural information on data as well as enable clustering. In this paper we present an elegant method for measuring the similarity between homogeneous datasets. The algorithm presented is efficient in storage and scale, has the ability to adjust to time constraints. and can provide the user with likely causes of similarity or dis-similarity.
One potential application of our similarity measure is in the distributed data mining domain. Using the notion of similarity across databases as a distance metric one cangenerate clusters of similar datasets. Once similar datasets are clustered, each cluster can be independently mined to generate the appropriate rules for a given cluster. The similarity measure is evaluated on a dataset from the Census Bureau, and synthetic datasets from IBM.
This work was supported in part by NSF grants CDA-9401142, CCR-9702466, CCR-9701911, CCR-9725021, INT-9726724, and CCR-9705594; and an external research grant from Digital Equipment Corporation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
C. Aggarwal and P. Yu. Online generation of association rules. In ICDE’98.
R. Agrawal, C. Faloutsos, and A. Swami. Efficient similarity search in sequence databases. In Foundations of Data Organization and Algorithms, 1993.
R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and A. Inkeri Verkamo. Fast discovery of association rules. In U. Fayyad and et al, editors, Advances in Knowledge Discovery and Data Mining, pages 307–328. AAAI Press, Menlo Park, CA, 1996.
R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In 20th VLDB Conf., 1994.
G. Das, H. Mannila, and P. Ronkainen. Similarity of attributes by external probes. In KDD 1998.
U. M. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. The KDD process of rextracing useful information from volumes of data. Communications of ACM, 39(11):27–34, 1996.
R. Goldman, N. Shivakumar, V. Suresh, and H. Garcia-Molina. Proximity search in databases. In VLDB Conf., 1998.
H. Jagadish, A. Mendelzon, and T. Milo. Similarity based queries. In PODS, 1995.
R. Subramonian. Defining diff as a data mining primitive. In KDD 1998.
H. Toivonen. Sampling large databases for association rules. In VLDB Conf., 1996.
M. J. Zaki, S. Parthasarathy, M. Ogihara, and W. Li. New algorithms for fast discovery of association rules. In KDD, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Parthasarathy, S., Ogihara, M. (2000). Exploiting Dataset Similarity for Distributed Mining. In: Rolim, J. (eds) Parallel and Distributed Processing. IPDPS 2000. Lecture Notes in Computer Science, vol 1800. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45591-4_52
Download citation
DOI: https://doi.org/10.1007/3-540-45591-4_52
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67442-9
Online ISBN: 978-3-540-45591-2
eBook Packages: Springer Book Archive