Erich Schubert
Jörg Sander
Martin Ester
Skip Abstract Section
Abstract
At SIGMOD 2015, an article was presented with the title “DBSCAN Revisited: Mis-Claim, Un-Fixability, and Approximation” that won the conference’s best paper award. In this technical correspondence, we want to point out some inaccuracies in the way DBSCAN was represented, and why the criticism should have been directed at the assumption about the performance of spatial index structures such as R-trees and not at an algorithm that can use such indexes. We will also discuss the relationship of DBSCAN performance and the indexability of the dataset, and discuss some heuristics for choosing appropriate DBSCAN parameters. Some indicators of bad parameters will be proposed to help guide future users of this algorithm in choosing parameters such as to obtain both meaningful results and good performance. In new experiments, we show that the new SIGMOD 2015 methods do not appear to offer practical benefits if the DBSCAN parameters are well chosen and thus they are primarily of theoretical interest. In conclusion, the original DBSCAN algorithm with effective indexes and reasonably chosen parameter values performs competitively compared to the method proposed by Gan and Tao.
- Pankaj K. Agarwal, Herbert Edelsbrunner, Otfried Schwarzkopf, and Emo Welzl. 1991. Euclidean minimum spanning trees and bichromatic closest pairs. Discrete 8 Computational Geometry 6, 1 (1991), 407--422. DOI:http://dx.doi.org/10.1007/BF02574698 Google Scholar
Digital Library
- Gennady L. Andrienko, Natalia V. Andrienko, Christophe Hurter, Salvatore Rinzivillo, and Stefan Wrobel. 2011. From movement tracks through events to places: Extracting and characterizing significant places from mobility data. In Proceedings of the 2011 IEEE Symposium on Visual Analytics Science and Technology (VAST). 161--170. DOI:http://dx.doi.org/10.1109/VAST.2011.6102454 Google ScholarCross Ref
- Mihael Ankerst, Markus M. Breunig, Hans-Peter Kriegel, and Jörg Sander. 1999. OPTICS: Ordering points to identify the clustering structure. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 49--60. DOI:http://dx.doi.org/10.1145/304182.304187 Google ScholarDigital Library
- Lars Arge, Mark de Berg, Herman J. Haverkort, and Ke Yi. 2004. The priority R-tree: A practically efficient and worst-case optimal R-tree. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 347--358. DOI:http://dx.doi.org/10.1145/1007568.1007608 Google ScholarDigital Library
- Sunil Arya and David M. Mount. 1993. Approximate nearest neighbor queries in fixed dimensions. In Proceedings of the 4th Annual ACM/SIGACT-SIAM Symposium on Discrete Algorithms (SODA). 271--280.Google Scholar
- Norbert Beckmann, Hans-Peter Kriegel, Ralf Schneider, and Bernhard Seeger. 1990. The R*-tree: An efficient and robust access method for points and rectangles. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 322--331. DOI:http://dx.doi.org/10.1145/93597.98741 Google ScholarDigital Library
- Jon Louis Bentley. 1975. Multidimensional binary search trees used for associative searching. Communications of the ACM 18, 9 (1975), 509--517. DOI:http://dx.doi.org/10.1145/361002.361007 Google ScholarDigital Library
- Stefan Berchtold, Christian Böhm, Bernhard Braunmüller, Daniel A. Keim, and Hans-Peter Kriegel. 1997. Fast parallel similarity search in multimedia databases. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 1--12. DOI:http://dx.doi.org/10.1145/253260.253263 Google ScholarDigital Library
- Kevin S. Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. 1999. When is “nearest neighbor” meaningful? In Proceedings of the 7th International Conference on Database Theory (ICDT). 217--235. DOI:http://dx.doi.org/10.1007/3-540-49257-7_15 Google ScholarCross Ref
- Alina Beygelzimer, Sham Kakade, and John Langford. 2006. Cover trees for nearest neighbors. In Proceedings of the 23rd International Conference on Machine Learning (ICML). 97--104. DOI:http://dx.doi.org/10.1145/1143844.1143857 Google ScholarDigital Library
- Ergun Biçici and Deniz Yuret. 2007. Locally scaled density based clustering. In Proceedings of the 8th International Conference on Adaptive and Natural Computing Algorithms (ICANNGA) 2007. 739--748. DOI:http://dx.doi.org/10.1007/978-3-540-71618-1_82 Google ScholarDigital Library
- Thomas Brinkhoff, Hans-Peter Kriegel, Ralf Schneider, and Bernhard Seeger. 1994. Multi-step processing of spatial joins. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 197--208. DOI:http://dx.doi.org/10.1145/191839.191880 Google ScholarDigital Library
- Ricardo J. G. B. Campello, Davoud Moulavi, and Jörg Sander. 2013. Density-based clustering based on hierarchical density estimates. In Proceedings of the 17th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD). 160--172. DOI:http://dx.doi.org/10.1007/978-3-642-37456-2_14 Google ScholarCross Ref
- Michel Marie Deza and Elena Deza. 2009. Encyclopedia of Distances (3rd ed.). Springer. Google ScholarCross Ref
- Jeff Erickson. 1995. On the relative complexities of some geometric problems. In Proceedings of the 7th Canadian Conference on Computational Geometry. 85--90.Google Scholar
- Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd ACM International Conference on Knowledge Discovery and Data Mining (KDD). 226--231.Google Scholar
- Christos Faloutsos and Ibrahim Kamel. 1994. Beyond uniformity and independence: Analysis of R-trees using the concept of fractal dimension. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 4--13. DOI:http://dx.doi.org/10.1145/182591.182593 Google ScholarDigital Library
- Junhao Gan and Yufei Tao. 2015. DBSCAN revisited: Mis-claim, un-fixability, and approximation. In Proceedings of the ACM International Conference on Management of Data (SIGMOD). 519--530. DOI:http://dx.doi.org/10.1145/2723372.2737792 Google ScholarDigital Library
- Ade Gunawan. 2013. A faster algorithm for DBSCAN. Master’s thesis. Technical University of Eindhoven. http://repository.tue.nl/760643.Google Scholar
- Mark A. Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H. Witten. 2009. The WEKA data mining software: An update. ACM SIGKDD Explorations 11, 1 (2009), 10--18. DOI:http://dx.doi.org/10.1145/1656274.1656278 Google ScholarDigital Library
- Jiawei Han, Micheline Kamber, and Jian Pei. 2011. Data Mining: Concepts and Techniques (3rd ed.). Morgan Kaufmann.Google ScholarDigital Library
- Joseph M. Hellerstein, Elias Koutsoupias, and Christos H. Papadimitriou. 1997. On the analysis of indexing schemes. In Proceedings of the 16th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. 249--256. DOI:http://dx.doi.org/10.1145/263661.263688 Google ScholarDigital Library
- Kirsten Hildrum, John Kubiatowicz, Sean Ma, and Satish Rao. 2004. A note on the nearest neighbor in growth-restricted metrics. In Proceedings of the 15th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 560--561.Google Scholar
- Alexander Hinneburg and Daniel A. Keim. 1998. An efficient approach to clustering in large multimedia databases with noise. In Proceedings of the 4th ACM International Conference on Knowledge Discovery and Data Mining (KDD). 58--65.Google Scholar
- Michael E. Houle, Hans-Peter Kriegel, Peer Kröger, Erich Schubert, and Arthur Zimek. 2010. Can shared-neighbor distances defeat the curse of dimensionality? In Proceedings of the 22nd International Conference on Scientific and Statistical Database Management (SSDBM). 482--500. DOI:http://dx.doi.org/10.1007/978-3-642-13818-8_34 Google ScholarCross Ref
- Piotr Indyk and Rajeev Motwani. 1998. Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proceedings of the 30th Annual ACM Symposium on Theory of Computing (STOC). 604--613. DOI:http://dx.doi.org/10.1145/276698.276876 Google ScholarDigital Library
- David R. Karger and Matthias Ruhl. 2002. Finding nearest neighbors in growth-restricted metrics. In Proceedings of the 34th Annual ACM Symposium on Theory of Computing (STOC). 741--750. DOI:http://dx.doi.org/10.1145/509907.510013 Google ScholarDigital Library
- Robert Krauthgamer and James R. Lee. 2004. Navigating nets: Simple algorithms for proximity search. In Proceedings of the 15th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 798--807.Google Scholar
- Hans-Peter Kriegel, Erich Schubert, and Arthur Zimek. 2016. The (black) art of runtime evaluation: Are we comparing algorithms or implementations?Knowledge and Information Systems (KAIS) (Oct. 2016). DOI:http://dx.doi.org/10.1007/s10115-016-1004-2 Google ScholarDigital Library
- Moshe Lichman. 2013. UCI Machine Learning Repository. Retrieved from http://archive.ics.uci.edu/ml.Google Scholar
- Shaaban Mahran and Khaled Mahar. 2008. Using grid for accelerating density-based clustering. In Proceedings of 8th IEEE International Conference on Computer and Information Technology (CIT’08). 35--40. DOI:http://dx.doi.org/10.1109/CIT.2008.4594646 Google ScholarCross Ref
- Son T. Mai, Ira Assent, and Martin Storgaard. 2016. AnyDBC: An efficient anytime density-based clustering algorithm for very large complex datasets. In Proceedings of the 22st ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD). 1025--1034. DOI:http://dx.doi.org/10.1145/2939672.2939750 Google ScholarDigital Library
- Yannis Manolopoulos, Alexandros Nanopoulos, Apostolos N. Papadopoulos, and Yannis Theodoridis. 2003. R-trees have Grown Everywhere. Technical Report. Retrieved from http://www.rtreeportal.org/.Google Scholar
- Jirí Matousek. 1991. Reporting points in halfspaces. In Proceedings of the 32nd Annual IEEE Symposium on Foundations of Computer Science (FOCS). 207--215. DOI:http://dx.doi.org/10.1109/SFCS.1991.185370 Google ScholarDigital Library
- Jirí Matousek. 1994. Geometric range searching. Computing Surveys 26, 4 (1994), 421--461. DOI:http://dx.doi.org/10.1145/197405.197408 Google ScholarDigital Library
- Apostolos Papadopoulos and Yannis Manolopoulos. 1997. Performance of nearest neighbor queries in R-trees. In Proceedings of the 6th International Conference on Database Theory (ICDT). 394--408. DOI:http://dx.doi.org/10.1007/3-540-62222-5_59 Google ScholarCross Ref
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine learning in python. Journal of Machine Learning Research 12 (2011), 2825--2830.Google ScholarDigital Library
- Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the 16th International Symposium on Wearable Computers (ISWC’12). 108--109. DOI:http://dx.doi.org/10.1109/ISWC.2012.13 Google ScholarDigital Library
- Vasilis Samoladas and Daniel P. Miranker. 1998. A lower bound theorem for indexing schemes and its application to multidimensional range queries. In Proceedings of the 17th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. 44--51. DOI:http://dx.doi.org/10.1145/275487.275493 Google ScholarDigital Library
- Jörg Sander, Martin Ester, Hans-Peter Kriegel, and Xiaowei Xu. 1998. Density-based clustering in spatial databases: The algorithm GDBSCAN and its applications. Data Mining and Knowledge Discovery 2, 2 (1998), 169--194. DOI:http://dx.doi.org/10.1023/A:1009745219419 Google ScholarDigital Library
- Erich Schubert, Alexander Koos, Tobias Emrich, Andreas Züfle, Klaus Arthur Schmid, and Arthur Zimek. 2015. A framework for clustering uncertain data. Proceedings of the VLDB Endowment 8, 12 (2015), 1976--1979. DOI:http://dx.doi.org/10.14778/2824032.2824115 https://elki-project.github.io/. Google ScholarDigital Library
- Erich Schubert, Arthur Zimek, and Hans-Peter Kriegel. 2013. Geodetic distance queries on r-trees for indexing geographic data. In Proceedings of the 13th International Symposium on Spatial and Temporal Databases (SSTD). 146--164. DOI:http://dx.doi.org/10.1007/978-3-642-40235-7_9 Google ScholarDigital Library
- Pang-Ning Tan, Michael Steinbach, and Vipin Kumar. 2006. Introduction to Data Mining. Addison Wesley.Google Scholar
- R Core Team. 2015. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Retrieved from http://www.r-project.org/.Google Scholar
- Wei Wang, Jiong Yang, and Richard R. Muntz. 1997. STING: A statistical information grid approach to spatial data mining. In Proceedings of the 23rd International Conference on Very Large Data Bases (VLDB). 186--195.Google Scholar
- Roger Weber, Hans-Jörg Schek, and Stephen Blott. 1998. A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In Proceedings of the 24th International Conference on Very Large Data Bases (VLDB). 194--205.Google Scholar
- Xiaowei Xu, Nurcan Yuruk, Zhidan Feng, and Thomas A. J. Schweiger. 2007. SCAN: A structural clustering algorithm for networks. In Proceedings of the 13th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD). 824--833. DOI:http://dx.doi.org/10.1145/1281192.1281280 Google ScholarDigital Library
- Arthur Zimek, Erich Schubert, and Hans-Peter Kriegel. 2012. A survey on unsupervised outlier detection in high-dimensional numerical data. Statistical Analysis and Data Mining 5, 5 (2012), 363--387. DOI:http://dx.doi.org/10.1002/sam.11161 Google ScholarDigital Library
Index Terms
- DBSCAN Revisited, Revisited: Why and How You Should (Still) Use DBSCAN
Recommendations
AA-DBSCAN: an approximate adaptive DBSCAN for finding clusters with varying densities
Clustering is a typical data mining technique that partitions a dataset into multiple subsets of similar objects according to similarity metrics. In particular, density-based algorithms can find clusters of different shapes and sizes while remaining ...
A new hybrid method based on partitioning-based DBSCAN and ant clustering
Clustering problem is an unsupervised learning problem. It is a procedure that partition data objects into matching clusters. The data objects in the same cluster are quite similar to each other and dissimilar in the other clusters. Density-based ...
Exact, Fast and Scalable Parallel DBSCAN for Commodity Platforms
ICDCN '17: Proceedings of the 18th International Conference on Distributed Computing and NetworkingDBSCAN is one of the most popular density-based clustering algorithm capable of identifying arbitrary shaped clusters and noise. It is computationally expensive for large data sets. In this paper, we present a grid-based DBSCAN algorithm, GridDBSCAN, ...
Comments