Ibrahim Louhi
ABSTRACT
In this paper we propose to process big data using a data streams approach. The data set is divided into subsets, each subsets is considered as a time window from a data stream. Our approach uses a neighborhood-based clustering. Instead of processing each new element one by one, we propose to process each group of new elements simultaneously. A clustering is applied on each new group using neighborhood graphs. The obtained clusters are then used to incrementally construct a representative graph of the data. The data graph is visualized in real time with specific visualizations that reflect the processing algorithm. To validate the approach, we apply it to different data streams and we compare it with known data stream clustering approaches.
- C. C. Aggarwal, J. Han, J. Wang, and P. S. Yu. A framework for clustering evolving data streams. In Proceedings of the 29th international conference on Very large data bases-Volume 29, pages 81--92. VLDB Endowment, 2003. Google Scholar
Digital Library
- C. C. Aggarwal and K. Subbian. Event detection in social streams. In SDM, volume 12, pages 624--635. SIAM, 2012.Google Scholar
- P. Berkhin. A survey of clustering data mining techniques. In Grouping multidimensional data, pages 25--71. Springer, 2006.Google ScholarCross Ref
- F. Cao, M. Ester, W. Qian, and A. Zhou. Density-based clustering over an evolving data stream with noise. In Proceedings of 2006 SIAM Conference on Data Mining, volume 6, pages 328--339. SIAM, 2006.Google ScholarCross Ref
- M. Chen, S. Mao, and Y. Liu. Big data: a survey. Mobile Networks and Applications, 19(2):171--209, 2014. Google ScholarDigital Library
- Y. Chen and L. Tu. Density-based clustering for real-time stream data. In Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133--142. ACM, 2007. Google ScholarDigital Library
- B. Desgraupes. Clustering indices. University of Paris Ouest-Lab Modal'X, pages 1--34, 2013.Google Scholar
- E. W. Forgy. Cluster analysis of multivariate data: efficiency versus interpretability of classifications. Biometrics, 21:768--769, 1965.Google Scholar
- B. Fritzke et al. A growing neural gas network learns topologies. Advances in neural information processing systems, 7:625--632, 1995. Google ScholarDigital Library
- K. R. Gabriel and R. R. Sokal. A new statistical approach to geographic variation analysis. Systematic Biology, 18(3):259--278, 1969.Google Scholar
- J. Gama. Knowledge discovery from data streams. CRC Press, 2010. Google ScholarDigital Library
- M. Ghesmoune, M. Lebbah, and H. Azzag. Clustering over data streams based on growing neural gas. pages 134--145, 2015.Google Scholar
- S. Guha, N. Mishra, R. Motwani, and L. O'Callaghan. Clustering data streams. In Proceedings of the 41st Annual Symposium on Foundations of Computer Science, FOCS '00, pages 359--366, Washington, DC, USA, 2000. IEEE Computer Society. Google ScholarDigital Library
- S. Guha, R. Rastogi, and K. Shim. Rock: A robust clustering algorithm for categorical attributes. In Data Engineering, 1999. Proceedings., 15th International Conference on, pages 512--521. IEEE, 1999. Google ScholarDigital Library
- G. Karypis, E.-H. Han, and V. Kumar. Chameleon: Hierarchical clustering using dynamic modeling. Computer, 32(8):68--75, 1999. Google ScholarDigital Library
- P. Kranen, I. Assent, C. Baldauf, and T. Seidl. Self-adaptive anytime stream clustering. In Data Mining, 2009. ICDM'09. Ninth IEEE International Conference on, pages 249--258. IEEE, 2009. Google ScholarDigital Library
- J. Lavergne, H. Azzag, C. Guinot, and G. Venturini. Incremental construction of neighborhood graphs using the ants self-assembly behavior. In Tools with Artificial Intelligence, 2007. ICTAI 2007. 19th IEEE International Conference on, volume 1, pages 399--406. IEEE, 2007. Google ScholarDigital Library
- M. Lichman(b). Uci machine learning repository, 2013.Google Scholar
- I. Louhi, L. Boudjeloud-Assala, and T. Tamisier. Incremental nearest neighborhood graph for data stream clustering. In International Joint Conference on Neural Networks, 2016. 2016 IEEE International Conference on, volume 1. IEEE, 2016.Google ScholarCross Ref
- S. Lühr and M. Lazarescu. Incremental clustering of dynamic data streams using connectivity based representative points. Data & Knowledge Engineering, 68(1):1--27, 2009. Google ScholarDigital Library
- J. MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281--297. Oakland, CA, USA., 1967.Google Scholar
- A. Y. Ng, M. I. Jordan, Y. Weiss, et al. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 2:849--856, 2002. Google ScholarDigital Library
- S. Nittel, K. T. Leung, and A. Braverman. Scaling clustering algorithms for massive data sets using data streams. In ICDE, volume 4, page 830, 2004. Google ScholarDigital Library
- M.-Y. Shih, J.-W. Jheng, and L.-F. Lai. A two-step method for clustering mixed categroical and numeric data. Tamkang Journal of Science and Engineering, 13(1):11--19, 2010.Google Scholar
- J. A. Silva, E. R. Faria, R. C. Barros, E. R. Hruschka, A. C. de Carvalho, and J. Gama. Data stream clustering: A survey. ACM Computing Surveys (CSUR), 46(1):13, 2013. Google ScholarDigital Library
- T. M. Thuy, H. A. Le Thi, and L. Boudjeloud-Assala. An efficient clustering method for massive dataset based on dc programming and dca approach. In International Conference on Neural Information Processing, pages 538--545. Springer, 2013. Google ScholarDigital Library
- G. T. Toussaint. The relative neighbourhood graph of a finite planar set. Pattern recognition, 12(4):261--268, 1980.Google ScholarCross Ref
- G. T. Toussaint. Some unsolved problems on proximity graphs. 1991.Google Scholar
- A. C.-C. Yao. On constructing minimum spanning trees in k-dimensional spaces and related problems. SIAM Journal on Computing, 11(4):721--736, 1982.Google ScholarDigital Library
Recommendations
A Framework for Clustering Massive-Domain Data Streams
ICDE '09: Proceedings of the 2009 IEEE International Conference on Data EngineeringIn this paper, we will examine the problem of clustering massive domain data streams. Massive-domain data streams are those in which the number of possible domain values for each attribute are very large and cannot be easily tracked for clustering ...
Mining big data streams using business analytics tools: a bird's eye view on MOA and SAMOA
Big data evolves as the prominent field in modern computing era. Big data analytics and its impact on extracting business intelligence is becoming indispensable for plethora of applications. The non-proprietary software revolution paved the way for ...
Requirements for clustering data streams
Scientific and industrial examples of data streams abound in astronomy, telecommunication operations, banking and stock-market applications, e-commerce and other fields. A challenge imposed by continuously arriving data streams is to analyze them and to ...
Comments