Aggeliki Koutsibella
ABSTRACT
Although online versions of several well known clustering algorithms have been proposed, in order to deal effectively with the big data issue, as well as with the case where the data are available in a streaming fashion, very few of them follow the stochastic gradient descent philosophy. In this paper a novel stochastic gradient descent possibilistic clustering algorithm, called O-PCM2 is introduced. The algorithm is presented in detail and a convergence proof of it is provided, based on the general convergence results established for the family of the stochastic gradient descent algorithms. Finally, the effectiveness of the proposed algorithm is assessed through extensive experimentation on both simulated and real data sets and in comparison with other related algorithms.
- [1] Ball G.H., Hall D.J.: A clustering technique for summarizing multivariate data. Behavioral Science, 12, 153–155 (1967).Google Scholar
Cross Ref
- [2] Lloyd S.P.: Least squares quantization in PCM. IEEE Transactions on Information Theory 28(2), 129–137 (1982).Google ScholarDigital Library
- [3] Duda R.O., Hart P.E., Stork D.: Pattern Classification. John Wiley & Sons (2001).Google Scholar
- [4] Bezdek J.C.: A convergence theorem for the fuzzy Isodata clustering algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 2(1), 1–8 (1980).Google ScholarDigital Library
- [5] Hathaway R.J., Bezdek J.C.: Local convergence of the fuzzy c-means algorithms. Pattern Recognition 19(6), 477–480 (1986).Google ScholarCross Ref
- [6] Krishnapuram R., Keller J.M.: A possibilistic approach to clustering. IEEE Transactions on Fuzzy Systems, 1, 98-110 (1993)Google ScholarDigital Library
- [7] Krishnapuram R., Keller J.M.: The possibilistic c-means algorithm: insights and recommendations. IEEE Transactions on Fuzzy Systems, 4, 385-393 (1996)Google ScholarDigital Library
- [8] Vidal R.: Subspace clustering. IEEE Signal Processing Magazine, 28(2), 52-68 (2011).Google ScholarCross Ref
- [9] Barbakh W., Fyfe C.: Online clustering algorithms. International Journal of Neural Systems, 18, 185–194 (2008).Google ScholarCross Ref
- [10] Hore P., Hall L.O., Goldgof D.B., Cheng W.: Online fuzzy c-means. NAFIPS2008 (2008).Google ScholarCross Ref
- [11] Xenaki S.D., Koutroumbas K.D., Rontogiannis A.A.: A Novel Online Generalized Possibilistic Clustering Algorithm for Big Data Processing. Proc. of the 26th European Signal Processing Conference (EUSIPCO), Rome, Sept. 2018 (2018).Google ScholarCross Ref
- [12] Xenaki S.D., Koutroumbas K.D., Rontogiannis A.A.: A Novel Adaptive Possibilistic Clustering Algorithm. IEEE Transactions on Fuzzy Systems. 24(4), 791-810 (2016).Google Scholar
- [13] Yang M.-S, Wu K.-L.: Unsupervised possibilistic clustering. Pattern Recognition 39, 5–21 (2006).Google ScholarDigital Library
- [14] Bottou L., Curtis F.E., Nocedal J.: Optimization methods for largescale machine learning. Siam Review 60(2), 223-311 (2018).Google ScholarCross Ref
- [15] Theodoridis S., Koutroumbas K.D.: Pattern Recognition. Academic Press (2008).Google Scholar
- Stochastic gradient descent possibilistic clustering
Recommendations
Runtime Analysis of Stochastic Gradient Descent
CSAE '20: Proceedings of the 4th International Conference on Computer Science and Application EngineeringStochastic gradient descent (SGD) is one of the most famous methods for minimization. However, there are few results about the runtime analysis of SGD because of its randomness. In this paper, we explain how to approximate SGD by stochastic differential ...
A new stochastic gradient descent possibilistic clustering algorithm
Highlights of AI Research in EuropeSeveral well known clustering algorithms have their own online counterparts, in order to deal effectively with the big data issue, as well as with the case where the data become available in a streaming fashion. However, very few of them follow the ...
Asynchronous parallel stochastic gradient descent: a numeric core for scalable distributed machine learning algorithms
MLHPC '15: Proceedings of the Workshop on Machine Learning in High-Performance Computing EnvironmentsThe implementation of a vast majority of machine learning (ML) algorithms boils down to solving a numerical optimization problem. In this context, Stochastic Gradient Descent (SGD) methods have long proven to provide good results, both in terms of ...
Comments