HPC Based Scalable Logarithmic Kernelized Fuzzy Clustering Algorithms for Handling Big Data

Abstract

With the advent of big data in the new millennium, the previous scalable clustering methods were no longer able to match the accuracy and efficiency requirements of big clustering. In light of this, we propose a fuzzy-based scalable incremental kernelized clustering algorithm for Big Data. First, we present the details of scalable kernelized fuzzy clustering techniques for Big Data that are based on the Radial Basis Function (RBF). Next, we define the membership degree and the cluster center for the logarithmic kernel function. For the purpose of managing Big Data, the Logarithmic Kernelized Scalable Random Sampling with Iterative Optimization Fuzzy c-Means (LKSRSIO-FCM) clustering algorithm has been developed on Apache Spark. These kernel functions translate the input data space non-linearly onto a high-dimensional feature space, so, these kernelized clustering approaches have developed in order to deal with non-linearly separable problems. Hence, our aim is to design and implement the logarithmic kernelized incremental fuzzy clustering algorithms on Apache Spark, which, as a result of its in-memory cluster computing methodology, is able to effectively perform the clustering of Big Data. Extensive experiments on a variety of datasets derived from the real world demonstrate that the proposed LKSRSIO-FCM has superior performance to the scalable kernelized fuzzy clustering algorithms based on the RBF kernel in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score, respectively. The results were obtained by comparing the LKSRSIO-FCM to the scalable kernelized fuzzy clustering algorithms.

This work is supported by National Supercomputing Mission, HPC Applications Development Funded Research Project by DST in collaboration with the Ministry of Electronics and Information Technology (MeiTY), Govt. of India.