Conventional scalable clustering-based Deep Neural Network (DNN) algorithms cluster linearly separable data, however non-linearly separable data in the feature space is harder to cluster. This paper proposes a novel Scalable Deep Neural Network Kernelized Literal Fuzzy C-Means (SDnnKLFCM) and Scalable Deep Neural Network Kernelized Random Sampling Iterative Optimization Fuzzy C-Means for Big Data (SDnnKRSIO-FCM). These kernelized clustering methods solve non-linear separable issues by non-linearly transforming the input data space into a high-dimensional feature space using a Cauchy Kernel Function (CKF). We create kernelized deep neural network fuzzy clustering methods using Apache Spark in-memory cluster computing technique to efficiently cluster Big Data on High-Performance Computing (HPC) machine. To demonstrate the effectiveness of the proposed (SDnnKLFCM) and (SDnnKRSIO-FCM) in comparison to previous scalable deep neural network clustering methods, extensive tests are carried out on a variety of large datasets. The reported experimental results show that the kernelized non-linear deep clustering algorithms in comparison with linear fuzzy clustering algorithms achieve significant improvement in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score, respectively.