Abstract
In this paper, we propose a categorization strategy to handle the incremental nature of data by identifying concepts of drift in the data stream. In the world of digitalization, the total amount of data created, captured, copied, and consumed is increasing rapidly, reaching a few zettabytes. Various fields of data mining and machine learning applications involve clustering as their principal component, considering the non-incremental nature of the data. However, many real-world machine learning algorithms need to adapt to this ever-growing global data sphere to continually learn new patterns. In addition, the model needs to be acquainted with the continuous change in the distribution of the input data. Towards this, we propose a clustering algorithm termed as Progressive Clustering to foresee the phenomenon of increase in data and sustain it until the pattern of the data changes considerably. We demonstrate the results of our clustering algorithm by simulating various instances of the incremental nature of the data in the form of a data stream. We demonstrate the results of our proposed methodology on benchmark MNIST and Fashion-MNIST datasets and evaluate our strategy using appropriate quantitative metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Castro, F.M., Marín-Jiménez, M.J., Mata, N.G., Schmid, C., Karteek, A.: End-to-end incremental learning. arXiv abs/1807.09536 (2018)
Chen, J., Zhang, L., Liang, Y.: Exploiting gaussian mixture model clustering for full-duplex transceiver design. IEEE Trans. Commun. 67(8), 5802–5816 (2019). https://doi.org/10.1109/TCOMM.2019.2915225
Du, X., Charan, G., Liu, F., Cao, Y.: Single-net continual learning with progressive segmented training. In: 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1629–1636 (2019). https://doi.org/10.1109/ICMLA.2019.00267
Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation, August 2017. https://doi.org/10.24963/ijcai.2017/243
Guo, X., Liu, X., Zhu, E., Yin, J.: Deep clustering with convolutional autoencoders. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) Neural Information Processing, ICONIP 2017. LNCS, vol. 10635, pp. 373–382. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70096-0_39
Guo, X., Zhu, E., Liu, X., Yin, J.: Deep embedded clustering with data augmentation. In: Zhu, J., Takeuchi, I. (eds.) Proceedings of The 10th Asian Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 95, pp. 550–565. PMLR, 14–16 November 2018. http://proceedings.mlr.press/v95/guo18b.html
Hoens, T., Polikar, R., Chawla, N.: Learning from streaming data with concept drift and imbalance: an overview. Prog. Artif. Intell. 1(1), 89–101 (2012). https://doi.org/10.1007/s13748-011-0008-0, Copyright: Copyright 2021 Elsevier B.V., All rights reserved
Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114(13), 3521–3526 (2017). https://doi.org/10.1073/pnas.1611835114, https://www.pnas.org/content/114/13/3521
Kuncheva, L.I.: Classifier ensembles for changing environments. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 1–15. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-25966-4_1
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
Li, Y., Wu, H.: A clustering method based on k-means algorithm. Physics Procedia 25, 1104–1109 (2012). https://doi.org/10.1016/j.phpro.2012.03.206
Mallya, A., Davis, D., Lazebnik, S.: Piggyback: adapting a single network to multiple tasks by learning to mask weights (2018)
Mallya, A., Lazebnik, S.: PackNet: adding multiple tasks to a single network by iterative pruning, pp. 7765–7773, June 2018. https://doi.org/10.1109/CVPR.2018.00810
Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: ICARL: incremental classifier and representation learning. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5533–5542 (2017). https://doi.org/10.1109/CVPR.2017.587
Rusu, A.A., et al.: Progressive neural networks (2016)
Tabib, R.A., et al.: Deep features for categorization of heritage images towards 3D reconstruction. Procedia Comput. Sci. 171, 483–490 (2020). https://doi.org/10.1016/j.procs.2020.04.051, https://www.sciencedirect.com/science/article/pii/S1877050920310176, Third International Conference on Computing and Network Communications (CoCoNet 2019)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms (2017)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis (2016)
Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks (2018)
Acknowledgement
This project is partly carried out under Department of Science and Technology (DST) through ICPS programme - Indian Heritage in Digital Space for the project “CrowdSourcing” (DST/ ICPS/ IHDS/ 2018 (General)) and “Digital Poompuhar” (DST/ ICPS/ Digital Poompuhar/ 2017 (General)).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Gunari, A., Kudari, S.V., Tabib, R.A., Mudenagudi, U. (2022). Progressive Clustering: An Unsupervised Approach Towards Continual Knowledge Acquisition of Incremental Data. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13364. Springer, Cham. https://doi.org/10.1007/978-3-031-09282-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-09282-4_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09281-7
Online ISBN: 978-3-031-09282-4
eBook Packages: Computer ScienceComputer Science (R0)