Abstract
Deep learning models often require a significant amount of data, which can be computationally intensive and architecturally complex. Efforts to address the challenge of handling large amounts of data in high-resolution scenarios have led to the development of techniques like data pruning and data diet approaches. We present a novel approach called Select Base on Intra-Class Similarity (SICS), distinguishes itself by measuring the similarity of samples within the same class and identifies the most informative samples that are most dissimilar from others, and introducing the novel concept of a distinctive-variant sample, vital for enhancing deep-learning classification tasks. We evaluated our method on several image classification benchmarks and compared it with existing techniques. Our results show that in high-resolution images and many class scenarios, SICS can achieve the same level of accuracy as the full data while using only about 80% of the training data, outperforming the ForgettingScore method by 20% to 90%. Additionally, our method maintains its robustness when switching to different training models. Our source code is publicly available at https://github.com/Gusicun/SICS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abramowitz, M., Stegun, I.A., Romer, R.H.: Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables (1988)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Castellani, A., Schmitt, S., Hammer, B.: Stream-based active learning with verification latency in non-stationary environments. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds.) Artificial Neural Networks and Machine Learning–ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol. 13532, pp. 260–272. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-15937-4_22
Coleman, C., et al.: Selection via proxy: efficient data selection for deep learning. arXiv preprint arXiv:1906.11829 (2019)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. In: International Conference on Machine Learning, pp. 1885–1894. PMLR (2017)
Manwar, A., Mahalle, H.S., Chinchkhede, K., Chavan, V.: A vector space model for information retrieval: a Matlab approach. Indian J. Comput. Sci. Eng. 3(2), 222–229 (2012)
Nosofsky, R.M.: Attention, similarity, and the identification-categorization relationship. J. Exp. Psychol. Gen. 115(1), 39 (1986)
Paul, M., Ganguli, S., Dziugaite, G.K.: Deep learning on a data diet: finding important examples early in training. Adv. Neural. Inf. Process. Syst. 34, 20596–20607 (2021)
Settles, B.: Active learning literature survey (2009)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Toneva, M., Sordoni, A., Combes, R.T., Trischler, A., Bengio, Y., Gordon, G.J.: An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 (2018)
Zhu, C., Chen, W., Peng, T., Wang, Y., Jin, M.: Hard sample aware noise robust learning for histopathology image classification. IEEE Trans. Med. Imaging 41(4), 881–894 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Diao, H., Liu, Z., Zhang, F., Huang, J., Zhou, F., U. Khan, S. (2023). Selecting Distinctive-Variant Training Samples Base on Intra-class Similarity. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14262. Springer, Cham. https://doi.org/10.1007/978-3-031-44201-8_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-44201-8_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44200-1
Online ISBN: 978-3-031-44201-8
eBook Packages: Computer ScienceComputer Science (R0)