Abstract
Analyzing neural network (NN) generalization is vital for ensuring effective performance on new, unseen data, beyond the training set. Traditional methods involve evaluating NN across multiple testing datasets, a resource-intensive process involving data acquisition, preprocessing, and labeling. The primary challenge is determining the optimal capacity for training observations, requiring adaptable adjustments based on the task and available data information. This paper leverages Algebraic Topology and relevance measures to investigate NN behavior during learning. We define NN on a topological space as a functional topology graph and compute topological summaries to estimate generalization gaps. Simultaneously, we assess the relevance of NN units, progressively pruning network units. The generalization gap estimation helps identify overfitting, enabling timely early-stopping decisions and identifying the architecture with optimal generalization. This approach offers a comprehensive insight into NN generalization and supports the exploration of NN extensibility and interpretability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ballester, R., Clemente, X.A., Casacuberta, C., Madadi, M., Corneanu, C.A., Escalera, S.: Towards explaining the generalization gap in neural networks using topological data analysis (2022). https://doi.org/10.48550/ARXIV.2203.12330, https://arxiv.org/abs/2203.12330
Chazal, F., Michel, B.: An introduction to topological data analysis: fundamental and practical aspects for data scientists (2017). https://doi.org/10.48550/ARXIV.1710.04019, https://arxiv.org/abs/1710.04019
Cohen-Steiner, D., Edelsbrunner, H., Harer, J., Mileyko, Y.: Lipschitz functions have l p-stable persistence. Found. Comput. Math. 10(2), 127–139 (2010)
Corneanu, C., Madadi, M., Escalera, S., Martinez, A.: Computing the testing error without a testing set (2020). https://doi.org/10.48550/ARXIV.2005.00450, https://arxiv.org/abs/2005.00450
Deng, W., Zheng, L.: Are labels always necessary for classifier accuracy evaluation? In: Proceedings of the CVPR (2021)
Dua, D., Graff, C.: UCI machine learning repository (2017). https://archive.ics.uci.edu/ml
Hensel, F., Moor, M., Rieck, B.: A survey of topological machine learning methods. Front. Artif. Intell. 4, 681108 (2021). https://doi.org/10.3389/frai.2021.681108
Jiang, Y., et al.: NeurIPS 2020 competition: predicting generalization in deep learning. arXiv preprint arXiv:2012.07976 (2020)
Lassance, C., Béthune, L., Bontonou, M., Hamidouche, M., Gripon, V.: Ranking deep learning generalization using label variation in latent geometry graphs. CoRR abs/2011.12737 (2020), https://arxiv.org/abs/2011.12737
Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theor. Probab. Appl. 16(2), 264–280 (1971)
Watanabe, S., Yamana, H.: Topological measurement of deep neural networks using persistent homology. CoRR abs/2106.03016 (2021), https://arxiv.org/abs/2106.03016
Yacoub, M., Bennani, Y.: HVS: a heuristic for variable selection in multilayer artificial neural network classifier. In: Dagli, C., Akay, M., Ersoy, O., Fernandez, B., Smith, A. (eds.) Intelligent Engineering Systems Through Artificial Neural Networks, vol. 7, pp. 527–532 (1997)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Barbara, A., Bennani, Y., Karkazan, J. (2024). On the Use of Persistent Homology to Control the Generalization Capacity of a Neural Network. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1962. Springer, Singapore. https://doi.org/10.1007/978-981-99-8132-8_21
Download citation
DOI: https://doi.org/10.1007/978-981-99-8132-8_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8131-1
Online ISBN: 978-981-99-8132-8
eBook Packages: Computer ScienceComputer Science (R0)