Abstract
Features in a deep neural network are only as robust as those present in the data provided for training. The robustness of features applies to not just the types of features and how they apply to various classes, known or unknown, but also to how those features apply to different octaves, or scales. Neural Networks trained at one octave have been shown to be invariant to other octaves, while neural networks trained on large robust datasets operate optimally at only the octaves that resonate best with the learned features. This may still discard features that existed in the data. Not knowing the octave a trained neural network is most applicable to can lead to sub-optimal results during prediction due to poor preprocessing. Recent work has shown good results in quantifying how the learned features in a neural network apply to objects. In this work, we follow up on work in feature applicability, using it to quantify which octaves the features in a trained neural network resonate best with.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Basu, S., Ganguly, S., Mukhopadhyay, S., DiBiano, R., Karki, M., Nemani, R.: Deepsat: a learning framework for satellite imagery. In: Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems, p. 37. ACM (2015)
Basu, S., et al.: Learning sparse feature representations using probabilistic quadtrees and deep belief nets. Neural Process. Lett. 45(3), 855–867 (2017)
Basu, S., et al.: Deep neural networks for texture classification – a theoretical analysis. Neural Netw. 97, 173–182 (2018)
Bhattacharya, U., Chaudhuri, B.B.: Handwritten numeral databases of indian scripts and multistage recognition of mixed numerals. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 444–457 (2008)
Bhattacharya, U., Shridhar, M., Parui, S.K., Sen, P., Chaudhuri, B.: Offline recognition of handwritten bangla characters: an efficient two-stage approach. Pattern Anal. Appl. 15(4), 445–458 (2012)
Collier, E., DiBiano, R., Mukhopadhyay, S.: Cactusnets: layer applicability as a metric for transfer learning. arXiv preprint arXiv:1804.07846 (2018)
Collier, E., et al.: Progressively growing generative adversarial networks for high resolution semantic segmentation of satellite images. In: Tong, H., Li, Z.J., Zhu, F., Yu, J. (eds.) 2018 IEEE International Conference on Data Mining Workshops, ICDM Workshops, Singapore, 17–20 November 2018, pp. 763–769. IEEE (2018)
Glorot, X., Bordes, A., Bengio, Y.: Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning (ICML-2011), pp. 513–520 (2011)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Liu, Q., et al.: Deepsat v2: feature augmented convolutional neural nets for satellite image classification. Remote Sens. Lett. 11(2), 156–165 (2020)
Liu, Q., Collier, E., Mukhopadhyay, S.: PCGAN-CHAR: progressively trained classifier generative adversarial networks for classification of noisy handwritten bangla characters. In: Jatowt, A., Maeda, A., Syn, S.Y. (eds.) ICADL 2019. LNCS, vol. 11853, pp. 3–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34058-2_1
Liu, Q., Mukhopadhyay, S.: Unsupervised learning using pretrained CNN and associative memory bank. In: 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, 8–13 July 2018, pp. 1–8. IEEE (2018)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)
Zhu, Y., et al.: Heterogeneous transfer learning for image classification. In: AAAI (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Collier, E., DiBiano, R., Mukhopadhyay, S. (2020). Using Applicability to Quantifying Octave Resonance in Deep Neural Networks. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Communications in Computer and Information Science, vol 1333. Springer, Cham. https://doi.org/10.1007/978-3-030-63823-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-63823-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63822-1
Online ISBN: 978-3-030-63823-8
eBook Packages: Computer ScienceComputer Science (R0)