Abstract
Catastrophic forgetting arises when a neural network is not capable of preserving the past learned task when learning a new task. There are already some methods proposed to mitigate this problem in artificial neural networks. In this paper we propose to improve upon our previous state-of-the-art method, SeNA-CNN, such as to enable the automatic recognition in test time of the task to be solved and we experimentally show that it has excellent results. The experiments show the learning of up to 4 different tasks with a single network, without forgetting how to solve previous learned tasks.
This work was supported by National Founding from the FCT- Fundação para a Ciência e a Tecnologia, through the UID/EEA/50008/2013 Project. The GTX Titan X used in this research was donated by the NVIDIA Corporation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, Software available from tensorflow.org
Acemoglu, D., Cao, D., Acemoglu, D., Cao, D.: MIT and CIFAR (2010)
Ahmed, K., Baig, M.H., Torresani, L.: Network of experts for large-scale image categorization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 516–532. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_32
Chollet, F., et al.: Keras (2015)
Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: 2004 Conference on Computer Vision and Pattern Recognition Workshop, p . 178, June 2004. https://doi.org/10.1109/CVPR.2004.109
Li, Z., Hoiem, D.: Learning without forgetting. CoRR abs/1606.09282 (2016). http://arxiv.org/abs/1606.09282
Liu, B.: Lifelong machine learning: a paradigm for continuous learning. Front. Comput. Sci. 1–3 (2016). https://doi.org/10.1007/s11704-016-6903-6
Mante, V., Sussillo, D., Shenoy, K.V., Newsome, W.T.: Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503(7474), 78–84 (2013). https://doi.org/10.1038/nature12742. http://europepmc.org/articles/PMC4121670
Mountcastle, V.B.: Modality and topographic properties of single neurons of cat’s somatic sensory cortex. J. Neurophysiol. 20(4), 408–434 (1957). https://doi.org/10.1152/jn.1957.20.4.408, pMID: 13439410
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)
Rafegas, I., Vanrell, M., Alexandre, L.A.: Understanding trained CNNs by indexing neuron selectivity. CoRR abs/1702.00382 (2017). http://arxiv.org/abs/1702.00382
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497
Shin, H., Lee, J.K., Kim, J., Kim, J.: Continual learning with deep generative replay. CoRR abs/1705.08690 (2017). http://arxiv.org/abs/1705.08690
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Technical report
Zacarias, A., Alexandre, L.A.: SeNA-CNN: overcoming catastrophic forgetting in convolutional neural networks by selective network augmentation. In: Pancioni, L., Schwenker, F., Trentin, E. (eds.) ANNPR 2018. LNCS (LNAI), vol. 11081, pp. 102–112. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99978-4_8
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zacarias, A., Alexandre, L.A. (2018). Improving SeNA-CNN by Automating Task Recognition. In: Yin, H., Camacho, D., Novais, P., Tallón-Ballesteros, A. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2018. IDEAL 2018. Lecture Notes in Computer Science(), vol 11314. Springer, Cham. https://doi.org/10.1007/978-3-030-03493-1_74
Download citation
DOI: https://doi.org/10.1007/978-3-030-03493-1_74
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03492-4
Online ISBN: 978-3-030-03493-1
eBook Packages: Computer ScienceComputer Science (R0)