Abstract
In this paper, a novel type of radial basis function network is proposed for multitask pattern recognition. We assume that recognition tasks are switched sequentially without notice to a learner and they have relatedness to some extent. We further assume that training data are given to learn one by one and they are discarded after learning. To learn a recognition system incrementally in such a multitask environment, we propose Resource Allocating Network for Multi-Task Pattern Recognition (RAN-MTPR). There are five distinguished functions in RAN-MTPR: one-pass incremental learning, task change detection, task categorization, knowledge consolidation, and knowledge transfer. The first three functions enable RAN-MTPR not only to acquire and accumulate knowledge of tasks stably but also to allocate classes to appropriate tasks unless task labels are not explicitly given. The fourth function enables RAN-MTPR to recover the failure in task categorization by minimizing the conflict in class allocation to tasks. The fifth function, knowledge transfer from one task to another, is realized by sharing the internal representation of a hidden layer with different tasks and by transferring class information of the most related task to a new task. The experimental results show that the recognition performance of RAN-MTPR is enhanced by introducing the two types of knowledge transfer and the consolidation works well to reduce the failure in task change detection and task categorization if the RBF width is properly set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Attar V, Sinha P, Wankhade K (2010) A fast and light classifier for data streams. Evol Syst 1: 199–207
Ban SW, Lee M (2007) Autonomous incremental visual environment perception based on visual selective attention. Proceedings of IEEE/INNS International Joint Conference on Neural Network. IJCNN, Budapest pp 1411–1416
Baxter J (1997) A Bayesian/information theoretic model of learning to learn via multiple task sampling. Mach Learn 28: 7–39
Carpenter GA, Grossberg S (1998) The ART of adaptive pattern recognition by a self-organizing neural network. IEEE Comput 21: 77–88
Caruana R (1997) Multitask learning. Mach Learn 28: 41–75
Caruana R, O’Sullivan J (1998) Multitask pattern recognition for autonomous robots. Proc IEEE/RSJ Int Conf on Intell Robotic Syst 1: 13–18
Chen S, He H (2011) Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach. Evol Syst 2: 35–50
Ghosn J, Bengio Y (2003) Bias learning, knowledge sharing. IEEE Trans on Neural Netw 14: 748–765
Hisada M, Ozawa S, Zhang K, Kasabov N (2010) Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems. Evol Syst 1: 17–27
Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans on Neural Netw 16: 57–67
Iglesias JA, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agents’ behaviors: a general approach. Evol Syst 1: 161–171
Jin F, Sun S (2008) A multitask learning approach to face recognition based on neural networks. In: Intelligent data engineering and automated Learning. Springer, Berlin, pp 24–31
Kasabov N (2002) Evolving connectionist systems: methods and applications in bioinformatics, brain study and intelligent machines. Springer, London
Okamoto K, Ozawa S, Abe S (2003) A fast incremental learning algorithm of RBF networks with long-term memory. Proceedings of IEEE/INNS International Joint Conference on Neural Network. IJCNN, Budapest, pp 102–107
Ozawa S, Toh SL, Abe S, Pang S, Kasabov N (2005) Incremental learning of feature space and classifier for face recognition. Neural Netw 18: 575–584
Ozawa S, Pang S, Kasabov N (2008) Incremental learning of chunk data for on-line pattern classification systems. IEEE Trans on Neural Netw 19: 1061–1074
Ozawa S, Roy A, Roussinov D (2009) A multitask learning model for online pattern recognition. IEEE Trans on Neural Netw 20: 430–445
Pang S, Ozawa S, Kasabov N (2005) Incremental linear discriminant analysis for classification of data streams. IEEE Trans on Syst Man and Cyber B 35: 905–914
Platt J (1991) A resource allocating network for function interpolation. Neural Comput 3: 213–225
Pratt L and Jennings B (1998) A survey of connectionist network reuse through transfer. In: Learning to learn. Kluwer Academic Publications, Boston, pp 19-43
Puchala E (2004) A Bayes algorithm for the multitask pattern recognition problem? direct and decomposed independent approaches. In: Computational science and its applications. LNCS 3046, springeer, Heidelberg, pp 39–45
Rubio JJ, Vàzquez DM, Pacheco J (2010) Backpropagation to train an evolving radial basis function neural network. Evol Syst 1: 173–180
Silver D and Mercer R (2001) Selective functional transfer: inductive bias from related tasks. Proceedings of IASTED International Conference on Artificial Intelligence and Soft Computing, pp 182–189
Thrun S, Pratt L (1998) Learning to learn. Kluwer Academic Publications, Boston
Tsumori K, Ozawa S (2003) Incremental learning in dynamic environments using neural network with long-term memory. Proceedings IEEE/INNS International Joint Conference on Neural Network. IJCNN, Budapest, pp 2583–2588
Blake C, Keogh E, Merz C (1998) UCI repository of machine learning databases. www.ics.uci.edu/mlearn/MLRepository.html
Yamauchi K, Yamaguchi N, Ishii N (1999) Incremental learning methods with retrieving of interfered patterns. IEEE Trans on Neural Netw 10: 1351–1365
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nishikawa, H., Ozawa, S. Radial Basis Function Network for Multitask Pattern Recognition. Neural Process Lett 33, 283–299 (2011). https://doi.org/10.1007/s11063-011-9178-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-011-9178-9