Abstract
In this paper, we propose a new incremental linear discriminant analysis (ILDA) for multitask pattern recognition (MTPR) problems in which a chunk of training data for a particular task are given sequentially and the task is switched to another related task one after another. The Pang et al.’s ILDA is extended such that a discriminant space of the current task is augmented with effective discriminant vectors that are selected from other tasks based on the class separability. We call this selective augmentation of discriminant vectors knowledge transfer of feature space. In the experiments, the proposed ILDA is evaluated for seven MTPR problems, each of which consists of three recognition tasks. The results demonstrate that the proposed ILDA with knowledge transfer outperforms the conventional ILDA and its naive extension to MTPR problems with regard to both class separability and recognition accuracy. We confirm that the proposed knowledge transfer works well to evolve effective feature spaces online in MTPR problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abu-Mostafa YS (1989) Learning from hints in neural networks. J Complex 6:192–198
Argyriou A, Evgeniou T, Pontil M (2007) Multi-task feature learning. In: Schölkopf B et al (eds) Advances in neural information processing systems 19, MIT Press, pp 41–48
Blake C, Keogh E, Merz C (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html
Caruana R (1995) Learning many related tasks at the same time with backpropagation. In: Tesauro G et al (eds) Advances in neural information processing systems 7, MIT Press, pp 657–664
Caruana R (1997) Multitask learning. Mach Learn 28:41–75
Carpenter GA, Grossberg S (1998) The ART of adaptive pattern recognition by a self-organizing neural network. IEEE Comput 21:77–88
Cauwenberghs G, Poggio T (2000) Incremental and decremental support vector machine learning. In: Leen T et al (eds) Advances in neural information processing systems 13, MIT Press
Kasabov N (2001) Evolving fuzzy neural networks for supervised/unsupervised on-line knowledge-based learning. IEEE Trans Syst Man Cybern B 31:902–918
Kasabov N (2002) Evolving connectionist systems: methods and applications in bioinformatics, brain study and intelligent machines. Springer, London
Okada T and Tomita S (1985) An optimal orthonormal system for discriminant analysis. Pattern Recognit 18:139–144
Ozawa S, Toh SL, Abe S, Pang S, Kasabov N (2005) Incremental learning of feature space and classifier for face recognition. Neural Netw 18:575–584
Ozawa S, Pang S, Kasabov N (2008) Incremental learning of chunk data for on-line pattern classification systems. IEEE Trans on Neural Netw 19:430–445
Ozawa S, Roy A, Roussinov D (2009) A multitask learning model for online pattern recognition. IEEE Trans Neural Netw 20:1061–1074
Pang S, Ozawa S, Kasabov N (2005) Incremental linear discriminant analysis for classification of data streams. IEEE Trans Syst Man Cybern B 35:905–914
Platt J (1991) A resource allocating network for function interpolation. Neural Comput 3:213–225
Roy A (2003) Neural networks: how do we make a widely used technology out of it? IEEE Neural Netw Soc Newsl 1:8–12
Silver D and Mercer R (2001) Selective functional transfer: inductive bias from related tasks. In: Proc IASTED Int Conf on Artif Intell and Soft Comput, pp 182–189
Takeuchi Y, Ozawa S, Abe S (2007) An efficient incremental kernel principal component analysis for online feature selection. In: Proc Int Joint Conf on Neural Netwo 2007, pp 1603–1608
Thrun S, Pratt L (1998) Learning to learn, Kluwer, Boston
Xiang C, Fan X, Lee T (2006) Face recognition using recursive fisher linear discriminant. IEEE Trans on Image Process 15:2097–2105
Zheng W (2006) Class-incremental generalized discriminant analysis. Neural Comput 18:979–1006
Acknowledgments
The authors would like to thank Professor Shigeo Abe for his helpful comments and discussions. This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Scientific Research (C) 205002205.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hisada, M., Ozawa, S., Zhang, K. et al. Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems. Evolving Systems 1, 17–27 (2010). https://doi.org/10.1007/s12530-010-9000-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-010-9000-3