Wojciech Jaskowski
Krzysztof Krawiec
ABSTRACT
We propose a method of knowledge reuse for an ensemble of genetic programming-based learners solving a visual learning task. First, we introduce a visual learning method that uses genetic programming individuals to represent hypotheses. Individuals-hypotheses process image representation composed of visual primitives derived from the training images that contain objects to be recognized. The process of recognition is generative, i.e., an individual is supposed to restore the shape of the processed object by drawing its reproduction on a separate canvas. This canonical method is extended with a knowledge reuse mechanism that allows a learner to import genetic material from hypotheses that evolved for the other decision classes (object classes). We compare the performance of the extended approach to the basic method on a real-world tasks of handwritten character recognition, and conclude that knowledge reuse leads to significant convergence speedup and, more importantly, significantly reduces the risk of overfitting.
- B. Bhanu, Y. Lin, and K. Krawiec. Evolutionary Synthesis of Pattern Recognition Systems. Springer--Verlag, New York, 2005. Google Scholar
Digital Library
- R. Caruana. Multitask learning. Mach. Learn., 28(1):41--75, 1997. Google ScholarDigital Library
- E. Galvan Lopez, R. Poli, and C. A. Coello Coello. Reusing code in genetic programming. In M. K. et al., editor, Genetic Programming 7th European Conference, EuroGP 2004, Proceedings, volume 3003 of LNCS, pages 359--368. Springer-Verlag, 2004.Google Scholar
- D. Howard, S. C. Roberts, and C. Ryan. Pragmatic genetic programming strategy for the problem of vehicle detection in airborne reconnaissance. Pattern Recognition Letters, 27(11):1275--1288, 2006. Google ScholarDigital Library
- W. H. Hsu, S. J. Harmon, E. Rodriguez, and C. Zhong. Empirical comparison of incremental reuse strategies in genetic programming for keep--away soccer. In M. Keijzer, editor, Late Breaking Papers at the 2004 Genetic and Evolutionary Computation Conference, Seattle, Washington, USA, 26 July 2004.Google Scholar
- W. Jaskowski. Genetic programming with cross-task knowledge sharing for learning of visual concepts. Master's thesis, Poznan University of Technology, Poznan, Poland, 2006.Google Scholar
- W. Jaskowski, K. Krawiec, and B. Wieloch. Genetic programming for cross-task knowledge sharing. In Genetic and Evolutionary Computation Conference GECCO, 2007. Google ScholarDigital Library
- W. Jaskowski, K. Krawiec, and B. Wieloch. Learning and recognition of hand-drawn shapes using generative genetic programming. In M. G. et al., editor, EvoWorkshops 2007, volume 4448 of LNCS, pages 281--290, Berlin Heidelberg, 2007. Springer-Verlag. Google ScholarDigital Library
- J. Koza. Genetic Programming. MIT Press, Cambridge, MA, 1992.Google Scholar
- J. Koza. Genetic programming -- 2. MIT Press, Cambridge, MA, 1994.Google Scholar
- J. R. Koza, F. H. Bennett III, D. Andre, and M. A. Keane. Reuse, parameterized reuse, and hierarchical reuse of substructures in evolving electrical circuits using genetic programming. In T. H. et al., editor, Proceedings of International Conference on Evolvable Systems: From Biology to Hardware (ICES-96), volume 1259 of Lecture Notes in Computer Science. Springer-Verlag, 1996. Google ScholarDigital Library
- K. Krawiec. Evolutionary learning of primitive-based visual concepts. In Proc. IEEE Congress on Evolutionary Computation, Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada July 16--21, pages 4451--4458, 2006.Google ScholarCross Ref
- K. Krawiec. Learning high-level visual concepts using attributed primitives and genetic programming. In F. R., editor, EvoWorkshops 2006, LNCS 3907, pages 515--519, Berlin Heidelberg, 2006. Springer--Verlag. Google ScholarDigital Library
- K. Krawiec and B. Bhanu. Visual learning by coevolutionary feature synthesis. IEEE Transactions on System, Man, and Cybernetics -- Part B, 35(3):409--425, June 2005. Google ScholarDigital Library
- B. Krishnapuram, C. M. Bishop, and M. Szummer. Generative models and bayesian model comparison for shape recognition. In IWFHR '04: Proceedings of the Ninth International Workshop on Frontiers in Handwriting Recognition (IWFHR'04), pages 20--25, Washington, DC, USA, 2004. IEEE Computer Society. Google ScholarDigital Library
- S. Louis. Case injected genetic algorithms for learning across problems. Engineering Optimization, 36:237--247(11), apr 2004.Google ScholarCross Ref
- S. Louis and J. McDonnell. Learning with case-injected genetic algorithms. Evolutionary Computation, IEEE Transactions on, 8(4):316--328, 2004. Google ScholarDigital Library
- S. J. Louis. Genetic learning from experience. In Proceedings of the International Congress on Evolutionary Computation, Canberra, Australia, 2003. IEEE Press.Google ScholarCross Ref
- S. Luke. ECJ evolutionary computation system, 2002. (http://cs.gmu.edu/ eclab/projects/ecj/).Google Scholar
- M. Maloof, P. Langley, T. Binford, R. Nevatia, and S. Sage. Improved rooftop detection in aerial images with machine learning. Mach. Learn., 53:157--191, 2003. Google ScholarDigital Library
- T. M. Mitchell. The discipline of machine learning. Technical Report CMU-ML-06-108, Machine Learning Department, Carnegie Mellon University, July 2006.Google Scholar
- K. M. W. Moya, M. R. and L. D. Hostetler. One-class classifier networks for target recognition applications. In Proceedings world congress on neural networks, pages 797--801, Portland, OR, 1993. International Neural Network Society.Google Scholar
- M. Revow, C. K. I. Williams, and G. E. Hinton. Using generative models for handwritten digit recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6):592--606, 1996. Google ScholarDigital Library
- M. Rizki, M. Zmuda, and L. Tamburino. Evolving pattern recognition systems. IEEE Transactions on Evolutionary Computation, 6:594--609, 2002. Google ScholarDigital Library
- S. C. Roberts, D. Howard, and J. R. Koza. Evolving modules in genetic programming by subtree encapsulation. In J. F. M. et al., editor, Genetic Programming, Proceedings of EuroGP'2001, volume 2038 of LNCS, pages 160--175. Springer-Verlag, 2001. Google ScholarDigital Library
- A. Teller and M. Veloso. PADO: A new learning architecture for object recognition. In K. Ikeuchi and M. Veloso, editors, Symbolic Visual Learning, pages 77--112. Oxford Press, New York, 1997. Google ScholarDigital Library
- B. Wieloch. Genetic programming with knowledge modularization for learning of visual concepts. Master's thesis, Poznan University of Technology, Poznan, Poland, 2006.Google Scholar
Index Terms
- Knowledge reuse in genetic programming applied to visual learning
Recommendations
Cross-task code reuse in genetic programming applied to visual learning
Selected Problems of Biomedical Engineering (special section, pp. 7 - 63), Marek Kowal and Józef Korbicz (Eds.)AbstractWe propose a method that enables effective code reuse between evolutionary runs that solve a set of related visual learning tasks. We start with introducing a visual learning approach that uses genetic programming individuals to recognize ...
A Comparison of three evolutionary strategies for multiobjective genetic programming
We report what we believe to be the first comparative study of multi-objective genetic programming (GP) algorithms on benchmark symbolic regression and machine learning problems. We compare the Strength Pareto Evolutionary Algorithm (SPEA2), the Non-...
Neural network crossover in genetic algorithms using genetic programming
AbstractThe use of genetic algorithms (GAs) to evolve neural network (NN) weights has risen in popularity in recent years, particularly when used together with gradient descent as a mutation operator. However, crossover operators are often omitted from ...
Comments