Skip to main content
SpringerLink
Log in

IOGA: An instance-oriented genetic algorithm

  • Modifications and Extensions of Evolutionary Algorithms Further Modifications and Extensionds
  • Conference paper
  • First Online:
Parallel Problem Solving from Nature — PPSN IV (PPSN 1996)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1141))

Included in the following conference series:

  • 147 Accesses

Abstract

Instance-based methods of classification are easy to implement, easy to explain and relatively robust. Furthermore, they have often been found in empirical studies to be competitive in accuracy with more sophisticated classification techniques (Aha et al., 1991; Weiss & Kulikowski, 1991; Fogarty, 1992; Michie et al., 1994). However, a twofold drawback of the simplest instance-based classification method (1-NNC) is that it requires the storage of all training instances and the use of all attributes or features on which those instances are measured — thus failing to exhibit the cognitive economy which is the hallmark of successful learning (Wolff, 1991). Previous researchers have proposed ways of adapting the basic 1-NNC algorithm either to select only a subset of training cases (‘prototypes’) or to discard redundant and/or ‘noisy’ attributes, but not to do both at once. The present paper describes a program (IOGA) that uses an evolutionary algorithm to select prototypical cases and relevant attributes simultaneously, and evaluates it empirically by application to a set of test problems from a variety of fields. These trials show that very considerable economization of storage can be achieved, coupled with a modest gain in accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Ackley, D.H. (1987). An Empirical Study of Bit Vector Function Optimization. In: L. Davis, ed., Genetic Algorithms & Simulated Annealing. Pitman, London.

    Google Scholar 

  • Afifi, A.A. & Azen, S.P. (1979). Statistical Analysis: a Computer Oriented Approach, 2nd. edition, Academic Press, New York.

    Google Scholar 

  • Aha, D.W., Kibler, D. & Albert, M.K. (1991). Instance-Based Learning Algorithms. Machine Learning, 6, 37–66.

    Google Scholar 

  • Allaway, S.L., Ritchie, C.D., Robinson, D. & Smolski, O.R. (1988). Detection of Alcohol-Induced Fatty Liver by Computerized Tomography. J. Royal Soc. Medicine, 81, 149–151.

    Google Scholar 

  • Althoff, K-D., Auriol, E., Barletta, R. & Manago, M. (1995). A Review of Industrial Case-Based Reasoning Tools. AI Intelligence, Oxford.

    Google Scholar 

  • Anderson, E. (1935). The Irises of the Gaspe Peninsula. Bulletin of the American Iris Society, 59, 2–5.

    Google Scholar 

  • Andrews, D.F. & Herzberg, A.M. (1985). Data: a Collection of Problems from Many Fields for the Student and Research Worker. Springer-Verlag, New York.

    Google Scholar 

  • Batchelor, B.G. (1974). Practical Approaches to Pattern Classification. Plenum Press, London.

    Google Scholar 

  • Batchelor, B.G. (1978) ed. Pattern Recognition: Ideas in Practice. Plenum Press, N.Y.

    Google Scholar 

  • Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and Regression Trees. Wadsworth, Monterey, California.

    Google Scholar 

  • Chang, C.L. (1974). Finding Prototypes for Nearest Neighbour Classifiers. IEEE Trans. on Computers, C-23(11), 1179–1184.

    Google Scholar 

  • Darwin, C.R. & Wallace, A.R. (1858). On the Tendency of Species to Form Varieties; and on the Perpetuation of Varieties and Species by Natural Means of Selection. Paper presented to the London Linnean Society, 1st July 1858. In: D.C. Porter & P.W.

    Google Scholar 

  • Graham (1993). The Portable Darwin. Penguin, London, 86–104.

    Google Scholar 

  • Dasarathy, B.V. (1991) ed. Nearest Neighbour (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos, California.

    Google Scholar 

  • Davis, L. (1991) ed. Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York.

    Google Scholar 

  • Devijer, P.A. & Kittler, J. (1982). Pattern Recognition: a Statistical Approach. Prentice-Hall, New Jersey.

    Google Scholar 

  • Fisher, R.A. (1936). The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7, 179–188.

    Google Scholar 

  • Fix, E. & Hodges, J.L. (1951). Discriminatory Analysis — Nonparametric Discrimination: Consistency Properties. Project 21-49-004, Report No. 4, USAF School of Aviation Medicine, Randolf Field, Texas, 261–279.

    Google Scholar 

  • Flury, B. & Riedwyl, H. (1988). Multivariate Statistics: a Practical Approach. Chapman & Hall, London.

    Google Scholar 

  • Fogarty, T.C. (1992). First Nearest Neighbor Classification on Frey & Slate's Letter Recognition Problem. Machine Learning, 9, 387–388.

    Google Scholar 

  • Forsyth, R.S. (1989) ed. Machine Learning: Principles & Techniques. Chapman & Hall, London.

    Google Scholar 

  • Forsyth, R.S. (1990). Neural Learning Algorithms: Some Empirical Trials. Proc. 3rd International Conf. on Neural Networks & their Applications, Neuro-Nimes-90. EC2, Nanterre.

    Google Scholar 

  • Forsyth, R.S. (1995). Stylistic Structures: a Computational Approach to Text Classification. Doctoral Thesis, University of Nottingham.

    Google Scholar 

  • Fukunaga, K. & Mantock, J.M. (1984). Nonparametric Data Reduction. IEEE Trans. on Pattern Analysis & Machine Intelligence, PAMI-6(1), 115–118.

    Google Scholar 

  • Gabor, G. (1975). The eta-NN Method: a Sequential Feature Selection for Nearest Neighbour Decision Rule. In: I. Csiszar & P. Elias, eds., Topics in Information Theory. North-Holland, Amsterdam.

    Google Scholar 

  • Geva, S. & Sitte, J. (1991). Adaptive Nearest Neighbor Pattern Classification. IEEE Trans. on Neural Networks, NN-2(2), 318–322.

    Article  Google Scholar 

  • Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization & Machine Learning. Addison-Wesley, Reading, Mass.

    Google Scholar 

  • Hand, D.J. & Batchelor, B.G. (1978). An Edited Nearest Neighbour Rule. Information Sciences, 14, 171–180.

    Article  Google Scholar 

  • Hart, P.E. (1968). The Condensed Nearest Neighbour Rule. IEEE Trans. on Info. Theory, IT-14(3), 515–516.

    Article  Google Scholar 

  • Holland, J.H. (1975). Adaptation in Natural & Artficial Systems. Univ. Michigan Press, Ann Arbor.

    Google Scholar 

  • James, M. (1985). Classification Algorithms. Collins, London.

    Google Scholar 

  • Kelly, J.D. & Davis, L. (1991). Hybridizing the Genetic Algorithm and the K Nearest Neighbors Classification Algorithm. In: R.K. Belew & L.B. Booker, eds., Proc. Fourth Internat. Conf. on Genetic Algorithms. Morgan-Kaufmann, San Mateo, California, 377–383.

    Google Scholar 

  • Kohonen, T. (1988). Self-Organization & Associative Memory, 2nd. edition. Springer-Verlag, Berlin.

    Google Scholar 

  • Kolodner, J.L. (1993). Case-Based Reasoning. Morgan Kaufmann, California.

    Google Scholar 

  • Manly, B.F.J. (1994). Multivariate Statistical Methods: a Primer. Chapman & Hall, London.

    Google Scholar 

  • McKenzie, D.P. & Forsyth, R.S. (1995). Classification by Similarity: An Overview of Statistical Methods of Case-Based Reasoning. Computers in Human Behavior, 11(2), 273–288.

    Article  Google Scholar 

  • McLachlan, G. (1992). Discriminant Analysis and Statistical Pattern Recognition. Wiley, New York.

    Google Scholar 

  • Michie, D., Spiegelhalter, D.J. & Taylor, C.C. (1994) eds. Machine Learning, Neural and Statistical Classification. Ellis Horwood, Chichester.

    Google Scholar 

  • Mosteller, F. & Tukey, J.W. (1977). Data Analysis and Regression. Addison-Wesley, Reading, Mass.

    Google Scholar 

  • Mosteller, F. & Wallace, D.L. (1984). Applied Bayesian and Classical Inference: the Case of the Federalist Papers. Springer-Verlag, New York.

    Google Scholar 

  • Pei, M., Goodman, E.D., Punch, W.F. & Ding, Y. (1995). Genetic Algorithms for Classification & Feature Extraction. Technical Report: Michican State Univeristy, GA Research Group, Engineering Faculty.

    Google Scholar 

  • Quinlan, J.R. (1987). Simplifying Decision Trees. Int. J. Man-Machine Studies, 27, 221–234.

    Google Scholar 

  • Reaven, G.M. & Miller, R.G. (1979). An Attempt to Define the Nature of Chemical Diabetes using a Multidimensional Analysis. Diabetologia, 16, 17–24.

    Article  PubMed  Google Scholar 

  • Rechenberg, I. (1973). Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution, Frommann-Halzboog, Stuttgart.

    Google Scholar 

  • Ritter, G.L., Woodruff, H.B., Lowry, S.R. & Isenhour, T.L. (1974). An Algorithm for a Selective Nearest Neighbour Decision Rule. IEEE Trans. on Info. Theory, IT-21(6), 665–669.

    Google Scholar 

  • Siedlecki, W. & Sklansky, J. (1989). A Note on Genetic Algorithms for Large-scale Feature Selection. Pattern Recognition Letters, 10, 335–347.

    Article  Google Scholar 

  • Smith, J.E., Fogarty, T.C. & Johnson, I.R. (1994). Genetic Selection of Features for Clustering and Classification. IEE Colloquium on Genetic Algorithms in Image Processing & Vision. London.

    Google Scholar 

  • Swonger, C.W. (1972). Sample Set Condensation for a Condensed Nearest Neighbour Decision Rule for Pattern Recognition. In: S. Watanabe, ed., Frontiers of Pattern Recognition. Academic Press.

    Google Scholar 

  • Tomek, I. (1976). An Experiment with the Edited Nearest-Neighbour Rule. IEEE Trans. on Systems, Man & Cybernetics, SMC-6(6), 448–452.

    Google Scholar 

  • Ullman, J.R. (1974). Automatic Selection of Reference Data for Use in a Nearest Neighbour Method of Pattern Classification. IEEE Trans. on Info. Theory, IT-20(4), 541–543.

    Article  Google Scholar 

  • Weiss, S.M. & Kulikowski, C.A. (1991). Computer Systems that Learn. Morgan Kaufmann, San Mateo, CA.

    Google Scholar 

  • Whitley, D. (1989). The GENITOR Algorithm and Selective Pressure: why Rank-Based Allocation of Reproductive Trials is Best. Proc. Third Internat. Conf. on GAs, 116–121, Morgan-Kaufmann, Palo Alto, CA.

    Google Scholar 

  • Wolff, J.G. (1991). Towards a Theory of Cognition and Computing. Ellis Horwood, Chichester.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, University of the West of England, BS16 1QY, Bristol, UK

    Richard S. Forsyth

Authors
  1. Richard S. Forsyth
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Hans-Michael Voigt Werner Ebeling Ingo Rechenberg Hans-Paul Schwefel

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Forsyth, R.S. (1996). IOGA: An instance-oriented genetic algorithm. In: Voigt, HM., Ebeling, W., Rechenberg, I., Schwefel, HP. (eds) Parallel Problem Solving from Nature — PPSN IV. PPSN 1996. Lecture Notes in Computer Science, vol 1141. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61723-X_1012

Download citation

  • DOI: https://doi.org/10.1007/3-540-61723-X_1012

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-61723-5

  • Online ISBN: 978-3-540-70668-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation