Skip to main content
Springer Nature Link
Log in

Inductive Learning: A Combinatorial Optimization Approach

  • Chapter
Advances in Machine Learning I

Part of the book series: Studies in Computational Intelligence ((SCI,volume 262))

  • 2231 Accesses

Abstract

We propose an improved inductive learning method to derive classification rules correctly describing (at least) most of the positive examples and do not correctly describe (at least) most of the negative examples. We start with a pre-analysis of data to assign higher weights to those values of attributes which occur more often in the positive than in the negative examples. The inductive learning problem is represented as a modification of the set covering problem which is solved by an integer programming based algorithm using elements of a greedy algorithm or a genetic algorithm, for efficiency. The results are very encouraging and are illustrated on a thyroid cancer data set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Balas, E.: Cutting planes from conditional bounds: a new approach to set covering. Mathematical Programming Study 12, 19–36 (1980)

    MATH  MathSciNet  Google Scholar 

  • Balas, E., Ho, A.: Set covering algorithms using cutting planes, heuristics and sub gradient optimisation: A computation study. Mathematical Programming 12, 37–60 (1980)

    MATH  MathSciNet  Google Scholar 

  • Balas, E., Padberg, M.W.: Set partitioning - A survey. In: Christofides, N. (ed.) Combinatorial Optimisation. Wiley, New York (1979)

    Google Scholar 

  • Beasley, J.E.: An algorithm for set covering problem. European Journal of Operational Research 31, 85–93 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  • Beasley, J.E.: A Lagrangian heuristic for set covering problem. Naval Research Logistics Quarterly 37, 151–164 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  • Beasley, J.E.: A genetic algorithm for the set covering problem. European Journal of Operational Research 94, 392–404 (1996)

    Article  MATH  Google Scholar 

  • Beasley, J.E., Jornsten, K.: Enhancing an algorithm for set covering problem. European Journal of Operational Research 58, 293–300 (1992)

    Article  MATH  Google Scholar 

  • Christofides, N., Korman, S.: A computational survey of methods for the set covering problem. Management Science 21, 591–599 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  • Chvatal, V.: A greedy heuristic for the set-covering problem. Mathematics of Operational Research 4(3), 233–235 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  • Cios, K.J., Liu, N.: An algorithm which learns multiple covers via integer linear programming. Part I - The CLILP2 Algorithm. Kybernetes 24, 29–50 (1995)

    Article  Google Scholar 

  • Cios, K.J., Wedding, D.K., Liu, N.: CLIP3: cover learning using integer programming. Kybernetes 26, 513–536 (1997)

    Article  Google Scholar 

  • Cios, K.J., Kurgan, L.: CLIP4: Hybrid inductive machine learning algorithm that generates inequality rules. Information Sciences 163, 37–83 (2004)

    Article  Google Scholar 

  • Etcheberry, J.: The set covering problem: A new implicit enumeration algorithm. Operations Research 25, 760–772 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  • Grossman, T., Wool, A.: Computational experience with approximation algorithms for the set covering problem. Working paper, Theoretical Division and CNLS, Los Alamos National Laboratory (1995)

    Google Scholar 

  • Homaifar, A., Lai, S., Qi, X.: Constrained optimization via genetic algorithms. Simulation 62, 242–254 (1994)

    Article  Google Scholar 

  • Iwański, C., Szkatuła, G.: Inductive learning supported by integer programming. Computers and Artificial Intelligence 10, 57–66 (1991)

    MathSciNet  Google Scholar 

  • Johnson, D.A.: Approximation algorithms for combinatorial problems. Journal of Computer System Science 9, 256–278 (1974)

    Article  MATH  Google Scholar 

  • Kacprzyk, J., Iwański, C.: Fuzzy logic with linguistic quantifiers in inductive learning. In: Zadeh, L.A., Kacprzyk, J. (eds.) Fuzzy Logic for the Management of Uncertainty, pp. 465–478. Wiley, New York (1992)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: Machine learning from examples under errors in data. In: Bouchon-Meunier, B., Yager, R.R., Zadeh, L.A. (eds.) IPMU 1994. LNCS, vol. 945, pp. 1047–1051. Springer, Heidelberg (1994)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: An algorithm for learning from erroneous and incorrigible examples. International Journal of Intelligent Systems 11, 565–582 (1996)

    Article  MATH  Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: An improved inductive learning algorithm with a preanalysis of data. In: Raś, Z.W., Skowron, A. (eds.) ISMIS 1997. LNCS, vol. 1325, pp. 157–166. Springer, Heidelberg (1997)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: Deriving IF-THEN rules for intelligent decision support via inductive learning. In: Kasabov, N., et al. (eds.) Progress in Connectionist-Based Information Systems (Proceedings of ICONIP 1997, ANZIIS 1997 and ANNES 1997 Conference), Dunedin, New Zealand, Singapore, vol. 2, pp. 818–821. Springer, Heidelberg (1997b)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: IP1 - An improved inductive learning procedure with a preprocessing of data. In: Proceedings of IDEAL 1998, Hong Kong, pp. 385–392. Springer, Heidelberg (1998)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: An inductive learning algorithm with a preanalysis of data. International Journal of Knowledge - Based Intelligent Engineering Systems 3, 135–146 (1999)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: An integer programming approach to inductive learning using genetic and greedy algorithms. In: Jain, L.C., Kacprzyk, J. (eds.) New Learning Paradigms in Soft Computing, pp. 322–366. Physica-Verlag, Heidelberg (2002a)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: An integer programming approach to inductive learning using genetic algorithm. In: Proceedings of CEC 2002 - The 2002 Congress on Evolutionary Computation, Honolulu, Hawaii, pp. 181–186 (2002b)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.: A softened formulation in inductive learning and application to coronary disease data. In: Atanassov, K.T., Kacprzyk, J., Krawczak, M., Szmidt, E. (eds.) Issues in the Representation and Processing of Uncertain and Imprecise Information. Fuzzy Sets, Intuitionistic Fuzzy Sets, Generalized Nets, and Related Topics, pp. 181–197. AOW EXIT, Warszawa (2005a)

    Google Scholar 

  • Kacprzyk, J., Szkatula, G.z.: A softened formulation of inductive learning and its use for coronary disease data. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 200–209. Springer, Heidelberg (2005)

    Google Scholar 

  • Kacprzyk, J., Szkatuła, G.z.: An inductive learning algorithm with a partial completeness and consistence via a modified set covering problem. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 661–666. Springer, Heidelberg (2005)

    Google Scholar 

  • Michalski, R.S.: Variable valued logic: system VLl. In: Proc. of 1974 International Symposium on Multiple Valued Logic and Pattern Recognition, West Virginia University, Morgantown, pp. 323–346 (1974)

    Google Scholar 

  • Michalski, R.S.: A theory and methodology of inductive learning. In: Michalski, R., Carbonell, J., Mitchell, T.M. (eds.) Machine Learning. Tioga Press (1983)

    Google Scholar 

  • Michalski, R.S., Mozetic, I., Hong, J., Lovrac, N.: The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In: Proceedings of 5th National Conference on Artificial Intelligence, Philadelphia, pp. 1041–1045. Morgan Kaufmann, San Francisco (1986)

    Google Scholar 

  • Nakache, J.P., Asselain, B.: Medical data set proposed for the workshop on data analysis. In: EIASM Workshop (April 1983)

    Google Scholar 

  • Paixao, J.: Algorithms for large scale set covering problems. PhD. Thesis, Department of Management Science, Imperial College (1984)

    Google Scholar 

  • Powell, D., Skolnick, M.M.: Using genetic algorithms in engineering design optimization with non-linear constraints. In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 424–430. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  • Schoenauer, M., Xanthakis, S.: Constrained GA optimization. In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 573–580. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  • Szkatuła, G.: Machine learning from examples under errors in data (In Polish), Ph.D. thesis, SRI PAS Warsaw, Poland (1996)

    Google Scholar 

  • Szkatuła, G., Kacprzyk, J.: An inductive learning algorithm with a partial completeness and consistency. In: Draminski, M., Grzegorzewski, P., Trojanowski, T., Zadrozny, S. (eds.) Issues in intelligent systems. Models and techniques, pp. 229–246. AOW EXIT, Warszawa (2005)

    Google Scholar 

  • Vercellis, C.: A probabilistic analysis of the set covering problem. Annals of Operational Research 1, 255–271 (1984)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Systems Research Institute, Polish Academy of Sciences, ul. Newelska 6, 01-447, Warsaw, Poland

    Janusz Kacprzyk & Grażyna Szkatuła

Authors
  1. Janusz Kacprzyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grażyna Szkatuła
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Polish Academy of Sciences, ul.Ordona 21, 01-237, Warsaw, Poland

    Jacek Koronacki  & Sławomir T. Wierzchoń  & 

  2. Woodward Hall 430C University of North Carolina, 9201 University City Blvd., N.C. 28223, Charlotte, USA

    Zbigniew W. Raś

  3. Systems Research Institute, Polish Academy of Sciences, ul.Newelska 6, 01-447, Warsaw, 01-447

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kacprzyk, J., Szkatuła, G. (2010). Inductive Learning: A Combinatorial Optimization Approach. In: Koronacki, J., Raś, Z.W., Wierzchoń, S.T., Kacprzyk, J. (eds) Advances in Machine Learning I. Studies in Computational Intelligence, vol 262. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05177-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-05177-7_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05176-0

  • Online ISBN: 978-3-642-05177-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics