Skip to main content
SpringerLink
Log in

Experiments on Data Classification Using Relative Entropy

  • Conference paper
  • First Online:
Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 403))

  • 976 Accesses

Abstract

Data classification is one of the basic tasks in data mining. In this paper, we propose a new classifier based on relative entropy, where data to particular class assignment is made by the majority good guess criteria. The presented approach is intended to be used when relations between datasets and assignment classes are rather complex, nonlinear, or with logical inconsistencies; because such datasets can be too complex to be classified by ordinary methods of decision trees or by the tools of logical analysis. The relative entropy evaluation of associative rules can be simple to interpret and offers better comprehensibility in comparison to decision trees and artificial neural networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kotsiantis, S.B.: Supervised machine learning: a review of classification techniques. Informatica 31, 249–268 (2007)

    MathSciNet  MATH  Google Scholar 

  2. Fürnkranz, J., Flach, P.A.: ROC ‘n’ rule learning—towards a better understanding of covering rules. Mach. Learn. 58, 39–77 (2005)

    Article  MATH  Google Scholar 

  3. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948)

    Google Scholar 

  4. Cover, T.M.: Elements of Information Theory. Wiley-Interscience, New York (1991)

    Book  MATH  Google Scholar 

  5. Quinlan, J.R.: Learning efficient classification procedures and their application to chess endgames. Machine Learning: An Artificial Inteligence Approach, pp. 463–482. Palo Alto, Tioga (1983)

    Chapter  Google Scholar 

  6. Cendrowska, J.: PRISM: an algorithm for inducing modular rules. Int. J. Man-Mach. Stud. 27, 349–370 (1987)

    Article  MATH  Google Scholar 

  7. Thabtah, F.A., Cowling, P.I.: A greedy classification algorithm based on association rule. Appl. Soft Comput. 7, 1102–1111 (2007)

    Article  Google Scholar 

  8. Li, J., Wong, L.: Using rules to analyse bio-medical data: a comparison between C4.5 and PCL. Adv. Web-Age Inf. Manag. 4, 254–265 (2003)

    Article  Google Scholar 

  9. Fano, R.M.: Transmission of Information. A Statistical Theory of Communications. M.I.T. Press, New York (1961)

    MATH  Google Scholar 

  10. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  11. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco (1993)

    Google Scholar 

  12. Cohen, W.: Fast effective rule induction. In: Proceedings of ICML-95, pp. 115–123 (1995)

    Google Scholar 

  13. Liu, B., Hsu, W., Ma, Y.: Integrating classification and association rule mining. In: Proceedings of the KDD, pp. 80–86. New York (1998)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the SGS in VSB—Technical University of Ostrava, Czech Republic, under the grant No. SP2015/146.

Author information

Authors and Affiliations

  1. FEECS, Department of Computer Science, VŠB-Technical University of Ostrava, 17.listopadu, 708 33, Ostrava, Poruba, Czech Republic

    Michal Vašinek & Jan Platoš

Authors
  1. Michal Vašinek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Platoš
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michal Vašinek .

Editor information

Editors and Affiliations

  1. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Robert Burduk

  2. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Konrad Jackowski

  3. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Marek Kurzyński

  4. Dept. of Systems and Computer Networks, Wrocław University of Technology, Wroclaw, Poland

    Michał Woźniak

  5. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Andrzej Żołnierek

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Vašinek, M., Platoš, J. (2016). Experiments on Data Classification Using Relative Entropy. In: Burduk, R., Jackowski, K., Kurzyński, M., Woźniak, M., Żołnierek, A. (eds) Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015. Advances in Intelligent Systems and Computing, vol 403. Springer, Cham. https://doi.org/10.1007/978-3-319-26227-7_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26227-7_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26225-3

  • Online ISBN: 978-3-319-26227-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation