Skip to main content
SpringerLink
Log in
Book cover

International Conference on Hybrid Artificial Intelligence Systems

HAIS 2016: Hybrid Artificial Intelligent Systems pp 647–658Cite as

The Multivariate Entropy Triangle and Applications

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9648))

Abstract

We extend a framework for the analysis of classifiers to encompass also the analysis of data sets. Specifically, we generalize a balance equation and a visualization device, the Entropy Triangle, for multivariate distributions, not only bivariate ones. With such tools we analyze a handful of UCI machine learning task to start addressing the question of how information gets transformed through machine learning classification tasks.

F.J. Valverde-Albacete—CPM & FVA have been partially supported by the Spanish Government-MinECo projects TEC2014-53390-P and TEC2014-61729-EXP.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/FJValverde/entropies.git.

References

  1. McGill, W.J.: Multivariate information transmission. Psychometrika 19(2), 97–116 (1954)

    Article  MathSciNet  MATH  Google Scholar 

  2. Shannon, C.E.: A mathematical theory of Communication. Bell Syst. Techn. J. XXVII(3), 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  3. Murphy, K.P.: Machine Learning. A Probabilistic Perspective. MIT Press, Cambridge (2012)

    MATH  Google Scholar 

  4. Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Academic Press, Orlando (2006)

    MATH  Google Scholar 

  5. Valverde-Albacete, F.J., Peláez-Moreno, C.: Two information-theoretic tools to assess the performance of multi-class classifiers. Pattern Recogn. Lett. 31(12), 1665–1671 (2010)

    Article  Google Scholar 

  6. Valverde-Albacete, F.J., Peláez-Moreno, C.: 100% classification accuracy considered harmful: the normalized information transfer factor explains the accuracy paradox. PLOS ONE 9(1), e84217 (2014)

    Article  Google Scholar 

  7. Valverde-Albacete, F.J., Carrillo-de-Albornoz, J., Peláez-Moreno, C.: A proposal for new evaluation metrics and result visualization technique for sentiment analysis tasks. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 41–52. Springer, Heidelberg (2013)

    Google Scholar 

  8. Brown, G., Pocock, A., Zhao, M.J., Luján, M.: Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J. Mach. Learn. Res. 13(1), 27–66 (2012)

    MathSciNet  MATH  Google Scholar 

  9. Gibaja, E., Ventura, S.: A tutorial on multilabel learning. ACM Comput. Surv. (CSUR) 47(3), 1–38 (2015)

    Article  Google Scholar 

  10. Mejía-Navarrete, D., Gallardo-Antolín, A., Peláez-Moreno, C., Valverde-Albacete, F.J.: Feature extraction assessment for an acoustic-event classification task using the entropy triangle. In: Interspeech 2010: 12th Annual Conference of the International Speech Communication Association (2011)

    Google Scholar 

  11. Han, T.S.: Linear dependence structure of the entropy space. Inf. Control 29, 337–368 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  12. Han, T.S.: Nonnegative entropy measures of multivariate symmetric correlations. Inf. Control 36(2), 133–156 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  13. Watanabe, S.: Information theoretical analysis of multivariate correlation. IBM Corp. J. Res. Dev. 4(1), 66–82 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  14. Studený, M., Vejnarová, J.: The multiinformation function as a tool for measuring stochastic dependence. In: Jordan, M.I. (ed.) Learning in Graphical Models. NATO ASI Series, vol. 89, pp. 261–297. Springer, Netherlands (1998)

    Chapter  Google Scholar 

  15. Abdallah, S.A., Plumbley, M.D.: A measure of statistical complexity based on predictive information with application to finite spin systems. Phys. Lett. A 376(4), 275–281 (2012)

    Article  MATH  Google Scholar 

  16. R Core Team: R A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2015)

    Google Scholar 

  17. Meyer, D., Zeileis, A., Hornik, K.: VCD: Visualizing Categorical Data. R package version 1.4-1 (2015)

    Google Scholar 

  18. Leisch, F., Dimitriadou, E.: mlbench: Machine Learning Benchmark Problems. R package version 2.1-1 (2010)

    Google Scholar 

  19. Lichman, M.: UCI Machine Learning Repository. University of California, Irvine (2013)

    Google Scholar 

  20. Hamilton, N.: ggtern: An Extension to ggplot2, for the Creation of Ternary Diagrams. R package version 1.0.6.1 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Teoría de la Señal y de las Comunicaciones, Universidad Carlos III de Madrid, 28911, Leganés, Spain

    Francisco José Valverde-Albacete & Carmen Peláez-Moreno

Authors
  1. Francisco José Valverde-Albacete
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carmen Peláez-Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francisco José Valverde-Albacete .

Editor information

Editors and Affiliations

  1. Universidad Pablo de Olavide, Sevilla, Spain

    Francisco Martínez-Álvarez

  2. Universidad Pablo de Olavide, Sevilla, Spain

    Alicia Troncoso

  3. University of Salamanca, Salamanca, Spain

    Héctor Quintián

  4. University of Salamanca, Salamanca, Spain

    Emilio Corchado

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Valverde-Albacete, F.J., Peláez-Moreno, C. (2016). The Multivariate Entropy Triangle and Applications. In: Martínez-Álvarez, F., Troncoso, A., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2016. Lecture Notes in Computer Science(), vol 9648. Springer, Cham. https://doi.org/10.1007/978-3-319-32034-2_54

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32034-2_54

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32033-5

  • Online ISBN: 978-3-319-32034-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation