Skip to main content
SpringerLink
Log in
Book cover

International Conference on Hybrid Artificial Intelligence Systems

HAIS 2022: Hybrid Artificial Intelligent Systems pp 186–194Cite as

Generalized Fisher Kernel with Bregman Divergence

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13469))

Abstract

The Fisher kernel has good statistical properties. However, from a practical point of view, the necessary distributional assumptions complicate the applicability. We approach the solution to this problem with the NMF (Non-negative Matrix Factorization) methods, which with adequate normalization conditions, provide stochastic matrices. Using the Bregman divergence as the objective function, formally equivalent solutions appear for the specific forms of the functionals involved. We show that simply by taking these results and plug-in into the general expression of the NMF kernel, obtained with purely algebraic techniques, without any assumptions about the distribution of the parameters, the properties of the Fisher kernel hold, and it is a convenient procedure to use this kernel the situations in which they are needed we derive the expression of the information matrix of Fisher. In this work, we have limited the study to the Gaussian metrics, KL (Kullback-Leibler), and I-divergence.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aggarwal, C.C., Clustering, C.R.D.: Algorithms and Applications. CRC Press Taylor and Francis Group, Boca Raton (2014)

    Google Scholar 

  2. Amari, S.I.: Information geometry of the EM and EM algorithms for neural networks. Neural Netw. 8(9), 1379–1408 (1995)

    Article  Google Scholar 

  3. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)

    Article  MathSciNet  Google Scholar 

  4. Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J., Lafferty, J.: Clustering with Bregman divergences. J. Mach. Learn. Res. 6(10), 1705–174 (2005)

    MathSciNet  MATH  Google Scholar 

  5. Chappelier, J.-C., Eckard, E.: PLSI: the true fisher kernel and beyond. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009. LNCS (LNAI), vol. 5781, pp. 195–210. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04180-8_30

    Chapter  Google Scholar 

  6. Chen, J.C.: The nonnegative rank factorizations of nonnegative matrices. Linear Algebra Appl. 62, 207–217 (1984)

    Article  MathSciNet  Google Scholar 

  7. Ding, C., Li, T., Peng, W.: On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Comput. Stat. Data Anal. 52(8), 3913–3927 (2008)

    Article  MathSciNet  Google Scholar 

  8. Dua, D., Graff, C.: UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences (2017). http://archive.ics.uci.edu/ml

  9. Figuera, P., García Bringas, P.: On the probabilistic latent semantic analysis generalization as the singular value decomposition probabilistic image. J. Stat. Theory Appl. 19, 286–296 (2020)

    Article  Google Scholar 

  10. Figuera, P., Bringas, P.G.: A non-parametric fisher kernel. In: Sanjurjo González, H., Pastor López, I., García Bringas, P., Quintián, H., Corchado, E. (eds.) HAIS 2021. LNCS (LNAI), vol. 12886, pp. 448–459. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86271-8_38

    Chapter  Google Scholar 

  11. Gudovskiy, D., Hodgkinson, A., Yamaguchi, T., Tsukizawa, S.: Deep active learning for biased datasets via fisher kernel self-supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9041–9049 (2020)

    Google Scholar 

  12. Hofmann, T.: Learning the similarity of documents: an information-geometric approach to document retrieval and categorization. In: Advances in Neural Information Processing Systems, pp. 914–920 (2000)

    Google Scholar 

  13. Hofmann, T., Schölkopf, B., Smola, A.J.: Kernel methods in machine learning. Ann. Stat. 36, 1171–1220 (2008)

    MathSciNet  MATH  Google Scholar 

  14. Jaakkola, T., Diekhans, M., Haussler, D.: A discriminative framework for detecting remote protein homologies. J. Comput. Biol. 7(1–2), 95–114 (2000)

    Article  Google Scholar 

  15. Jaakkola, T.S., Haussler, D., et al.: Exploiting generative models in discriminative classifiers. In: Advances in Neural Information Processing Systems, pp. 487–493 (1999)

    Google Scholar 

  16. Kim, D.: KFDA: kernel fisher discriminant analysis (2017). https://CRAN.R-project.org/package=kfda, R package version 1.0.0

  17. Langrené, N., Warin, X.: Fast and stable multivariate kernel density estimation by fast sum updating. J. Comput. Graph. Stat. 28(3), 596–608 (2019)

    Article  MathSciNet  Google Scholar 

  18. Lee, H., Cichocki, A., Choi, S.: Kernel nonnegative matrix factorization for spectral EEG feature extraction. Neurocomputing 72(13–15), 3182–3190 (2009)

    Article  Google Scholar 

  19. Mercer, J.: XVI. Functions of positive and negative type, and their connection the theory of integral equations. Philos. Trans. R. Soc. Lond. Ser. A Containing Papers Math. Phys. Character 209(441–458), 415–446 (1909)

    Google Scholar 

  20. Rao, C.R.: Differential metrics in probability spaces. Diff. Geom. Stat. Inference 10, 217–240 (1987)

    Article  Google Scholar 

  21. Salazar, D., Rios, J., Aceros, S., Flórez-Vargas, O., Valencia, C.: Kernel joint non-negative matrix factorization for genomic data. IEEE Access 9, 101863–101875 (2021)

    Article  Google Scholar 

  22. Scholkopft, B., Mullert, K.R.: Fisher discriminant analysis with kernels. Neural Netw. Signal Process. IX 1(1), 1 (1999)

    Google Scholar 

  23. Sharma, A., Paliwal, K.K.: Rotational linear discriminant analysis technique for dimensionality reduction. IEEE Trans. Knowl. Data Eng. 20(10), 1336–1347 (2008)

    Article  Google Scholar 

  24. Tsuda, K., Akaho, S., Kawanabe, M., Müller, K.R.: Asymptotic properties of the fisher kernel. Neural Comput. 16(1), 115–137 (2004)

    Article  Google Scholar 

  25. Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)

    Article  Google Scholar 

  26. Zhang, D., Zhou, Z.-H., Chen, S.: Non-negative matrix factorization on kernels. In: Yang, Q., Webb, G. (eds.) PRICAI 2006. LNCS (LNAI), vol. 4099, pp. 404–412. Springer, Heidelberg (2006). https://doi.org/10.1007/978-3-540-36668-3_44

    Chapter  Google Scholar 

  27. Zhou, Y., Shi, J., Zhu, J.: Nonparametric score estimators. In: International Conference on Machine Learning, pp. 11513–11522. PMLR (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. D4K Group, University of Deusto, Bilbao, Spain

    Pau Figuera & Pablo García Bringas

  2. iDEA Lab, University of Calabria, Rende, Italy

    Alfredo Cuzzocrea

Authors
  1. Pau Figuera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alfredo Cuzzocrea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pablo García Bringas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pau Figuera .

Editor information

Editors and Affiliations

  1. University of Deusto, Bilbao, Spain

    Pablo García Bringas

  2. University of León, León, Spain

    Hilde Pérez García

  3. University of La Rioja, Logroño, La Rioja, Spain

    Francisco Javier Martínez de Pisón

  4. University of Oviedo, Oviedo, Spain

    José Ramón Villar Flecha

  5. Data Science and Big Data Analytics Lab, Pablo de Olavide University, Sevilla, Spain

    Alicia Troncoso Lora

  6. Department of Computer Science, University of Oviedo, Oviedo, Spain

    Enrique A. de la Cal

  7. Applied Computational Intelligence, University of Burgos, Burgos, Burgos, Spain

    Álvaro Herrero

  8. Universidad Pablo de Olavide, Seville, Spain

    Francisco Martínez Álvarez

  9. DIGIP, University of Bergamo, Dalmine, Bergamo, Italy

    Giuseppe Psaila

  10. Department of Industrial Engineering, University of A Coruña, Ferrol, Spain

    Héctor Quintián

  11. University of Salamanca, Salamanca, Spain

    Emilio Corchado

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Figuera, P., Cuzzocrea, A., Bringas, P.G. (2022). Generalized Fisher Kernel with Bregman Divergence. In: García Bringas, P., et al. Hybrid Artificial Intelligent Systems. HAIS 2022. Lecture Notes in Computer Science(), vol 13469. Springer, Cham. https://doi.org/10.1007/978-3-031-15471-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15471-3_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15470-6

  • Online ISBN: 978-3-031-15471-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation