Skip to main content
SpringerLink
Log in

Evaluation in Learning from Label Proportions: An Approximation to the Precision-Recall Curve

  • Conference paper
  • First Online:
Advances in Artificial Intelligence (CAEPIA 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11160))

Included in the following conference series:

Abstract

In the last decade, the learning from label proportions problem has attracted the attention of the machine learning community. Many learning methodologies have been proposed, although the evaluation with real label proportions data has hardly been explored. This paper proposes an adaptation of the area under the precision-recall curve metric to the problem of learning from label proportions. The actual performance is bounded by minimum and maximum approximations. Additionally, an approximate estimation which takes advantage of low-uncertain bags is proposed. The benefits of this proposal are illustrated by means of an empirical study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.sc.ehu.es/ccwbayes/members/jeronimo/aucpr_llp/.

References

  1. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2011). http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/

  2. Chen, S., Liu, B., Qian, M., Zhang, C.: Kernel k-means based framework for aggregate outputs classification. In: IEEE ICDM Workshops, pp. 356–361 (2009)

    Google Scholar 

  3. Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of 23rd ICML, pp. 233–240 (2006)

    Google Scholar 

  4. Dery, L.M., Nachman, B., Rubbo, F., Schwartzman, A.: Weakly supervised classification in high energy physics. J. High Energy Phys. 2017(5), 145 (2017)

    Article  Google Scholar 

  5. Fish, B., Reyzin, L.: On the complexity of learning from label proportions. In: Proceedings of 26th IJCAI, pp. 19–25 (2017)

    Google Scholar 

  6. Frank, A., Asuncion, A.: UCI machine learning repository (2010). http://archive.ics.uci.edu/ml

  7. Hernández-González, J., Inza, I., Crisol-Ortiz, L., Guembe, M.A., Iñarra, M.J., Lozano, J.A.: Fitting the data from embryo implantation prediction: learning from label proportions. Stat. Methods Med. Res. 27(4), 1056–1066 (2018)

    Article  MathSciNet  Google Scholar 

  8. Hernández-González, J., Inza, I., Lozano, J.A.: Learning Bayesian network classifiers from label proportions. Pattern Recognit. 46(12), 3425–3440 (2013)

    Article  Google Scholar 

  9. Hernández-González, J., Inza, I., Lozano, J.A.: Weak supervision and other non-standard classification problems: a taxonomy. Pattern Recognit. Lett. 69, 49–55 (2016)

    Article  Google Scholar 

  10. Hübner, D., Verhoeven, T., Schmid, K., Müller, K.R., Tangermann, M., Kindermans, P.J.: Learning from label proportions in brain-computer interfaces: online unsupervised learning with guarantees. PLoS One 12(4), e0175856 (2017)

    Article  Google Scholar 

  11. Kück, H., de Freitas, N.: Learning about individuals from group statistics. In: Proceedings of 21st UAI, pp. 332–339 (2005)

    Google Scholar 

  12. Mitchell, T.: Machine Learning. McGraw Hill, New York City (1997)

    MATH  Google Scholar 

  13. Musicant, D.R., Christensen, J.M., Olson, J.F.: Supervised learning by training on aggregate outputs. In: Proceedings of 7th IEEE ICDM, pp. 252–261 (2007)

    Google Scholar 

  14. Patrini, G., Nock, R., Rivera, P., Caetano, T.: (Almost) no label no cry. In: Proceedings of NIPS 27, pp. 190–198 (2014)

    Google Scholar 

  15. Pérez-Ortiz, M., Gutiérrez, P.A., Carbonero-Ruz, M., Hervás-Martínez, C.: Adapting linear discriminant analysis to the paradigm of learning from label proportions. In: Proceedings of IEEE SSCI, pp. 1–7 (2016)

    Google Scholar 

  16. Qi, Z., Meng, F., Tian, Y., Niu, L., Shi, Y., Zhang, P.: Adaboost-LLP: a boosting method for learning with label proportions. IEEE Trans. Neural Netw. Learn. Syst., 1–12 (2017, early access)

    Google Scholar 

  17. Quadrianto, N., Smola, A.J., Caetano, T.S., Le, Q.V.: Estimating labels from label proportions. J. Mach. Learn. Res. 10, 2349–2374 (2009)

    MathSciNet  MATH  Google Scholar 

  18. Rüping, S.: SVM classifier estimation from group probabilities. In: Proceedings of 27th ICML, pp. 911–918 (2010)

    Google Scholar 

  19. Stolpe, M., Morik, K.: Learning from label proportions by optimizing cluster model selection. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6913, pp. 349–364. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23808-6_23

    Chapter  Google Scholar 

  20. Sun, T., Sheldon, D., O’Connor, B.: A probabilistic approach for learning with label proportions applied to the US presidential election. In: IEEE ICDM (2017)

    Google Scholar 

  21. Yu, F.X., Liu, D., Kumar, S., Jebara, T., Chang, S.: \(\propto \)SVM for learning with label proportions. In: Proceedings of 30th ICML, pp. 504–512 (2013)

    Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the Basque Government (IT609-13, Elkartek BID3A), and the Spanish Ministry of Economy and Competitiveness (TIN2016-78365-R).

Author information

Authors and Affiliations

  1. Intelligent Systems Group, University of the Basque Country UPV/EHU, Donostia, Spain

    Jerónimo Hernández-González

Authors
  1. Jerónimo Hernández-González
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jerónimo Hernández-González .

Editor information

Editors and Affiliations

  1. Andalusian Research Institute on Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain

    Francisco Herrera

  2. Andalusian Research Institute on Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain

    Sergio Damas

  3. Andalusian Research Institute on Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain

    Rosana Montes

  4. Andalusian Research Institute on Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain

    Sergio Alonso

  5. Andalusian Research Institute on Data Science and Computational Intelligence (DaSCI), University of Granada, Granada, Spain

    Óscar Cordón

  6. Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain

    Antonio González

  7. School of Engineering, Pablo de Olavide University, Seville, Spain

    Alicia Troncoso

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hernández-González, J. (2018). Evaluation in Learning from Label Proportions: An Approximation to the Precision-Recall Curve. In: Herrera, F., et al. Advances in Artificial Intelligence. CAEPIA 2018. Lecture Notes in Computer Science(), vol 11160. Springer, Cham. https://doi.org/10.1007/978-3-030-00374-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00374-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00373-9

  • Online ISBN: 978-3-030-00374-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation