Skip to main content
SpringerLink
Log in

Possibilistic Reject-Classification Based on Contrastive Learning in Vector Quantization Networks

  • Conference paper
  • First Online:
Data Mining and Big Data (DMBD 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1744))

Included in the following conference series:

  • 501 Accesses

Abstract

In this paper, while considering rejection as an option, we attempt to tackle the problem of multiclass classification and the uncertainty that arises from the class possibility assignment of data. To address the challenge of classification based on possible class assignments, we use the likelihood ratio, which helps us develop a holistic approach that considers all the positive and negative effects of assigning a particular class as opposed to others to a data point. To this end, we propose a possibilistic variant of the contrastive-learning function, inspired by RSLVQ [20], and a class-wise decision rule based on it. The latter is used to define the total cost function. In addition, with the help of likelihood ratio, an error-rejection trade-off inspired by Chow [3], is proposed. Finally, modification of the cost function and integration of rejection into it result in an interpretable model whose capabilities in both aspects (classification/rejection) are demonstrated by application to different data sets.

Supported by Die Bundesanstalt für Arbeitsschutz und Arbeitsmedizin (BAuA).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bousquet, O., Elisseeff, A.: Stability and generalization. J. Mach. Learn. Res. 2, 499–526 (2002)

    MATH  Google Scholar 

  2. Bradley, A.P.: The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recogn. 30(7), 1145–1159 (1997)

    Article  Google Scholar 

  3. Chow, C.: On optimum recognition error and reject tradeoff. IEEE Trans. Inf. Theory 16(1), 41–46 (1970)

    Article  MATH  Google Scholar 

  4. Devarakota, P.R., Mirbach, B., Ottersten, B.: Confidence estimation in classification decision: a method for detecting unseen patterns. In: Advances in Pattern Recognition, pp. 290–294. World Scientific (2007)

    Google Scholar 

  5. Fischer, L., Nebel, D., Villmann, T., Hammer, B., Wersing, H.: Rejection strategies for learning vector quantization – a comparison of probabilistic and deterministic approaches. In: Villmann, T., Schleif, F.-M., Kaden, M., Lange, M. (eds.) Advances in Self-Organizing Maps and Learning Vector Quantization. AISC, vol. 295, pp. 109–118. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07695-9_10

    Chapter  Google Scholar 

  6. Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Netw. 15(8–9), 1059–1068 (2002)

    Article  Google Scholar 

  7. Kohavi, R., et al.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, Montreal, Canada, vol. 14, pp. 1137–1145 (1995)

    Google Scholar 

  8. Kohonen, T.: Learning vector quantization. In: Kohonen, T. (ed.) Self-Organizing Maps, pp. 175–189. Springer, Heidelberg (1995). https://doi.org/10.1007/978-3-642-97610-0_6

    Chapter  Google Scholar 

  9. Kohonen, T., Hynninen, J., Kangas, J., Laaksonen, J., Torkkola, K.: LVQ PAK: the learning vector quantization program package. Technical report (1996)

    Google Scholar 

  10. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MATH  Google Scholar 

  11. Musavishavazi, S., Mohannazadeh Bakhtiari, M., Villmann, T.: A mathematical model for optimum error-reject trade-off for learning of secure classification models in the presence of label noise during training. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2020. LNCS (LNAI), vol. 12415, pp. 547–554. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61401-0_51

    Chapter  Google Scholar 

  12. Musavishavazi, S., Kaden, M., Villmann, T.: Possibilistic classification learning based on contrastive loss in learning vector quantizer networks. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2021. LNCS (LNAI), vol. 12854, pp. 156–167. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87986-0_14

    Chapter  Google Scholar 

  13. Nebel, D., Hammer, B., Villmann, T.: A median variant of generalized learning vector quantization. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8227, pp. 19–26. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42042-9_3

    Chapter  Google Scholar 

  14. Provost, F., Fawcett, T.: Robust classification for imprecise environments. Mach. Learn. 42(3), 203–231 (2001)

    Article  MATH  Google Scholar 

  15. Raschka, S.: Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808 (2018)

  16. Ravichandran, J., Kaden, M., Saralajew, S., Villmann, T.: Variants of dropconnect in learning vector quantization networks for evaluation of classification stability. Neurocomputing 403, 121–132 (2020)

    Article  Google Scholar 

  17. Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 400–407 (1951)

    Google Scholar 

  18. Saralajew, S., Holdijk, L., Rees, M., Asan, E., Villmann, T.: Classification-by-components: probabilistic modeling of reasoning over a set of components. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  19. Sato, A., Yamada, K.: Generalized learning vector quantization. In: NIPS, vol. 95, pp. 423–429 (1995)

    Google Scholar 

  20. Seo, S., Obermayer, K.: Soft learning vector quantization. Neural Comput. 15(7), 1589–1604 (2003)

    Article  MATH  Google Scholar 

  21. Varma, S., Simon, R.: Bias in error estimation when using cross-validation for model selection. BMC Bioinform. 7(1), 1–8 (2006)

    Article  Google Scholar 

  22. Villmann, A., Kaden, M., Saralajew, S., Hermann, W., Villmann, T.: Reliable patient classification in case of uncertain class labels using a cross-entropy approach. In: ESANN (2018)

    Google Scholar 

  23. Villmann, A., Kaden, M., Saralajew, S., Villmann, T.: Probabilistic learning vector quantization with cross-entropy for probabilistic class assignments in classification learning. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds.) ICAISC 2018. LNCS (LNAI), vol. 10841, pp. 724–735. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91253-0_67

    Chapter  Google Scholar 

  24. Villmann, T., et al.: Self-adjusting reject options in prototype based classification. In: Merényi, E., Mendenhall, M.J., O’Driscoll, P. (eds.) Advances in Self-Organizing Maps and Learning Vector Quantization. AISC, vol. 428, pp. 269–279. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28518-4_24

    Chapter  Google Scholar 

  25. Villmann, T., Kaden, M., Nebel, D., Biehl, M.: Learning vector quantization with adaptive cost-based outlier-rejection. In: Azzopardi, G., Petkov, N. (eds.) CAIP 2015. LNCS, vol. 9257, pp. 772–782. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23117-4_66

    Chapter  Google Scholar 

  26. Zadeh, L.A.: Probability measures of fuzzy events. J. Math. Anal. Appl. 23(2), 421–427 (1968)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Die Bundesanstalt für Arbeitsschutz und Arbeitsmedizin (BAuA), Fabricestraße 8, 01099, Dresden, Germany

    Seyedfakhredin Musavishavazi & Maryam Alipour

Authors
  1. Seyedfakhredin Musavishavazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maryam Alipour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyedfakhredin Musavishavazi .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Ying Tan

  2. Southern University of Science and Technology, Shenzhen, China

    Yuhui Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Musavishavazi, S., Alipour, M. (2022). Possibilistic Reject-Classification Based on Contrastive Learning in Vector Quantization Networks. In: Tan, Y., Shi, Y. (eds) Data Mining and Big Data. DMBD 2022. Communications in Computer and Information Science, vol 1744. Springer, Singapore. https://doi.org/10.1007/978-981-19-9297-1_25

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-9297-1_25

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-9296-4

  • Online ISBN: 978-981-19-9297-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation