Skip to main content
SpringerLink
Log in

A multi-metric small sphere large margin method for classification

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Multi-metric learning is important for improving performance of learners. For complex data, multi metric learning algorithms need intensive research. Moreover, the existing multi-metric learning methods may lead to the distance not being comparable. To solve these shortcomings and characterize better complexity data, we propose a novel multi-metric learning framework, where each class is divided into several clusters, and then a local metric and two concentric hypers-pheres are trained jointly in a cluster, such that the samples of the same cluster distribute within one hypersphere, and the classification margin are as large as possible simultaneously. This will leads to intra-class compactness and inter-class dispersion. During the test phase, the relative distance in learned metric space is designed to make classification decisions. A new example is classified to the class of its closest hyper-sphere center. This ensures that the comparison of distances is meaningful and avoids effectively the limitation of k-nearest neighbors (kNN) classifiers. Moreover,some important properties the proposed algorithm are analyzed theoretically. Further, an alternating iterative algorithm is developed to solve the problem. Numerical experiments are carried out on different scales and types datasets. Experiment results confirm the feasibility and effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data that support the findings of this study will available on reasonable request from the corresponding author.

Notes

  1. https://www.mathworks.com/

  2. https://codeload.github.com/gabeos/lmnn/zip/master.

  3. https://codeload.github.com/PouriaZ/GMML/zip/refs/heads/master.

  4. https://users.ugent.be/bacnguye/DML-dc.v1.0.zip

  5. https://users.ugent.be/bacnguy/DMLMJ.zip.

  6. https://github.com/bacnguyencong/CMML.

  7. http://researchers.lille.inria.fr/abellet/code.html..

  8. https://sci2s.ugr.es/keel/datasets.php.

References

  1. Cover TM, Hart PE (2003) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  MATH  Google Scholar 

  2. Likas A, Vlassis N, Verbeek JJ (2002) The global K-means clustering algorithm. Pattern Recognit 36(2):451–461

    Article  Google Scholar 

  3. Wang CS (2011) Theoretical analysis for solution of support vector data description. Neural Netw 24:360–369

    Article  MATH  Google Scholar 

  4. Jian Yu, Yujian Feng (2019) Quadruplet-center loss for face verification. In 2019 Chinese Automation Congress (CAC), 5034–5039

  5. Yin F, Liu CL (2009) Handwritten Chinese text line segmentation by clustering with distance metric learning. Pattern Recognit 42(12):3146–3157

    Article  MATH  Google Scholar 

  6. Tang J, Li D, Tian Y (2022) Image classification with multi-view multi-instance metric learning. Expert Syst Appl 189:116117

    Article  Google Scholar 

  7. Chang X, Ma Z, Wei X, Hong X, Gong Y (2020) Transductive semi-supervised metric learning for person re-identification. Pattern Recognit 108:107569

    Article  Google Scholar 

  8. Ysc A, Myw A, Lang HB et al (2020) Joint deep semantic embedding and metric learning for person re-identification. Pattern Recognit Lett 130:306–311

    Article  Google Scholar 

  9. Xiang S, Nie F, Zhang C (2008) Learning a Mahalanobis distance metric for data clustering and classification. Pattern Recognit 41(12):3600–3612

    Article  MATH  Google Scholar 

  10. Weinberger Kilian Q, Saul Lawrence K (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(9):207–244

    MATH  Google Scholar 

  11. Ye H, Zhan D, Li N, Jiang Y (2020) Learning multiple local metrics: global consideration helps. IEEE Trans Pattern Anal Mach Intell 42(7):1698–1712

    Article  Google Scholar 

  12. Ye H, Zhan D, Jiang Y, Zhou Z (2019) What makes objects similar: a unified multi-metric learning approach. IEEE Trans Pattern Anal Mach Intell 41(5):1257–1270

    Article  Google Scholar 

  13. Yuan Y, Xiao Y, Hao Z, Liu B (2021) A nearest-neighbor search model for distance metric learning. Inf Sci 552:261–277

    Article  MathSciNet  MATH  Google Scholar 

  14. Yuan Y, Xiao Y, Hao Z, Liu B (2021) A convex model for support vector distance metric learning. IEEE Trans Neural Netw Learn Syst 2021:1–14

    Google Scholar 

  15. Schultz M , Joachims T. Learning a distance metric from relative comparisons. Advances in Neural Information Processing Systems, NIPS 2003, MIT Press

  16. Xu Y, Miao C, Liu Y et al (2020) Kernel-target alignment based non-linear metric learning. Neurocomputing 411:54–66

    Article  Google Scholar 

  17. Bac N, Ferri FJ, Carlos M, de Baets B (2019) An efficient method for clustered multi-metric learning. Inf. Sci. 471:149–163

    Article  MathSciNet  MATH  Google Scholar 

  18. Frome A,Singer Y, Sha F, Malik J. Learning globally-consistent local distance functions for shape-based image retrieval and classification. in: 2007 IEEE 11th international conference on computer vision, 2007,pp 1–8

  19. Wang J, Woznica A, Kalousis A (2012) Parametric local metric learning for nearest neighbor classification. Adv Neural Inf Process Syst 2:1601–1609

    Google Scholar 

  20. Amand J S , Huan J . Sparse compositional local metric learning. in: Acm Sigkdd international conference on knowledge discovery and data mining, ACM, 2017: 1097–1104

  21. Wu M, Ye J (2009) A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans Pattern Anal Mach Intell 31(11):2088–2092

    Article  Google Scholar 

  22. Le T , Tran D , Ma W , Sharma D. An optimal sphere and two large margins approach for novelty detection. in: The 2010 international joint conference on neural networks (IJCNN), 2010, pp 18-23

  23. Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  24. Vapnik VN (1995) The nature of statistical learning theory. Springer

    Book  MATH  Google Scholar 

  25. Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27

    Article  MathSciNet  MATH  Google Scholar 

  26. Zhao Y, Yang L (2023) Distance metric learning based on the class center and nearest neighbor relationship. Inpress, Neural Networks

    Book  Google Scholar 

  27. Tseng P (2001) Convergence of a block coordinate descent method for nondifferentiable minimization. J Optim Theory Appl 109(3):475–494

    Article  MathSciNet  MATH  Google Scholar 

  28. Gass Saul I (2003) Linear programming: methods and applications. Courier Corporation

    MATH  Google Scholar 

  29. Yang L, Qian Y (2016) A sparse logistic regression framework by difference of convex functions programming. Appl Intell 45(2):1–14

    Article  Google Scholar 

  30. Bac N, De BB (2018) An approach to supervised distance metric learning based on difference of convex functions programming. Pattern Recognit 81:562–574

    Article  Google Scholar 

  31. Sardarmehni T, Heydari A (2018) Sub-optimal scheduling in switched systems with continuous-time dynamics: a gradient descent approach. Neurocomputing 285:10–22

    Article  Google Scholar 

  32. Zadeh PH, Hosseini R, and Sra S Geometric mean metric learning. In: Proceedings of the 33rd international conference on machine learning, 2016,48: pp 2464–2471

  33. Bamakan S, Wang H, Shi Y (2017) Ramp loss K-support vector classification-Regression; a robust and sparse multi-class approach to the intrusion detection problem. Knowled-based Syst 2017:126

    Google Scholar 

  34. Bac N, Carlos M, De BB (2017) Supervised distance metric learning through maximization of the Jeffrey divergence. Pattern Recognit 64:215–225

    Article  MATH  Google Scholar 

  35. Demiar J, Schuurmans D (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information and Electrical Engineering, China Agricultural University, Beijing, Haidian, 100083, China

    Yifeng Zhao & Liming Yang

  2. College of Science, China Agricultural University, Beijing, Haidian, 100083, China

    Yifeng Zhao & Liming Yang

Authors
  1. Yifeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liming Yang.

Ethics declarations

Conflict of interest

There are no conflicts of interest in this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Yang, L. A multi-metric small sphere large margin method for classification. Pattern Anal Applic 26, 1615–1629 (2023). https://doi.org/10.1007/s10044-023-01188-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-023-01188-2

Keywords

Search

Navigation