Skip to main content

Advertisement

Springer Nature Link
Log in

Performance Analysis for SVM Combining with Metric Learning

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

This paper analyses the performance of combining Support Vector Machines (SVMs) and metric learning, in order to evaluate the effect of metric learning on improving SVM. First, we establish the sufficient condition under which the performance of SVM cannot be improved by metric learning. Second, to verify whether the sufficient condition holds, we develop a two-step metric learning strategy by learning an orthonormal matrix and a diagonal matrix respectively. Third, we analyze the case when the sufficient condition holds after the two-step metric learning, and therefore demonstrate the practicability of improving the accuracy of SVM. Finally, we provide some experiments, and also apply metric learning into SVM for 3D object classification and face recognition. The experimental results demonstrate the effectiveness of improving the SVM classification performance by metric learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php.

References

  1. Baghshah MS, Shouraki SB (2009) Semi-supervised metric learning using pairwise constraints. In: Proceedings of the 21st international joint conference on artificial intelligence. Pasadena, California, USA, pp 1217–1222

  2. Bilenko M, Basu S, Mooney RJ (2004) Integrating constraints and metric learning in semi-supervised clustering. In: Proceedings of the 21st international conference on machine learning. Banff, Alberta, Canada, pp 81–88

  3. Chang C, Lin C (2011) LIBSVM: a library for support vector machines. ACM Trans Intel Syst Technol 2(3):1–27

    Article  Google Scholar 

  4. Collobert R, Bengio S, Williamson C (2001) SVMTorch: support vector machines for large-scale regression problems. J Mach Learn Res 1:143–160

    MathSciNet  MATH  Google Scholar 

  5. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  6. Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: Proceedings of the 24th international conference on machine learning. Corvalis, Oregon, USA, pp 209–216

  7. Ding Z, Suh S, Han JJ, Choi C, Fu Y (2015) Discriminative low-rank metric learning for face recognition. In: Proceedings of the 11th IEEE international conference and workshops on automatic face and gesture recognition, vol 1. Ljubljana, Slovenia, pp 1–6

  8. Do H, Kalousis A, Wang J, Woznica A (2012) A metric learning perspective of SVM: on the relation of SVM and LMNN. In: Proceedings of the 15th international conference on artificial intelligence and statistics, vol. 22. San Diego, CA, USA, pp 308–317

  9. Du S, Guo Y, Sanroma G, Ni D, Wu G, Shen D (2015) Building dynamic population graph for accurate correspondence detection. Med Image Anal 26(1):256–267

    Article  Google Scholar 

  10. Gao Y, Ji R, Cui P, Dai Q, Hua G (2014) Hyperspectral image classification through bilayer graph-based learning. IEEE Trans Image Process 23(7):2769–2778

    Article  MathSciNet  Google Scholar 

  11. Gao Y, Wang M, Ji R, Wu X, Dai Q (2014) 3D object retrieval with Hausdorff distance learning. IEEE Trans Ind Electron 61(4):2088–2098

    Article  Google Scholar 

  12. Gao Y, Wang M, Zha ZJ, Shen J, Li X, Wu X (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376

    Article  MathSciNet  Google Scholar 

  13. Goldberger J, Hinton GE, Roweis ST, Salakhutdinov RR (2005) Neighbourhood components analysis. In: Advances in neural information processing systems 18. MIT Press, Vancouver, Canada. pp 513–520

  14. Hoi SCH, Liu W, Chang S (2010) Semi-supervised distance metric learning for collaborative image retrieval and clustering. ACM Trans Multimed Comput Commun Appl 6(3):1–26

    Article  Google Scholar 

  15. Huang Z, Wang R, Shan S, Chen X (2015) Face recognition on large-scale video in the wild with hybrid Euclidean-and-Riemannian metric learning. Pattern Recog 48(10):3113–3124

    Article  Google Scholar 

  16. Huang Z, Wang R, Shan S, Chen X (2015) Projection metric learning on Grassmann manifold with application to video based face recognition. In: Proceedings of the 28th IEEE conference on computer vision and pattern recognition. Boston, MA, USA, pp 140–149

  17. Joachims T (1998) Making large-scale SVM learning practical. In: Advances in kernel methods-support vector learning. MIT Press, Cambridge, MA, pp 169–184

  18. Joachims T (1998) Text categorization with support vector machines: learning with many relevant features. In: Proceedings of the 10th European conference on machine learning. Dorint-Parkhotel, Chemnitz, Germany, pp 137–142

    Chapter  Google Scholar 

  19. Joachims T (2006) Training linear SVMs in linear time. In: Proceedings of the 12th ACM conference on knowledge discovery and data mining. Philadelphia, PA, USA, pp 217–226

  20. Khan B, Han F, Wang Z, Masood RJ (2016) Bio-inspired approach to invariant recognition and classification of fabric weave patterns and yarn color. Assem Autom 36(2):152–158

    Article  Google Scholar 

  21. Kulis B (2012) Metric learning: a survey. Found Trends Mach Learn 5(4):287–364

    Article  Google Scholar 

  22. Lee KC, Ho J, Kriegman DJ (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intel 27(5):684–698

    Article  Google Scholar 

  23. Li H, Jiang T, Zhang K (2006) Efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 17(1):157–165

    Article  Google Scholar 

  24. Li S, Fu Y (2013) Low-rank coding with b-matching constraint for semi-supervised classification. In: Proceedings of the 23rd international joint conference on artificial intelligence. Beijing, China, pp 1472–1478

  25. Liu J, Chen S, Tan X, Zhang D (2007) Comments on “efficient and robust feature extraction by maximum margin criterion”. IEEE Trans Neural Netw 18(6):1862–1864

    Article  Google Scholar 

  26. Liu W, Liu J, Ma S, Liu P, Tao D (2010) Semi-supervised sparse metric learning using alternating linearization optimization. In: Proceedings of 16th the ACM conference on knowledge discovery and data mining. Washington, DC, USA, pp 1139–1148

  27. Mahalanobis PC (1936) On the generalized distance in statistics. Proc Natl Inst Sci India 2:49–55

    MATH  Google Scholar 

  28. Mei J, Liu M, Karimi HR, Gao H (2014) Logdet divergence-based metric learning with triplet constraints and its applications. IEEE Trans Image Process 23(11):4920–4931

    Article  MathSciNet  Google Scholar 

  29. Niu G, Dai B, Yamada M, Sugiyama M (2014) Information-theoretic semi-supervised metric learning via entropy regularization. Neural Comput 26(8):1717–1762

    Article  MathSciNet  Google Scholar 

  30. Platt JC (1998) Sequential minimal optimization: a fast algorithm for training support vector machines. Tech. Rep. MSR-TR-98-14, Microsoft Research

  31. Schultz M, Joachims T (2004) Learning a distance metric from relative comparisons. In: Advances in neural information processing systems 17. MIT Press, Vancouver, Canada, pp 41–48

  32. Shen B (2016) Bio-inspired manipulation and robotics. Assem Autom 36(2):109–110

    Article  Google Scholar 

  33. Shi B, Liu J (2015) Nonlinear metric learning for kNN and SVMs through geometric transformations. arXiv:1508.01534 pp 1–9

  34. Shoushtari L, Mazzoleni S, Dario P (2016) Bio-inspired kinematical control of redundant robotic manipulators. Assem Autom 36(2):200–215

    Article  Google Scholar 

  35. Tang Y (2013) Deep learning using linear support vector machines. arXiv:1306.0239 pp 1–6

  36. Wang H, Wang Y, Sun Y, Pu Q, Lu X (2016) On the small fiber-coupled laser controller for animal robot. Assem Autom 36(2):146–151

    Article  Google Scholar 

  37. Wang Q, Yuen PC, Feng G (2013) Semi-supervised metric learning via topology preserving multiple semi-supervised assumptions. Pattern Recog 46(9):2576–2587

    Article  Google Scholar 

  38. Wang Q, Zuo W, Zhang L, Li P (2014) Shrinkage expansion adaptive metric learning. In: Proceedings of the 13th European conference on computer vision. Zurich, Switzerland, pp 456–471

    Google Scholar 

  39. Weinberger KQ, Blitzer J, Saul LK (2006) Distance metric learning for large margin nearest neighbor classification. In: Advances in neural information processing systems 19. MIT Press, Vancouver, Canada, pp 1473–1480

  40. Wolf L, Levy N (2013) The SVM-minus similarity score for video face recognition. In: Proceedings of the 26th IEEE conference on computer vision and pattern recognition. Portland, OR, USA, pp 3523–3530

  41. Xing EP, Jordan MI, Russell SJ, Ng AY (2003) Distance metric learning with application to clustering with side-information. In: Advances in neural information processing systems 15. MIT Press, Vancouver, Canada, pp 521–528

  42. Ying SH, Wen ZJ, Shi J, Peng YX, Peng JG, Qiao H (2017) Manifold preserving: an intrinsic approach for semi-supervised distance metric learning. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2017.2691005

    Article  Google Scholar 

  43. Zhong G, Huang K, Liu CL (2011) Low rank metric learning with manifold regularization. In: Proceedings of 11th IEEE international conference on data mining. Vancouver, Canada, pp 1266–1271

Download references

Acknowledgements

This work is supported by National Science Foundation of China (Nos. 11771276, 11471208, 61731009 and 61273298), and the Open Research Fund of Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, China.

Author information

Authors and Affiliations

  1. Shanghai University, Shanghai, China

    Lingfang Hu & Yaxin Peng

  2. Shanghai Electric Central Research Institute, Shanghai, China

    Juan Hu & Zhen Ye

  3. East China Normal University, Shanghai, China

    Chaomin Shen

  4. Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China

    Yaxin Peng

Authors
  1. Lingfang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chaomin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaxin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaxin Peng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, L., Hu, J., Ye, Z. et al. Performance Analysis for SVM Combining with Metric Learning. Neural Process Lett 48, 1373–1394 (2018). https://doi.org/10.1007/s11063-017-9771-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-017-9771-7

Keywords

Search

Navigation