Skip to main content
SpringerLink
Log in

A soft-margin convex polyhedron classifier for nonlinear task with noise tolerance

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

As a special form of piecewise linear classifier, the convex polyhedron classifier is simple to implement and achieves rapid response in real-time classification. However, it usually performs badly in the case of high noise where severe boundary intrusion exists. Inspired by the scheme of soft margin in support vector machine, in this paper we propose a soft-margin convex polyhedron classifier for nonlinear classification task. The base (linear) classifier is first generalized to its soft-margin version through kernelization process and slack variables. In each local region, the soft-margin base classifier learns a decision hyperplane with noise tolerance. Then, a series of learned hyperplanes are structurally integrated into a convex polyhedron classifier, which is essentially a convex polyhedron that encloses one class and excludes the other class outside. Experimental results on fifteen benchmark datasets show the proposed soft-margin convex polyhedron classifier is comparable to linear support vector machine and four piecewise linear classifiers, but does not perform as well as the support vector machine with radial basis function kernel in general. When random noises are added to datasets, the soft-margin convex polyhedron classifier achieves similar or better accuracies with the well-known classifiers used for comparison, implying its promising ability of noise tolerance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tenmoto H, Kudo M, Shimbo M (1998) Piecewise linear classifiers with an appropriate number of hyperplanes. Pattern Recogn 31(11):1627–1634

    Article  MATH  Google Scholar 

  2. Huang X, Mehrkanoon S, Suykens JAK (2013) Support vector machines with piecewise linear feature mapping. Neurocomputing 117:118–127

    Article  Google Scholar 

  3. Kostin A (2006) A simple and fast multi-class piecewise linear pattern classifier. Pattern Recogn 39(11):1949–1962

    Article  MATH  Google Scholar 

  4. Webb D (2010) Efficient piecewise linear classifiers and applications [PhD thesis]. Graduate School of Information Technology and Mathematical Sciences, University of Ballarat

  5. Mangasarian OL (1968) Multisurface method of pattern separation. IEEE Trans Inf Theory 14 (6):801–807

    Article  MATH  Google Scholar 

  6. Smith FW (1968) Pattern classifier design by linear programming. IEEE Trans Comput 100 (4):367–372

    Article  Google Scholar 

  7. Herman GT, Yeung KTD (1992) On piecewise-linear classification. IEEE Trans Pattern Anal Mach Intell 14(7):782–786

    Article  Google Scholar 

  8. Nakayama H, Kagaku N (1998) Pattern classification by linear goal programming and its extensions. J Glob Optim 12(2):111–126

    Article  MathSciNet  MATH  Google Scholar 

  9. Oladunni OO, Singhal G (2009) Piecewise multi-classification support vector machines. In: Proceedings of 2009 International Joint Conference on Neural Networks, pp 2323–2330

  10. Garcia-Palomares UM (2012) Novel linear programming approach for building a piecewise nonlinear binary classifier with a priori accuracy. Decis Support Syst 52(3):717–728

    Article  Google Scholar 

  11. Sklansky J, Michelotti L (1980) Locally trained piecewise linear classifiers. IEEE Trans Pattern Anal Mach Intell (2):101–111

  12. Park Y, Sklansky J (1989) Automated design of multiple-class piecewise linear classifiers. J Classif 6(1):195–222

    Article  MathSciNet  MATH  Google Scholar 

  13. Gai K, Zhang C (2010) Learning discriminative piecewise linear models with boundary points. In: Proceedings of the 24th AAAI Conference on Artificial Intelligence, pp 444–450

  14. Chai B, Huang T, Zhuang X, Zhao Y, Sklansky J (1996) Piecewise linear classifiers using binary tree structure and genetic algorithm. Pattern Recogn 29(11):1905–1917

    Article  Google Scholar 

  15. Wang D, Zhang X, Fan M, Ye X (2016) Hierarchical mixing linear support vector machines for nonlinear classification. Pattern Recogn 59:255–267

    Article  MATH  Google Scholar 

  16. Ozkan H, Vanli N, Kozat S (2016) Online classification via self-organizing space partitioning. IEEE Trans Signal Process 64(15):3895–3908

    Article  MathSciNet  MATH  Google Scholar 

  17. Bagirov AM (2015) Max-min separability. Optim Methods Softw 20(2-3):277–296

    Article  MathSciNet  MATH  Google Scholar 

  18. Bagirov AM, Ugon J, Webb D (2011) An efficient algorithm for the incremental construction of a piecewise linear classifier. Inf Syst 36(4):782–790

    Article  Google Scholar 

  19. Bagirov AM, Kasimbeyli R, Öztürk G, Ugon J (2014) Piecewise linear classifiers based on nonsmooth optimization approaches. In: Optimization in Science and Engineering. Springer, New York, pp 1–32

  20. Ozturk G, Bagirov AM, Kasimbeyli R (2015) An incremental piecewise linear classifier based on polyhedral conic separation. Mach Learn 101(1-3):397–413

    Article  MathSciNet  MATH  Google Scholar 

  21. Osadchy M, Hazan T, Keren D (2015) K-hyperplane hinge-minimax classifier. In: Proceedings of the 32nd International Conference on Machine Learning, pp 1558–1566

  22. Raviv D, Hazan T, Osadchy M (2018) Hinge-minimax learner for the ensemble of hyperplanes. J Mach Learn Res 19(1):2517–2546

    MathSciNet  MATH  Google Scholar 

  23. Maji S, Berg AC, Malik J (2008) Classification using intersection kernel support vector machines is efficient. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–8

  24. Maji S, Berg AC, Malik J (2012) Efficient classification for additive kernel SVMs. IEEE Trans Pattern Anal Mach Intell 35(1):66–77

    Article  Google Scholar 

  25. Huang X, Suykens JAK, Wang S, Hornegger J, Maier A (2018) Classification with truncated ℓ1 distance kernel. IEEE Trans Neural Netw Learn Syst 29(5):2025–2030

    Article  MathSciNet  Google Scholar 

  26. Ladicky L, Torr PHS (2011) Locally linear support vector machines. In: Proceedings of the 28th International Conference on Machine Learning (ICML), pp 985–992

  27. Mao X, Fu Z, Wu O, Hu W (2015) Optimizing locally linear classifiers with supervised anchor point learning. In: Proceedings of the 24 International Joint Conference on Artificial Intelligence (IJCAI), pp 3699–3706

  28. Xu B, Shen S, Shen F, Zhao J (2019) Locally linear SVMs based on boundary anchor points encoding. Neural Netw 117:274– 284

    Article  Google Scholar 

  29. Takács G (2009) Convex polyhedron learning and its applications [PhD thesis]. Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics

  30. Kantchelian A, Tschantz MC, Huang L, Bartlett PL, Joseph A D, Tygar JD (2014) Large-margin convex polytope machine. In: Advances in Neural Information Processing Systems, pp 3248–3256

  31. Manwani N, Sastry PS (2010) Learning polyhedral classifiers using logistic function. In: Proceedings of the 2nd Asian Conference on Machine Learning, pp 17–30

  32. Astorino A, Fuduli A (2015) Support vector machine polyhedral separability in semisupervised learning. J Optim Theory Appl 164(3):1039–1050

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhou Y, Jin B, Spanos CJ (2015) Learning convex piecewise linear machine for data-driven optimal control. In: Proceedings of the 14th International Conference on Machine Learning and Applications (ICMLA), pp 966–972

  34. Li Y, Liu B, Yang X, Fu Y, Li H (2011) Multiconlitron: a general piecewise linear classifier. IEEE Trans Neural Netw 22(2):276–289

    Article  Google Scholar 

  35. Li Y, Leng Q (2015) Alternating multiconlitron: a novel framework for piecewise linear classification. Pattern Recogn 48(3):968–975

    Article  MATH  Google Scholar 

  36. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  37. Wu Q, Zhou D (2005) SVM Soft margin classifiers: linear programming versus quadratic programming. Neural Comput 17(5):1160–1187

    Article  MathSciNet  MATH  Google Scholar 

  38. Funaya H, Ikeda K (2012) A statistical analysis of soft-margin support vector machines for non-separable problems. In: Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), pp 1–7

  39. Norton M, Mafusalov A, Uryasev S (2017) Soft margin support vector classification as buffered probability minimization. J Mach Learn Res 18(1):2285–2327

    MathSciNet  MATH  Google Scholar 

  40. Rätsch G, Onoda T, Müller K R (2001) Soft margins for AdaBoost. Mach Learn 42 (3):287–320

    Article  MATH  Google Scholar 

  41. Díez J, del Coz JJ, Bahamonde A, Luaces O (2009) Soft margin trees. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp 302–314

  42. Hu W, Chung FL, Wang S, Ying W (2012) Scaling up minimum enclosing ball with total soft margin for training on large datasets. Neural Netw 36:120–128

    Article  MATH  Google Scholar 

  43. Wang Y, Chen S (2013) Soft large margin clustering. Inf Sci 232:116–129

    Article  MathSciNet  Google Scholar 

  44. Aizenberg I (2015) MLMVN With soft margins learning. IEEE Trans Neural Netw Learn Syst 25(9):1632–1644

    Article  Google Scholar 

  45. Franc V, Hlavác̆ V (2003) An iterative algorithm learning the maximal margin classifier. Pattern Recogn 36(9):1985–1996

    Article  MATH  Google Scholar 

  46. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167

    Article  Google Scholar 

  47. Friess TT, Harisson R (1998) Support vector neural networks: The kernel adatron with bias and soft-margin. Technical Report ACSE-TR-752, Dept ACSE. University of Sheffield. UK

  48. Mavroforakis ME, Theodoridis S (2006) A geometric approach to support vector machine (SVM) classification. IEEE Trans Neural Netw 17(3):671–682

    Article  Google Scholar 

  49. Tao Q, Wu G, Wang J (2008) A general soft method for learning SVM classifiers with L1-norm penalty. Pattern Recogn 41(3):939–948

    Article  MATH  Google Scholar 

  50. Liu Z, Liu J, Pan C, Wang G (2009) A novel geometric approach to binary classification based on scaled convex hulls. IEEE Trans Neural Netw 20(7):1215–1220

    Article  Google Scholar 

  51. Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KR (2000) A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Trans Neural Netw 11(1):124–136

    Article  Google Scholar 

  52. Bennett KP, Bredensteiner EJ (2000) Duality and Geometry in SVM Classifiers. In: Proceedings of the 17th International Conference on Machine Learning, pp 57–64

  53. Astorino A, Gaudioso M (2002) Polyhedral separability through successive LP. J Optim Theory Appl 112(2):265–293

    Article  MathSciNet  MATH  Google Scholar 

  54. Lever J, Krzywinski M, Altman N (2016) Points of significance: model selection and overfitting. Nat Methods 13(9):703–704

    Article  Google Scholar 

  55. Frank A, Asuncion A (2010) UCI machine learning repository [Online]. Available: http://archive.ics.uci.edu/ml

  56. Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2(3):27

    Article  Google Scholar 

  57. Platt JC (1999) Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods, pp 185–208

Download references

Acknowledgements

The authors would like to thank all the editors and the anonymous reviewers for their valuable comments. The authors would also like to thank Dr. Huang in Shanghai Jiao Tong University, Dr. Xu and Prof. Shen in Nanjing University for sharing their source codes.

This work was supported in part by the National Natural Science Foundation of China under grants 61602056, 61572082, the Natural Science Foundation of Liaoning Province of China under grants 2019-ZD-0493, 20180550525.

Author information

Authors and Affiliations

  1. College of Information Science and Technology, Bohai University, Jinzhou, 121000, China

    Qiangkui Leng, Zuowei He & Yuqing Liu

  2. College of Engineering, Bohai University, Jinzhou, 121000, China

    Yuping Qin

  3. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Yujian Li

Authors
  1. Qiangkui Leng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zuowei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuqing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuping Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yujian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiangkui Leng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Leng, Q., He, Z., Liu, Y. et al. A soft-margin convex polyhedron classifier for nonlinear task with noise tolerance. Appl Intell 51, 453–466 (2021). https://doi.org/10.1007/s10489-020-01854-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01854-6

Keywords

Search

Navigation