Skip to main content
SpringerLink
Log in

Unconstrained convex minimization based implicit Lagrangian twin extreme learning machine for classification (ULTELMC)

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The recently proposed twin extreme learning machine (TELM) requires solving two quadratic programming problems (QPPs) in order to find two non-parallel hypersurfaces in the feature that brings in the additional requirement of external optimization toolbox such as MOSEK. In this paper, we propose implicit Lagrangian TELM for classification via unconstrained convex minimization problem (ULTELMC) and further suggest iterative convergent schemes which eliminates the requirement of external optimization toolbox generally required in solving the quadratic programming problems (QPPs) of TELM. The solutions to the dual variables of the proposed ULTELMC are obtained using iterative schemes containing ‘plus’ function which is not differentiable. To overcome this shortcoming, the generalized derivative approach and smooth approximation approaches are suggested. Further, to test the performance of the proposed approaches, classification performances are compared with support vector machine (SVM), twin support vector machine (TWSVM), extreme learning machine (ELM), twin extreme learning machine (TELM) and Lagrangian extreme learning machine (LELM). Moreover, non-requirement to solve QPPs makes the iterative schemes find the solution faster as compared to the reported methods that finds the solution in dual space. Computational times required in finding the solutions are also presented for comparison.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Avci D, Dogantekin A (2016) An Expert diagnosis system for Parkinson disease based on genetic algorithm-wavelet kernel-extreme learning machine. Parkinson’s Dis 2016:5264743

    Google Scholar 

  2. Balasundaram S, Gupta D (2014) 1-norm extreme learning machine for regression and multiclass classification using Newton method. Neurocomputing 128:4–14

    Article  Google Scholar 

  3. Balasundaram S, Gupta D (2016) On optimization based extreme learning machine in primal for regression and classification by functional iterative method. Int J Mach Learn Cybern 7(5):707–728

    Article  Google Scholar 

  4. Balasundaram S, Gupta D, Prasad SC (2017) A new approach for training Lagrangian twin support vector machine via unconstrained convex minimization. Appl Intell 46(1):124–134

    Article  Google Scholar 

  5. Bi J, Zhang C (2018) An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme. Knowl-Based Syst 158:81–93

    Article  Google Scholar 

  6. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  7. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(Jan):1–30

    MathSciNet  MATH  Google Scholar 

  8. Deng W, Zheng Q, Chen L (2009, March). Regularized extreme learning machine. In: IEEE symposium on computational intelligence and data mining, 2009. CIDM’09. IEEE, pp 389–395

  9. Drucker H, Burges CJ, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. In: Advances in neural information processing systems, pp 155–161

  10. Gupta D, Borah P, Prasad M (2017) A fuzzy based Lagrangian twin parametric-margin support vector machine (FLTPMSVM). In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, pp 1–7. https://doi.org/10.1109/SSCI.2017.8280964

  11. Gupta D, Richhariya B (2018) Entropy based fuzzy least squares support vector machine for class imbalance learning. Appl Intell 48:4212–4231. https://doi.org/10.1007/s10489-018-1204-4

    Article  Google Scholar 

  12. Gupta D, Richhariya B, Borah P (2018) A fuzzy twin support vector machine based on information entropy for class imbalance learning. Neural Comput Applic 31:7153–7164. https://doi.org/10.1007/s00521-018-3551-9

    Article  Google Scholar 

  13. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062

    Article  Google Scholar 

  14. Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3):155–163

    Article  Google Scholar 

  15. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  16. Huang GB, Zhu QY, Siew CK (2004, July) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of 2004 IEEE international joint conference on neural networks, 2004, vol 2. IEEE, pp 985–990

  17. Ismaeel S, Miri A, Chourishi D (2015, May) Using the extreme learning machine (ELM) technique for heart disease diagnosis. In: Humanitarian Technology Conference (IHTC2015), 2015 IEEE Canada International. IEEE, pp 1–3

  18. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  19. Joachims T (1999) Making large-scale SVM learning practical. In: Schölkopf B, Burges C, Smola A (eds) Advances in kernel methods – support vector learning. MIT Press, Cambridge

    Google Scholar 

  20. Lee YJ, Mangasarian OL (2001) SSVM: a smooth support vector machine for classification. Comput Optim Appl 20(1):5–22

    Article  MathSciNet  Google Scholar 

  21. Li Q, Chen H, Huang H, Zhao X, Cai Z, Tong C et al (2017) An enhanced grey wolf optimization based feature selection wrapped kernel extreme learning machine for medical diagnosis. Comput Math Methods Med 2017:9512741

    Google Scholar 

  22. Ma J, Wen Y, Yang L (2019) Lagrangian supervised and semi-supervised extreme learning machine. Appl Intell 49(2):303–318

    Article  Google Scholar 

  23. Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Mach Learn Res 1(Mar):161–177

    MathSciNet  MATH  Google Scholar 

  24. Mangasarian OL (2004) A Newton method for linear programming. J Optim Theory Appl 121:1–18 ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/02-02.ps

    Article  MathSciNet  Google Scholar 

  25. Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162

    Article  Google Scholar 

  26. Mitra SK, Rao CR (1971) Generalized inverse of matrices and its applications. Wiley, New York

    MATH  Google Scholar 

  27. Musicant DR, Feinberg A (2004) Active set support vector regression. IEEE Trans Neural Netw 15(2):268–275

    Article  Google Scholar 

  28. Murphy PM, Aha DW (1992) UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine

    Google Scholar 

  29. Muthusamy H, Polat K, Yaacob S (2015) Improved emotion recognition using gaussian mixture model and extreme learning machine in speech and glottal signals. Math Probl Eng 2015:394083

    Article  Google Scholar 

  30. Ning K, Liu M, Dong M, Wu C, Wu Z (2015) Two efficient twin ELM methods with prediction interval. IEEE Trans Neural Netw Learn Syst 26(9):2058–2071

    Article  MathSciNet  Google Scholar 

  31. Peng X (2010) Primal twin support vector regression and its sparse approximation. Neurocomputing 73:2846–2858

    Article  Google Scholar 

  32. Peng Y, Wang S, Long X, Lu BL (2015) Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 149:340–353

    Article  Google Scholar 

  33. Rastogi R, Sharma S, Chandra S (2018) Robust parametric twin support vector machine for pattern classification. Neural Process Lett 47(1):293–323

    Article  Google Scholar 

  34. Richhariya B, Tanveer M (2018) A robust fuzzy least squares twin support vector machine for class imbalance learning. Appl Soft Comput Elsevier 71:418–432

    Article  Google Scholar 

  35. Ripley BD (2007) Pattern recognition and neural networks. Cambridge university press, Cambridge

    MATH  Google Scholar 

  36. Rozza A, Manzo M, Petrosino A (2014, August) A novel graph-based fisher kernel method for semi-supervised learning. In: 2014 22nd International Conference on Pattern Recognition (ICPR). IEEE, pp 3786–3791

  37. Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  38. Uçar A, Demir Y, Güzeliş C (2016) A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering. Neural Comput Applic 27(1):131–142

    Article  Google Scholar 

  39. Wan Y, Song S, Huang G, Li S (2017) Twin extreme learning machines for pattern classification. Neurocomputing 260:235–244

    Article  Google Scholar 

  40. Xue Z, Zhang R, Qin C, Zeng X (2018) A rough ν-twin support vector regression machine. Appl Intell 48:1–24

    Article  Google Scholar 

  41. Zhou F, Yang S, Fujita H, Chen D, Wen C (2019) Deep learning fault diagnosis method based on global optimization GAN for unbalanced data. Knowl-Based Syst

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, National Institute of Technology, Yupia, Arunachal Pradesh, India

    Parashjyoti Borah & Deepak Gupta

Authors
  1. Parashjyoti Borah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deepak Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak Gupta.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Borah, P., Gupta, D. Unconstrained convex minimization based implicit Lagrangian twin extreme learning machine for classification (ULTELMC). Appl Intell 50, 1327–1344 (2020). https://doi.org/10.1007/s10489-019-01596-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01596-0

Keywords

Search

Navigation