Skip to main content
SpringerLink
Log in

Regularized based implicit Lagrangian twin extreme learning machine in primal for pattern classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper, we suggest a novel approach termed as regularized based implicit Lagrangian twin extreme learning machine in primal as a pair of unconstrained convex minimization problem (RILTELM) where regularization term is added to follow the structural risk minimization principle. Here, we consider 2-norm of the slack vector of variables to make the problem strongly convex which results in a unique solution. Since it has non-smooth plus functions in their objective function, so we find an approximate solution by replacing the non-smooth plus function with smooth approximation function because to find an approximation solution in primal space is always superior to its dual. Due to non-smooth plus function, we solve the problem by either smooth approximation approach or generalized derivative approach. In addition, a functional iterative scheme is also suggested to find the optimal solution. Hence, no external optimization toolbox is required unlike in twin extreme learning machine (TELM) and twin support vector machine (TWSVM). The numerical experiments are demonstrated on artificial and real-world datasets and compared with TWSVM, ELM, TELM and LSTELM to establish the efficacy and applicability of proposed RILTELM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Aslan MF, Sabanci A, Durdu A (2017) Different Wheat Species Classifier Application of ANN and ELM. J Multidiscipl Eng Sci Technol 4(9):8194–8198

    Google Scholar 

  2. Balasundaram S, Gupta D (2016) On optimization based extreme learning machine in primal for regression and classification by functional iterative method. Int J Mach Learn Cybern 7(5):707–728

    Google Scholar 

  3. Balasundaram S, Gupta D, Kapil S (2014) 1-Norm extreme learning machine for regression and multiclass classification using Newton method. Neurocomputing 128:4–14

    Google Scholar 

  4. Balasundaram S, Gupta D, Prasad SC (2017) A new approach for training Lagrangian twin support vector machine via unconstrained convex minimization. Appl Intell 46:124–134. https://doi.org/10.1007/s10489-016-0809-8

    Article  Google Scholar 

  5. Balasundaram S, Tanveer M (2013) On Lagrangian twin support vector regression. Neural Comput Appl 22(1):257–267

    Google Scholar 

  6. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  7. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mac Learn Rese 7:1–30

    MathSciNet  MATH  Google Scholar 

  8. Dua D, Karra Taniskidou E (2017) UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science (2017)

  9. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proc. Int. Conf. Knowl. Discov. Data Mining, San Francisco, CA, (2001): 77–86

  10. Fung G, Mangasarian OL (2003) Finite Newton method for Lagrangian support vector machine classification. Neurocomputing 55(1–2):39–55

    Google Scholar 

  11. Gautam C, Tiwari A, Tanveer M (2019) "OCKELM+: kernel extreme learning machine based one-class classification using privileged information (or KOC+: Kernel Ridge Regression or Least Square SVM with zero bias based One-class Classification using Privileged Information)." arXiv preprint arXiv:1904.08338

  12. Gu X, Chung F-L, Wang S (2020) Extreme vector machine for fast training on large data. Int J Mach Learn Cybern 11(1):33–53

    Google Scholar 

  13. Gu Y, Chen Y, Liu J, Jiang X (2015) Semi-supervised deep extreme learning machine for Wi-Fi based localization. Neurocomputing 166:282–293

    Google Scholar 

  14. Gupta U, Gupta D (2019) Lagrangian twin-bounded support vector machine based on L2-norm. In: Kalita J, Balas V, Borah S, Pradhan R (eds) Recent developments in machine learning and data analytics, vol 740. Springer, Singapore, pp 431–444

    Google Scholar 

  15. Gupta U, Gupta D, Prasad M (2018) Kernel target alignment based fuzzy least square twin bounded support vector machine. In: IEEE Symposium series on computational intelligence (SSCI). IEEE, pp 228–235

  16. Hiriart-Urruty J-B, Strodiot J-J, Nguyen VH (1984) Generalized Hessian matrix and second-order optimality conditions for problems with C1, data. Applied mathematics and optimization 11(1):43–56

    MathSciNet  MATH  Google Scholar 

  17. Horata P, Chiewchanwattana S, Sunat K (2013) Robust extreme learning machine. Neurocomputing 102:31–44

    Google Scholar 

  18. Huang G, Huang G-B, Song S, You K (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48

    MATH  Google Scholar 

  19. Huang G, Song S, Gupta JND, Wu C (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 44(12):2405–2417

    Google Scholar 

  20. Huang G-B, Zhou H, Ding X, Zhang R (2011) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybernetics) 42(2):513–529

    Google Scholar 

  21. Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062

    Google Scholar 

  22. Huang G-B, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18):3460-3468

    Google Scholar 

  23. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Google Scholar 

  24. Huang G-B, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3):155–163

    Google Scholar 

  25. Jayadeva R Khemchandani, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910. https://doi.org/10.1109/TPAMI.2007.1068

    Article  MATH  Google Scholar 

  26. Jia X, Wang R, Liu J, Powers DMW (2016) A semi-supervised online sequential extreme learning machine method. Neurocomputing 174:168–178

    Google Scholar 

  27. Kongsorot Y, Horata P, Musikawan P, Sunat K (2019) Kernel extreme learning machine based on fuzzy set theory for multi-label classification. Int J Mach Learn Cybern 10(5):979–989

    Google Scholar 

  28. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Google Scholar 

  29. Lan Y, Soh YC, Huang G-B (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73(16–18):3191–3199

    Google Scholar 

  30. Lee Y-J, Mangasarian OL (2001) SSVM: A smooth support vector machine for classification. Comput Optim Appl 20(1):5–22

    MathSciNet  MATH  Google Scholar 

  31. Li K, Kong X, Zhi Lu, Wenyin L, Yin J (2014) Boosting weighted ELM for imbalanced learning. Neurocomputing 128:15–21

    Google Scholar 

  32. Li S, Song S, Wan Y (2018) Laplacian twin extreme learning machine for semi-supervised classification. Neurocomputing 321:17–27

    Google Scholar 

  33. Liu J, Patwary MJA, Sun XY, Tao K (2019) An experimental study on symbolic extreme learning machine. Int J Mach Learn Cybern 10(4):787–797

    Google Scholar 

  34. Luo X, Li Y, Wang W, Ban X, Wang J-H, Zhao W (2020) A robust multilayer extreme learning machine using kernel risk-sensitive loss criterion. Int J Mach Learn Cybern 11(1):197–216

    Google Scholar 

  35. Lv F, Han M (2019) Hyperspectral image classification based on multiple reduced kernel extreme learning machine. Int J Mach Learn Cybern 10(12):3397–3405

    Google Scholar 

  36. Ma J, Yang L, Wen Y, Sun Q (2019) Twin minimax probability extreme learning machine for pattern recognition. Knowl-Based Syst 187:104806

    Google Scholar 

  37. Mangasarian OL (2001). Data mining via support vector machines. In: IFIP Conference on system modeling and optimization, pp. 91–112. Springer, Boston, MA, 2001

  38. Mangasarian OL, Musicant DR (2001a) Lagrangian support vector machines. Journal of Mach Learn Res 1:161–177

    MathSciNet  MATH  Google Scholar 

  39. Mangasarian OL, Musicant DR (2001b) Active support vector machine classification. Adv Neural Inf Process Syst 13:577–583

    Google Scholar 

  40. Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2009) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162

    Google Scholar 

  41. Musicant DR, Feinberg A (2004) Active set support vector regression. IEEE Trans Neural Netw 15(2):268–275

    Google Scholar 

  42. Peng X (2010) Primal twin support vector regression and its sparse approximation. Neurocomputing 73(16–18):2846–2858

    Google Scholar 

  43. Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53

    MATH  Google Scholar 

  44. Raghuwanshi BS, Shukla S (2019) Classifying imbalanced data using ensemble of reduced kernelized weighted extreme learning machine. Int J Mach Learn Cybern 10(11):3071–3097

    Google Scholar 

  45. Rao CR, Mitra SK (1971) Further contributions to the theory of generalized inverse of matrices and its applications. Sankhyā Indian J Stat Ser A 33(3):289–300

    MathSciNet  MATH  Google Scholar 

  46. Rastogi R, Bharti A (2019) Least squares twin extreme learning machine for pattern classification. In: Deb D, Balas V, Dey R (eds) Innovations in infrastructure, vol 757. Springer, Singapore, pp 561–571

    Google Scholar 

  47. Rastogi R, Sharma S, Chandra S (2018) Robust parametric twin support vector machine for pattern classification. Neural Process Lett 47(1):293–323

    Google Scholar 

  48. Richhariya B, Gupta D (2019) Facial expression recognition using iterative universum twin support vector machine. Appl Soft Comput 76:53–67

    Google Scholar 

  49. Richhariya B, Tanveer M (2020) A reduced universum twin support vector machine for class imbalance learning. Pattern Recognit. https://doi.org/10.1016/j.patcog.2019.107150

    Article  Google Scholar 

  50. Ripley BD (1994) Neural networks and related methods for classification. J R Stat Soc Ser B (Methodological) 56:409–456

    MathSciNet  MATH  Google Scholar 

  51. Rozza A, Manzo M, Petrosino A (2014) A novel graph-based fisher kernel method for semi-supervised learning. In: 2014 22nd International Conference on Pattern Recognition (ICPR). IEEE, 2014

  52. Sattar AMA, Ertuğrul ÖF, Gharabaghi B, McBean EA, Cao J (2019) Extreme learning machine model for water network management. Neural Comput Appl 31(1):157–169

    Google Scholar 

  53. Shao Y-H, Zhang C-H, Wang X-B, Deng N-Y (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Google Scholar 

  54. Shi L-C, Lu B-L (2013) EEG-based vigilance estimation using extreme learning machines. Neurocomputing 102:135–143

    Google Scholar 

  55. Sun X, Wang Z, Hu J (2017) Prediction interval construction for byproduct gas flow forecasting using optimized twin extreme learning machine. Math Probl Eng 55:1–12

    MathSciNet  MATH  Google Scholar 

  56. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Google Scholar 

  57. Tanveer M (2015) Robust and sparse linear programming twin support vector machines. Cognit Comput 7(1):137–149

    Google Scholar 

  58. Tanveer M, Khan MA, Ho S-S (2016) Robust energy-based least squares twin support vector machines. Applied Intelligence 45:174–186. https://doi.org/10.1007/s10489-015-0751-1

    Article  Google Scholar 

  59. Tran H-N, Cambria E (2018) Ensemble application of ELM and GPU for real-time multimodal sentiment analysis. Memetic Comput 10(1):3–13

    Google Scholar 

  60. Wan Y, Song S, Huang G, Li S (2017) Twin extreme learning machines for pattern classification. Neurocomputing 260:235–244

    Google Scholar 

  61. Wang W, Gan Y, Vong CM et al (2020) Homo-ELM: fully homomorphic extreme learning machine. Int J Mach Learn Cyber 11:1531–1540. https://doi.org/10.1007/s13042-019-01054-w

    Article  Google Scholar 

  62. Wong SY, Yap KS, Yap HJ (2016) A Constrained Optimization based Extreme Learning Machine for noisy data regression. Neurocomputing 171:1431–1443

    Google Scholar 

  63. Yang Z-X, Wang X-B, Wong PK (2018) Single and Simultaneous Fault Diagnosis with Application to a Multistage Gearbox: A Versatile Dual-ELM Network Approach. IEEE Trans Industr Inf 14:5245–5255

    Google Scholar 

  64. Yu Q, Miche Y, Eirola E, Van Heeswijk M, SéVerin E, Lendasse A (2013) Regularized extreme learning machine for regression with missing data. Neurocomputing 102:45–51

    Google Scholar 

  65. Yuan Y, Wang Y, Cao F (2011) Optimization approximation solution for regression problem based on extreme learning machine. Neurocomputing 74(16):2475–2482

    Google Scholar 

  66. Zhou W, Qiao S, Yi Y, Han N, Chen Y, Lei G (2020) Automatic optic disc detection using low-rank representation based semi-supervised extreme learning machine. Int J Mach Learn Cybern 11(1):55–69

    Google Scholar 

  67. Zhu Q-Y, Kai Qin A, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recognit 38(10):1759–1763

    MATH  Google Scholar 

  68. Zhang G, Cui D, Mao S et al (2020) Unsupervised feature learning with sparse Bayesian auto-encoding based extreme learning machine. Int J Mach Learn Cyber 11:1557–1569. https://doi.org/10.1007/s13042-019-01057-7

    Article  Google Scholar 

  69. Zong W, Huang G-B, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Arunachal Pradesh, Papum Pare, Arunachal Pradesh, India

    Umesh Gupta & Deepak Gupta

Authors
  1. Umesh Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deepak Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak Gupta.

Ethics declarations

Conflict of interest

Authors have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, U., Gupta, D. Regularized based implicit Lagrangian twin extreme learning machine in primal for pattern classification. Int. J. Mach. Learn. & Cyber. 12, 1311–1342 (2021). https://doi.org/10.1007/s13042-020-01235-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01235-y

Keywords

Search

Navigation