Skip to main content
SpringerLink
Log in

Accelerating the Training of an LP-SVR Over Large Datasets

  • Conference paper
  • First Online:
Artificial Intelligence XXXVII (SGAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12498))

  • 695 Accesses

Abstract

This paper presents a learning speedup method based on the relationship between the support vectors and the within-class Mahalanobis distances among the training set. We explain how statistical properties of the data can be used to pre-rank the training set. Then we explain the relationship among the pre-ranked training set indices, convex hull indices, and the support vector indices. We also explain how this method has better efficiency than those approaches based on the convex hull, especially at large-scale problems. At the end of the paper we conclude by explaining the findings of the experimental results over the speedup alternative.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bennett, K.P., Bredensteiner, E.J.: Duality and geometry in SVM classifiers. ICML 2000, 57–64 (2000)

    Google Scholar 

  2. Cawley, G.C.: Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs. In: The 2006 IEEE International Joint Conference on Neural Network Proceedings, pp. 1661–1668. IEEE (2006)

    Google Scholar 

  3. Chen, X., Xiao, Y.: Geometric projection twin support vector machine for pattern classification. Multimedia Tools Appl. 58, 1–17 (2020)

    Google Scholar 

  4. Cherkassky, V., Ma, Y.: Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw. 17(1), 113–126 (2004)

    Article  Google Scholar 

  5. Collobert, R., Bengio, S.: Svmtorch: Support vector machines for large-scale regression problems. J. Mach. Learn. Res. 1, 143–160 (2001)

    Google Scholar 

  6. Darsow, W.F., Nguyen, B., Olsen, E.T., et al.: Copulas and Markov processes. Ill. J. Math 36(4), 600–642 (1992)

    Article  MathSciNet  Google Scholar 

  7. Gorman, R.P., Sejnowski, T.J.: Analysis of hidden units in a layered network trained to classify sonar targets. Neural Netw. 1(1), 75–89 (1988)

    Article  Google Scholar 

  8. Gu, X., Chung, F.l., Wang, S.: Fast convex-hull vector machine for training on large-scale ncRNA data classification tasks. Knowl. Based Syst. 151, 149–164 (2018)

    Google Scholar 

  9. Joachims, T.: Making large-scale SVM learning practical. Technical Report (1998)

    Google Scholar 

  10. Karsaz, A., Mashhadi, H.R., Mirsalehi, M.M.: Market clearing price and load forecasting using cooperative co-evolutionary approach. Int. J. Electr. Power Energy Syst. 32(5), 408–415 (2010)

    Article  Google Scholar 

  11. Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.: A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Trans. Neural Netw. 11(1), 124–136 (2000)

    Article  Google Scholar 

  12. Kinzett, D., Zhang, M., Johnston, M.: Using numerical simplification to control bloat in genetic programming. In: Li, X., et al. (eds.) SEAL 2008. LNCS, vol. 5361, pp. 493–502. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89694-4_50

    Chapter  Google Scholar 

  13. Liu, Z., Liu, J., Pan, C., Wang, G.: A novel geometric approach to binary classification based on scaled convex hulls. IEEE Trans. Neural Netw. 20(7), 1215–1220 (2009)

    Article  Google Scholar 

  14. Mangasarian, O.L., Musicant, D.R.: Large scale kernel regression via linear programming. Mach. Learn. 46(1–3), 255–269 (2002)

    Article  Google Scholar 

  15. McGarry, K.J., Wermter, S., MacIntyre, J.: Knowledge extraction from radial basis function networks and multilayer perceptrons. In: IJCNN’99, International Joint Conference on Neural Networks, Proceedings, vol. 4, pp. 2494–2497. IEEE (1999)

    Google Scholar 

  16. Mehr, A.D., Nourani, V., Khosrowshahi, V.K., Ghorbani, M.A.: A hybrid support vector regression-firefly model for monthly rainfall forecasting. Int. J. Environ. Sci. Technol. 16(1), 335–346 (2019)

    Article  Google Scholar 

  17. Mezghani, D.B.A., Boujelbene, S.Z., Ellouze, N.: Evaluation of SVM kernels and conventional machine learning algorithms for speaker identification. Int. J. Hybrid Inf. Technol. 3(3), 23–34 (2010)

    Google Scholar 

  18. Niu, B., Jin, Y., Lu, W., Li, G.: Predicting toxic action mechanisms of phenols using adaboost learner. Chemom. Intell. Lab. Syst. 96(1), 43–48 (2009)

    Article  Google Scholar 

  19. Osuna, E., De Castro, O.: Convex hull in feature space for support vector machines. In: Garijo, F.J., Riquelme, J.C., Toro, M. (eds.) IBERAMIA 2002. LNCS (LNAI), vol. 2527, pp. 411–419. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-36131-6_42

    Chapter  Google Scholar 

  20. Peng, X.: TSVR: An efficient twin support vector machine for regression. Neural Netw. 23(3), 365–372 (2010)

    Article  Google Scholar 

  21. Platt, J.C.: Using analytic QP and sparseness to speed training of support vector machines. In: Advances in Neural Information Processing Systems, pp. 557–563 (1999)

    Google Scholar 

  22. Rivas-Perea, P.: Southwestern us and northwestern mexico dust storm modeling trough moderate resolution imaging spectroradiometer data: a machine learning perspective. Technical report, NASA/UMBC/GEST GSSP (2009)

    Google Scholar 

  23. Rivas Perea, P.: Algorithms for training large-scale linear programming support vector regression and classification. The University of Texas at El Paso (2011)

    Google Scholar 

  24. Rivas-Perea, P., Cota-Ruiz, J.: An algorithm for training a large scale support vector machine for regression based on linear programming and decomposition methods. Pattern Recogn. Lett. 34(4), 439–451 (2013)

    Article  Google Scholar 

  25. Santamaría-Bonfil, G., Reyes-Ballesteros, A., Gershenson, C.: Wind speed forecasting for wind farms: A method based on support vector regression. Renew. Energy 85, 790–809 (2016)

    Article  Google Scholar 

  26. Smola, A., Scholkopf, B., Ratsch, G.: Linear programs for automatic accuracy control in regression. In: 1999 Ninth International Conference on Artificial Neural Networks ICANN 99, (Conf. Publ. No. 470), vol. 2, pp. 575–580. IET (1999)

    Google Scholar 

  27. Trzciński, T., Rokita, P.: Predicting popularity of online videos using support vector regression. IEEE Transactions on Multimedia 19(11), 2561–2570 (2017)

    Article  Google Scholar 

  28. Vapnik, V., Golowich, S., Smola, A.: Support vector method for function approximation, regression estimation, and signal processing. Adv. Neural Inf. Process. Syst. 9, 281–287 (1997)

    Google Scholar 

  29. Wang, D., Qiao, H., Zhang, B., Wang, M.: Online support vector machine based on convex hull vertices selection. IEEE Trans. Neural Netw. Learn. Syst. 24(4), 593–609 (2013)

    Article  Google Scholar 

  30. Xu, Z., Huang, K., Zhu, J., King, I., Lyu, M.R.: A novel kernel-based maximum a posteriori classification method. Neural Netw. 22(7), 977–987 (2009)

    Article  Google Scholar 

  31. Zhang, L., Zhou, W.: On the sparseness of 1-norm support vector machines. Neural Netw. 23(3), 373–385 (2010)

    Article  Google Scholar 

  32. Zhong, H., Wang, J., Jia, H., Mu, Y., Lv, S.: Vector field-based SVR for building energy consumption prediction. Appl. Energy 242, 403–414 (2019)

    Article  Google Scholar 

  33. Zhou, X., Jiang, W., Tian, Y., Shi, Y.: Kernel subclass convex hull sample selection method for SVM on face recognition. Neurocomputing 73(10–12), 2234–2246 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science, Baylor University, Waco, TX, USA

    Pablo Rivas

Authors
  1. Pablo Rivas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pablo Rivas .

Editor information

Editors and Affiliations

  1. School of Computing, University of Portsmouth, Portsmouth, UK

    Max Bramer

  2. RKE Consulting, Micheldever, UK

    Richard Ellis

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rivas, P. (2020). Accelerating the Training of an LP-SVR Over Large Datasets. In: Bramer, M., Ellis, R. (eds) Artificial Intelligence XXXVII. SGAI 2020. Lecture Notes in Computer Science(), vol 12498. Springer, Cham. https://doi.org/10.1007/978-3-030-63799-6_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63799-6_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63798-9

  • Online ISBN: 978-3-030-63799-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation