Skip to main content
SpringerLink
Log in

A New Method for Classifying Random Variables Based on Support Vector Machine

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

In this paper, a new version of Support Vector Machine (SVM) is proposed which any of training samples are considered the random variables. Hence, in order to achieve robustness, the constraint in SVM must be replaced with probability of constraint. In this new model, by applying the nonparametric statistical methods, we obtain the optimal separating hyperplane by solving a quadratic optimization problem. Afterwards, we present the least squares model of our proposed method. The efficiency of our proposed method is shown by several examples for both cases (linear and nonlinear) with probabilistic constraints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • AVIDAN, S. (2000), “Support Vector Tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1064–1072.

    Article  Google Scholar 

  • BACHE, K., and LICHMAN, M. (2013), UCIMachine Learning Repository, http://archive.ics.uci.edu/ml/index.php.

  • BARNETT, N.S., and DRAGOMIR, S.S. (2002), “Some Inequalities for Probability, Expectation and Variance of Random Variables Defined Over a Finite Interval”, Computers and Mathematics with Applications, 43, 1319–1357.

    Article  MathSciNet  MATH  Google Scholar 

  • BEN-TAL, A., BHADRA, S., BHATTACHARYYA, C., and NATH, J.S. (2011), “Chance Constrained Uncertain Classification via Robust Optimization”, Mathematical Programming, 127(1), 145–173.

    Article  MathSciNet  MATH  Google Scholar 

  • BOSCH, P., LOPEZ, J., RAMIREZ, H., and ROBOTHAM, H. (2013), “Support Vector Machine Under Uncertainty: An Application For Hydroacoustic Classification of Fish-Schools in Chile”, Expert Systems with Applications, 40, 4029–4034.

    Article  Google Scholar 

  • CHEN, Y., and WANG, J.Z. (2003), “Support Vector Learning for Fuzzy Rule-Based Classification Systems”, IEEE Transactions on Fuzzy Systems, 11(6), 716–728.

    Article  Google Scholar 

  • CHEN, X., YANG, J., LIANG, J., and YE, Q. (2012), “Recursive Robust Least Squares Support Vector Regression Based on Maximum Correntropy Criterion”, Neurocomputing, 97, 63–73.

    Article  Google Scholar 

  • CHIANG, J.H., and HAO, P.Y. (2004), “Support Vector Learning Mechanism for Fuzzy Rule-Based Modeling: A New Approach”, IEEE Transactions on Fuzzy Systems, 12(1), 1–12.

    Article  Google Scholar 

  • CORTES, C., and VAPNIK, V. (1995), “Support Vector Networks”, Machine Learning, 20, 273–297.

    MATH  Google Scholar 

  • EFRON, B. (1979), “Bootstrap Methods: Another Look at the Jackknife”, Annals of Statistics, 7, 1-26.

    Article  MathSciNet  MATH  Google Scholar 

  • GAO, J.B., GUNN, S.R., HARRIS, C.J., and BROWN, M. (2002), “A Probabilistic Framework for SVM Regression and Error Bar Estimation”, Machine Learning, 46, 71–89.

    Article  MATH  Google Scholar 

  • HUANG, H.P., and LIU, Y.H. (2002), “Fuzzy Support Vector Machines for Pattern Recognition and Data Mining”, International Journal of Fuzzy Systems, 4, 826–835.

    MathSciNet  Google Scholar 

  • HUANG, G., SONG, S.,WU, C., and YOU, K. (2012a), “Robust Support Vector Regression for Uncertain Input and Output Data”, IEEE Transactions on Neural Networks and Learning Systems, 23(11), 1690–1700.

    Article  Google Scholar 

  • HUANG, G.B., ZHOU, H., DING, X.J., and ZHANG, R. (2012b), “Extreme Learning Machine for Regression and Multiclass Classification”, IEEE Transactions on Systems, Man, and Cybernetics, Part B (TSMC), 42(2), 513–529.

    Article  Google Scholar 

  • HUANG, G.B., ZHU, Q.Y., and SIEW, C.K. (2006), “Extreme Learning Machine: Theory and Applications”, Neurocomputing, 70, 489–501.

    Article  Google Scholar 

  • JINGLIN, Y., LI, H.X., and YONG, H. (2011), “A Probabilistic SVM Based Decision System for Pain Diagnosis”, Expert Systems with Applications, 38, 9346–9351.

    Article  Google Scholar 

  • JOACHIMS, T. (1998), “Text Categorization with Support Vector Machines: Learning with Many Relevant Features”, in Proceedings European Conference on Machine Learning, eds. C. Nedellec, and C. Rouveirol, London UK: Springer, pp. 137–142.

  • KARR, A.F. (1992), Probability, New York: Springer.

    MATH  Google Scholar 

  • KIM, S., KAVURI, S., and LEE, M. (2013), “Deep Network with Support Vector Machines”, in Neural Information Processing, Springer, pp. 458–465.

  • KIM, S., YUA, Z., KIL, R.M., and LEE, M. (2015), “Deep Learning of Support Vector Machines with Class Probability Output Networks”, Neural Networks, Special Issue, pp. 19–28.

  • KWOK, J.T.Y. (2000), “The Evidence Framework Applied to Support Vector Machines”, IEEE Transactions on Neural Networks, 11, 1162–1173.

    Article  Google Scholar 

  • LANCKRIET, G.R.G., GHAOUI, L.E., BHATTACHARYYA, CH., and JORDAN, M.I. (2002), “A Robust Minimax Approach to Classification”, Journal of Machine Learning Research, 3, 555–582.

    MathSciNet  MATH  Google Scholar 

  • LECUN, Y., BOTOU, L., JACKEL, L., DRUCKER, H., CORTES, C., DENKER, J., GUYON, I., MULLER, U., SACKINGER, E., SIMARD, P., and VAPNIK, V. (1995), “Learning Algorithms for Classification: A Comparison on Handwritten Digit Recognition”, in Neural Networks: The Statistical Mechanics Perspective, World Scientific, pp. 261–276.

  • LEE, Y.J., and HUANG, S.Y. (2007), “Reduced Support Vector Machines: A Statistical Theory”, IEEE Transactions on Neural Networks, 18, 1–13.

    Article  Google Scholar 

  • LI, H., YANG, J., ZHANG, G., and FAN, B. (2013), “Probabilistic Support Vector Machines for Classification of Noise Affected Data”, Information Sciences, 221, 60–71.

    Article  Google Scholar 

  • LIANG, N.Y., HUANG, G.B., SARATCHANDRAN, P., and SUNDARARAJAN, N. (2006), “A Fast and Accurate Online Sequential Learning Algorithm for Feed Forward Networks”, IEEE Transactions on Neural Networks, 17(6), 1411–1423.

    Article  Google Scholar 

  • LIN, C.F., and WANG, S.D. (2002), “Fuzzy Support Vector Machine”, IEEE Transactions on Neural Networks, 13, 464–471.

    Article  Google Scholar 

  • LIU, C., NAKASHIMA, K., SAKO, H., and FUJISAWA, H. (2003), “Handwritten Digit Recognition: Bench-Marking of State-of-the-Art Techniques”, Pattern Recognition, 36, 2271–2285.

    Article  MATH  Google Scholar 

  • LIU, W.Y., YUE, K., and GAO, M.H. (2011), “Constructing Probabilistic Graphical Model from Predicate Formulas for Fusing Logical and Probabilistic Knowledge”, Information Sciences, 181(18), 3828–3845.

    Article  MathSciNet  Google Scholar 

  • LOBO, M., VANDENBERGHE, L., BOYD, S., and LEBRET, H. (1998), “Applications of Second-Order Cone Programming”, Linear Algebra Its Applications, 284, 193–228.

    Article  MathSciNet  MATH  Google Scholar 

  • MEHROTRA, S. (1992), ”On the Implementation of a Primal-Dual Interior Point Method”, SIAM Journal on Optimization, 2(4), 575–601.

    Article  MathSciNet  MATH  Google Scholar 

  • OSUNA, E., FREUND, R., and GIROSI, F. (1997), “Training Support Vector Machines: An Application to Face Detection”, in IEEE Conference Computer Vision Pattern Recognition, pp. 130–136.

  • PARK, W.J., and KIL, R.M. (2009), “Pattern Classification with Class Probability Output Network”, IEEE Transactions on Neural Networks, 20, 1659–1673.

    Article  Google Scholar 

  • PLATT, J.C. (1999), “Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods”, in Advances in Large Margin Classifiers, MIT Press.

  • QI, Z., TIAN, Y., and SHI, Y. (2013), “Robust Twin Support Vector Machine for Pattern Classification”, Pattern Recognition, 46, 305–316.

    Article  MATH  Google Scholar 

  • RINALDI, F. (2009), “Mathematical Programming Methods forMinimizing the Zero Norm Over Polyhedral Sets,” Ph.D. thesis, Operations Research at Sapienza University of Rome.

  • SADOGHI YAZDI, H., EFFATI, S., and SABERI, Z. (2007), “The Probabilistic Constraints in the Support Vector Machine”, Applied Mathematics and Computation, 194, 467–479.

    Article  MathSciNet  MATH  Google Scholar 

  • SEBALD, D.J., and BUCKLEW, J.A. (2000), “Support Vector Machine Techniques for Nonlinear Equalization”, IEEE Transactions on Signal Processing, 48(11), 3217–3226.

    Article  Google Scholar 

  • SOLLICH, P. (2002), “Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities”, Machine Learning, 46, 21–52.

    Article  MATH  Google Scholar 

  • SUN, Y., YUAN, Y., and WANG, G. (2014), “Extreme Learning Machine for Classification Over Uncertain Data”, Neurocomputing, 128, 500–506.

    Article  Google Scholar 

  • SUYKENS, J.A.K., DE BRABANTER, J., LUKAS, L., and VDANEWALLE, J. (2002), “Weighted Least Squares Support Vector Machines: Robustness and Sparse Approximation”, Neurocomputing, 48, 85–105.

    Article  MATH  Google Scholar 

  • SUYKENS, J.A.K., and VANDEWALLE, J. (1999), “Least Squares Support Vector Machines Classifiers”, Neural Processing Letters, 9, 293–300.

    Article  Google Scholar 

  • THI, H.A.L., VO, X.T., and DINH, T.P. (2014), “Feature Selection for Linear SVMs Under Uncertain Data: Robust Optimization Based on Difference of Convex Functions Algorithms”, Neural Networks, 59, 36–50.

    Article  MATH  Google Scholar 

  • TRAFALIS, T.B., and GILBERT, R.C. (2006), “Robust Classification and Regression Using Support Vector Machines”, European Journal of Operational Research, 173(3), 893–909.

    Article  MathSciNet  MATH  Google Scholar 

  • VAPNIK, V. (1995), The Nature of Statistical Learning Theory, New York: Springer-Verlag.

    Book  MATH  Google Scholar 

  • WANG, Y.Q.,WANG, S.Y., and LAI, K.K. (2005), “A New Fuzzy Support Vector Machine to Evaluate Credit Risk”, IEEE Transactions on Fuzzy Systems, 13, 820–831.

    Article  Google Scholar 

  • YANG, X., TAN, L., and HE, L. (2014), “A Robust Least Squares Support Vector Machine for Regression and Classification with Noise”, Neurocomputing, 140, 41–52.

    Article  Google Scholar 

  • YOU, L., JIZHEN, L., and YAXIN, Q. (2011), “A New Robust Least Squares Support Vector Machine for Regression with Outliers”, Procedia Engineering, 15, 1355–1360.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, Ferdowsi University of Mashhad, 9177948974 Azadi Sq, Mashhad, Khorasan Razavi, Iran

    Maryam Abaszade & Sohrab Effati

Authors
  1. Maryam Abaszade
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sohrab Effati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sohrab Effati.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abaszade, M., Effati, S. A New Method for Classifying Random Variables Based on Support Vector Machine. J Classif 36, 152–174 (2019). https://doi.org/10.1007/s00357-018-9282-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-018-9282-x

Keywords

Search

Navigation